FluffyBench

Unicode inspector

Inspect Unicode code points and UTF-16 offsets, or compare NFC, NFD, NFKC and NFKD normalization forms with the original.

FluffyShield Protected workspace
From
To

Choose text conversion
FluffyShield

Isolated workspace: other page scripts can't read your files. How FluffyShield works

How it works
  1. Under To, choose Code points or a normalization form
  2. Paste your text; short text updates as you type
  3. Read the rows, or copy the normalized text

Limits Over 100,000 characters, the result waits for you: press the button that appears, such as Inspect or Normalize NFC.

Text looks identical, but an exact comparison says it's different?

Two copies of a word can look the same on screen and still be stored differently, so a search or an exact match fails. FluffyBench's Unicode inspector shows the stored values behind each letter, so you can see where the copies differ, and it can rewrite text into one consistent spelling.

FluffyBench · 6 min read · Reviewed

Why can text look identical but compare differently?

Say you search a document for café and get nothing, even though café is right there on the screen. What you see is the finished letter; what the computer compares is the list of parts it was built from. An accented letter can be stored as one ready-made piece, or as a plain e followed by an accent that attaches to it.

Here are both ways of storing é, side by side, with the values behind each one.

Left: é stored as one piece, U+00E9, 1 code point, UTF-8 bytes C3 A9. Right: é stored as U+0065 plus U+0301, a plain e and an accent, 2 code points, UTF-8 bytes 65 CC 81. They look the same, and an exact comparison says they're different.
Same letter on screen, two different sets of stored values. A search for one spelling won't find the other. How we reviewed

Doing it in FluffyBench: inspect the stored values

Paste the smallest piece of text that shows the problem, such as the one word that won't match. On a phone, Input and Result sit on two tabs; on a wider screen they sit side by side.

Here's how to look inside one copy of the text, using the é that's typed as e plus an accent.

  1. FluffyBench Unicode inspector with From set to Text and To set to Code points.
    FluffyBench Unicode inspector with From set to Text and To set to Code points.
    Step 1. From stays on Text. Under To, choose Code points.
    1. From: Text, the kind of input you paste.
    2. To: choose Code points.
  2. Input box holding an e with a combining accent, which reads 2 characters, 3 bytes, with the Input and Result tabs above it.
    Input box holding an e with a combining accent, which reads 2 characters, 3 bytes, with the Input and Result tabs above it.
    Step 2. Paste one copy of the text. Short text is inspected as you type, so there's nothing else to press.
    1. Paste one copy, here é typed as e + accent.
    2. On a phone, open the Result tab.
  3. Code points result: 2 UTF-16 code units; 2 code points. Rows: offset 0, U+0065, decimal 101, 1 code unit; offset 1, U+0301, decimal 769, 1 code unit.
    Code points result: 2 UTF-16 code units; 2 code points. Rows: offset 0, U+0065, decimal 101, 1 code unit; offset 1, U+0301, decimal 769, 1 code unit.
    Step 3. Read the rows: this é is stored as two values, U+0065 (e) and U+0301 (the accent).
    1. The result is a list of code points.
    2. One row per stored value: e, then the accent.

Now clear the box and paste the other copy, the one-piece é. FluffyBench inspects one text at a time, so you compare the two lists yourself.

Code points result for the one-piece é: 1 UTF-16 code unit; 1 code point. One row: offset 0, U+00E9, decimal 233, 1 code unit.

The one-piece é under Code points

One row, U+00E9, where the other copy had two. That extra accent row is the difference an exact comparison trips over.

Read the code-point rows in FluffyBench

The first line gives two totals. Each row after it is one stored value, called a code point. Code point is its Unicode number, such as U+0065 for e, and Decimal is the same number written in ordinary digits. UTF-16 offset is where that value starts in your text, counted from 0. Code units says how much room the value takes: 1 for most letters, 2 for many emoji. Note stays empty unless your text holds half of a two-unit value on its own, such as half an emoji; then it reads Lone surrogate.

Emoji show why the offset can skip. Here's a smiley face followed by x: the x is the second thing you see, but its row starts at offset 2.

The text 😀x takes 3 UTF-16 code units, numbered 0 to 2. The emoji fills units 0 and 1, so its row reads offset 0, U+1F600, 2 code units. The x sits in unit 2, so its row reads offset 2, U+0078, 1 code unit.
The emoji needs two UTF-16 units, so everything after it starts one place later than you'd count by eye.

Rows aren't letters. The rows count stored values, not the letters a reader sees, so the é typed as e plus an accent is one letter on screen and two rows here. They don't show how a font draws the text, either.

Compare normalization with your original text

Once you've found the difference, you can make both copies use the same spelling. That's called normalization: like agreeing that everyone writes dates the same way before you compare them. NFC joins a letter and its accent into one piece; NFD splits them apart. Pick the form the system you're matching expects.

Here's the e-plus-accent é rewritten as NFC, the one-piece spelling, and taken away as text.

  1. FluffyBench Unicode inspector with From set to Text and To set to NFC.
    FluffyBench Unicode inspector with From set to Text and To set to NFC.
    Step 1. Under To, choose NFC, the form that joins a letter and its accent.
    1. To: choose NFC.
  2. Input box holding é typed as e plus accent, reading 2 characters, 3 bytes, with the Input and Result tabs above it.
    Input box holding é typed as e plus accent, reading 2 characters, 3 bytes, with the Input and Result tabs above it.
    Step 2. Paste the text. Input keeps it exactly as you pasted it, and the NFC copy appears under Result.
    1. Your original: 2 characters, e + accent.
    2. Open Result for the NFC copy.
  3. NFC result pane showing é with a size readout of 1 character.
    NFC result pane showing é with a size readout of 1 character.
    Step 3. Check the count: the same-looking é is now 1 character.
    1. NFC copy: 1 character, down from 2.
  4. FluffyBench Copy and Download buttons, shown close up.
    FluffyBench Copy and Download buttons, shown close up.
    Step 4. Take the NFC copy to another app with Copy, or save it with Download.
    1. Copy puts it on your clipboard.
    2. Download saves a text file.

There are four forms to choose from. NFC and NFD only join or split letters and accents. NFKC and NFKD also swap formatting look-alikes, such as the one-character ligature fi and a superscript ², for plain ones.

Two inputs under each form. é typed as e plus accent: NFC and NFKC change it to the one-piece é, NFD and NFKD leave it unchanged. fi² (a ligature and a superscript two): NFC and NFD leave it unchanged, NFKC and NFKD change it to fi2.
NFKC and NFKD make more text match, but they throw away formatting. When the exact characters matter, such as a quotation or a signed text, keep your original and use NFC or NFD.

Greyed-out Copy, or a job for another FluffyBench tool

Copy and Download are greyed out. That happens when your text holds a lone surrogate, half of a two-unit value such as an emoji. The rows keep it visible, but plain text can't carry it, so FluffyBench points you to the Base encoder's UTF-16 byte views to copy it exactly.

The Unicode inspector shows code points, not bytes. For bytes, a domain name or a search pattern, these FluffyBench tools fit the job better.

Need to convert a domain name to its IDN form?

Need to encode the inspected text as bytes?

Need to test a JavaScript regular expression?

Search FluffyBench