Reference
How software reads text
The surprising thing about strings is that “length” is not one number. The moment a word leaves the keyboard, the software measures it by a UNIT you chose — and the unit changes the answer. Here’s the model. Open the strings lab →
- A string is measured by a UNIT
There is no single “length” of a word. A person counts LETTERS; a program can count grapheme clusters, Unicode code points, or UTF-8 bytes — and get three different numbers for the same word. Decide which unit you mean before you count, compare, or truncate.
- Grapheme cluster = what a person calls a letter
A grapheme cluster is a base letter plus any marks that ride on it. Vietnamese “ệ” is one cluster even though it carries a circumflex and a dot. Use grapheme clusters (Intl.Segmenter) whenever you mean “letters” — for counting, cursor movement, or a character limit.
- Normalization: one letter, two encodings
The same letter can be stored “composed” (NFC, one code point) or “decomposed” (NFD, a base letter plus combining marks). They look identical but have different lengths and won’t match with ==. Normalize (usually to NFC) before you compare or store.
- Plural is a CATEGORY, not “add an s”
Software picks a word-form by CLDR category (one · few · many · other · …). English has two categories, Vietnamese has one, Polish has four, Arabic six. Never hard-code an English pluralize() — ask Intl.PluralRules for the user’s language.
- Casing and sorting are locale-specific
Turkish “i” upper-cases to a dotted “İ”; Swedish sorts Ö after Z; German sorts it as O. A naïve A–Z uppercase or a raw string sort is wrong for real users — pass the locale to toLocaleUpperCase and Intl.Collator.
- Layout must survive translation
Translations are often 30–50% longer (German especially) — and some scripts are taller. Never size a control to the English word; let text grow and wrap, and test with a long language before you ship.
The units & the tools
grapheme clusterWhat a person calls a letter (base + its marks). Count with Intl.Segmenter. Use for character limits and “length in letters.”Unicode scalarA single code point. [...string] iterates these. A decomposed accented letter is several scalars — more than the eye sees.UTF-8 byteWhat’s stored/sent. ASCII = 1 byte; accented and CJK letters = 2–4. A byte limit is not a letter limit.NFC / NFDComposed vs decomposed encoding of the same text. Normalize before comparing so “é” always equals “é”.plural categoryone · few · many · other — Intl.PluralRules picks the right one for the language and number.collationLocale sort order. Intl.Collator, not a raw string sort — the alphabet is not universal.
Measure your name
A button that really overflows
The same Save button, in three languages. German needs more than twice the room of English.
And one the lab leaves for later: right-to-left
Arabic, Hebrew, Persian and Urdu read right-to-left, and their whole layout mirrors: a “next” arrow points left, the back button moves to the right, padding flips from the leading edge to the trailing edge. Numbers and embedded Latin words stay left-to-right inside the mirrored page. Write your layout in leading/trailing terms, never fixed left/right, and it flips for free. Read a string →
Where this idea comes from
Quốc ngữ: Vietnamese in the Latin alphabet
Vietnamese was once written with Chinese characters and with chữ Nôm, characters made for Vietnamese words. In the 1600s Portuguese and other Jesuit missionaries worked out a way to write it in the Latin alphabet, with marks for its sounds and tones.
One thing to hold: Alexandre de Rhodes’s Vietnamese–Portuguese–Latin dictionary, printed in Rome in 1651. Its spelling is close to the quốc ngữ (“national script”) used today, which replaced characters in schools and newspapers in the early 1900s.
Why it matters: Vietnamese has six tones. Five are written as marks on the vowel (à, á, ả, ã, ạ) and one has no mark, and a vowel can carry a tone mark and another mark at once, as in ệ. That is why one Vietnamese letter can be one code point or three, and three bytes or five: the history is in the byte count.
What we don’t know: How much of the system de Rhodes invented and how much he took from earlier Jesuits, such as Francisco de Pina, is debated by historians.
Dates:
- 1651 De Rhodes’s dictionary is printed in Rome.
- 1919 The old character-based civil examinations end; quốc ngữ spreads in schools.