Real cases
A case · you decide
Zeta and the long tail
Zeta — a teen counting words in a big English text for a word-game project
Zeta ran a word count on a long novel. "The" is the most common word, at about 60,000 times. A teammate guesses the tenth most common word shows up about half as often.
In large texts, word frequency roughly follows Zipf's law: the word at rank r appears about 1/r as often as the top word.
It is an approximation, not a law of physics, but it holds surprisingly well across languages.
About how many times should the tenth-ranked word appear?
The team prices rare words high and common words low. The tile values now follow the curve they measured.
A case · you decide
Grapha and Freqa count the letters
Grapha & Freqa — a script specialist working with the frequency counter
Freqa's script counts Russian "щи" (cabbage soup) as 4 units. Grapha frowns. It is a two-letter word.
In UTF-8, each Cyrillic letter takes 2 bytes, so a byte count doubles the word's length.
A word game is played in letters. Frequencies have to be counted in the units a reader actually sees.
How many tiles should "щи" take in the Russian bag?
The Russian bag now has one "щ" tile, rare and worth a lot, just as the corpus says.
A case · you decide
Rara and Zeta price the rare tiles
Rara & Zeta — a 17-year-old designing a word game for a language club
Rara is setting the bag for an English version of her tile game. Her first draft gives every letter the same number of tiles and one point each.
Zeta's count: in running English text, E shows up about 120 times for every 1 Q.
Rara's rule says the rare, hard-to-use units should carry the most points, so a player who lands one is rewarded.
Which setting fits the count?
Zeta shows where the words are. Rara prices the ones that are hard to reach.