Ready to begin
No records have been processed. Start with Next step.
SECTION 05 / 15
Test an idea: are frequent words short?
On this page 2 headings
A hypothesis is an idea we can test with data. Looking at a few familiar words suggests that frequent words may be short. We need clear definitions and all relevant words before deciding what the evidence says.
Make the definitions precise
Frequent means occurring more than once. Infrequent means occurring once. Add all stored letter counts: 329 letters across 65 word occurrences gives an average length of 329/65 ≈ 5.0615. Thus integer lengths 5 or less are short; lengths 6 or more are long. The average here is over occurrences, while the classification table below counts each different word once.
Normalize case and remove punctuation at word ends before counting: Monday and Monday. use the same key. Keep internal hyphens and use the supplied LetterCount. First finish the frequency counts; then place each distinct word into exactly one of four boxes.
REAL COURSE DATA · FOLLOW THE PROCEDURE
Test short words against word frequency
Read the current record, check the explanation, and watch the variables. Use Back to revisit any step.
Ready
What we remember
- Processed
- 0
Start at the first step. Watch the current row and the values we remember.
| Final frequency | Short (≤5) | Long (≥6) | Total |
|---|---|---|---|
| Frequent (>1) | 8 | 1 | 9 |
| Infrequent (=1) | 15 | 23 | 38 |
| Total | 23 | 24 | 47 |
Tends to is different from always
Monday has six letters and occurs three times. It disproves “every frequent word is short”. However, 8 of the 9 frequent word types are short: about 88.9%. Of the 38 infrequent types, 23 are long: about 60.5%. The two matching boxes contain 8 + 23 = 31 of 47 types, about 66.0%.
These are observations about this paragraph, not proof about all English text, and not a claim that word length causes frequency. Reverse statements also differ: 8/9 frequent words are short, but only 8/23 short word types are frequent. The supplied counts resolve the transcript’s spoken arithmetic slips.
Try it yourself
Is Monday short under this definition?
Show explained answer
No. Six letters makes it long; it is the frequent-long counterexample.
Try it yourself
Why does the table total 47 instead of 65?
Show explained answer
It counts each distinct normalized word once, rather than all occurrences.
Try it yourself
What fraction of frequent word types are short?
Show explained answer
8/9, about 88.9%.
Try it yourself
Does 8/9 prove that short words cause repetition?
Show explained answer
No. It describes an association in this dataset, not a causal rule.
Try it yourself
Why is the average not slightly below 5?
Show explained answer
329/65 is approximately 5.0615, which is above 5. The transcript’s quick spoken estimate slips, but the integer short/long boundary remains 5/6.