← Errata: Built to Last

The Commons Year · The case · ages 15-18

1. They Don't Make Them Like They Used To

The hammer rang once. A spark jumped off the anvil and did not go out. It waited. They followed.

The April issue of the Green Gazette had a feature across the middle pages, and Tara was proud of it. It was the kind of story people liked to read on a Saturday with a cup of tea: warm, a bit funny, full of pictures of people holding spanners.

Ollie in Year 10 had written it. It was his second big story, after the one about the energy drinks. He had spent three Saturday mornings at the repair café in the church hall, where volunteers fixed toasters and lamps and jumpers and bikes for free, as long as you stayed and watched and learned how. Mrs Kaur ran it. She had a toolbox the size of a suitcase and a laugh you could hear from the car park.

Ollie had noticed something. The old bikes that came in, the heavy ones from the nineteen-seventies with steel frames and real leather saddles, could always be mended. Always. A new chain, a bit of oil, a patch on the tyre, and off they went. But the newer bikes, the light shiny ones from the last few years, were often too far gone. A cracked frame. A gear system nobody could get the parts for. Mrs Kaur would shake her head and say it wasn't worth it.

He had asked the volunteers about it, and they had all said the same thing, more or less. Mrs Kaur had said it best, and Ollie had used it for the headline:

THEY DON'T MAKE THEM LIKE THEY USED TO

Underneath, the story said that every single 1970s bike brought to the repair café this year had been mended, while a quarter of the new bikes had been sent away as not worth fixing. There was a photograph of Mr Pettit, who had written to the Gazette about crossings and floods and anything else that annoyed him, sitting proudly on his 1978 bicycle, which he rode to the paper shop every morning.

People loved it. A man stopped Mrs Hallam outside the bakery to say he'd been saying that for years.

On Saturday there was a letter in the Gazette box.

It was typed, on paper with a small drawing of a bicycle at the top: Daley's Cycles, the Green, est. 1961.

My grandfather opened this shop, it said. My father ran it after him, and now I do. Since your April paper, four customers have told me that new bikes are rubbish and they would rather buy an old one off the internet. One of them was a mother buying her daughter's first bike. I sell good bikes. I mend them too. I would like to know how your reporter decided that old ones are better, because that is not what I see in my workshop.

Tara pinned it to the board above the back table at the Whetstone café. She read it twice.

She had been proud on Tuesday. Now she was uneasy, and she did not know why yet.

2. The Whole Sheet

"I was going to say every bike in the story is real," Tara said.

She had brought the letter to the back table. Danya was there, sticking a puncture patch onto the inner tube of his own bike, which he had brought in a carrier bag and which smelled strongly of rubber. Across from him sat Sana, who had spent last summer as an intern at a lab and still checked every sample sheet twice, out of habit, even when it was a shopping list.

"I was going to say it like this," Tara said. "Ollie counted the bikes himself, at the repair café, on three Saturdays. He didn't make anything up. Every old bike was mended. A quarter of the new ones weren't. I can write back to Mr Daley and say so."

She already had the sentence half written in her head. Thank you for your letter. Our reporter counted every bike himself.

It felt good. It felt like the end of the conversation.

Sana read the feature slowly, and then the letter, and then the feature again.

"Show me the whole sheet, not the tidy one," said Sana. "Ollie's count is the tidy one. Thirty-something old bikes, all fine. What I want to know is who got left out."

"Nobody got left out," said Tara. "He counted every bike that came in."

"Every bike that came in," said Sana. "Which ones didn't come in? Which ones did we drop, and why?"

"Says who that any got dropped?" said Danya, without looking up from his patch.

"Nobody says it," said Sana. "That's what worries me. Nobody's even asked."

Tara opened her mouth. Then she shut it again.

"How do we know which bikes didn't come in?" Danya asked.

Nobody knew. That was the problem.

3. The Log

Deed came in at eleven, with a bag of second-hand books and a printout of the council's minutes about the new cycle path. Deed read council minutes for fun. She kept every planning application on her phone, sorted into folders.

Tara showed her the feature and the letter.

"Where are the numbers?" Deed asked.

"In the story."

"No, the original document," said Deed. "The repair café will keep a log. They have to, for the insurance. A summary of a summary drifts, and Ollie's story is a summary of three Saturdays. I want the log."

Tara texted Mrs Kaur. Mrs Kaur sent back a photograph of a big blue exercise book, open, with columns ruled in pencil: date, item, age, fixed? Underneath she had written: All of this year so far. Tell Ollie well done.

Deed turned the phone round so everyone could see, and counted with her finger.

"Read it where it was first written down," said Deed. "Old bikes, from the seventies: thirty-six this year. Fixed: thirty-six. Every one."

"So that's true," said Tara.

"That's true," said Deed. "Newer bikes, from the last ten years: a hundred and twenty. Not worth mending: thirty." She did the sum on her phone. "Thirty out of a hundred and twenty. A quarter. Ollie's right about that too."

"Show me the source, then," said Danya, leaning over. "Thirty-six and a hundred and twenty. So the old ones really are better."

"The numbers are right," said Deed slowly. "Page one, column four, here." She tapped the column. "I'm not sure they say that."

4. Where Are the Others?

Sana had been quiet. Now she put her finger on the number thirty-six.

"Thirty-six old bikes," she said. "Thirty-six bikes from the nineteen-seventies came to the repair café this year. How many bikes were there in the nineteen-seventies?"

"Loads," said Danya. "Everyone had a bike. My grandad says the whole street had bikes."

"So where are they?" said Sana.

Nobody answered.

"The ones you threw away are data too," said Sana. "If there were thousands of bikes on the Green in nineteen seventy-eight, and thirty-six of them came in this year, then the thirty-six aren't the old bikes. They're the old bikes that are still here. The ones that rusted through, or got a cracked frame, or were left out in the rain and thrown on the tip. Where are those?"

"In the tip," said Danya.

"In the tip, forty years ago," said Sana. "They don't come to the repair café. They can't. So the café only ever sees the tough ones."

Tara thought about it.

"But we don't know how many there were," she said.

"Mr Daley might," said Deed. "Est. 1961. His grandfather opened the shop."

Tara texted Mr Daley. He did not answer for twenty minutes, and then he sent a photograph of his own: a long, thin, cloth-bound book, very old, with a faded label on the front. Sales, 1978.

My grandfather wrote down every bike he sold, the message said. I've got one for every year. I've never thought about it before. This is the year of Mr Pettit's bike. I looked him up. He's on page 31.

Deed scrolled through the photos he sent, page after page of neat old handwriting, and added up the totals at the bottom of each page.

"Nine hundred bikes," she said. "From one shop, in one year."

"Who got left out?" said Sana. "Thirty-six still riding out of all the bikes sold in the seventies. That's not even from one year. But even if every one of them was from nineteen seventy-eight..." She did it on her phone. "Thirty-six out of nine hundred is four in a hundred. Four per cent. And there were years before and after."

"Eight hundred and sixty-four of that year's bikes are gone," said Deed. "From one shop. In one year."

"Does the sample look like the whole?" said Sana. "The café's sample is the four in a hundred that lasted. Of course they're tough. They're the tough ones. That's why they're still here."

Danya had stopped patching.

"Who checked that before it went in?" he said. "That's like saying everyone who's still alive at a hundred must have been really healthy. Of course they were. The others aren't there to ask."

"Exactly like that," said Sana.

5. Compared With What

Widen arrived with a takeaway hot chocolate and his notebook of what things cost, from a bus ticket to a bridge. He read the feature and Mrs Kaur's log and Mr Daley's ledger, one after another.

"Compared with what, though?" said Widen. "Widen the view. Ollie's comparing two groups. What's in each one?"

"Old bikes and new bikes," said Tara.

"No," said Widen. "Look properly. One group is the old bikes that survived forty years. Thirty-six, out of thousands. Forty years did the sorting. Every weak one was thrown away long ago." He wrote it in his notebook. "The other group is every new bike that broke this year. The good ones and the bad ones, all mixed in, before anything has sorted them. Of course a quarter of those won't mend. Some of them are the bikes that will be in the tip in forty years."

"So it's not old against new," said Tara slowly.

"One number on its own tells you almost nothing," said Widen. "It's the toughest four in a hundred of the old ones, against all of the new ones. That isn't a fair race. It was never the same kind of count."

He drew two boxes in his notebook. In one he wrote 36 survivors of 900+. In the other he wrote 120 new, unsorted. He drew a line between them and a question mark on the line.

"Is that a lot, or does it just sound like a lot?" said Widen. "A quarter of new bikes not worth mending sounds bad. But nobody knows how many of the 1978 bikes were not worth mending in 1979. Nobody wrote that down."

"So we don't actually know whether old bikes were better," said Tara.

"We know the ones that lasted are good," said Widen. "We don't know whether they were typical. Probably they weren't. That's the whole problem."

Tara looked from the log to the ledger to the notebook.

"Can I say one thing?" Tara asked. "So the feature was wrong?"

"Not wrong," said Deed. "Every number in it is true. It drew the wrong lesson from them."

6. The Box

Attest came last, carrying the Gazette's red pen. Attest subbed every story before it went to print and drew a red circle round any number that did not have a name next to it. The April feature had come back with one circle on it, round the words every 1970s bike. Next to the circle Attest had written: every one that came in?

"Where did that number come from? I wondered whether anyone would read my note," said Attest.

"I read it," said Tara. "I thought you meant check the count."

"I meant: which ones," said Attest. "But I didn't think it through either. Nobody did, until Sana asked where the others were."

"Let me listen first," Tara said, before anyone could start explaining. She had learned that in the autumn, from the photo of the flooded underpass, and again last month, from the bar chart. She took out a pad and wrote three headings across the top: what we printed, what the log says, what is missing.

Under the first she wrote old bikes are built to last. Under the second she wrote 36 old, all mended; 120 new, 30 not. The third column filled up quickly this time: the 864, and then the ones in the tip, and then how many new ones will still ride in 40 years? (nobody knows).

"Hm. So what does a clarification like this have to carry?" Tara asked.

"Every claim names its source," said Attest. "The repair café's log, for both counts. Then it has to say what the old bikes are: the ones that lasted, out of many more that were sold. Mr Daley's ledger, for one year, if he agrees. And that nobody knows yet how today's bikes will do over forty years, because nobody has had forty years to find out."

"Do we take the headline back? Say Ollie got it wrong?"

"No," said Attest. "Ollie's numbers are right, and he named his source. The volunteers really said it. Put a name next to it, and say what kind of count it was."

"Hm. That's a lot for one box," said Tara.

"It's two sentences," said Attest. "Maybe three. Possibly four, if Mr Pettit insists on being in it."

Tara looked at Mr Daley's letter again. One of them was a mother buying her daughter's first bike.

"I think we missed someone," Tara said slowly. "Mrs Kaur. She said the line we used for the headline. If the box says it's a mistake, it'll look like we're calling her a fool. She isn't. Old bikes really are easier to mend. She knows more about bikes than all of us."

Nobody answered at once.

"A correction names its source too," said Attest. "Who you name is your call. That's your bit."

7. Four People and a Pen

Mrs Hallam, who advised the Gazette and signed off every issue, came in at noon. Ollie came with her, looking worried. Mrs Kaur came ten minutes later with oil on her hands and a lamp under her arm that she had been mending. Mr Daley came last, in a fleece with the shop's name on it, carrying the old cloth-bound ledger in a plastic bag. Mr Pettit was not invited, and came anyway, on his bike.

Ollie spoke first.

"I counted every bike," he said. "Three Saturdays. Every old one got fixed. I watched. And the volunteers all said it, not just Mrs Kaur. And old steel frames really are strong, I looked it up. If I'd left that out, I'd have been hiding what I saw."

It was the strongest version of his argument, and everything he had seen was true.

Mrs Kaur spoke next.

"I stand by what I said," she said cheerfully. "Old bikes are easier to mend. Simple parts. You can get a spanner on everything. Half the new ones I can't open without a special tool that costs more than the bike. That's real. You can print that."

Then Mr Daley put the ledger on the table, very gently, as if it might fall apart.

"I'm not asking you to say new bikes never break," he said. "They do. I mend them every day. I'm asking you to understand what your story did. My grandfather sold nine hundred bikes the year he sold Mr Pettit his. Most of them are gone. Not because nobody loved them. Because bikes wear out, and these were the ones that lasted. Now a mother has bought her daughter a forty-year-old bike off the internet with brakes I wouldn't trust, because she read that old ones are better."

"I didn't mean that," said Ollie.

"I know," said Mr Daley. "You didn't. A count did."

Mr Pettit cleared his throat.

"My bike," he said, "is the best bike ever made. I have ridden it every day since nineteen seventy-eight. I am not a statistic."

"You're one of the thirty-six," said Danya.

Mr Pettit considered this.

"I am one of the thirty-six," he agreed, with dignity. "I would like that noted."

Nobody said anything for a while.

Mrs Hallam took the corrections box out of her bag. It was a small grey panel at the bottom of page two, with a border round it and room for a few lines. She had used it seven times that year. She set it on the table in front of Tara, with the April feature beside it.

On the table in front of Tara were Mrs Kaur's blue log, with its column of thirty-six ticks. There was Mr Daley's ledger from nineteen seventy-eight, with nine hundred bikes written in by hand, and Mr Pettit on page thirty-one. There was Widen's notebook, with two boxes and a question mark between them. There was Attest's red note: every one that came in? And there was one letter, on paper with a small drawing of a bicycle at the top.

"You're the editor," said Mrs Hallam. "What does it say?"

She handed Tara the pen.

Tara looked at the empty box.

 

 

(Write the line Tara prints in the corrections box.)

 

The spark dimmed to a warm dot and settled back on the anvil. The Green was just the Green again.

Back on the Green

That week Lan, who was seven, said that all the conkers in the park were really big this year.

Danya asked her where she had looked. Lan said under the big tree, where the biggest ones were, because the little ones were no good.

Danya asked whether Lan had picked up the little ones too, before deciding they were all big.

Lan thought about it for a long time. Then she went back to the park with a bag and picked up every conker under the tree, big and small, and laid them out on the kitchen table in a line.

Most of them were quite small. Lan said that was all right. She liked the small ones too.

Afterword. The idea in this case is survivorship bias. If you only look at the things that made it through some long test, like old bikes that have lasted forty years, you are looking at the toughest few, not at a fair sample of all of them. The ones that did not survive are missing from the count, and they were usually the majority. The repair café's numbers were true: every old bike that came in could be mended. But the old bikes that came in were four in a hundred of those sold, and the new bikes were everyone's, before time had sorted them. A fair comparison counts the ones that were thrown away too, or compares things at the same age. Good reporting says what kind of count it is, names its source, and never takes a true number back. You can practise asking who got left out of a sample in InquiryForge, and putting a number next to another in NewsForge.

Who you met

  • Sana Before you trust a result, check who was counted and who was left out from InquiryForge
  • Deed Go to the original document, not a summary of a summary from NewsForge
  • Danya a child of the Green Asks "how do we know?"
  • Widen One number means little without something to compare it with from NewsForge
  • Attest Every claim names its source, and a correction names one too from NewsForge
  • Tara a child of the Green Listens longest, speaks last

Every recurring character has a short tune, so you can tell who is on stage before a word is said. The tunes play only when you tap, and the Reading Access sound switch turns them off.

The Honest Page

Which parts of this story are true, stretched or made up?

True

  • Survivorship bias is the mistake of looking only at the things that made it through some test, and not at the ones that did not, so the survivors seem typical when they are the exceptions. Source
  • In the Second World War, the statistician Abraham Wald pointed out that the bullet holes on returning planes showed where a plane could be hit and still come home; the places with no holes were where planes that did not return had been hit. Source
  • In the case, 30 of 120 newer bikes were not worth mending, a quarter; 36 old bikes still riding out of 900 sold in one year is four in a hundred, and 864 of that year's bikes are gone.

Stretched

  • In the case, a 1978 sales ledger is found and read in one afternoon. What we simplified: Real records of old sales are often lost, and there may be many more old bikes still riding that never come to a repair café. That uncertainty is part of the point: the ones you can see are not a fair sample.

Made up

  • The repair café, its log, the bike shop and its ledger, Ollie, Mrs Kaur, Mr Daley and the Green Gazette's April feature are made up. The idea of survivorship bias is real.