Validity, Reproducibility and Uncertainty: a skill companion
A small set of reusable sheets that grow one inquiry skill: judging whether a method measures what it claims and could be repeated, then weighing a conclusion against its hidden assumptions, any conflicting evidence and the conditions it never tested. Print the scaffolds once and slot them into the science lessons you already teach.
What a skill companion is
Inquiry skills are not a topic of their own. They grow inside the science units a class teaches all year, from Body Systems to Chemical Equations. So this pack is not a full term of lessons. It is three reusable scaffolds, a map of where they fit, and a short stand-alone lesson for teaching the skill on its own first.
Start here: five minutes
- Read the pairing map on the next page: it shows a claim to test in each topic.
- Print the claim checker whenever a class meets a conclusion, its own or someone else’s, and the two-studies sheet as practice.
- Cut out the cards once. They are reused all year.
- Open the free interactive unit on your board when you want a worked example of the skill.
- Run the one-page mini-lesson first if you want to teach the skill before folding it into a topic.
The step up from Year 8
In Year 8 students looked for errors in a method. In Year 9 they judge the method as a whole (is it valid, and is it reproducible?) and then the conclusion drawn from it: what it quietly assumes, what other careful evidence says, and how far beyond the tested conditions it reaches. The aim is a fair restatement: the claim the evidence really supports.
Slot the skill into your science lessons
Every topic produces claims worth testing. This map gives one claim for each Year 9 topic, the question to ask of its method, an assumption hiding in it, and the uncertainty that remains.
| When you teach | A claim to test | Validity question | Hidden assumption | Uncertainty that remains |
|---|---|---|---|---|
| Body Systems and Negative Feedback (AC9S9U01) | “The heart rate always returns to resting within two minutes.” | Was the pulse counted the same way, for the same time, by everyone? | Everyone was equally fit | Only light exercise was tested |
| Reproduction in Animals and Plants (AC9S9U02) | “Cuttings with more leaves always grow more roots.” | Were the cuttings kept in the same light and water? | All cuttings were the same length | Only up to four leaves, one kind of plant |
| The Carbon Cycle (AC9S9U03) | “House plants clean the air in a room.” | A sealed jar is not a room: does the method measure the claim? | The lamp gave the same light all day | One plant of one kind, in one jar |
| Wave and Particle Models of Energy (AC9S9U04) | “A tighter slinky always carries a pulse faster.” | Was each pulse started the same way? | Every flick was the same size | Stretches beyond 3 m were not tested |
| Conservation of Energy and Efficiency (AC9S9U05) | “Thicker wool keeps any drink warmer.” | Was the final temperature read the same way in every run? | Every run started at the same temperature | Other containers, and times longer than ten minutes |
| The Atom and Radioactive Decay (AC9S9U06) | “Half the dice are always gone after four throws.” | Was each six removed every time? | Every die is fair | Small runs vary a lot by chance |
| Chemical Equations and Conservation of Mass (AC9S9U07) | “Mass is always lost when a reaction fizzes.” | In an open cup the gas escapes: does the method weigh everything? | The balance read zero each time | Other reactions, and a closed container |
The three moves, and the picture that backs each one
When you want a worked example on the board, open the interactive unit and use the picture that matches the move.
- Separate the evidence from the over-claim and the assumption: “Which evidence actually supports the claim?”.
- Judge reproducibility from the repeats: “Spot the run that exposes a method flaw”.
- Choose a fix, knowing what it costs: “How should the method be tightened?”.
Does the evidence support the claim?
Judge the method first, then the conclusion. A claim that stays inside what a valid, reproducible method measured is strong; a claim that reaches past it rests on assumptions and uncertainty.
The claim, word for word
Check it
| Valid? Does the method measure what the claim is about, with other influences held steady? | |
| Reproducible? Is the method written so others could repeat it? Did repeats or other groups agree? | |
| Hidden assumptions What was taken as true but never checked? | |
| Conflicting evidence Does other careful work disagree? What differs in its method? | |
| Areas of uncertainty Which conditions did the data never reach? |
A fair restatement
Rewrite the claim so it says only what the evidence supports: the conditions, the size of the effect, and how sure.
Teacher note: a fair restatement is not a weaker claim, it is a more accurate one. “In our jug, over ten minutes, 10 mm of wool kept water about 4 °C warmer than one layer of foil” is worth more than “wool is best”.
Two careful studies, opposite answers
When two careful studies disagree, the scientific move is not to pick a side but to compare the methods. A difference that favours one side can explain the disagreement; a difference that changes both sides alike cannot.
A. The results
Both schools started a jug of warm water at 50 °C and read its temperature after 10 minutes.
| School | Wool-wrapped jug (°C) | Foil-wrapped jug (°C) |
|---|---|---|
| School A | 44.8 | 41.0 |
| School B | 40.2 | 43.1 |
B. The methods
For each difference, decide whether it could explain why the schools disagree, and why.
| School A | School B | Could it explain the disagreement? Why? |
|---|---|---|
| Wool 10 mm thick; one flat layer of foil | Wool 2 mm thick (a thin sock); three layers of crumpled foil with air between them | |
| A 500 mL jug | A 250 mL jug | |
| A still room | Beside an open window | |
| Read with a digital probe | Read with a glass thermometer |
C. Settle it
Describe one test that would settle which wrap keeps water warmer. What would you keep the same?
D. What stays uncertain
Even after your test, name one condition the conclusion would still not cover.
Assumption, conflict, or uncertainty?
Cut out the cards. Lay down the three grey heading cards, then sort the statements under them. On the blank cards, write one from your own investigation for each heading.
Teacher note: the piles are not the same size. The answer sheet lists which card is which, and why.
Check a claim, then test it fairly
Use this stand-alone lesson to teach the skill on its own, before you fold it into a science topic. An advertisement says: “Our cardboard cup sleeve keeps your drink hot three times longer!” Its test: one cup in a sleeve, filled with a hot drink, left on a warm desk; one bare cup with a cooler drink, left by an open window; one run each. The class checks the claim, runs a fairer version, compares groups, and rewrites the claim so it says only what the evidence supports.
We are learning to
- judge whether a method is valid and reproducible,
- find the assumptions, conflicting evidence and areas of uncertainty behind a claim,
- restate a claim so it fits the evidence.
Success criteria
- I can name two reasons the advertised test is not valid.
- I can write a fair restatement of the claim from our class’s results.
You need
- for each group: two paper cups of the same kind, a cardboard sleeve or a strip of corrugated card and a rubber band, a thermometer, a measuring jug and a stopwatch; warm water from the hot tap (an adult pours; not from a kettle),
- the claim checker (scaffold 1), one per student, and the cards from scaffold 3, one set per table. Wipe up spills straight away.
Lesson flow (about 30 minutes)
| 5 min | Read the claim Read the advertisement and its test aloud. Students fill the top of the claim checker. Ask: “The sleeved cup started hotter and sat somewhere warmer. What can the result tell us about the sleeve?” |
| 10 min | A fairer test Groups pour 150 mL of warm water into each cup, read both starting temperatures, sleeve one cup, and read both again after 5 minutes, side by side on the same desk. While the cups cool, tables sort the cards. Ask: “Which assumption of the advertisement have we now checked?” |
| 10 min | Compare groups Groups write the difference between their cups on the board. Show the board (see the box). Ask: “Two groups disagree. Before choosing one, what differed in how they ran it?” |
| 5 min | Restate the claim Each student writes a fair restatement: the conditions, the size of the difference, and how sure the class can be. |
Running it shorter? Check the claim on paper, and run the fair test and restate the claim in your next science lesson.
Watch for these ideas
- Trusting a claim because it has a number in it, like “three times longer”.
- Picking the result you like when two groups disagree, instead of comparing their methods.
- Thinking that naming uncertainty makes a conclusion worthless.
Make it easier, make it bigger
- Easier: give the claim checker with the “Valid?” row filled in as a model.
- Bigger: plan how the advertiser could test “three times longer” properly, and what “longer” would have to mean.
Answers and look-fors
The next sheet has the worked comparison of the two schools, the card key, a checked version of the cup-sleeve claim, and a quick three-level guide.
Answers and look-fors
Two studies disagree
School A found wool 3.8 °C warmer than foil; School B found foil 2.9 °C warmer than wool. Both were careful, so this is conflicting evidence: compare the methods.
| The difference | Explains it? | Why |
|---|---|---|
| Wool 10 mm thick; one flat layer of foil against wool 2 mm thick (a thin sock); three layers of crumpled foil with air between them | Yes | B gave the foil much more trapped air than the wool, so the wraps were not compared at equal thickness |
| A 500 mL jug against a 250 mL jug | No | a smaller jug cools faster in both wraps, so it cannot turn the result around |
| A still room against beside an open window | Yes | a draught blows through a thin knitted sock but not through foil, so it favours the foil |
| Read with a digital probe against read with a glass thermometer | No | both jugs at each school were read the same way |
A test that settles it: the same jug, the same volume and starting temperature, wool and foil wraps of the same thickness, side by side in still air and then in a draught, three runs each. What stays uncertain: other containers, other wraps, and times longer than ten minutes.
Evidence cards
| Card | Statement | Pile | Why |
|---|---|---|---|
| A | The water started at the same temperature in every run, though nobody measured it. | Hidden assumption | Taken as true, never checked; an uneven start could explain a difference. |
| B | Nobody tested what happens over an hour instead of ten minutes. | Area of uncertainty | A time the data never reached. |
| C | One study found wool keeps water warmest; another, equally careful, found foil did. | Conflicting evidence | Two careful results that disagree: compare the methods. |
| D | Every die in the box is fair. | Hidden assumption | Never tested; a loaded die would change the pattern. |
| E | The balance read zero before each cup was weighed. | Hidden assumption | Assumed, not recorded; a balance that was not zeroed gives precise, wrong masses. |
| F | Two thermometers in the same cup at the same moment read 41 °C and 44 °C. | Conflicting evidence | Two readings of one thing that disagree: check both instruments. |
| G | The bounce was measured only from drops of up to 2 m. | Area of uncertainty | Higher drops were never tested. |
| H | Last year’s class found more leaves gave more roots; this year’s class found no difference. | Conflicting evidence | Two results that disagree: look for what changed between the years. |
| I | All the cuttings were the same length when they went into the water. | Hidden assumption | Assumed, not measured. |
| J | One class timed the slinky pulse at 0.60 s; another, with the same stretch, timed 0.75 s. | Conflicting evidence | The same test, different results: compare how each class timed it. |
| K | The sealed-jar test used one plant of one kind. | Area of uncertainty | Other plants were never tested. |
| L | The lamp gave out the same light all day. | Hidden assumption | Assumed; a light meter would check it. |
The cup-sleeve claim, checked
| Valid? | No: the sleeved cup started hotter and sat somewhere warmer, so the result mixes three differences, not one. |
| Reproducible? | No: one run each, and the method gives no volumes, temperatures or times. |
| Hidden assumptions | that both drinks started equally hot, and that the room was the same for both cups. |
| Conflicting evidence | the class’s fair tests, if groups find a small difference or none. |
| Areas of uncertainty | other cups and drinks, longer times, and what “three times longer” was measured to. |
A fair restatement at standard: “In our classroom, a cardboard sleeve kept 150 mL of warm water in a paper cup about _ °C warmer after 5 minutes than a bare cup; groups differed, so we are only fairly sure.” The gap comes from the class’s own results.
A quick three-level guide
| Move | Working towards | At standard | Beyond |
|---|---|---|---|
| Judge the method | accepts a method because it has numbers | says whether the method measures the claim and could be repeated | names the change to the method that would fix each weakness |
| Weigh the conclusion | agrees or disagrees with the claim as a whole | finds the hidden assumptions, the conflicting evidence and the areas of uncertainty | explains conflicting evidence from a difference in method |
| Restate it fairly | drops the claim, or keeps it unchanged | restates the claim inside the tested conditions, with the size of the effect | says how sure the restated claim is, and what would make it surer |