Evaluating Studies

When you read a study, for example one claiming a link between rising temperature and a change in a species, you need to judge how far its conclusions can be trusted. Ask these questions.Who did it and where was it published? Work in a peer-reviewed journal has been checked by experts. Check who funded it, as funding from an interested group could introduce bias.Was the method valid? A valid study measures what it claims to measure. Were other variables controlled, or at least monitored? In field studies many variables cannot be controlled, so a control site or a comparison over time is needed.How large was the sample? A large, randomly chosen sample is more representative and reduces the effect of chance. Results from one site or one year are weak evidence.Are the results repeatable and reproducible? Repeatable means the same investigator gets similar results using the same method; reproducible means other scientists, possibly using different methods, get similar results.Do the data support the conclusion? Look at the spread of the data, the error bars and whether a statistical test shows a significant difference or correlation. If error bars overlap a lot, the difference between the means may not be significant. Check that the conclusion does not go beyond the data, for example by extrapolation far outside the range measured, or by claiming that a correlation proves a cause.Finally, consider whether the study agrees with other evidence. A single study rarely settles a question; confidence grows when many independent studies agree.