The first question “can experiments detect differences that matter?” is important because if your experiment isn’t going to yield any useful information then it’s kind of a shot in the dark, it is essentially useless till you find out that the results matter. The second question “does the study show causation or correlation?” really brings to light the importance of the data, causation is very hard to prove scientifically, where as correlation is much easier. However, correlation doesn’t have the same large impact that causation does. The third question “what is the mechanism?” is important because without the mechanism we aren’t truly sure of the science behind the data. The mechanism is useful in determining how we can use the acquired data to our benefit, to positively impact human health. The fourth question “how much do experiments reflect reality?” is a strong question because if the data you’re acquiring isn’t realistic in the real world then it’s not a good model for how the real world works. The whole point of experiments is to be able to put together a model of what happens in the real world. The last question “could anything else explain the results?” brings about the potential for confounding variables. If the experiments conducted did not have good controls while keeping all other variables stable then the results could be due to other variables, meaning the data is essentially useless to the hypothesis being tested. I feel that the last question is most important when discussing controversy because if your data isn’t done with solid methods, then the data is essentially useless in terms of discussing controversy objectively.
- Hanage W. 2014. Microbiome science needs a healthy dose of scepticism. Nature 512:247–248.
