Not all evidence is created equal. There is a hierarchy of evidence. The best evidence often comes from high quality, randomized, controlled clinical trials that are well designed and well conducted, and without major methodological flaws. Other types of evidence are also useful, but the quality of the evidence must always be carefully considered before making conclusions on nutrition products, plans, or programs.

• Objective evidence is far more compelling than subjective evidence:
  Objective evidence (e.g., scientific studies, lab experiments) can be examined and evaluated by anyone. On the other hand, subjective evidence is based on personal opinion and cannot be independently evaluated. Typical forms of subjective evidence include testimonials, anecdotes, “success stories,” intuition, speculation, expert opinion, “best educated guesses,” observational reports, surveys, and clinical experience. These subjective forms of evidence are certainly important in the scientific method; however, they are only a start. Final conclusions must include rigorous objective evidence to back them up.
  • Be wary of “educated” opinions: Even expert knowledge is not enough. Evidence-based nutrition requires “the integration of individual expertise with the best available external evidence from systematic research.”
• Clinical trials are generally considered the highest form of evidence
  • Controlled studies are more reliable than uncontrolled studies:
    In controlled studies, two (or more) groups of subjects are used: an experimental group, which receives the intervention being tested; and a control group, which receives a comparison intervention, often a placebo. The two groups must be identical in every way possible (similar ages, genders, backgrounds, weights, etc), except for the introduction of the intervention that is being studied. For example, if vitamin E is being tested to see if it decreases the risk of heart disease, two groups of patients could be evaluated: One group that takes vitamin E pills for a specified period of time and another group that takes a control pill, such as vitamin C, selenium, or a placebo.
  • Randomized studies are preferred: In randomized studies, subjects are randomly assigned to either the experimental group or the control group in order to ensure that both groups are as similar as possible (eg, similar average age, cholesterol levels, blood pressures, etc).
  • A blinded study is one in which the subjects do not know whether they are receiving the experimental intervention or the control intervention (often a placebo). This decreases the risk of bias (whether intentional or unintentional). A double-blind study is one in which neither the subjects nor the conductors of the study know which subjects are in the experimental group and which are in the control group. This further decreases the risk of bias.
  • Large studies are more reliable than smaller studies: The results of smaller studies must always be weighed against the risk that they are due to chance. For example, if a coin is flipped 10 times, it is not unusual for 7 or 8 "heads" to occur. Of course, this does not mean that there is a 70-80% chance of heads. Rather, it is only due to chance that heads occurred more often than tails. If the coin is flipped 10,000 times, the results would approach 50/50.
    
    The same is true of human studies. Consider a study consisting of 10 subjects who use an herbal weight loss product, and 8 of the 10 patients lose weight. The product label may say "studies show that 80% of people who use this product lose weight!" However, this is hardly a sufficient sample size to make this claim. There may be hundreds or thousands of other users who lost no weight at all.
  • Studies of long duration are more reliable than shorter studies: Long-term safety and efficacy cannot be evaluated with studies that last only a few weeks or months. This situation often takes the form of an advertisement for a diet plan that boasts "He lost 10 pounds in 3 weeks!" What happens after those 3 weeks is anyone's guess.
• Human studies are more accurate than animal studies: Important lessons can be learned from animal studies, but the results can be very different in humans.
• In vivo studies are more accurate than in vitro ("test tube") experiments: In vivo studies use live human subjects and are far more accurate than in vitro studies. The results of in vitro studies may or may not correspond to similar results in real people. Beta-carotene is a good example: In test tubes it acts as a powerful antioxidant, which has prompted much speculation about possible anti-cancer effects; however, in vivo studies have failed to show much benefit of beta-carotene supplementation.
• A peer review process is essential: Peer review is a standard process in medical experimentation that affords the opportunity to identify and correct errors, oversights, and other flaws in published (or soon to be published) studies. In order to appropriately evaluate the validity of a study or other form of evidence, independent experts who have no stake in the results should be able to review it and comment accordingly. Evidence that has not undergone thorough peer review by independent experts may not be reliable and should be taken with skepticism.
• Consistency. Consistent, repeatable results are more reliable than any one piece of evidence, regardless of its quality.
• Bias, whether intentional or unintentional, negates or limits the effectiveness of any type of evidence: Bias may come in the form of overt conflicts of interest, inappropriate study design, "spin", or other positive or negative influences on a set of data. Bias should be thoroughly considered and evaluated for all evidence.