When can you believe the studies?

We are being bombarded with study results from every which way on daily basis. It’s impossible not to notice a few patterns in this never-ending stream. They flip-flop more often than a politician trying to appease various audiences; one day we read about red wine preventing heart disease, next day they tell us the opposite. Cheese goes from bad to good to tolerable in moderation, milk follows the suit, and don’t even get me started on red meat. You must eat your breakfast according to one study even if you are not hungry, until the next one tells you to stop it at once. Saturated fat kills you on sight or cures most of the known diseases. It goes on and on.

Is there some guidance about how we should take this onslaught of contradictory nutritional advice? Let’s try to formulate a few guiding principles.

• Any article titled “New study shows” should be regarded with skeptical brow-raising. The reason is simple: there is a lot of context to any study, the methodology used in it is a matter of careful research, and mainstream media articles often sensationalize and distort the actual findings without giving you much insight into all the detail. You may, for instance, read a scary headline about “red meat increasing cancer risk by 60%” (!!) not realizing that it a)may mean increase of the risk from 1 to 1.6 percent (quite a different thing, even though headline is formally correct, right?) and b)lumps together red meat and processed meat. To sum it up:

Without reading an actual study while having enough experience in deciphering it, don’t take mass media reports on faith.

• Any study defending a low-fat high-carb diet should be regarded with “This again?” Last 50 years of following this dietary advice led to an explosion of obesity, diabetes, and many other health complications tied to metabolic problems.And what about all the studies this advice was based on? Going over them is not unlike an Easter egg hunt – what has been done in this one to twist the facts toward the “fat is bad” conclusion? Hunt that has been successful in EVERY SINGLE CASE. Find it hard to believe? Allow me to refer you to The Big Fat Surprise by Nina Teicholz. Going over all the studies that underpin the low-fat paradigm, this book dismantles them one by one, demonstrating that there has never been a solid scientific case for cutting down saturated fat in our diet. To sum it up:

Verifying that study methodology is solid and conclusions can be trusted is tricky at best. It takes a sizeable effort, a lot of time and skill, and requires access to information we laymen rarely have.

•  A study blaming sugar and carbs should be regarded with care so we don’t repeat what was done to fat without enough evidence. The problem with low-fat paradigm encompassing our society diet for decades is multifold, and one of the most important aspects of it is taking it on faith without scientific evidence, simply because it “made sense.” From there it elevated to an unchallengeable dogma. Let’s not add to a mistake by correcting it with the same methodology that led to a mistake in a first place. The goal at this point should be more than getting our diet back on the right track; preventing scientifically unsound approach turning into a nation-wide experiment is just as important. To sum it up:

Whatever you believe in your heart and whatever makes sense to you, rigorous science is the only way to confirm any notion.

• Every time you see that a study uses words “A is associated (or linked, or correlates) with B,” know that you are looking at a so-called epidemiological study. Those do not prove causation – events might be linked in some way but you can’t conclude that one causes another. The best example of that is a classic “Whenever ice cream sales rise, so do shark attacks.” Is there a third factor that causes both (like summertime in the example with sharks)? Are the two just symptoms of some larger factor? Will the correlation hold in other locales, times, and circumstances? To sum it up:

Beware of correlation studies as far as guidance is concerned. The clinical randomized double-blind study is the only reliable way to prove causation.

• Apply common sense to study results – to a degree! Sometimes you are looking at the conclusions and think “this just doesn’t mesh with reality,” and you are correct. For instance, study result concluding that something we ate from the beginning of times is bad for us, while some chemical concoction that never existed before is beneficial to our health deserves to be shrugged off as non-sensical. It’s not always the case, though – remember the infamous “eating fat makes us fat, eating cholesterol raises our cholesterol.” Just how intuitive was that? The fact that dietary fat and cholesterol do not directly translate into fat and cholesterol in our cells and bloodstream just never registered with many scientists, let alone us mere mortals. Sadly, here too we have to refrain from simple answers. Let’s remember this eternal wisdom:

“For every complex problem, there is an answer that is clear, simple, and wrong.” – H. L. Mencken

So, when can you believe a study? I am afraid, there is no simple answer to that. It’s tricky. You can rely more on a large clinical study with a solid methodology, reviewed by peers, confirmed by repeated studies, set up by an independent body that is not financed by interested parties. However, confirming all that is often beyond the layman’s capabilities. I suggest checking it all out to the best of your abilities, yet standing to be corrected.

Let’s finish with an almost comical example of the conflict of interest. The screenshot below shows a complaint about “propaganda” of eggs consumption by the egg producers’ lobby. Have a look at the source of financing of complainers in the footnote. Can you say Irony Incorporated?

(Thanks, Martin Andreae MD‏@M_Andreae for the screenshot)