Not a week goes by when someone doesn’t bring up his/her mistrust of science. No area of science has less trust these days than those associated with nutrition and health. And this mistrust is perfectly valid. Nutrition science is a mess. How many times have you asked why information about nutrition is so contradictory and why it changes on an almost daily basis? This is especially true of the ever-evolving notion of disease-states caused by nutrition. While we definitely know a lot of concrete things about nutrition, most nutrition science out there today is, in fact, junk. That is why nutrition science is one of the best ways to find examples of junk science.
Before we get into it, you may be wondering what I mean by junk science. Is this the same as pseudoscience? The answer is no. Junk science is bad science. That is, it is real science, but science done badly. Pseudoscience is not science at all. So, yes, there is a distinction but this article is meant to help you learn to recognize bad science, not pseudoscience, per se.
Nutrition for Health and Health Care has a list of junk science red flags attributed The Food and Nutrition Science Alliance(FANSA). This list is so useful it forms the basis of this post. They very well sum up how to be on guard against junk science in the nutrition world and, of course, junk science in the health industry and in the broader sense. Remember that a “red flag” does not automatically mean that something is amiss, it means that you should have your hackles raised a bit because you’ve encountered a warning sign. Now, the more red flags you see in one piece of information, the more you can be assured that it is junk. I will expand on some of them.
FANSA is a joining of members from seven professional scientific societies. The organization speaks on food and nutrition science issues. FANSA’s combined membership includes more than 100,000 food, nutrition, and medical practitioners and scientists. The societies are:
- American College of Nutrition
- American Dietetic Association
- American Society for Clinical Nutrition
- American Society for Parenteral and Enteral Nutrition
- American Society of Nutritional Sciences
- Institute of Food Technologists
- Society for Nutrition Education
Here are the red flags of FANSA. Although this list is universally attributed to FANSA, I have been unable to locate an original document or website.
10 Red Flags of Nutrition-Related Junk Science
1. The information promises a quick and easy fix. This should eliminate half of what is out there.
A ready example is any one of many quick fat loss products or programs that promise that you’ll lose weight via some method or pill that “tricks” your metabolism so that you lose weight without dieting or exercise. Nothing of the sort exists.
2. Fearful messages or warnings of great danger from a single food, product, diet, etc. You’ll find that the easy fixes are often accompanied by these types of tactics against the competition. I have an easy example in this typical bashing of a single product, namely casein, just sent to me recently. Of course, this guy knows something I don’t: He knows how to trigger the biological mechanisms that transform your body and extend your life. Should we ask how much older than 100 he is? And how many 115-year-olds he has on his personal success list? Moving on…
3. It says what most everybody wants to hear. In other words, it is too good to be true. Let’s see: Eat all the fatty red meat you want and avoid fruit and gluten (Wheat Belly?). You’ll live longer. Yep, that fits the bill nicely.
4. Simple (simple-minded?) conclusions drawn from a complex study. Most of the time, the data from a highly involved and complex study does not lend itself to one easy and simple conclusion. In fact, more often than not, the data just brings up the need for further investigation into certain areas brought to light by the limitation of the data.
5. Making recommendations based on a single study.
6. Dramatic statements or claims that are refuted by major scientific organizations. This is a lesson that most lay-people never seem to grasp, and it is owed to the many old stories of amazing scientific discoveries that turned accepted knowledge on its head. These stories are the exception, not the rule. Saying something different than everyone else does not make you right!
7. Lists of good and bad foods! These are the articles I hate most of all and they are a dime a dozen. There are no bad foods, nutritionists sometimes say, only bad diets. Also, this is a good time to bring up the difference between a food, a dish, or a product. Anyone can make a list of prepared food products with less than stellar nutrition. However, this is not bad food, it is a bad combination of food ingredients. Check the blatant tactic in this article. The “bad” foods are prepared products, and the good foods are almost all fresh vegetables and fruits, salmon, and few exceptions like a Greek yogurt and some whole grain crackers. These are completely random products compared to completely random foods that are picked out of a hat.
8. You may sometimes find products linked in nutrition articles, whether they be foods, books, etc. This is okay. But when the article is making claims designed to sell that particular product, you should be cautious. “This food is delicious” is probably more credible than “this is a superfood.”
9. Study results released to the media before peer review, and sometimes before publication! Claims or recommendations based on studies without peer review, such as when a study is reported to the media and recommendations are made based on the study before other scientists have a chance to review the work is one of the biggest problems in science today. This absolutely undermines the scientific process. Many times, you will know from how the work is reported whether it has been peer-reviewed, but sometimes this is a tough one for lay people. Today, since there are hundreds, if not thousands of non-peer reviewed journals, the problem is even tougher. This is not to say that peer-review is not without its downside. There are many problems with scientific publishing in general. However, when a study reaches the public before it reaches the scientific community, then the flood-gates for bad science open up wide.
How should we know if a study was peer reviewed? And if it is peer-reviewed, what did the other scientists have to say? Well, the truth is, most fantastical claims made about the healing or marked disease-fighting properties of foods, when based on studies, are based on quite shoddy and discredited studies. Too good to be true is simply that, too – good – to – be – true. A familiar example is pomegranate juice, with its studies claiming pronounced cardiovascular effects. Do your best to read the studies and look for references to the studies so that you can tease out what experts have to say about it. It can be a lot of work, especially since most articles do not properly reference such studies. Usually, though, when the claims being made seem a bit far-fetched, you can feel fairly assured that any studies referenced were not peer-reviewed, were shoddy, or, as is often the case, not even related to the specifics of the claims.
Remember, ANY scientific results that are released to the public prior to being released and reviewed by the rest of the scientific community should be considered suspect until otherwise proven. Among many reasons this is done, one of the main ones is to secure grant money from private institutions so that a University can continue research. If a study can be reported to the media and sensationalized in advance of any scientific review (which may discredit the study), there may be a better chance of securing this money. Not all companies or organizations that have money to contribute to investing in research actually know good science from bad, or whether a certain scientific avenue is worth pursuing. Many millions of dollars are thrown at junk science every year.
10. Recommendations from studies that ignore differences among individuals or groups. This would be “sweeping conclusions” being made based on one sample from one particular group of individuals. What applies to someone else may not apply to you or may not apply to you in the same way. When Morgan Spurlock had his health go the crap after eating exclusively McDonald’s, well, that really happened. But this does not mean that you would be affected in the exact same way at the same rate. Perhaps your baseline fitness is greater, for example. By the same token, when some fitness or health-and-wellness personalities, being in grand shape, try to disprove his results by doing the experiment on themselves, to find less pronounced ill effects, they are making the same mistake in reverse. Of course, these examples are not really scientific studies but are more like shoddy case studies (by lay-persons) on one individual. Case studies cannot be used as evidence of an effect or lack of effect, despite the fact that they can be very illuminating when done properly by actual scientists.
You Aren’t Aware of the NEW Science? My Personal Big Red Flag
I have one big red flag and it has everything to do with science and how it works, as well as the profession that is dietetics or nutrition. This is a kind of statement which, to me, once uttered, should be considered to be a deal breaker because it is the height of dishonesty. The kind of statement I have in mind involves ideas about “modern” nutrition or the new science of nutrition. Many fraudulent huxters use this idea of a new wave of science to discount the mainstream. They say that the new science of nutrition has left dieticians and nutritionists behind. This is, of course, a load of crap. What they really mean is that they have a pet theory that nobody thinks holds any weight and they would like you to think that the reason nobody agrees with it is that they are not privy to the new science. They want you to think they are ahead of the curve and the rest of the scientific community just hasn’t caught up. Says this person, they don’t understand! They cannot see what I see because I have a special understanding they lack! “They don’t understand,” is never a valid defense of a theory: It is no defense at all. Once you hear it, you should know to move on.
Nutrition Article on News Sites: How the Media Misrepresents Nutrition Research
Number five and nine, above, deserve more coverage. After all, I began this article by saying that people mistrust science because science constantly contradicts itself. When we hear about every minor nutrition study published in every obscure journal then why wouldn’t science seem to be contradictory? And, we are deluged with such studies.
In truth, science cannot contradict itself. It’s a concept, a way of gathering knowledge. Scientists can, however, contradict each other. And that is a good thing! That is part of how science works. Scientists review and question other scientists findings. They may seek to replicate the results of a certain study or experiment. And now we’re on to something.
Surprising New Egg Yolk Finding
Replication of results is a cornerstone of the scientific process. Recently, a study was reported which found egg yolk eating to be as bad as cigarette smoking in terms of cholesterol-raising. It made the rounds on Facebook and, as you can guess, most everybody was quite tired of this whole egg yolk fiasco.
This single study was reported, for instance on a site called “ScienceDaily” which constantly reports single new studies: Egg Yolk Consumption Almost as Bad as Smoking When It Comes to Atherosclerosis, Study Suggests.
Notice that the title says that the study “suggests” this thing. However, the small blurb underneath the title uses language quite different:
Newly published research led by Western’s Dr. David Spence shows that eating egg yolks accelerates atherosclerosis in a manner similar to smoking cigarettes.
Now, the study SHOWS that eating egg yolks is like smoking cigarettes in terms of atherosclerosis. Not suggests anymore. It shows.
A single study, even a very good one, cannot show anything. It can only suggest that maybe, just maybe. Replication of results.
The Problem With Media Reporting
The problem, however, is that the media, be it internet websites, news organizations, magazine, etc. have no understanding of the scientific process whatsoever. In fact, even if they hire a person with scientific credentials, it may not help.
Yet most people get the majority of their nutrition and health information from television news, magazine articles, internet news websites, etc. These organizations have deadlines and often rush to “print” without checking the facts.
An even bigger problem, at present, however, is the constant reporting of single studies as if a single study represents a new nutrition paradigm. They report these findings prematurely, without any interpretation, without peer replication, and without peer review. These “SURPRISING FINDINGS” from single studies, just released the day before, if not the same day, are the bane of accurate and reliable nutrition education and info.
The reporters and news organizations are not as concerned with the accuracy of the information as they are with reporting “exciting” new things, hopefully before another organization does. The conclusions and findings are twisted and even when the findings are not conclusive, they are reported as if they are meaningful.
Number One Criteria to Report on a New Study
To add to our problems and our frustration, the number one criteria that any news organization uses to choose which, of the thousands of studies published, to report about. is this: Does the study run contrary to current health and nutrition recommendations? All the other studies that do not run contrary to current recommendations, but further support them, they are not reported on! If you only get one take-home point from this article, this would be an excellent one. Even if you are going to report on a contrary study, it is your responsibility to say something about how it fits with the other evidence on the subject. This is the most important question we can ask!
How Can We Trust Science with Scientists Always Contradicting Each Other?
Well, perhaps you can see now that often, scientists (plural) are not contradicting each other. One scientist is contradicting most other scientists. A trick of psychology is to believe that one dissenting viewpoint outweighs the popular viewpoint, giving these lone voices more power than they sometimes deserve.
Of course, with so many new studies being published every day, and with so many of them being reported by the media, these surprising new headlines often contradict each other! When the scientists themselves, the university or other organization they work for, reports these findings to the media before any peer review the problem becomes worse. How can we trust science, given all this?
I’ll answer that. The scientific big picture rarely changes dramatically. There are certain things that, once you are aware of them, you can count on them. The “science,” in other words, is fairly static, with only small changes in our understanding happening. These small changes slowly contribute to a fundamental shift in understanding.
Nutrition is nothing like the laws of physics, but you can think of it like that. That is, you can’t always count on scientists, but you can count on the laws of physics.
Back to the Egg Debacle
However, the article about the egg study makes no attempt whatsoever to consider any other findings which may shed light on this single study. There is no interpretation, and there is no attempt to provide any skeptical counterpoint from a qualified peer. Certainly, there is no thought given whatsoever to the validity of the methods used in the study, which would be covered by peer review, should it exist. What do we do with this information, as consumers?
Hardly anyone that I know took it seriously. But you can bet many many other people did. Why shouldn’t they? That is what we are here to find out.
Similar to what I’ve already mentioned, many science types will say, well, how was the methodology? Was it a “quality” study? That is certainly important. Other scientists, reviewing the paper, will ask these questions and answer them with their take on the soundness of the methodology and the appropriateness of the conclusions. However…
I’ll let you in on a little secret. There are thousands of crappy studies published all the time. People have this vision that when some controversial new study, overturning everything that came before it, comes along, hundreds of distinguished scientists go on a crusade to check all the P’s and Q’s and a big battle ensues.
Well, most of the time, when they see some clearly flawed piece of crap study, you know what they do? They ignore it! Why? Everybody needs priorities, man! Not everything is worthy of our time. This is the same reason I don’t examine every new workout program or fitness book that comes out. Even though people expect me to. Whether this particular study is being ignored I do not know. But it is not likely to be receiving a lot of attention.
The egg yolk study has been reported on hundreds of web sites by this time. Out of those hundreds, perhaps one or two have the knowledge and ability to analyze the paper. Yet, all you have to go by are the sensational reports of this surprising new finding.
Furthermore, to actually read this study in its entirety, you will have to pay 32 bucks. Who is going to do that? Do you think all the websites reporting this story paid the 32 dollars to buy the full text? I can assure you that they did not. They simply reported the very limited information that was available in the preview. This consists of a very brief summary report of the background, methods, results, and interpretation. That and a quote from a study author is all you’re usually getting from these online articles.
There are many very important details you do not know, and you may not be able to interpret them if you did. Also, you have no sources. For instance, what sources did the author(s) use to come to their hypothesis? Where do we read about the link between cigarette smoking and arteriosclerosis? How does this compare to another food? Why wasn’t another food compared? What would be considered a “control” in this study? Where did the information come from for this study and can we consider it accurate and reliable? How was it determined that egg yolks build up plaque “2/3’s” as much as smoking? What is P<0.0001? What is multiple regression? Do you know? Do the reporters know?
Do you trust that the reason to trust this study is that the person who performed it is an authority figure? Or do you think that because it was reported on news sites it must be accurate and important? I think I have probably shown you that neither of these assumptions are safe.
Yet, if you have seen one of these articles, you probably found yourself, once again, getting a little fed up with the constantly conflicting information. After all, you have been led to believe, as of late, that moderate consumption of egg yolks (and 3 egg yolks per week is certainly moderate) is not bad for you and may even be healthy. You may have read that certain people may be more sensitive to dietary cholesterol than others but that there is no particular need to be concerned. And, you probably have read that eggs are generally considered a healthy food, yolks and all. Then comes new “science” to confuse you, and piss you off. Before it was milk, now it’s eggs.
Well, now you know. All that stuff you have read and been told, it hasn’t changed! Because all the evidence that lead to these very moderate views on egg consumption is still the preponderance of evidence on the subject. This was ONE study. It does not overturn everything that came before it. That is not how it works. When and if our basic understanding of egg consumption changes in a “big picture” way, you will know.
Your mantra, when dealing with these large and sudden changes in the scientific status-quo must become: Science is a slow and deliberate process.
Is Nutrition Science Mostly Junk?
Since I began this article with the provocative statement that nutrition science is mostly junk, perhaps I should end on it. I’m not the only person to think so. Dr. Edward Archer, in his paper, The Failure to Measure Dietary Intake Engendered a Fictional Discourse on Diet-Disease Relations has basically put the science of nutrition on trial. Although he goes too far in his pronouncements, which make it seem as if we know nothing about nutrition, he has identified something in the history of nutrition research which may have created a kind of bottle-neck. As I explained in the article Are Most Diseases Caused by Eating the Wrong Food the early history of nutrition science was about disease states. It was discovered that many theretofore devastating diseases were caused by deficiencies of a certain vitamin. In fact, many, but not all, vitamins were discovered this way. But, as Dr. Archer points out, as our knowledge increased and the public-health improved, the number of big discoveries of nutrition-related diseases decreased. We aren’t likely to discover anything like scurvy or beri-beri again! Yet, nutrition science seems to be a desperate race to discover just such a new disease. So, if you’re wondering why egg yolks are bad yesterday, but good today, you may have your answer.