Analysis: Vegetables are good. The meat is bad. Here’s how meta-studies can be “interpreted” to provide simplistic results

The researchers, who report in Nature Medicine, start here.

Lifelong exposure to risk leads to a wide variety of outcomes. Objectively judging the relative impact of these risks on individual and population health is fundamental to individual survival and societal prosperity. Existing mechanisms for quantifying and classifying the magnitude of these myriad effects and the uncertainty of their estimation are largely subjective…

There are a number of reasons why meta-analysis, the aggregation of many studies, may be uncertain, including the choices of studies to report, the size of those studies, as well as specific characteristics of the population and the results taken into account. At best, the meta-analysis considers a wide variety of apples and, at worst, apples and oranges. Research contribution is a new methodology to reduce uncertainty and bias.

The Burden of Evidence Risk Function (BPRF)

“[BPRF] reflects both the magnitude of the risk and the extent of the uncertainty…”.

The underlying mathematical techniques are offered as a “complementary” technique to those already in hand. It takes the average risk of an outcome, measured by looking at the response of the middle 15% to 85% of respondents, eliminating extremes or “outliers”. He then characterizes this result as the “smallest level of excess risk consistent with the data” – they interpret the data conservatively. More importantly, it does not assume the linear nature of risk, altering the “dose makes the poison” argument, recognizing that a particular risk may plateau and cause no further deleterious effects or even a small amount may not not be harmful, but useful. (You can read more about linear “thresholds” in biological systems here.)

They assigned stars to their findings to make their work more accessible to non-expert. One and two stars indicate that “conservative interpretation of the evidence may suggest” little or no evidence of a five-star protective effect where there was strong evidence of a protective effect of more than 46%. To demonstrate the methodology, they considered four risk factors, smoking, hypertension, and consumption of red meat and vegetables, along with various outcomes, creating 180 risk-outcome (RO) pairs. Here’s what they found.

Each graph illustrates the studies considered and their results as well as the derived BPRF, with shaded areas representing the uncertainty of their conclusions. Both graphs show the relative risk or benefit; the one on the left has a logarithmic scale.

Smoking

371 observations from 78 studies since 1980. “On average, we found a very strong relationship between pack-years of smoking and the log of the relative risk of lung cancer…smoking significantly increases the risk of lung cancer. ” 5 star risk-result pair.

Hypertension

189 observations from 41 studies of systolic blood pressure and ischemic heart disease. 5 star risk-result pair.

Heart disease and vegetable consumption

78 observations from 17 studies. “The relationship is not log-linear. We found that, on average, vegetable consumption was protective, with the relative risk of ischemic heart disease being 0.81 (0.74-0.89) at 100 grams of vegetable consumption per day versus 0 grams per day. The calculated BPRF suggests a lower risk of heart disease of about 12%. 2 star risk-result pair.

Heart disease and red meat consumption

43 observations from 11 studies. “For unprocessed red meat and ischemic heart disease, the average exposure BPRF is 0.01, essentially on the zero threshold.” The calculated BPRF suggests a 1% lower risk of heart disease. 2 star risk-result pair. (I think that might still be too generous, but that’s my bias.)

Nearly two-thirds of the results were rated one or two stars; only 22 have been rated four or five stars. They pointed out that certain risks, such as smoking and hypertension, had high marks for several outcomes, “which should be considered when making individual decisions about risk exposure.” And while one-star ratings may receive less attention, there may be “small but significant benefits to individuals if their exposure is reduced.”

Personal health vs public health

While star ratings are intended for non-experts, the researchers believe they serve a complementary and valuable purpose for experts with a more nuanced and “inherently subjective” understanding. Researchers rely on two public health risk assumptions.

• “The precautionary principle implies that public policy should pay attention to all potential risks.”
• “…public policy should pay attention not only to risk functions that are supported by evidence, but to the prevalence of exposure to those risks.”

Your personal decision regarding these risks should also be based on the prevalence of your exposure and your risk tolerance. It would be foolish to ignore these five-star risks, but for the one- and two-star risks, a little moderation, especially in food choices, could serve you just as well.

The editorial accompanying this study makes two remarks limiting this meta-analysis or any other meta-analysis. First, it does not address the bias in studies that use “lifestyle questionnaires to record risk factors and self-report health outcomes.” Second, many of these studies come from “well endowed” regions, i.e. first world countries, and may not be generalizable globally. This is also true for the applicability of in-depth ‘Third World’ studies to ‘First World’ countries.

Last word from the authors,

One of the goals of this new star rating system is to clear up confusion and help consumers make informed decisions about diet, exercise and other activities that can affect their long-term health.

— Dr. Christopher Murray, Director of the Institute for Health Metrics and Evaluation

For those interested in all of their findings, they have provided the data in an easily viewable manner here.

Sources:

Burden of Evidence Studies: Assessing Nature Medicine Risk Evidence DOI: 10.1038/s41591-022-01973-2

Dr. Charles Dinerstein, MD, MBA, FACS is a Senior Medical Fellow at the American Council on Science and Health. He has over 25 years of experience as a vascular surgeon. He received his MBA with honors from George Washington University’s MBA in Healthcare program and has consulted to hospitals. Although no longer clinically active, his writings have been presented to KevinMD and Doximity. Follow him on Twitter @CRDtoday

A version of this article was originally published on the American Council on Science and Health and is used here with permission. Check out the American Council on Science and Health on Twitter @ACSHorg