Critique of AHRQ report: “Meditation Programs for Stress and Well-being"”

David Orme-Johnson, Ph.D.

This is a critique of a comparative effectiveness review entitled “Meditation Programs for Stress and Well-being,” which was prepared for the U.S. Government's Agency for Healthcare Research and Quality (AHRQ) and posted on their website from December 4, 2012 through January 2, 2013 (1).

The AHRQ’s non-scientific process
The main problem with this report (“Meditation Programs for Stress and Well-being,”) is that the AHRQ review process does not adhere to even the minimal standards of science that any professional journal requires.  

Peer-reviewed journals send submitted papers to independent outside reviewers to critique, and the authors of the submission must address the weaknesses and flaws identified by the reviewers and incorporate changes into the submission to the satisfaction of the reviewers before the paper is published in the journal.  AHRQ does invite outside professional as well as public reviews. However, study authors are not required to make changes that satisfy the reviewers’ criticisms before they publish their reports. They only promise to make revisions, “as appropriate” through some opaque internal process to be posted three months after the review is finalized. In any journal review, the researchers may even have to go back and do more research and analyses and completely revise the paper, with the reviewer’s final signing off on it before it is acceptable for publication. The AHRQ reviews do not have such a transparent process of interaction with the reviewers in place, and consequently, there is no real accountability to the scientific community (2). As a scientist and taxpayer who has paid for this report, as well as paid for all the salaries of the AHRQ personnel, I have to say that the AHRQ review process is a sham, blatantly ignoring the most basic tenets of the scientific process, making it completely open to bias and vulnerable to the agendas of special interest groups.

Bias in the inclusion criteria 
Meta-analysis is a completely objective process, as far as the mathematics of quantifying the effects of a body of studies is concerned. There are indeed many choices and decisions on how to conduct it, but these are explicitly stated and transparent. Where subjectivity and bias can creep in is in the selection of what studies to include  (3, 4).
 
Omission of the most relevant studies
The guiding principle for what studies to include should be the best controlled and most relevant ones for addressing the major question being posed by the analysis: “This report reviews the efficacy of meditation programs on stress-related outcomes among those with a clinical condition.” Yet this report excludes meditation studies on hypertension, chronic heart failure, arterial sclerosis and other aspects of cardiovascular disease, which are arguably the conditions most well-documented to be stress-related (5-7). The omitted meditation studies in this area used highly objective outcome measures, such as decreased blood pressure in hypertensive patients (8-13), arterial blockage in patients with blocked arteries (14), decreased mortality due to cardiovascular disease and by all causes in hypertensive patients over an 18-year period (13, 15, 16), reduction of enlarged hearts in patients with left ventricular hypertrophy (17, 18), and decreased strokes, heart attacks and death due to all causes over a ten-year period in patients with at least 50% blockage of one or more of the major arteries to their heart (19). 

All these studies used active treatment control groups to control non-specific effects, such as expectation, attention, social support, amount of contact time with the instructors, and other factors. All were on the Transcendental Meditation technique (TM) and there are no such studies on mindfulness. Yet, studies on mindfulness on much more subjective outcomes, such as pain perception, were included. The selection process of what studies to include in this AHRQ report suggests a bias that is not in the national interest. The review was initially presented as being on all types of meditation, yet the name given in the download of the preliminary report is simply “Mindfulness Meditation.” 

Another exclusion criterion not favorable to the TM technique but favorable to mindfulness techniques was excluding studies on adolescents, who are not “children” using different techniques, as the report asserts. Learning to meditate in early adulthood could potentially reduce stress-related problems and diseases and increase the quality of life across the lifespan (6, 17, 20-22).  The report also misclassifies the TM technique as “concentration” meditation, even though it is consistently characterized as an effortless technique requiring no concentration (23, 24), and recently as automatic self-transcending (25).
 
There are advantages and limitations of active controls in behavioral research and using cross-validation to solve the problem. The AHRQ report only included studies that used active control groups to control for non-specific effects, which is good. But there can be problems interpreting such studies. For example, in a study on anxiety prominently cited in the review as evidence that TM does not work,  Smith (1976) carefully constructed a control group that had received all same the expectation fostering features and procedural details as the TM program and found that both TM and the control group reduced anxiety (26). Does this mean that TM is just a placebo? Not necessarily. TM’s reduction of anxiety is cross-validated by studies showing it reduces autonomic correlates of anxiety, such as respiratory rate, skin resistance, and plasma lactate, compared to sitting comfortably with eyes closed as is done in TM practice  (27). It also reduces cortisol, a major stress hormone in humans (28), and reduces stress reactivity (22, 29, 30). Coronary heart disease is a correlate of anxiety (31, 32) and TM practice reduces coronary heart disease (5). Physiological cross-validating evidence should be included in evaluating outcomes such as Smith’s. The abstract of the AHRQ report states: “We need more research using adequately powered high-quality randomized controlled trials that address the effects of meditation programs on stress and its correlates (emphasis added).” Yet, the review ignored precisely that information, the physiological and medical correlates of stress.

The review should also be broadened to take into account the results of previous meta-analyses. To continue with the example of Smith’s study, it is relevant that a recent meta-analysis, conducted by researchers at Chemnitz University in Germany, who are completely independent of any TM organization, found that TM practice reduces anxiety more than mindfulness and other meditation techniques (33).  The studies included were not limited to randomized controlled trails (RCTs), but they do replicate an earlier meta-analysis, which also found that TM practice reduced anxiety more than other meditation and relaxation techniques, even when only RTC’s conducted by researchers who were neutral or negative towards the TM technique were included (34). The other meditation and relaxation treatments that the TM technique has been compared with in these meta-analyses provides a wide range of controls for attention, expectation, social support, etc. that support the conclusion that TM has non-specific effects on reducing anxiety, regardless of the conclusions from Smith’s study.

Tunnel vision 
This AHRQ report used a set of exclusion/inclusion criteria that severely limited its perspective on the current status of meditation research on stress and well-being, which led to highly distorted conclusions. I was invited to be a key informant at the beginning of this study, and I emphasized to the study group that they needed to include studies on objective outcomes on stress and its correlates, such as cardiovascular disease, and that they needed to examine cross-validating physiological evidence of stress reduction. Apparently, they had another agenda, and chose not to provide a balanced picture of the evidence.
 
References:
1.           Goyal M, Singh S. Meditation programs for stress and well-being: Draft comparative effectiveness review Rockville, MD: Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services; 2012; Available from: http://effectivehealthcare.ahrq.gov/index.cfm/search-for-guides-reviews-and-reports/?productid=1343&pageaction=displayproduct.
2.           Orme-Johnson DW. Commentary on the AHRQ report on research on meditation practices in health. The Journal of Alternative and Complementary Medicine. 2008 Dec;14(10):1215-21.
3.           Borenstein M, Hedges L, Higgins J, Rothstein H. Introduction to Meta-Analysis. Southern Gate, Chichester, UK: John Wiley & Sons, Ltd; 2009.
4.           Rothstein HR, Sutton AJ, Borenstein M, editors. Publication Bias in Meta-Analysis – Prevention, Assessment and Adjustments. Hoboken, NJ John Wiley & Sons, Ltd; 2005.
5.           Orme-Johnson DW, Barnes VA, Schneider RH. Transcendental Meditation for the Prevention of Coronary Heart Disease In: Allan R, Fisher J, editors. Heart & Mind: the Practice of Cardiac Psychology (2nd edition). Washington, DC: American Psychological Association; 2011.
6.           Barnes VA, Orme-Johnson DW. Prevention and treatment of cardiovascular disease in adolescents and adults
through the Transcendental Meditation program®: A research review update. Current Hypertension Reviews. 2012;(in press).
7.           Lewis G. Psychological distress and death from cardiovascular disease. British Journal of Medicine. 2012;345:e5177(31 July 2012):1-2.
8.           Alexander CN, Schneider RH, Staggers F, Sheppard W, Clayborne M, Rainforth MV, et al. A trial of stress reduction for hypertension in older African Americans (Part II): Sex and risk factor subgroup analysis. Hypertension. 1996;28(1):228-37.
9.           Rainforth MV, Schneider RH, Nidich SI, Gaylord-King C, Salerno JW, Anderson JW. Stress reduction programs in patients with elevated blood pressure: A systematic review and meta-analysis. Current Hypertension Reports. 2007;9(6):520-8.
10.         Schneider RH, Alexander CN, Staggers F, Orme-Johnson DW, Rainforth MV, Salerno JW, et al. A randomized controlled trial of stress reduction in the treatment of hypertension in African Americans during one year. American Journal of Hypertension. 2005;18(1):88-98.
11.         Schneider RH, Staggers F, Alexander CN, Sheppard W, Rainforth MV, Kondwani K, et al. A randomized controlled trial of stress reduction for hypertension in older African Americans. Hypertension. 1995;26:820-7.
12.         Anderson JW, Liu CH, Kryscio RJ. Blood pressure response to Transcendental Meditation: A meta-analysis. American Journal of Hypertension. 2008;21(3):310-6.
13.         Alexander CN, Langer EJ, Newman RI, Chandler HM, Davies JL. Transcendental Meditation, mindfulness, and longevity: an experimental study with the elderly. Journal of Personality and Social Psychology. 1989;57(6):950-64.
14.         Castillo-Richmond A, Schneider RH, Alexander CN, Cook R, Myers H, Nidich SI, et al. Effects of stress reduction on carotid atherosclerosis in hypertensive African Americans. Stroke. 2000;31:568-73.
15.         Schneider RH, Alexander CN, Staggers F, Rainforth MV, Salerno JW, Hartz A, et al. Long-term effects of stress reduction on mortality in persons ≥ 55 years of age with systemic hypertension. American Journal of Cardiology. 2005;95(9):1060-4.
16.         Barnes V, Schneider R, Alexander C, Rainforth M, Salerno J, Kondwani K, et al. Impact of Transcendental Meditation on Mortality in Older African Americans with hypertension—Eight-Year follow-up. Journal of Social Behavior and Personality. 2005;17(1):201-16.
17.         Barnes VA, Kapuku G, Treiber FA. Impact of Transcendental Meditation® on Left Ventricular Mass in African American Adolescents. eCAM 2012. 2012;2012:1-6.
18.         Schneider R, Alexander C, Orme-Johnson D, Castillo-Richmond A, Rainforth M, Nidich S, et al. A controlled trial of effects of stress reduction on left ventricular mass in hypertensive African Americans. Ethnicity and Disease. 2004;14(Autumn):S2-54.
19.         Schneider RH, Grim CE, Rainforth MA, Kotchen TA, Nidich SI, Gaylord-King C, et al. Stress reduction in the secondary prevention of cardiovascular disease: Randomized controlled trial of Transcendental Meditation and health education in Blacks. Circulation: Cardiovascular Quality and Outcomes. 2012;2(5).
20.         So KT, Orme-Johnson DW. Three randomized experiments on the holistic longitudinal effects of the Transcendental Meditation technique on cognition. Intelligence. 2001;29(5):419-40.
21.         Barnes VA, Bauza LB, Treiber FA. Impact of stress reduction on negative school behavior in adolescents. Health Qual Life Outcomes. 2003 Apr 23;1(1):10.
22.         Barnes VA, Treiber FA, Davis H. Impact of Transcendental Meditation on cardiovascular function at rest and during acute stress in adolescents with high normal blood pressure. Journal of Psychosomatic Research. 2001;51(4):597-605.
23.         Alexander CN. Transcendental Meditation. In: Corsini RJ, editor. Encyclopedia of Psychology. 2nd ed. New York: Wiley Interscience; 1994. p. 545-6.
24.         Travis FT. Transcendental Meditation technique. In: Craighead WE, Nemeroff CB, editors. The Corsini Encyclopedia of Psychology and Behavioral Science, 3rd edition. New York: John Wiley & Sons; 2001. p. 705-1706.
25.         Travis FT, Shear J. Focused attention, open monitoring and automatic self-transcending: Categories to organize meditations from Vedic, Buddhist and Chinese traditions. Consciousness and Cognition. 2010;19(4):1110-8.
26.         Smith JC. Psychotherapeutic effects of transcendental meditation with controls for expectation of relief and daily sitting. J Consult Clin Psychol. 1976 Aug;44(4):630-7.
27.         Dillbeck MC, Orme-Johnson DW. Physiological differences between Transcendental Meditation and rest. American Psychologist. 1987;42:879–81.
28.         Walton KG, Schneider RH, Nidich SI, Salerno JW, Nordstrom CK, Bairey-Merz CN. Psychosocial stress and cardiovascular disease 2: Effectiveness of the Transcendental Meditation technique in treatment and prevention. Behavioral Medicine. 2002;28(3):106-23.
29.         Orme-Johnson DW. Autonomic stability and Transcendental Meditation. Psychosomatic Medicine. 1973;35:341-9.
30.         Travis FT, Haaga D, Hagelin JS, Tanner M, Nidich SI, King CG, et al. Effects of Transcendental Meditation practice on brain functioning and stress reactivity in college students. International Journal of Psychophysiology. 2009;71(2):170-6.
31.         Kolzet JA, Inra M. Anxiety. In: Allan R, Fisher J, editors. Heart and Mind: The Practice of Cardiac Psychology. 2nd ed. Washington, DC: American Psychological Association; 2012.
32.         Kubzansky LD, Cole SR, Kawachi I, Vokonas P, Sparrow D. Shared and unique contributions of anger, anxiety, and depression to coronary heart disease: A prospective study in the Normative Aging Study. Annals of Behavioral Medicine. 2006;31:21-9.
33.         Sedlmeier P, Eberth J, Schwarz M, Zimmermann D, Haarig F. The Psychological Effects of Meditation: A Meta-Analysis. Psychological Bulletin. 2012;Online First Publication, May 14(doi: 10.1037/a0028168).
34.         Eppley K, Abrams AI, Shear J. Differential effects of relaxation techniques on trait anxiety: A meta-analysis. Journal of Clinical Psychology. 1989;45(6):957–74.