Editor's Note: In addition to this interview, you can find several resources regarding this study at www.acupuncturetoday.com, including a PDF of the Australian study published in JAMA, an article in response to the flaws seen in this study, the Letter to the Editor of JAMA referenced in this interview and a complaint letter from the Traditional Chinese Medicine American Alumni Association (TCMAAA).
.
A recent Australian clinical trial, published in the Journal of the American Medical Association (JAMA) in 2014 by Rana Hinman, et el., evaluating the effectiveness of both needle and laser acupuncture for chronic knee pain, caught the attention of Dr.'s Hongjian He, AP, MD, PhD; Lixing Lao, PhD, MB; Wing-Fai Yeung, BCM, PhD; and Yong Ming Li, MD, PhD. They were astonished to read that the conclusions of the study stated, "In patients older than 50 years with moderate or severe chronic knee pain, neither laser nor needle acupuncture conferred benefit over sham for pain or function. Our findings do not support acupuncture for these patients." Obviously, that statement didn't reflect their combined clinical experience. Upon further investigation, they felt there were serious design flaws in the study and wrote letters to the editor of JAMA expressing their concerns that were published in February 2015 along with the Australian researchers' responses.
AT: Gentlemen – Can you please introduce yourselves one at a time and provide our readers with your backgrounds?
Dr. Lixing Lao: I have trained in acupuncture and traditional Chinese medicine (TCM) at the Shanghai University of Traditional Chinese Medicine, Shanghai, China. I obtained my PhD in physiology in the University of Maryland in 1992, and had been appointed as an assistant professor, associate professor and professor at the University of Maryland, School of Medicine during my 21-year tenure from 1992 to 2013. I have been conducting acupuncture/TCM research in the U.S. until 2013, when I was appointed as the director of the University of Hong Kong School of Chinese Medicine. I have participated, conducted and designed a number of clinical trials on acupuncture including the trial of acupuncture for knee osteoarthritis published in 2004 (Berman, Lao, et al., Annals of Internal Medicine, 2004).
Dr. Hongjian He: I graduated from Guangzhou Medical College of Traditional Chinese Medicine in 1983 with a degree in Western and Traditional Chinese Medicine. I then completed three years of residency training in Internal Medicine at the China-Japan Friendship Hospital located in Beijing. I moved to Clearwater, Fla. in 1988, and have been developing my passion and dedication for practicing Chinese Medicine ever since.
I frequently travel nationally and internationally to study, teach, and exchange knowledge with the best and most respected experts in the field. In addition to my Conventional and Chinese Medical training in China, I have also earned a PhD in Oriental Medicine. I received the Acupuncturist of the Year award from the American Association of Acupuncture and Oriental Medicine in 1999. I have also been elected for six terms as the President of the Florida Acupuncture Association.
Dr. Arthur Yin Fan: I graduated from the Nanjing University of Chinese Medicine in 1986, and was awarded a PhD in Internal medicine in 1998. I was an NIH fellow in Chinese medicine in 2001 and performed my postdoc at Georgetown University and the University of Maryland. I have been involved in studies in acupuncture and Chinese herbology safety, efficacy and their mechanism in China and the U.S. I was an internist and neurologist in a large university hospital for 15 years before I came to the U.S. Since 2005, I've worked full-time as a licensed acupuncturist and certified oriental medicine practitioner in Vienna, Virginia. I am also a reviewer for government grants and several academic journals. Currently, I serve as a coordinator of the Traditional Chinese Medicine American Alumni Association (TCMAAA) in handling Hinman JAMA Article response.
Dr. Yong Ming Li: I am a board-certified pathologist, dermatopathologist, and acupuncturist currently practicing in New Jersey. I had a BM from Liaoning College of TCM, China and PhD from the University of Illinois at Urbana-Champaign. After several years of medical research, I completed residency at North Shore University Hospital and fellowship at Ackerman Academy-Columbia University. I was the past President of American Traditional Chinese Society. For more than 10 years, I served as Study Section Member to review grants for clinical study at NCCAM-NIH and received the NIH Merit Award in 2011.
AT: Can one of you provide an overview about the study?
Dr. Fan: In the October 2014 publication of JAMA, Dr. Hinman and colleagues in physiotherapy Medicine, University of Melbourne, published an acupuncture clinical study entitled "Acupuncture for Chronic Knee Pain: A Randomized Clinical Trial" and concluded that, "in patients older than 50 years with moderate or severe chronic knee pain, neither laser nor needle acupuncture conferred benefit over sham for pain or function. Our findings do not support acupuncture for these patients."
In the study, patients were separated into four groups: control (71 patients), acupuncture (70 patients), laser acupuncture (71 patients), and sham acupuncture (70 patients). The 282 community volunteers (aged>50 years with chronic knee pain) were treated by family physician acupuncturists, where both participants and acupuncturists were blinded to laser and sham laser acupuncture. Control participants were unaware of the trial. Zelen-design was applied to this clinical trial (randomization occurred before informed consent), in Victoria, Australia (February 2010-December 2012). Dr. Hinman et al., claimed that their objective was to determine the efficacy of laser and needle acupuncture for chronic knee pain. In main outcomes and measures, she wrote: "the primary outcomes were average knee pain (numeric rating scale, 0 [no pain] to 10 [worst pain possible]; minimal clinically important difference [MCID], was identified as 1.8 units) and physical function (Western Ontario and McMaster Universities Osteoarthritis Index, 0 [no difficulty] to 68 [extreme difficulty]; MCID, 6 units) at 12 weeks. Secondary outcomes included other pain and function measures, quality of life, global change, and 1-year follow-up. Analyses were by intention-to-treat using multiple imputation for missing outcome data. The results showed that at "12 weeks and 1 year, 26 (9%) and 50 (18%) participants were lost to follow-up, respectively. Analyses showed neither needle nor laser acupuncture significantly improved pain (mean difference; –0.4 units; 95%CI, –1.2 to 0.4, and –0.1; 95%CI, –0.9 to 0.7, respectively) or function (–1.7; 95%CI, –6.1 to 2.6, and 0.5; 95%CI, –3.4 to 4.4, respectively) compared with sham at 12 weeks. Compared with control, needle and laser acupuncture resulted in modest improvements in pain (–1.1; 95%CI, –1.8 to –0.4, and –0.8; 95%CI, –1.5 to –0.1, respectively) at 12 weeks, but not at 1 year. Needle acupuncture resulted in modest improvement in function compared with control at 12 weeks (–3.9; 95%CI,–7.7 to –0.2), but was not significantly different from sham (–1.7; 95%CI, –6.1 to 2.6) and was not maintained at 1 year. There were no differences for most secondary outcomes and no serious adverse events."
Notice that the author used acupuncture to refer to both acupuncture and laser acupuncture in the conclusion and the title, which causes readers' great confusion; in one clinical trial with only 282 patients, four groups, to test two factors, acupuncture and laser acupuncture. However, laser acupuncture obviously differs from the (needle) acupuncture, and should not be mixed in with the acupuncture.
AT: What are the key issues related to this acupuncture trial?
Dr. Li: First, I have no problem to accept a well-designed clinical trial showing a negative result on acupuncture treatment in a specific clinical setting. The problem is the quality of the study and the integrity of the researchers.
After reading the reply from Dr. Hinman and other authors to my letter, my concerns regarding the validation and accuracy of this trial still remain, which leads me to further question the authors' academic integrity in this matter.
The key issue is that this clinical trial was originally designed to test laser acupuncture and use needle acupuncture for a positive control, as clearly stated in three of their own prior publications. Therefore, the power of the test and other resources were only optimized to evaluate one testing treatment in the trial. After collecting undesirable data showing that the positive control did not work, the authors amended their original aims and stated in their final report that they are going to evaluate two treatments in one trial, namely, "laser acupuncture" and needle acupuncture. There is no evidence to suggest that the original design and objective was to test two unknowns.
In the authors reply, they said, "He (Dr. Li) is correct that we did not explicitly state that we would compare needles with sham laser" and the further explanations by the authors are non-comprehensive. The authors' "[apologies] for any confusion" is insufficient and not acceptable as the medical community and public readers may act upon these misleading findings. For these reasons, I do not think Dr. Hinman's report should be considered as negative evidence in the future evaluation of acupuncture therapy for knee pain.
AT: Will you tell us what some of the weaknesses of the study are and the implications to the outcome?
Dr. Fan: There is a fatal flaw in the primary testing factor in this RCT: the laser acupuncture should be the primary testing factor, instead of the needle acupuncture.
Hinman's trial was a failed clinical trial for laser acupuncture. It is unethical to publish a "professional" paper, with a group of almost scrapped data and confusing logic that misleads the readers, including the general public, physicians and policy makers, as well as fellow researchers. Hinman appears to have the intention to mislead the editors and readers. The evidences we can see are from Hinman's article itself, from Hinman's former article (Supported by Dr. Lixing Lao's letter to the editor JAMA), from Australia New Zealand Clinical Trials Registry: "Acupuncture for chronic knee pain trial" [registry identifier: ACTRN12609001001280. 2015-02-22. Supported by Dr. Yongming Li's letter to editor JAMA), and Hinman and other major authors are all PhDs, and should understand the importance to follow the ethics in a scientific study. However, they violated the ethics.
And, from a statistical perspective, if acupuncture has better results along with a significant difference over the primary control (no-treatment group), p<0.05, we can conclude that "acupuncture is effective"- no matter what the results are from the comparison to the secondary control, such as "sham laser acupuncture," but Hinman intentionally does not report this effectiveness in her conclusion; if acupuncture has better results over "laser acupuncture" and "sham laser acupuncture," without significant in statistics, p>0.05, we can conclude that "acupuncture is better than the laser acupuncture, and sham laser acupuncture, but may require more studies to confirm". We can't conclude that "acupuncture is not effective" because that there are no significant differences in statistics between acupuncture and "laser acupuncture," or between acupuncture and "sham acupuncture" does not mean there is no difference between these treatments clinically. Hinman et al. misinterprets the results and violates the basic principle of statistics.
AT: Are there large-scale clinical trials of acupuncture for knee pain that conflict with the Australian study?
Dr. Lao: Yes, there are a number of clinical trials whose results conflict with this trial: Our study published in 2014 (Berman, Lao, et al. 2004) on acupuncture for knee OA shows that in a large sample sized trial, (570 patients), acupuncture was significantly superior to a sham control in reducing pain and increasing function of the knee. Furthermore, a meta-analysis published in one of the JAMA's archives, Archive of Internal Medicine by Vickers et al., 2012 demonstrated that in a summary of 29 clinical trials with nearly 18,000 patients, acupuncture treatment is superior to the conventional usual care and to sham acupuncture for four types of pain: osteoarthritis, back and neck pain, headache and shoulder pain.
AT: How were non-steroidal anti-inflammatory drugs (NSAIDs) applied to the patients and their potential effect on the study outcome?
Dr. Lao: A review paper on NSAIDs published by Zhang, et al., in the Journal of Osteoarthritis and Cartilage in 2010, 18:476-499, shows that the overall effectiveness of NSAIDs is much smaller than that of acupuncture, meaning acupuncture is more effective than this class of anti-inflammatory drugs.
Dr. Fan: However, there was bias in use of NSAID medications among groups. The author did not report in detail the usage methods and dosage of such medications in this acupuncture clinical trial. This bias may affect the evaluation of acupuncture effectiveness.
AT: What did the Australian researchers define as "clinically significant," and do you feel it was a statistically accurate assessment?
Dr. Fan: The author ignored the evidence that the acupuncture group did improve patients' overall pain score, functional improvement (both p<0.05), and stated that there was no clinical significance because this improvement was moderate at week 12; and there was no significant improvement at one year. If we assume their statistics were correct, we should point out that there were several flaws in effectiveness judgments:
First, the acupuncture intervention dosage was sub-optimal: patients in the acupuncture group received only 8-12 treatments (20 minutes per time), during 12 weeks, which means zero to one treatment per week, the frequency was much less than being treated by an acupuncturist in a real practice and reported literature; there were no detailed description of needling depth, de-qi, and did not employ electric acupuncture.
Second, a lack of basic information about patients using the pain and anti-inflammatory medications, if in control groups patients received more pain and anti-inflammatory medications, will cause a large bias and make the acupuncture group response appear less effective.
Third, the criteria of clinical significance used in this study was higher than that in many pain medication studies, which is unfair to an acupuncture study. And finally, the judgment for acupuncture effectiveness at one year is not acceptable and does not make any sense because chronic knee pain due to the knee OA is a degenerative process, using a suboptimal dose of acupuncture (just 8-12 sessions) will not play any significant role for delay such a degeneration.
However, there was a major methodological flaw in that study masking the acupuncture effectiveness about 31.27%. Hinman used Zelen design, and had a high drop-out rate and dilution effect. So, if we consider the dilution rate, even the acupuncture group had a sub-optimal dose, acupuncture effectiveness should increase by 31.27%.
AT: Do you feel that the laser parameters (frequency/wavelength, power output, acupoint exposure time) were clinically relevant to effectively treat chronic knee pain?
Dr. Lao: The intensity of the laser parameters in this trial is far below the required dosage reported by others on knee OA, e.g. Zhao et al., Photomed Laser Surgery, 2010 June.
AT: Was the general health and Body Mass Index (BMI) of the patients participating in this trial taken into account?
Dr. Fan: The article did not describe details of the general health and body mass index of the participants. However, according to Chinese medicine, the energy level is very important related to pain and inflammation recovery. If there were significant differences among groups in terms of general health and BMI, it is not comparable. Additionally, obese patients tend to have poor prognoses compared to slimmer patients, because the heavier patients may have more knee pain, due to more pressure on their knee, and larger BMI may cause alignment changes of the tibia and further exacerbate knee pain.
AT: What was the average experience level of the acupuncturists chosen to participate in this trial?
Dr. Fan: No licensed acupuncturist participated in that study, instead they used physician practitioners.
AT: Were the acupuncturists performing the treatments allowed to differentially diagnose the patient and choose their own acupoint "prescription?" Did the researchers define needle type and size, manipulation techniques (including deqi sensation) and angles of insertion/depth of penetration?
Dr. He: There are eight family physicians registered as acupuncturists to treat 57 patients in several different clinics. Some physicians only treated one patient and some treated up to 12 patients.
The acupuncture treatment design appears flawed. Specifically, the acupuncture points were non-standardized (over 32 points to be chosen. On the initial visit they choose six points out of more than 32 points. In subsequent treatments, additional points were allowed to be added, but no specific points were indicated) and the study lacked the details necessary to ascertain whether the provided interventions were representative of acupuncture sessions appropriate for chronic knee pain.
First, the acupuncture regimen was not consistent in the study. Dr. Hinman and colleagues failed to report how many patients received 1 or 2 treatments per week. The commonly used frequency of acupuncture treatments for chronic knee pain due to osteoarthritis is 2 treatments per week for 8 weeks, followed by 2 weeks of 1 treatment per week, then 4 weeks of 1 treatment every other week, and finally 12 weeks of 1 treatment per month.
Furthermore, no details were provided regarding depth of insertion or whether subjective deqi sensation was experienced by the patient. Hinman and colleagues also did not provide acupuncture with electrical stimulation, which not only has a dose-dependent effect on the degree of analgesia, but also induces differential neurotransmitter responses depending on the electrical frequency applied.
AT: Do you believe that the frequency and duration of the treatments were clinically relevant?
Dr. Fan: The frequency and duration of the treatments are very important, i.e. dose (either acupuncture or laser acupuncture) is one of the fundamental keys for clinical effectiveness.
AT: How do you feel about the accuracy of the results of this clinical trial? Do you believe your concerns were adequately addressed by the authors' response?
Dr. He: The results of this clinical trial are not accurate, valid or acceptable, and the author did not adequately address my concerns.
Here's how Dr. Hinman and colleagues responded: "Dr. He suggests lack of acupuncture standardization, treatment infrequency, and no electrical stimulation may explain our findings. However, when comparing acupuncture with sham treatment, a meta-analysis found no evidence that needle number or placement; use of electrical stimulation; or number, frequency, or duration of treatments influence acupuncture outcomes."
We read the article Dr. Hinman cited (MacPherson et al., PLoS 2013) and they obviously did not read it very carefully. The results section states: "This analysis showed that trials allowing electrical stimulation had a significantly stronger effect of acupuncture compared to sham controls (β= 0.27; 95% CI 0.03, 0.51; p=0.027) and those with a longer average treatment session duration had a smaller effect compared to sham controls (β=-0.14 per 5 minutes; 95% CI -0.22, -0.06; p=0.001)." In addition, "...acupuncture effects increased in comparison to non-acupuncture controls when more needles were used (increase in effect size per 5 needles of 0.33; 95% CI 0.08, 0.58; p=0.010)."
So, according to the article that Dr. Hinman herself cited, electrical stimulation, treatment duration, and acupuncture needle number may certainly influence clinical outcome, and it was an oversight by Dr. Hinman and colleagues to not report such specifics.
AT: Do you have any closing remarks for our readers?
Dr. He: The only conclusion that can be drawn from this study is that unsystematic acupuncture regimens did not result in significant clinical benefit to patients with chronic knee pain. Many well designed studies, including one referred to by Dr. Hinman, (MacPherson et al., PLoS 2013) and Dr. Lao referred to show statistically significant decreases in pain when comparing acupuncture to sham acupuncture.
Dr. Fan: Based on the evidence we find inside and outside of this article, I believe that Dr. Hinmen and her colleagues deliberately adjusted the study objectives, published for conflicting interests, and intentionally made errors to create a negative result regarding acupuncture for knee pain; and may have the intention to harm the acupuncture profession. Our practitioners and scholars should be more alert on this issue and more involved in countering such an intention.
Click here for more information about Bill Reddy, LAc, Dipl. Ac..