#CECEM Bridging the Gap between Evidence Based Practice and Practice Based Evidence

15 06 2009

cochrane-symbol A very interesting presentation at the CECEM was given by the organizer of this continental Cochrane meeting, Rob de Bie. De Bie is Professor of Physiotherapy Research and director of Education of the Faculty of Health within the dept. of Epidemiology of the Maastricht University. He is both a certified physiotherapist and an epidemiologist. Luckily he kept the epidemiologic theory to a minimum. In fact he is a very engaging speaker who keeps your attention to the end.

Guidelines

While guidelines were already present in the Middle Ages in the form of formalized treatment of daily practice, more recently clinical guidelines have emerged. These are systematically developed statements which assists clinicians and patients in making decisions about appropriate treatement for specific conditions.

Currently, there are 3 kinds of guidelines, each with its own shortcomings.

  • Consensus based. Consensus may be largely influenced by group dynamics
    Consensus = non-sensus and Consensus guidelines are guidelies.
  • Expert based. Might be even worse than consensus. It can have all kind of biases, like expert and opinion bias or external financing.
  • Evidence based. Guideline recommendations are based on best available evidence, deals with specific interventions for specific populations and are based on a systematic approach.

The quality of Evidence Based Guidelines depends on whether the evidence is good enough, transparent, credible, available, applied and not ‘muddled’ by health care insurers.
It is good to realize that some trials are never done, for instance because of ethical considerations. It is also true that only part of what you read (in the conclusions) has actually be done and some trials are republished several times, each time with a better outcome…

Systematic reviews and qualitatively good trials that don’t give answers.

Next Rob showed us the results of a study ( Jadad and McQuay in J. Clin. Epidemiol. ,1996) with efficacy as stated in the review plotted on the X-axis and the Quality score on the Y-axis. Surprisingly meta-analysis of high quality were less likely to produce positive results. Similar results were also obtained by Suttorp et al in 2006. (see Figure below)

12066264  rob de bie CECEM

Photo made by Chris Mavergames

There may be several reasons why good trials not always give good answers. Well known reasons are  the lack of randomization or blinding. However Rob focused on a less obvious reason. Despite its high level of evidence, a Randomized Controlled Trial (RCT) may not always be suitable to provide good answers applicable to all patients, because RCT’s often fail to reflect the true clinical practice. Often, the inclusion of patients in RCT’s is selective: middle-aged men with exclusion of co-morbidity. Whereas co-morbidity occurs in > 20% of the people of 60 years and older and in >40% of the people of 80 years and older (André Knottnerus in his speech).

Usefulness of a Nested Trial Cohort Study coupled to an EHR to study interventions.

Next, Rob showed that a nested Trial cohort study can be useful to study the effectiveness of  interventions. He used this in conjunction with an EHR (electronic health record), which could be accessed by practitioner and patient.

One of the diseases studied in this way, was Intermittent Claudication. Most commonly Intermittent Claudication is a manifestation of  peripheral arterial disease in the legs, causing pain and cramps in the legs while walking (hence the name). The mortality is high: the 5 year mortality rates are in between those of colorectal cancer and Non-Hodgkin Lymphoma. This is related to the underlying atherosclerosis.

There are several risk factors, some of which cannot be modified, like hereditary factors, age and gender. Other factors, like smoking, diet, physical inactivity and obesity can be tackled. These factors are interrelated.

Rob showed that, whereas there may be an overall null effect of exercise in the whole population, the effect may differ per subgroup.

15-6-2009 3-06-19 CI 1

  • Patients with mild disease and no co-morbidity may directly benefit from exercise-therapy (blue area).
  • Exercise has no effect on smokers, probably because smoking is the main causative factor.
  • People with unstable diabetes first show an improvement, which stabilized after a few weeks due to hypo- or hyperglycaemia induced by the exercise,
  • A similar effect is seen in COPD patients, the exercise becoming less effective because the patients become short of breath.

It is important to first regulate diabetes or COPD before continuing the exercise therapy. By individually optimizing the intervention(s) a far greater overall effect is achieved: 191% improval in the maximal (pain-free) walking distance compared to for instance <35% according to a Cochrane Systematic Review (2007).

Another striking effect: exercise therapy affects some of the prognostic factors: whereas there is no effect on BMI (this stays an important risk factor), age and diabetes become less important risk factors.

15-6-2009 3-35-10 shift in prognostic factors

Because guidelines are quickly outdated, the findings are directly implemented in the existing guidelines.

Another astonishing fact: the physiotherapists pay for the system, not the patient nor the government.

More information can be found on https://www.cebp.nl/. Although the presentation is not (yet?) available on the net, I found a comparable presentation here.

** (2009-06-15) Good news: the program and all presentations can now be viewed at: https://www.cebp.nl/?NODE=239

Advertisements




#CECEM David Tovey -the Cochrane Library’s First Editor in Chief

13 06 2009

cochrane-symbolThis week I was attending another congress, the Continental European Cochrane Entities Meeting (CECEM).

This annual meeting is meant for staff from Cochrane Entities, thus Centre Staff, RGC’s (Review Group Coordinators), TSC’s (Trial Search Coordinators) and other staff members of the Cochrane Collaboration based in Continental Europe.

CECEM 2009 was held in Maastricht, the beautiful old Roman city in the South of the Netherlands. The city where my father was born and where I spend many holidays.

One interesting presentation was by Cochranes’ 1st Editor in chief, David Tovey, previously GP in an urban practice in London for 14 years and  Editorial Director of the BMJ Group’s ‘Knowledge’ division (responsible for BMJ Clinical Evidence and its sister product Best Treatments, see announcement in Medical News Today)

David began with saying that the end user is really the key person and that the impact of the Cochrane Reviews is most important.

“How is it that a Senior health manager in the UK may shrug his shoulders when you ask him if he has ever heard of Cochrane?”

“How do we make sure that our work had impact? Should we make use of user generated content?”

Quality is central, but quality depends on four pillars. Cochrane reviews should be reliable, timely, relevant and accessible.

Cochrane Tovey wit

How quality is perceived is dependent on the end users. There are several kinds of end users, each with his own priorities.

  1. doctor: wants comprehensive and up-to-date info, wants to understand and get answers quickly.
  2. patient: trustworthiness, up-to-date, wants to be able to make sense of it.
  3. scientist: wants to see how the conclusions are derived.
  4. policy and guideline-makers.

Reliable: Several articles have shown Cochrane Systematic Reviews to be more reliable then other systematic reviews  (Moher, PLOS BMJ)*

Timely: First it takes time to submit a title of a Cochrane Review and then it takes at least 2 years before a protocol becomes a review. Some reviews take even longer than 2 years. So there is room for improvement.

Patients are also very important as end user. Strikingly, the systematic review about the use of cranberry to prevent recurrent urinary tract infection is the most frequently viewed article,- and this is not because the doctors are most interested in this particular treatment….

Doctors: Doctors often rely on their colleagues for a quick and trustworthy answer. Challenge: “can we make consulting the Cochrane Library as easy as asking a colleague: thus timely and easy?”

Solutions?

  • making plain language summaries more understandable
  • Summary of Findings
  • podcasts of systematic reviews (very successful till now), .e. see an earlier post.
  • Web 2.0 innovations

Key challenges:

  • ensure and develop consistent quality
  • (timely) updating
  • putting the customer first: applicability & prioritization
  • web delivery
  • resources (not every group has the same resources)
  • make clear what an update means and how important this update is: are there new studies found? are these likely to change conclusions or not? When was the last amendment to the search?

I found the presentation very interesting. What I also liked is that David stayed with us for two days -also during the social program- and was easy approachable. I support the idea of a user-centric approach very much. However, I had expected the emphasis to be less on the timeliness (of updates for instance), but more on how users (patients, doctors) can get more involved and how we review the subjects that are most urgently needed. Indeed, when I twittered that Tovey suggested that we “make consulting the Cochrane Library as easy as asking a colleague”, Jon Brassey of TRIP answered that a lot has to be done to fulfill this, as the Cochrane only answers 2 out of 350+ questions asked by GPs in the UK, a statement that appeared to be based on his own experience (Jon is founder of the TRIP-database).

But in principle I think that Jon is correct. Right now too few questions (in the field of interventions) are directly answered by Cochrane Systematic Reviews and too little is done to reach and involve the Cochrane Library users.

13-6-2009 15-43-17 twitter CECEM discussion

click to enlarge

During the CECEM other speakers addressed some of these issues in more detail. André Knottnerus, Chair of the Dutch Health Council, discussed “the impact of Cochrane Reviews”, and Rob the Bie of the Rehabilitation & Related Therapies field discussed “Bridging the  gap between evidenced based practice and practice based evidence”, while Dave Brooker launched ideas about how to implement Web 2.0 tools. I hope to summarize these (and other) presentations in a blogpost later on.

*have to look this up

NOTE (2009-11-10).

I had forgotten about this blank “citation” till this post was cited quite in another context (see comment: http://e-patients.net/archives/2009/11/tell-the-fda-the-whole-story-please.html) and someone commented that the asterisk to the “the amazing statement” had still to be looked up,  indirectly arguing that this statement thus was not reliable- and continuing by giving an example of a typically flawed Cochrane Review that hit the headlines 4 years ago, a typical exception to the rule that “Cochrane systematic reviews are more reliable than other systematic reviews”. Of course when it is said that A is more trustworthy than B it is meant on average. I’m a searcher, and on average the Cochrane searchers are excellent, but when I do my best I surely can find some that are not good at all. Without doubt that also pertains to other parts of Cochrane Systematic Reviews.
In addition -and that was the topic of the presentation- there is room for improvement.

Now about the asterisk, which according to Susannah should have been (YIKES!) 100 times bigger. This was a post based on a live presentation and I couldn’t pick up all the references on the slides while making notes. I had hoped that David Tovey would have made his ppt public, so I could have checked the references he gave. But he didn’t and so I forgot about it. Now I’ve looked some references up, and, although they might not be identical to the references that David mentioned, they are in line with what he said:

  1. Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG, 2007. Epidemiology and Reporting Characteristics of Systematic Reviews. PLoS Med 4(3): e78. doi:10.1371/journal.pmed.0040078 (free full text)
  2. The PLoS Medicine Editors 2007 Many Reviews Are Systematic but Some Are More Transparent and Completely Reported than Others. PLoS Med 4(3): e147. doi:10.1371/journal.pmed.0040147 (free full text; editorial coment on [1]
  3. Tricco AC, Tetzlaff J, Pham B, Brehaut J, Moher D, 2009. Non-Cochrane vs. Cochrane reviews were twice as likely to have positive conclusion statements: cross-sectional study. J Clin Epidemiol. Apr;62(4):380-386.e1. Epub 2009 Jan 6. [PubMed -citation]
  4. Anders W Jørgensen, Jørgen Hilden, Peter C Gøtzsche, 2006. Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review BMJ  2006;333:782, doi: 10.1136/bmj.38973.444699.0B (free full text)
  5. Alejandro R Jadad, Michael Moher, George P Browman, Lynda Booker, Christopher Sigouin, Mario Fuentes, Robert Stevens (2000) Systematic reviews and meta-analyses on treatment of asthma: critical evaluation BMJ 2000;320:537-540, doi: 10.1136/bmj.320.7234.537 (free full text)

In previous posts I regularly discussed that (Merck’s Ghostwriters, Haunted Papers and Fake Elsevier Journals and One Third of the Clinical Cancer Studies Report Conflict of Interest) that pharma-sponsored trials rarely produce results that are unfavorable to the companies’ products [e.g. see here for an overview, and many papers of Lisa Bero].

Also pertinent to the abovementioned discussion at E-patient-Net is my earlier post: The Trouble with Wikipedia as a Source for Medical Information. (references still not in the correct order. Yikes!)

Reblog this post [with Zemanta]