An Evidence Pyramid that Facilitates the Finding of Evidence

20 03 2010

Earlier I described that there are so many search- and EBM-pyramids that it is confusing. I described  3 categories of pyramids:

  1. Search Pyramids
  2. Pyramids of EBM-sources
  3. Pyramids of EBM-levels (levels of evidence)

In my courses where I train doctors and medical students how to find evidence quickly, I use a pyramid that is a mixture of 1. and 2. This is a slide from a 2007 course.

This pyramid consists of 4 layers (from top down):

  1. EBM-(evidence based) guidelines.
  2. Synopses & Syntheses*: a synopsis is a summary and critical appraisal of one article, whereas synthesis is a summary and critical appraisal of a topic (which may answer several questions and may cover many articles).
  3. Systematic Reviews (a systematic summary and critical appraisal of original studies) which may or may not include a meta-analysis.
  4. Original Studies.

The upper 3 layers represent “Aggregate Evidence”. This is evidence from secondary sources, that search, summarize and critically appraise original studies (lowest layer of the pyramid).

The layers do not necessarily represent the levels of evidence and should not be confused with Pyramids of EBM-levels (type 3). An Evidence Based guideline can have a lower level of evidence than a good systematic review, for instance.
The present pyramid is only meant to lead the way in the labyrinth of sources. Thus, to speed up to process of searching. The relevance and the quality of evidence should always be checked.

The idea is:

  • The higher the level in the pyramid the less publications it contains (the narrower it becomes)
  • Each level summarizes and critically appraises the underlying levels.

I advice people to try to find aggregate evidence first, thus to drill down (hence the drill in the Figure).

The advantage: faster results, lower number to read (NNR).

During the first courses I gave, I just made a pyramid in Word with the links to the main sources.

Our library ICT department converted it into a HTML document with clickable links.

However, although the pyramid looked quite complex, not all main evidence sources were included. Plus some sources belong to different layers. The Trip Database for instance searches sources from all layers.

Our ICT-department came up with a much better looking and better functioning 3-D pyramid, with databases like TRIP in the sidebar.

Moving the  mouse over a pyramid layer invokes a pop-up with links to the databases belonging to that layer.

Furthermore the sources included in the pyramid differ per specialty. So for the department Gynecology we include POPLINE and MIDIRS in the lowest layer, and the RCOG and NVOG (Dutch) guidelines in the EBM-guidelines layer.

Together my colleagues and I decide whether a source is evidence based (we don’t include UpToDate for instance) and where it  belongs. Each clinical librarian (we all serve different departments) then decides which databases to include. Clients can give suggestions.

Below is a short You Tube video showing how this pyramid can be used. Because of the rather poor quality, the video is best to be viewed in full screen mode.
I have no audio (yet), so in short this is what you see:

Made with Screenr:  http://screenr.com/8kg

The pyramid is highly appreciated by our clients and students.

But it is just a start. My dream is to visualize the entire pathway from question to PICO, checklists, FAQs and database of results per type of question/reason for searching (fast question, background question, CAT etc.).

I’m just waiting for someone to fulfill the technical part of this dream.

————–

*Note that there may be different definitions as well. The top layers in the 5S pyramid of Bryan Hayes are defined as follows: syntheses & synopses (succinct descriptions of selected individual studies or systematic reviews, such as those found in the evidence-based journals), summaries, which integrate best available evidence from the lower layers to develop practice guidelines based on a full range of evidence (e.g. Clinical Evidence, National Guidelines Clearinghouse), and at the peak of the model, systems, in which the individual patient’s characteristics are automatically linked to the current best evidence that matches the patient’s specific circumstances and the clinician is provided with key aspects of management (e.g., computerised decision support systems).

Begin with the richest source of aggregate (pre-filtered) evidence and decline in order to to decrease the number needed to read: there are less EBM guidelines than there are Systematic Reviews and (certainly) individual papers.
Advertisements




Time to weed the (EBM-)pyramids?!

26 09 2008

Information overload is a major barrier in finding that particular medical information you’re really looking for. Search- and EBM-pyramids are designed as a (search) guidance both for physicians, med students and information specialists. Pyramids can be very handy to get a quick overview of which sources to use and which evidence to look for in which order.

But look at the small collection of pyramids I retrieved from Internet plus the ones I made myself (8,9)………

ALL DIFFERENT!!!!

What may be particularly confusing is that these pyramids serve different goals. As pyramids look alike (they are all pyramids) this may not be directly obvious.

There are 3 main kinds of pyramids (or hierarchies):

  1. Search Pyramid (no true example, 4, 5 and 6 come closest)
    Guiding searches to answer a clinical question as promptly as possible. Begin with the easiest/richest source, for instance UpToDate, Harrison’s (books), local hospital protocols or useful websites. Search aggregate evidence respectively the best original studies if answer isn’t found or doubtful.
  2. Pyramid of EBM-sources (3 ,4, 8 )
    Begin with the richest source of aggregate (pre-filtered) evidence and decline in order to to decrease the number needed to read: there are less EBM guidelines than there are Systematic Reviews and (certainly) individual papers.
  3. Pyramid of EBM-levels (1, 2, 5, 7, 9)
    Begin to look for the original papers with the highest level of evidence.
    Often only individual papers/original research, including Systematic Reviews, are considered (1, 9), but sometimes the pyramid is a mixture of original and aggregated literature (2,5)
  4. A mixture of 2, 3 and/or 4 (2,5)

Further discrepancies:

  • Hierarchies.
    • Some place Cochrane Systematic Reviews higher than ‘other systematic reviews’, others place meta-analysis above Systematic reviews (2,6). This is respectively unnecessary or wrong. (Come back to that in another post).
    • Sometimes Systematic reviews are on top, sometimes Systems (never found out what that is), sometimes meta-analysis or Evidence based Guidelines
    • Synopses (critically appraised individual articles) may be placed above or below Syntheses (critically appraised topics).
    • Textbooks and Reviews may at the base of the pyramid or a little more up.
    • etcetera
  • Nomenclature
    • Evidence Summaries ?= Summaries of the evidence? = Evidence Syntheses? = critically appraised topics?
    • Etcetera
  • Categorization
    • UpToDate is sometimes placed at the top of the pyramid in Summaries (4) OR at the base in Textbooks (5), where I think it should belong in terms of evidence levels, but not in terms of usefulness.
    • DARE is considered a review, but it is really a synopsis (critical appraised summary) of a Systematic Review.

Isn’t it about time to weed the pyramids rigorously?

Are pyramids really serving the aim of making it easier for the meds to find their information?

Like to hear your thoughts about this.

What my thoughts are? I will give a hint: I would rather guide the informationseeker through different routes, dependent on his background, question, available time and goal. The pyramid of evidence sources and the levels of evidence would just be part of that scheme, ideally.

Will be continued….