Problems with Disappearing Set Numbers in PubMed’s Clinical Queries

18 10 2010

In some upcoming posts I will address various problems related to the changing interfaces of bibliographic databases.

We, librarians and end users, are overwhelmed by a flood of so-called upgrades, which often fail to bring the improvements that were promised….. or which go hand-in-hand with temporary glitches.

Christina of Christina’s Lis Rant even made rundown of the new interfaces of last summer. Although she didn’t include OVID MEDLINE/EMBASE, the Cochrane Library and Reference manager in her list, the total number of changed interfaces reached 22 !

As a matter of fact, the Cochrane Library was suffering some outages yesterday, to repair some bugs. So I will postpone my coverage of the Cochrane bugs a little.

And OVID send out a notice last week: This week Ovid will be deploying a software release of the OvidSPplatform that will add new functionality and address improvements to some existing functionality.”

In this post I will confine myself to the PubMed Clinical Queries. According to Christina PubMed changes “were a bit ago”, but PubMed continuously tweaks  its interface, often without paying much attention to its effects.

Back in July, I already covered that the redesign of the PubMed Clinical Queries was no improvement for people who wanted to do more than a quick and dirty search.

It was no longer possible to enter a set number in the Clinical Queries search bar. Thus it wasn’t possible to set up a search in PubMed first and to then enter the final set number in the Clinical Queries. This bug was repaired promptly.

From then on, the set number could be entered again in the clinical queries.

However, one bug was replaced by another: next, search numbers were disappearing from the search history.

I will use the example I used before: I want to know if spironolactone reduces hirsutism in women with PCOS, and if it works better than cyproterone acetate.

Since little is published about this topic,  I only search for  hirsutism and spironolactone. These terms  map correctly with  MeSH terms. In the MeSH database I also see (under “see also”) that spironolactone belongs to the aldosterone antagonists, so I broaden spironolactone (#2) with “Aldosterone antagonists”[pharmacological Action] using “OR” (set #7). My last set (#8) consists of #1 (hirsutism) AND #7 (#2 OR #6)

Next I go to the Clinical Queries in the Advanced Search and enter #8. (now possible again).

I change the Therapy Filter from “broad”  to “narrow”, because the broad filter gives too much noise.

In the clinical queries you see only the first five results.

Apparently even the clinical queries are now designed to just take a quick look at the most recent results, but of course, that is NOT what we are trying to achieve when we search for (the best) evidence.

To see all results for the narrow therapy filter I have to go back to the Clinical Queries again and click on see all (27) [5]

A bit of a long way about. But it gets longer…


The 27 hits, that result from combining the Narrow therapy filter with my search #8 appears. This is set #9.
Note it is a lower number than set #11 (search + systematic review filter).

Meanwhile set #9 has disappeared from my history.

This is a nuisance if I want to use this set further or if I want to give an overview of my search, i.e. for a presentation.

There are several tricks by which this flaw can be overcome. But they are all cumbersome.

1. Just add set number (#11 in this case, which is the last search (#8) + 3 more) to the search history (you have to remember the search set number though).

This is the set number remembered by the system. As you see in the history, you “miss” certain sets. #3 to #5 are for instance are searches you performed in the MeSH-database, which show up in the History of the MeSH database, but not in PubMed’s history.

The Clinical query set number is still there, but it doesn’t show either. Apparently the 3 clinical query-subsets yield a separate set number, whether the search is truly performed or not. In this case  #11 for (#8) AND systematic[sb], #9 for (#8) AND (Therapy/Narrow[filter]). And #10 for (#8) AND the medical genetics filter.

In this way you have all results in your history. It isn’t immediately clear, however, what these sets represent.

2. Use the commands rather than going to the clinical queries.

Thus type in the search bar: #8 AND systematic[sb]

And then: #8 AND (Therapy/Narrow[filter])

It is easiest to keep all filters in Word/Notepad and copy/paste each time you need the filter

3. Add clinical queries as filters to your personal NCBI account so that the filters show up each time you do a search in PubMed. This post describes how to do it.

Anyway these remain just tricks to try to make something right that is wrong.

Furthermore it makes it more difficult to explain the usefulness of the clinical queries to doctors and medical students. Explaining option 3 takes too long in a short course, option 1 seems illogical and 2 is hard to remember.

Thus we want to keep the set numbers in the history, at least.

A while ago Dieuwke Brand notified the NLM of this problem.

Only recently she received an answer saying that:

we are aware of the continuing problem.  The problem remains on our programmers’ list of items to investigate.  Unfortunately, because this problem appears to be limited to very few users, it has been listed as a low priority.

Only after a second Dutch medical librarian confirmed the problem to the NLM, saying it not only affects one or two librarians, but all the students we teach (~1000-2000 students/university/yearly), they realized that it was a more widespread problem than Dieuwke Brand’s personal problem. Now the problem has a higher priority.

Where is the time that a problem was taken for what it was? As another librarian sighed: Apparently something is only a problem if many people complain about it.

Now I know this (I regarded Dieuwke as a delegate of all Dutch Clinical Librarians), I realize that I have to “complain” myself, each time I and/or my colleagues encounter a problem.

Related Articles





A Filter for Finding “All Studies on Animal Experimentation in PubMed”

29 09 2010

ResearchBlogging.orgFor  an introduction to search filters you can first read this post.

Most people searching PubMed try to get rid of publications about animals. But basic scientists and lab animal technicians just want to find those animal studies.

PubMed has built-in filters for that: the limits. There is a limit  for “humans” and a limit for “animals”. But that is not good enough to find each and every article about humans, respectively animals. The limits are MeSH, Medical Subject Headings or index-terms and these are per definition not added to new articles, that haven’t been indexed yet. To name the main disadvantage…
Thus to find all papers one should at least search for other relevant MeSH and textwords (words in title and abstract) too.

A recent paper published in Laboratory Animals describes a filter for finding “all studies on animal experimentation in PubMed“, to facilitate “writing a systematic review (SR) of animal research” .

As the authors rightly emphasize, SR’s are no common practice in the field of animal research. Knowing what already has been done can prevent unnecessary duplication of animal experiments and thus unnecessary animal use. The authors have interesting ideas like registration of animal studies (similar to clinical trials registers).

In this article they describe the design of an animal filter for PubMed. The authors describe their filter as follows:

“By using our effective search filter in PubMed, all available literature concerning a specific topic can be found and read, which will help in making better evidencebased decisions and result in optimal experimental conditions for both science and animal welfare.”

Is this conclusion justified?

Design of the filter

Their filter is subjectively derived: the terms are “logically” chosen.

[1] The first part of the animal filter consists of only MeSH-terms.

You can’t use animals[mh] (mh=Mesh) as a search term, because MeSH are automatically exploded in PubMed. This means that narrower terms (lower in the tree) are also searched. If “Animals” were allowed to explode, the search would include the MeSH, “Humans”, which is at the end of one tree (primates etc, see Fig. below)

Therefore the MeSH-parts of their search consists of:

  1. animals [mh:noexp]: only articles are found that are indexed with “animals”, but not its narrower terms. Notably, this is identical to the PubMed Limit: animals).
  2. Exploded Animal-specific MeSH-terms not having humans as a narrow term, i.e. “fishes”[MeSH Terms].
  3. and non-exploded MeSH in those cases that humans occurred in the same branch. Like “primates”[MeSH Terms:noexp]
  4. In addition two other MeSH are used: “animal experimentation”[MeSH Terms] and “models, animal”[MeSH Terms]

[2] The second part of the search filter consist of terms in the title and abstract (command: [tiab]).

The terms are taken from relevant MeSH, two reports about animal experimentation in the Netherlands and in Europe, and the experience of the authors, who are experts in the field.

The authors use this string for non-indexed records (command: NOT medline[sb]). Thus this part is only meant to find records that haven’t (yet) been indexed, but in which (specific) animals are mentioned by the author in title or text. Synonyms and spelling variants have been taken into account.

Apparently the authors have chosen NOT to search for text words in indexed records only. Presumably it gives too much noise, to search for animals mentioned in non-indexed articles. However, the authors do not discuss why this was necessary.

This search string is extremely long. Partly because truncation isn’t used with the longer words: i.e. nematod*[tiab] instead of nematoda[Tiab] OR nematode[Tiab] OR nematoda[Tiab] OR nematode[Tiab] OR nematodes[Tiab]. Partly because they aim for completeness. However the usefulness of the terms as such hasn’t been verified (see below).

Search strategies can be freely accessed here.

Validation

The filter is mainly validated against the PubMed Limit “Animals”.

The authors assume that the PubMed Limits are “the most easily available and most obvious method”. However I know few librarians or authors of systematic reviews who would solely apply this so called ‘regular method’. In the past I have used exactly the same MeSH-terms (1) and the main text words (2) as included in their filter.

Considering that the filter includes the PubMed limit “Animals” [1.1] it does not come as a surprise that the sensitivity of the filter exceeds that of the PubMed limit Animals…

Still, the sensitivity (106%) is not really dramatic: 6% more records are found, the PubMed Limit “animals” is set as 100%.

Apparently records are very well indexed with the MeSH “animals”. Few true animal records are missed, because “animals” is a check tag. A check tag is a MeSH that is looked for routinely by indexers in every journal article. It is added to the record even if it isn’t the main (or major) point of an article.

Is an increased sensitivity of appr. 6% sufficient to conclude that this filter “performs much better than the current alternative in PubMed”?

No. It is not only important that MORE is found but to what degree the extra hits are relevant. Surprisingly, the authors ONLY determined SENSITIVITY, not specificity or precision.

There are many irrelevant hits, partly caused by the inclusion of animal population groups[mesh], which has some narrower terms that often not used for experimentation, i.e. endangered species.

Thus even after omission of animal population groups[mesh], the filter still gives hits like:

These are evidently NOT laboratory animal experiments and mainly caused by the inclusion invertebrates  like plankton.

Most other MeSH are not extremely useful either. Even terms as animal experimentation[mh] and models, animal[mh] are seldom assigned to experimental studies lacking animals as a MeSH.

According to the authors, the MeSH “Animals” will not retrieve studies solely indexed with the MeSH term Mice. However, the first records missed with mice[mesh] NOT animals[mh:noexp] are from 1965, when they apparently didn’t use “animals” as a check tag in addition to specific ‘animal’ MeSH.

Thus presumably the MeSH-filter can be much shorter and need only contain animal MeSH (rats[mh], mice[mh] etc) when publications older than 1965 are also required.

The types of vertebrate animals used in lab re...

Image via Wikipedia

Their text word string (2) is also extremely long.  Apart from the lack of truncation, most animal terms are not relevant for most searches. 2/3 of the experiments are done with rodents (see Fig). The other animals are often used for specific experiments (zebra-fish, Drosophila) or in another context, not related to animal experiments, such as:

swine flu, avian flu, milk production by cows, or allergy by milk-products or mites, stings by insects and bites by dogs and of course fish, birds, cattle and poultry as food, fetal calf serum in culture medium, but also vaccination with “mouse products” in humans. Thus most of the terms produce noise for most topics. An example below (found by birds[mesh] 🙂

On the other hand strains of mice and rats are missing from the search string: i.e. balb/c, wistar.

Extremely long search strings (1 page) are also annoying to use. However, the main issue is whether the extra noise matters. Because the filter is meant to find all experimental animal studies.

As Carlijn Hooijmans notices correctly, the filters are never used on their own, only in combination with topic search terms.

Hooijmans et al have therefore “validated” their filter with two searches. “Validated” between quotation marks because they have only compared the number of hits, thus the increase in sensitivity.

Their first topic is the use of probiotics in experimental pancreatitis (see appendix).

Their filter (combined with the topic search) retrieved 37 items against 33 items with the so called “regular method”: an increase in sensitivity of 21,1%.

After updating the search I got  38 vs 35 hits. Two of the 3 extra hits obtained with the broad filter are relevant and are missed with the PubMed limit for animals, because the records haven’t been indexed. They could also have been found with the text words pig*[tiab] or dog*[tiab]. Thus the filter is ok for this purpose, but unnecessary long. The MeSH-part of the filter had NO added value compared to animals[mh:noexp].

Since there are only 148 hits without the use of any filters, researchers could also use screen all hits. Alternatively there is a trick to safely exclude human studies:

NOT (humans[mh] NOT animals[mh:noexp])

With this double negation you exclude PubMed records that are indexed with humans[mh], as long as these records aren’t indexed with animals[mh:noexp] too. It is far “safer” than limiting for “animals”[mesh:noexp] only. We use a similar approach to ” exclude”  animals when we search for human studies.

This extremely simple filter yields 48 hits, finding all hits found with the large animal filter (plus 10 irrelevant hits).

Such a simple filter can easily be used for searches with relatively few hits, but gives too many irrelevant hits in case of  a high yield.

The second topic is food restriction. 9280 Records were obtained with the Limit: “Animals”, whereas this strategy combined with the complete filter retrieved 9650 items. The sensitivity in this search strategy was therefore 104%. 4% extra hits were obtained.

The MeSH-search added little to the search. Only 21 extra hits. The relevant hits were (again) only from before 1965.

The text-word part of the search finds relevant new articles, although there are quite some  irrelevant findings too, i.e. dieting and obtaining proteins from chicken.

4% isn’t a lot extra, but the aim of the researchers is too find all there is.

However, it is the question whether researchers want to find every single experiment or observation done in the animal kingdom. If I were to plan an experiment on whether food restriction lowers the risk for prostate cancer in a transgenic mice, need I know what the effects are of food restriction on Drosophila, nematodes, salmon or even chicken on whatever outcome? Would I like to screen 10,000 hits?

Probably most researchers would like separate filters for rodents and other laboratory animals (primates, dogs) and for work on Drosophila or fish. In some fields there might also be a need to filter clinical trials and reviews out.

Furthermore, it is not only important to have a good filter but also a good search.

The topic searches in the current paper are not ideal: they contain overlapping terms (food restriction is also found by food and restriction) and misses important MeSH (Food deprivation, fasting and the broader term of caloric restriction “energy intake” are assigned more often to records about food deprivation than caloric restriction).

Their search:

(“food restriction”[tiab] OR (“food”[tiab] AND “restriction”[tiab]) OR “feed restriction”[tiab] OR (“feed”[tiab] AND “restriction”[tiab]) OR “restricted feeding”[tiab] OR (“feeding”[tiab] AND “restricted”[tiab]) OR “energy restriction”[tiab] OR (“energy”[tiab] AND “restriction”[tiab]) OR “dietary restriction”[tiab] OR (dietary”[tiab] AND “restriction”[tiab]) OR “caloric restriction”[MeSH Terms] OR (“caloric”[tiab] AND “restriction”[tiab]) OR “caloric restriction”[tiab])
might for instance be changed to:

Energy Intake[mh] OR Food deprivation[mh] OR Fasting[mh] OR food restrict*[tiab] OR feed restrict*[tiab] OR restricted feed*[tiab] OR energy restrict*[tiab] OR dietary restrict*[tiab] OR  caloric restrict*[tiab] OR calorie restrict*[tiab] OR diet restrict*[tiab]

You do not expect such incomplete strategies from people who repeatedly stress that: most scientists do not know how to use PubMed effectively” and that “many researchers do not use ‘Medical Subject Headings’ (MeSH terms), even though they work with PubMed every day”…..

Combining this modified search with their animal filter yields 21920 hits instead of 10335 as found with their “food deprivation” search and their animal filter. A sensitivity of 212%!!! Now we are talking! 😉 (And yes there are many new relevant hits found)

Summary

The paper describes the performance of a subjective search filter to find all experimental studies performed with laboratory animals. The authors have merely “validated”  this filter against the Pubmed Limits: animals. In addition, they only determined sensitivity:  on average 7% more hits were obtained with the new animal filter than with the PubMed limit alone.

The authors have not determined the specificity or precision of the filter, not even for the 2 topics where they have applied the filter. A quick look at the results shows that the MeSH-terms other than the PubMed limit “animals” contributed little to the enhanced sensitivity. The text word part of the filter yields more relevant hits. Still -depending on the topic- there are many irrelevant records found, because  it is difficult to separate animals as food, allergens etc from laboratory animals used in experiments and the filter is developed to find every single animal in the animal kingdom, including poultry, fish, nematodes, flies, endangered species and plankton. Another (hardly to avoid) “contamination” comes from in vitro experiments with animal cells, animal products used in clinical trials and narrative reviews.

In practice, only parts of the search filter seem useful for most systematic reviews, and especially if these reviews are not meant to give an overview of all findings in the universe, but are needed to check if a similar experiment hasn’t already be done. It seems impractical if researchers have to make a systematic review, checking, summarizing and appraising  10,000 records each time they start a new experiment.

Perhaps I’m somewhat too critical, but the cheering and triumphant tone of the paper in combination with a too simple design and without proper testing of the filter asked for a critical response.

Credits

Thanks to Gerben ter Riet for alerting me to the paper. He also gave the tip that the paper can be obtained here for free.

References

  1. Hooijmans CR, Tillema A, Leenaars M, & Ritskes-Hoitinga M (2010). Enhancing search efficiency by means of a search filter for finding all studies on animal experimentation in PubMed. Laboratory animals, 44 (3), 170-5 PMID: 20551243

———————-





Thoughts on the PubMed Clinical Queries Redesign

7 07 2010

Added 2010-07-09:  It is possible to enter the set numbers again, but the results are not yet reliable. They are probably working on it.

Last Wednesday (June 30th 2010) the PubMed Clinical Queries were redesigned.

Clinical Queries are prefab search filters that enable you to find aggregate evidence (Systematic Reviews-filter) or articles in a certain domain (Clinical study category-filters: like diagnosis and therapy), as well as papers in the field of  Medical Genetics (not shown below).

This was how it looked:

Since there were several different boxes you had to re-enter your search each time you tried another filter.

Now the Clinical Queries page has been reconfigured with columns to preview the first five citations of the results for all three research areas.

So this is how it looks now (search= PCOS spironolactone cyproterone hirsutism (PubMed automatically connects with “AND”))

Click to enlarge

Most quick responses to the change are “Neat”, “improved”, “tightened up”…….

This change might be a stylistic improvement for those who are used to enter words in the clinical queries without optimizing the search. At least you see “what you get”, you can preview the results of 3 filters, and you can still see “all” results by clicking on “see all”.  However, if you want to see the all results of another filter, you still have to go back to the clinical queries again.

But… I was not pleased to notice that it is no longer possible to enter a set number (i.e. #9) in the clinical queries search bar.

….Especially since the actual change was just before the start of an EBM-search session. I totally relied on this feature….

  1. Laika (Jacqueline)
    laikas Holy shit. #Pubmed altered the clinical queries, so that I can’t optimize my search first and enter the setnumber in the clin queries later.
  2. Laika (Jacqueline)
    laikas Holy shit 2 And I have a search class in 15 minutes. Can’t prepare changes. I hate this #pubmed #fail
  3. Mark MacEachern
    markmac perfect timing (for an intrface chnge) RT @laikas Holy shit 2 And I have a search class in 15 min. Can’t prepare changes. #pubmed #fail

this quote was brought to you by quoteurl

Furthermore the clinical study category is now default on “therapy broad” instead of narrow. This means a lot more noise: the broad filter searches for (all) clinical trials, while the narrow filter is meant to find randomized controlled trials only.

Normally I optimize the search first before entering the final search set number into the clinical queries.(see  Tip 9 of  “10+1 Pubmed tips for residents and their instructors“).  For instance, the above search would not include PCOS (which doesn’t map to the proper MeSH and isn’t required) and cyproterone, but would consist of hirsutism AND spironolactone (both mapping to the appropriate MeSH).

The set number of the “optimized” search is then entered in the search box of the Systematic Review filter. This yields 9 more hits, including Cochrane systematic reviews. The narrow therapy filter gives more hits, that are more relevant as well (24).

The example that is shown in the NLM technical bulletin (dementia stroke) yields 142 systematic reviews and 1318 individual trials of which only the 5 most recent trials are shown. Not very helpful to doctors and scientists, IMHO.

Anyway, we “lost” a (roundabout) way- to optimize the search before entering it into the search box.

The preview of 3 boxes is o.k., the looks are o.k. but why is this functionality lost?

For the moment I decided to teach my class another option that I use myself: adding clinical queries to your personal NCBI account so that the filters show up each time you perform a search in PubMed ( this post describes how to do it).

It only takes some time to make NCBI accounts and to explain the procedure to the class, time you would like to save for the searches themselves  (in a 1-2 hr workshop). But it is the most practical solution.

We notified PubMed, but it is not clear whether they plan to restore this function.

Note: 2010-07-09:  It is possible to enter the set numbers again, but the results are not yet reliable. They are probably working on it.

Still, for advanced users, adding filters to your NCBI may be most practical.

——-

*re-entering spironolactone and hirsutism in the clinical queries is doable here, but often the search is more complex and different per filter. For instance I might add a third concept when looking for an individual trial.





PubMed versus Google Scholar for Retrieving Evidence

8 06 2010

ResearchBlogging.orgA while ago a resident in dermatology told me she got many hits out of PubMed, but zero results out of TRIP. It appeared she had used the same search for both databases: alopecea areata and diphenciprone (a drug with a lot of synonyms). Searching TRIP for alopecea (in the title) only, we found a Cochrane Review and a relevant NICE guideline.

Usually, each search engine has is its own search and index features. When comparing databases one should compare “optimal” searches and keep in mind for what purpose the search engines are designed. TRIP is most suited to search aggregate evidence, whereas PubMed is most suited to search individual biomedical articles.

Michael Anders and Dennis Evans ignore this “rule of the thumb” in their recent paper “Comparison of PubMed and Google Scholar Literature Searches”. And this is not the only shortcoming of the paper.

The authors performed searches on 3 different topics to compare PubMed and Google Scholar search results. Their main aim was to see which database was the most useful to find clinical evidence in respiratory care.

Well quick guess: PubMed wins…

The 3 respiratory care topics were selected from a list of systematic reviews on the Website of the Cochrane Collaboration and represented in-patient care, out-patient care, and pediatrics.

The references in the three chosen Cochrane Systematic Reviews served as a “reference” (or “golden”) standard. However, abstracts, conference proceedings, and responses to letters were excluded.

So far so good. But note that the outcome of the study only allows us to draw conclusions about interventional questions, that seek to find controlled clinical trials. Other principles may apply to other domains (diagnosis, etiology/harm, prognosis ) or to other types of studies. And it certainly doesn’t apply to non-EBM-topics.

The authors designed ONE search for each topic, by taking 2 common clinical terms from the title of each Cochrane review connected by the Boolean operator “AND” (see Table, ” ” are not used). No synonyms were used and the translation of searches in PubMed wasn’t checked (luckily the mapping was rather good).

“Mmmmm…”

Topic

Search Terms

Noninvasive positive-pressure ventilation for cardiogenic pulmonary edema “noninvasive positive-pressure ventilation” AND “pulmonary edema”
Self-management education and regular practitioner review for adults with asthma “asthma” AND “education”
Ribavirin for respiratory syncytial virus “ribavirin” AND “respiratory syncytial virus”

In PubMed they applied the narrow methodological filter, or Clinical Query, for the domain therapy.
This prefab search strategy (randomized controlled trial[Publication Type] OR (randomized[Title/Abstract] AND controlled[Title/Abstract] AND trial[Title/Abstract]), developed by Haynes, is suitable to quickly detect the available evidence (provided one is looking for RCT’s and doesn’t do an exhaustive search). (see previous posts 2, 3, 4)

Google Scholar, as we all probably know, does not have such methodological filters, but the authors “limited” their search by using the Advanced option and enter the 2 search terms in the “Find articles….with all of the words” space (so this is a boolean “AND“) and they limited it the search to the subject area “Medicine, Pharmacology, and Veterinary Science”.

They did a separate search for publications that were available at their library, which has limited value for others, subscriptions being different for each library.

Next they determined the sensitivity (the number of relevant records retrieved as a proportion of the total number of records in the gold standard) and the precision or positive predictive value, the  fraction of returned positives that are true positives (explained in 3).

Let me guess: sensitivity might be equal or somewhat higher, and precision is undoubtedly much lower in Google Scholar. This is because (in) Google Scholar:

  • you can often search full text instead of just in the abstract, title and (added) keywords/MeSH
  • the results are inflated by finding one and the same references cited in many different papers (that might not directly deal with the subject).
  • you can’t  limit on methodology, study type or “evidence”
  • there is no automatic mapping and explosion (which may provide a way to find more synonyms and thus more relevant studies)
  • has a broader coverage (grey literature, books, more topics)
  • lags behind PubMed in receiving updates from MEDLINE

Results: PubMed and Google Scholar had pretty much the same recall, but for ribavirin and RSV the recall was higher in PubMed, PubMed finding 100%  (12/12) of the included trials, and Google Scholar 58% (7/12)

No discussion as to the why. Since Google Scholar should find the words in titles and abstracts of PubMed I repeated the search in PubMed but only in the title, abstract field, so I searched ribavirin[tiab] AND respiratory syncytial virus[tiab]* and limited it with the narrow therapy filter: I found 26 papers instead of 32. These titles were missing when I only searched title and abstract (between brackets: [relevant MeSH (reason why paper was found), absence of abstract (thus only title and MeSH) and letter], bold: why terms in title abstract are not found)

  1. Evaluation by survival analysis on effect of traditional Chinese medicine in treating children with respiratory syncytial viral pneumonia of phlegm-heat blocking Fei syndrome.
    [MesH:
    Respiratory Syncytial Virus Infections/]
  2. Ribavarin in ventilated respiratory syncytial virus bronchiolitis: a randomized, placebo-controlled trial.
    [MeSH:
    Respiratory Syncytial Virus Infections/[NO ABSTRACT, LETTER]
  3. Study of interobserver reliability in clinical assessment of RSV lower respiratory illness.
    [MeSH:Respiratory Syncytial Virus Infections*]
  4. Ribavirin for severe RSV infection. N Engl J Med.
    [MeSH: Respiratory Syncytial Viruses
    [NO ABSTRACT, LETTER]
  5. Stutman HR, Rub B, Janaim HK. New data on clinical efficacy of ribavirin.
    MeSH: Respiratory Syncytial Viruses
    [NO ABSTRACT]
  6. Clinical studies with ribavirin.
    MeSH: Respiratory Syncytial Viruses
    [NO ABSTRACT]

Three of the papers had the additional MeSH respiratory syncytial virus and the three others respiratory syncytial virus infections. Although not all papers (2 comments/letters) may be relevant, it illustrates why PubMed may yield results, that are not retrieved by Google Scholar (if one doesn’t use synonyms)

In Contrast to Google Scholar, PubMed translates the search ribavirin AND respiratory syncytial virus so that the MeSH-terms “ribavirin”, “respiratory syncytial viruses”[MeSH Terms] and (indirectly) respiratory syncytial virus infection”[MeSH] are also found.

Thus in Google Scholar articles with terms like RSV and respiratory syncytial viral pneumonia (or lack of specifications, like clinical efficacy) could have been missed with the above-mentioned search.

The other result of the study (the result section comprises 3 sentences) is that “For each individual search, PubMed had better precision”.

The Precision was 59/467 (13%) in PubMed and 57/80,730 (0.07%)  in Google Scholar (p<0.001)!!
(note: they had to add author names in the Google Scholar search to find the papers in the haystack 😉

Héhéhé, how surprising. Well why would it be that no clinician or librarian would ever think of using Google Scholar as the primary, let alone the only, source to search for medical evidence?
It should also ring a bell, that [QUOTE**]:
In the Cochrane reviews the researchers retrieved information from multiple databases, including MEDLINE, the Cochrane Airways Group trial register (derived from MEDLINE)***, CENTRAL, EMBASE, CINAHL, DARE, NHSEED, the Acute Respiratory Infections Group’s specialized register, and LILACS… ”
Note
Google Scholar isn’t mentioned as a source! Google Scholar is only recommendable to search for work citing (already found) relevant articles (this is called forward searching), if one hasn’t access to Web of Science or SCOPUS. Thus only to catch the last fish.

Perhaps the paper could have been more interesting if the authors had looked at any ADDED VALUE of Google Scholar, when exhaustively searching for evidence. Then it would have been crucial to look for grey literature too, (instead of excluding it), because this could be a possible strong point for Google Scholar. Furthermore one could have researched if forward searching yielded extra papers.

The specificity of PubMed is attributed to the used therapy-narrow filter, but the vastly lower specificity of Google Scholar is also due to the searching in the full text, including the reference lists.

For instance, searching for ribavirin AND respiratory syncytial virus in PubMed yields 523 hits. This can be reduced to 32 hits when applying the narrow therapy filter. This means a reduction by a factor of 16.
Yet a similar search in Google Scholar yield
4,080 hits. Thus without the filter there is still an almost 8 times higher yield from Google Scholar than from PubMed.

That evokes another  research idea: what would have happened if randomized (OR randomised) would have been added to the Google Scholar search? Would this have increased the specificity? In case of the above search it lowers the yield with a factor 2, and the first hits look very relevant.

It is really funny but the authors bring down their own conclusion that “These results are important because efficient retrieval of the best available scientific evidence can inform respiratory care protocols, recommendations for clinical decisions in individual patients, and education, while minimizing information overload.” by saying elsewhere that “It is unlikely that users consider more than the first few hundred search results, so RTs who conduct literature searches with Google Scholar on these topics will be much less likely to find references cited in Cochrane reviews.”

Indeed no one would take it into ones head to try to find the relevant papers out of those 4,080 hits retrieved. So what is this study worth from a practical point of view?

Well anyway, as you can ask for the sake of asking you can research for the sake of researching. Despite being an EBM-addict I prefer a good subjective overview on this topic over a weak scientific, quasi-evidence based, research paper.

Does this mean Google Scholar is useless? Does it mean that all those PhD’s hooked on Google Scholar are wrong?

No, Google Scholar serves certain purposes.

Just like the example of PubMed and TRIP, you need to know what is in it for you and how to use it.

I used Google Scholar when I was a researcher:

  • to quickly find a known reference
  • to find citing papers
  • to get an idea of how much articles have been cited/ find the most relevant papers in a quick and dirty way (i.e. by browsing)
  • for quick and dirty searches by putting words string between brackets.
  • to search full text. I used quite extensive searches to find out what methods were used (for instance methods AND (synonym1 or syn2 or syn3)). An interesting possibility is to do a second search for only the last few words (in a string). This will often reveal the next words in the sentence. Often you can repeat this trick, reading a piece of the paper without need for access.

If you want to know more about the pros and cons of Google Scholar I recommend the recent overview by the expert librarian Dean Giustini: “Sure Google Scholar is ideal for some things” [7]”. He also compiled a “Google scholar bibliography” with ~115 articles as of May 2010.

Speaking of librarians, why was the study performed by PhD RRT (RN)’s and wasn’t the university librarian involved?****

* this is a search string and more strict than respiratory AND syncytial AND virus
**
abbreviations used instead of full (database) names
*** this is wrong, a register contains references to controlled clinical trials from EMBASE, CINAHL and all kind of  databases in addition to MEDLINE.
****other then to read the manuscript afterwards.

References

  1. Anders ME, & Evans DP (2010). Comparison of PubMed and Google Scholar Literature Searches. Respiratory care, 55 (5), 578-83 PMID: 20420728
  2. This Blog: https://laikaspoetnik.wordpress.com/2009/11/26/adding-methodological-filters-to-myncbi/
  3. This Blog: https://laikaspoetnik.wordpress.com/2009/01/22/search-filters-1-an-introduction/
  4. This Blog: https://laikaspoetnik.wordpress.com/2009/06/30/10-1-pubmed-tips-for-residents-and-their-instructors/
  5. NeuroDojo (2010/05) Pubmed vs Google Scholar? [also gives a nice overview of pros and cons]
  6. GenomeWeb (2010/05/10) Content versus interface at the heart of Pubmed versus Scholar?/ [response to 5]
  7. The Search principle Blog (2010/05) Sure Google Scholar is ideal for some things.




Ten Years of PubMed Central: a Good Thing that’s Only Going to Get Better.

26 05 2010

PubMed Central (PMC) is a free digital archive of biomedical and life sciences journal literature at the U.S. National Institutes of Health (NIH), developed and managed by NIH’s National Center for Biotechnology Information (NCBI) in the National Library of Medicine (NLM) (see PMC overview).
PMC is a central repository for biomedical peer reviewed literature in the same way as NCBI’s GenBank is the public archive of DNA sequences. The idea behind it “that giving all users free access to the material in PubMed Central is the best way to ensure the durability and utility of the electronical archive as technology changes over time and to integrate the literature with other information resources at NLM”.
Many journals are already involved, although most of them adhere to restrictions (i.e. availability after 1 year). For list see http://www.ncbi.nlm.nih.gov/pmc/journals/

PMC, the brain child of Harold Varmus, once the Director of the National Institutes of Health, celebrated its 10 year anniversary earlier this year.

For this occasion Dr. Lipman, Director of the NCBI, gave an overview of past and future plans for the NIH’s archive of biomedical research articles. See videotape of the Columbia University Libraries below:

Vodpod videos no longer available.

more about “Ten Years of PubMed Central | Scholar…“, posted with vodpod

The main points raised by David Lipman (appr. time given if you want to learn more about it; the text below is not a transcription, but a summary in my own words):

PAST/PRESENT

  • >7:00. BiomedCental (taken over by Spinger) and PLoS ONE show that Open Access can be a sustaining way in Publishing Science.
  • 13:23 Publisher keeps the copyright. He may stop depositing but the content already deposited remains in PMC.
  • 13:50 PMC is also an obligatory repository for author manuscripts under various funding agencies mandates, like the NIH and the UK welcome trust.
  • 14:31 One of the ideas from the beginning was to crosslink the literature with the underlying molecular and other databases. For instance NCBI is capable of mining out the information in the archived text and connecting it to the compound and the protein structure database.
  • 16:50 There is a back issue digitization for the journals that are participating, enabling to find research that you wouldn’t have easily found otherwise.
  • PMC has become international (not restricted to USA)
  • The PMC archive becomes more useful if it becomes more comprehensive
  • Before PMC you could do a Google Scholar search and find a paper in PubMed, that appeared funded by NIH, but then you had to pay $30 for it in order to get it. That’s hard to explain to the taxpayers (Lipman had a hard time explaining it to his dad who was looking for medical information online). This was the impetus for making the results of NIH-sponsored results freely available.

PRESENT/FUTURE

  • 23:00 Discovery initiative: is the use of tracking tools to find out which changes to the website work for users and which don’t. Thus modifications should lead to alterations in users behavior (statistics is easy with millions of users). Discovery initiative led to development and improvement of sensors, like sensors for disease names, drug names, genes and citations. What is being measured is if people click through (if it isn’t interesting, they usually don’t) and how quickly they find results. Motto: train the machine, not the users.
  • 30:37 We changed the looks of PMC. Planning to make a better presentation on the i-phone and on broad monitors.
  • 31:40. There are almost 2 million articles in PubMed Central, 585 journals fully participate in PMC
  • 32.30 It takes very long to publish a paper, even in Open Access papers. Therefore a lot of people are not publishing little discoveries, which are not important enough to put a lot of time in. Publishing should be almost as easy as writing a blog, but with peer review. This requires a new type of journal, with peer review, but with instant feedback from readers and reviewers and rapid response to comments. The Google Knol authoring system offers a fast and simple authoring system where authors (with a Google profile) can collaborate and compose the article on the server. Uploading of documents and figures is easy, the article updates are simple and fast, there is a simple workflow for moderators. After the paper is accepted you press a button, the paper is immediately available and the next day PMC automatically gets the XML content. There is also a simple Reference Manager included to paste citations.
  • Principle: How you can start a journal with this system (see Figure). Till now: 60 articles in PLOS Currents Influenza. There are also plans for other journals: the CDC is announcing a Systematic Reviews journal, for instance.

QUESTIONS (>39:30):

  • Process by which “KNOL-journal” is considered for inclusion in NLM?
    • Decide: is it in scope?, implicit policy (health peer review being done), who are the people involved, look at a dozen articles.
  • As the content in PMC increases, will it become possible to search in the full text, just like in Google Scholar?
    • Actually the full text is searchable in PMC as apposed to PubMed, but we are not that happy with the full text retrieval. Even with a really good approach, searching full text works just a little bit better than searching PubMed.
      We are incorporating more of the information of PMC into PubMed, and are working on a separate image database with all the figures from books and articles in PMC (with other search possibilities). Subsets of book(chapter)s (like practice guidelines) will get PubMed abstracts and become searchable in PubMed as well.
  • Are there ways to track a full list of our institutions OA articles in PMC (not picking up everything in PubMed)
    • Likely NIH will be contacting offices responsible for research to let them know what articles are out of compliance,  get their assistance in making sure that those get in.
    • Authors can easily update the electornic My Bibliography (in My NCBI in PubMed).
    • Author ID project, involves computational disambiguation. Where you are asked if you are the author of a paper if you didn’t include it. It may also be possible to have automatic reporting to the institutions.
  • What did it took politically to get the appropriation bill passed (PMC initiative)?
    • Congress always pushed more open access, because it was already spending money on the research. Most of the initiative came more from librarians (i.e. small libraries not having sufficient access) and government, than from the NIH.
  • Is there way to narrow down to NIH, free full text papers from PMC?
    • In PubMed, you can filter free full text articles in general via the limits.
  • Are all the articles deposited in PMC submitted the final manuscript?
    • Generally, yes.

HT: @bentoth on Twitter





An Evidence Pyramid that Facilitates the Finding of Evidence

20 03 2010

Earlier I described that there are so many search- and EBM-pyramids that it is confusing. I described  3 categories of pyramids:

  1. Search Pyramids
  2. Pyramids of EBM-sources
  3. Pyramids of EBM-levels (levels of evidence)

In my courses where I train doctors and medical students how to find evidence quickly, I use a pyramid that is a mixture of 1. and 2. This is a slide from a 2007 course.

This pyramid consists of 4 layers (from top down):

  1. EBM-(evidence based) guidelines.
  2. Synopses & Syntheses*: a synopsis is a summary and critical appraisal of one article, whereas synthesis is a summary and critical appraisal of a topic (which may answer several questions and may cover many articles).
  3. Systematic Reviews (a systematic summary and critical appraisal of original studies) which may or may not include a meta-analysis.
  4. Original Studies.

The upper 3 layers represent “Aggregate Evidence”. This is evidence from secondary sources, that search, summarize and critically appraise original studies (lowest layer of the pyramid).

The layers do not necessarily represent the levels of evidence and should not be confused with Pyramids of EBM-levels (type 3). An Evidence Based guideline can have a lower level of evidence than a good systematic review, for instance.
The present pyramid is only meant to lead the way in the labyrinth of sources. Thus, to speed up to process of searching. The relevance and the quality of evidence should always be checked.

The idea is:

  • The higher the level in the pyramid the less publications it contains (the narrower it becomes)
  • Each level summarizes and critically appraises the underlying levels.

I advice people to try to find aggregate evidence first, thus to drill down (hence the drill in the Figure).

The advantage: faster results, lower number to read (NNR).

During the first courses I gave, I just made a pyramid in Word with the links to the main sources.

Our library ICT department converted it into a HTML document with clickable links.

However, although the pyramid looked quite complex, not all main evidence sources were included. Plus some sources belong to different layers. The Trip Database for instance searches sources from all layers.

Our ICT-department came up with a much better looking and better functioning 3-D pyramid, with databases like TRIP in the sidebar.

Moving the  mouse over a pyramid layer invokes a pop-up with links to the databases belonging to that layer.

Furthermore the sources included in the pyramid differ per specialty. So for the department Gynecology we include POPLINE and MIDIRS in the lowest layer, and the RCOG and NVOG (Dutch) guidelines in the EBM-guidelines layer.

Together my colleagues and I decide whether a source is evidence based (we don’t include UpToDate for instance) and where it  belongs. Each clinical librarian (we all serve different departments) then decides which databases to include. Clients can give suggestions.

Below is a short You Tube video showing how this pyramid can be used. Because of the rather poor quality, the video is best to be viewed in full screen mode.
I have no audio (yet), so in short this is what you see:

Made with Screenr:  http://screenr.com/8kg

The pyramid is highly appreciated by our clients and students.

But it is just a start. My dream is to visualize the entire pathway from question to PICO, checklists, FAQs and database of results per type of question/reason for searching (fast question, background question, CAT etc.).

I’m just waiting for someone to fulfill the technical part of this dream.

————–

*Note that there may be different definitions as well. The top layers in the 5S pyramid of Bryan Hayes are defined as follows: syntheses & synopses (succinct descriptions of selected individual studies or systematic reviews, such as those found in the evidence-based journals), summaries, which integrate best available evidence from the lower layers to develop practice guidelines based on a full range of evidence (e.g. Clinical Evidence, National Guidelines Clearinghouse), and at the peak of the model, systems, in which the individual patient’s characteristics are automatically linked to the current best evidence that matches the patient’s specific circumstances and the clinician is provided with key aspects of management (e.g., computerised decision support systems).

Begin with the richest source of aggregate (pre-filtered) evidence and decline in order to to decrease the number needed to read: there are less EBM guidelines than there are Systematic Reviews and (certainly) individual papers.




Practicing Medicine in the Web 2.0 Era

29 01 2010

Many people don’t get Web 2.0 – and certainly not Medicine 2.0.

Just the other day a journalist asked me if the redesigned PubMed could be called PubMed 2.0.
I said: “well no….no… not at all” ….Web 2.0 is not merely tools or fancy looks, it is another way of producing and sharing information and new web tools facilitate that. It is not only simplicity, it is participation. PubMed has changed it looks, but it is not an interactive platform, where you can add or exchange information.

Well anyway, I probably didn’t succeed to explain in just a few sentences what Web 2.0 is and what it isn’t. For those that are unfamiliar with Web 2.0 and/or how it changes Medicine, I highly recommend the following presentation by Bertalan Mesko (of ScienceRoll and Webicina), who explains in a clear and nontechnical way what it is all about.

By the way Bertalan is a finalist with ScienceRoll in the 2009 Medical Weblog Awards (category Best Medical Technologies/Informatics Weblog). He could surely use your vote. (here you can vote in this category). You can see all Finalist here.

Reblog this post [with Zemanta]




Cochrane Evidence Aid for Catastrophes like Haiti’s Earthquake. “Helping by doing what we do best”

24 01 2010

How it started [1]
2005. December 26th. Someone* working for the Cochrane Collaboration was on the Internet when he accidentally saw the AOL’s home page mentioning a powerful earthquake in the Indian Ocean, triggering a powerful tsunami that swept the coasts of neighboring countries. The story and the horror unfolded over the next hours and days. From the first reports of a few thousand dead to, within a day, a few thousand dead and tens of thousands missing in one part of Indonesia alone.

“What can we do?” he thought “Aid needs evidence on what works and what doesn’t work. It is no good and, worse, might be harmful, to deliver health care that is ineffective. The Cochrane Library already contains several reviews of relevance. There are Cochrane reviews on overcoming the effects of dehydration and the treatment of injuries, both physical and psychological. Those of us who work in the production of evidence can, therefore, deliver our own form of aid: information. The provision of reliable information on the effects of health care is the way that many of us can contribute to alleviating its effects. We need to recognize the privileged position that we are in: we can help by doing what we do best.”[1]

As reader of this blog, you probably know that the Cochrane Collaboration (http://www.cochrane.org/) is an international not-for-profit and independent organization, dedicated to making up-to-date, accurate information about the effects of healthcare readily available worldwide. It produces and disseminates systematic reviews of healthcare interventions  in the Cochrane Library, which is available through subscription. The information on which these are based is drawn together collaboratively by a global network of dedicated volunteers, supported by a small staff.

Evidence Aid: what it is and what it does. [2, 3, 4, 5]
That Christmas, the idea was born to set up “Evidence Aid”.
A working party was established early January 2005 of people in the region and elsewhere.  Emails were send to people from the affected countries to express sympathy and support, and to ask for suggestions on how The Cochrane Collaboration might help.

Then, a list of over 200 interventions relevant to health care in the aftermath of the tsunami was made in consultation with all Cochrane entities, Cochrane members from affected countries, and members of other agencies such as the World Health Organization, Oxfam (one of the main UK charities working in the region), and the publishers of BMJ’s Clinical Evidence (http://clinicalevidence.com).

A prioritization was made, and subsequently lists were made of topics for which up-to-date Cochrane reviews were available and lists for which reviews were not yet available (see updated lists of  available and not currently available topics).

Concise summaries of evidence on the priority topics were offered in one place with “one-click” access to all contents, available free of charge (http://www.cochrane.org/docs/tsunamiresp​onse, now changed into http://www.cochrane.org/evidenceaid/index.htm)

The summaries link to the full evidence, which is already available on the Cochrane Library. If a summary is not currently available but there is a relevant Cochrane review in the Cochrane Library, a link takes people straight to that review. If a suitable Cochrane review is not available, links are included to other identified sources of evidence, in particular, to topics in Clinical Evidence .

In addition The Cochrane Library (http://www.thecochranelibrary.org) was made freely available in the effected countries for a six-month period. (This was before the Cochrane Library became freely available in India through funding)

Evidence matters, an example [4,5,6]
One helpful Cochrane Review was the Cochrane systematic review on the effects of brief “debriefing” [6], which is a procedure aimed to reduce immediate psychological distress and to prevent the subsequent development of psychological disorders, notably Post Traumatic Stress Disorder (PTSD). The review shows that this strategy is unlikely to be helpful and may even be harmful and cause an increase in PTSD.
After the tsunami, many teams of well-meaning people rushed to one of the worst hit areas in India, offering brief debriefing to survivors in each village, and then rushing on to the next of the 93 tsunami-affected villages in the district. Prathap Tharyan, Professor of Psychiatry and Coordinator of the South Asian Cochrane Network, found the relevant Cochrane review on debriefing and urged that this type of single session debriefing should not be provided. This message was incorporated into the content of counselor training workshops, along with evidence for interventions supported by the results of systematic reviews and other high quality research.[5]

Evidence Aid for Haiti [7]
After the tsunami it was decided to continue with Evidence Aid in natural disasters and other healthcare emergencies, drawing on knowledge gathered.

Tweets of @cochranecollab about various Evidence Aid Reviews for Haiti

Following the devastating earthquake in Haiti, The Cochrane Collaboration is working with colleagues in the World Health Organization (WHO), Pan American Health Organization (PAHO), the Centre for Reviews & Dissemination (UK), Cochrane Review Groups and others to identify Cochrane reviews and other systematic reviews of immediate importance. These, along with available Evidence Update summaries, were made available in a special Evidence Aid collection on Cochrane.org on 15 January, and have been shared with WHO and PAHO.

The information has been translated into French (thanks to the Cochrane Francophone Network) and Spanish (thanks to the Iberoamerican Cochrane Centre). At the moment, the collection includes reviews from several Cochrane Review Groups, including the Bone, Joint and Muscle Trauma Group; Depression, Anxiety and Neurosis Group; Infectious Diseases Group; Injuries Group; Renal Group and Wounds Group. [7]

Access to Evidence Aid resources for Haiti: The summaries are available at http://www.cochrane.org/evidenceaid/haiti/index.html and The Cochrane Library is freely available in the region through a variety of means. One is the Biblioteca Cochrane Plus via the Virtual Health Library BIREME interface (in English, Spanish or Portuguese).  Also, the PDF versions of all the highlighted Cochrane reviews are now available free to all on The Cochrane Library website.[7]

Is this enough?[4]
A PLOS article on Evidence Aids in 2005 already concluded: “No, not nearly enough”.[4]

Not all topics on the list have been covered by an up-to-date, good-quality systematic review. And, similar as in 2005, not all reviews have conclusions that can guide practice, because of a lack of relevant good-quality studies. After all, reviews are only as good as the studies they review. Therefore it is important to fill the gaps with good quality reviews and new practical trials on the most urgent topics.
Although things have certainly changed, i.e. more topics are now covered, there still remains room for further improvement.

If you would like to suggest additional material not yet covered, please contact Mike Clarke (mclarke@cochrane.ac.uk). You can also contribute to Evidence aid in other ways.

* This person signed the Gem [1] with “Insider”. It is not difficult to gather that the Insider is Mike Clarke, professor of clinical epidemiology at the University of Oxford, director of the UK Cochrane Centre and convenor of the working group which has set up the initiative.

Afterword: Last Monday, tweets mentioning Cochrane Evidence Aid topics appeared in my twitterstream (see Fig). As I was not profoundly familiar with this initiative, I wanted to gain more knowledge about it and summarize my findings in a post. I’m thankful to Mike Clarke and Nick Royle for instantly responding to my request for more information and Mike in particular for sending me the draft he compiled for CC-info [7] and an older cochrane gem [1], that explained how Evidence Aid arose.
Disclaimer: I’m employed as a Trial Search Coordinator of the Dutch Cochrane Centre for one day per week. The opinions expressed at this blog, however, are my own.

References:

  1. Cochrane Gem for week commencing 4 January 2005, written by “Insider”. Gems are weekly highlights one of new reviews or sometimes important news. Gems are available at the CKS database here.
  2. http://news.cochrane.org/view/item/review_one.jsp?j=177 assessed 24-01-2010
  3. Lynn Eaton (2005) Evidence based research for coping in emergencies goes online BMJ 330(7497):926 (23 April), doi:10.1136/bmj.330.7497.926-a
  4. Tharyan P, Clarke M, Green S (2005) How the Cochrane Collaboration Is Responding to the Asian Tsunami. PLoS Med 2(6): e169. doi:10.1371/journal.pmed.0020169
  5. World Health Organization (2005) Three months after the Indian Ocean earthquake-tsunami: Health consequences and WHO’s response. Available: http://www.who.int/hac/crises/internatio​nal/asia_tsunami/3months/en/index.html . Accessed 24-01-2010.
  6. Rose SC, Bisson J, Churchill R, Wessely S. Psychological debriefing for preventing post traumatic stress disorder (PTSD). Cochrane Database of Systematic Reviews 2002, Issue 2. Art. No.: CD000560. DOI: 10.1002/14651858.CD000560. Edited (no change to conclusions), published in Issue 1, 2009.
  7. Draft written for CC-INFO (January 21, 2010) by Mike Clarke. It will become available at the CC-info archive.
Reblog this post [with Zemanta]




When more is less: Truncation, Stemming and Pluralization in the Cochrane Library

5 01 2010

I’m on two mail lists of the Cochrane Collaboration, one is the TSC -list (TSC=Trials Search Coordinator) and the other the IRMG-list. IMRG stands for Information Retrieval Methods Group (of the Cochrane). Sometimes, difficult search problems are posted on the list. It is challenging to try to find the solutions. I can’t remember that a solution was not found.

A while ago a member of the list was puzzled why he got the following retrieval result from the Cochrane Library:

ID Search Hits
#1 (breast near tumour* ) ….. 254
#2 (breast near tumour) …… 640
#3 (breast near tumor*) ….. 428
#4 (breast near tumor) …… 640

where near = adjacent (thus breast should be just before tumour) and the asterisk * is the truncation symbol.  At the end of the word an asterisk is used for all terms that begin with that basic word root. Thus tumour* should find: tumours and tumour and thus broaden the search.

The results are odd, because #2 (without truncation) gives more hits than #1 (with truncation), and the same is true for #4 versus #3. One would expect truncation to give more results. What could be the reason behind it?

I suspected the problem had to do with the truncation. I searched for breast and tumour with or without truncation (#1 to #4) and only tumour* gave odd results: tumour* gave much less results than tumour. (to exclude that it had to do with the fields being searched I only searched the fields ti (title), ab (abstract) and kw (keywords))

Records found with tumour, not with tumour*, contained the word tumor (not shown). Thus tumour automatically searches for tumor (and vice versa). This process is called stemming.

According to the Help-function of the Cochrane Library:

Stemming: The stemming feature within the search allows words with small spelling variants to be matched. The term tumor will also match tumour.

In addition, as I realized later, the Cochrane has pluralization and singularization features.

Pluralization and singularization matches Pluralized forms of words also match singular versions, and vice versa. The term drugs will find both drug and drugs. To match either just the singular or plural form of a terms, use an exact match search and include the word in quotation marks.

Indeed (tumor* OR tumour*) (or shortly tumo*r*) retrieves a little more than tumor OR tumour: words like tumoral, tumorous, tumorectomy. Not particularly useful, although it might not be disadvantagous when used adjacent to breast, as this will filter most noise.

tumor spelling variants searched in the title (ti) only: it doesn't matter how you spell tumor (#8, #9, #10,#11), as long as you don't truncate (while using a single variant)

Thus stemming, pluralization and singularization only work without truncation. In case of truncation you should add the spelling variants yourselves if case stemming/pluralization takes place. This is useful if you’re interested in other word variants that are not automatically accounted for.

Put it another way: knowing that stemming and pluralization takes place you can simply search for the single or plural form, American or English spelling. So breast near tumor (or simply breast tumor) would have been o.k. This is the reason why these features were introduced in the first way. 😉

By the way, truncation and stemming (but not pluralization) are also features in PubMed. And this can give similar and other problems. But this will be dealt with in another blogpost.

Reblog this post [with Zemanta]




Adding Methodological Filters to MyNCBI

26 11 2009

Idea: Arnold Leenders
Text: “Laika”

Methodological Search Filters can help to narrow down a search by enriching for studies with a certain study design or methodology. PubMed has build-in methodological filters, the so called Clinical Queries for domains (like therapy and diagnosis) and for evidence based papers (like theSystematic Review subset” in Pubmed). These searches are often useful to quickly find evidence on a topic or to perform a CAT (Critical Appraised Topic). More exhaustive searches require broader  filters not incorporated in PubMed. (See Search Filters. 1. An Introduction.).

The Redesign of PubMed has made it more difficult to apply Clinical Queries after a search has been optimized. You can still go directly to the clinical queries (on the front page) and fill in some terms, but we rather advise to build the strategy first, check the terms and combine your search with filters afterwards.

Suppose you would like to find out whether spironolactone effectively reduces hirsutism in a female with PCOS (see 10+ 1 Pubmed Tips for Residents and their Instructors, Tip 9). You first check that the main concepts hirsutism and spironactone are o.k. (i.e. they map automatically with the correct MeSH). Applying the clinical queries at this stage would require you to scroll down the page each time you use them.

Instead you can use filters in My NCBI for that purpose. My NCBI is your (free) personal space for saving searches, results, PubMed preferences, for creating automatic email alerts and for creating Search Filters.
The My NCBI-option is at the upper right of the PubMed page. You first have to create a free account.

To activate or create filters, go to [1] My NCBI and click on [2] Search Filters.

Since our purpose is to make filters for PubMed, choose [3] PubMed from the list of NCBI-databases.

Under Frequently Requested Filters you find the most popular Limit options. You can choose any of the optional filters for future use. This works faster than searching for the appropriate limit each time. You can for instance use the filter for humans to exclude animals studies.

The Filters we are going to use are under “Browse Filters”, Subcategory Properties….

….. under Clinical Queries (Domains, i.e. therapy) and Subsets (Systematic Review Filters)

You can choose any filter you like. I choose the Systematic Review Filter (under Subsets) and the Therapy/Narrow Filter under  Clinical Queries.

In addition you can add custom filters. For instance you might want to add a sensitive Cochrane RCT filter, if you perform broad searches. Click Custom Filters, give the filter a name and copy/paste the search string you want to use as filter.

Control via “Run Filter” if the Filter works (the number of hits are shown) and SAVE the filter.

Next you have to activate the filters you want to use. Note there is a limit of five 15 filters (including custom filters) that can be selected and listed in My Filters. [edited: July 5th, hattip Tanya Feddern-Bekcan]

Under  My Filters you now see the Filters you have chosen or created.

From now on I can use these filters to limit my search. So lets go to my original search in “Advanced Search”. Unfiltered, search #3 (hirsutism  AND spironolactone) has 197 hits.

When you click on the number of hits you arrive at the results page.
At the right are the filters with the number of results of your search combined with these filters (between brackets).

When you click at the Systematic Reviews link you see the 11 results, most of them very relevant. Filters (except the Custom Filters) can be appended to the search (and thus saved) by clicking the yellow + button.

Each time you do a search (and you’re logged in into My NCBI)  the filtered results are automatically shown at the right.

Clinical Queries zijn vaak handig als je evidence zoekt of een CAT (Critical Appraised Topic) maakt. In de nieuwe versie van PubMed zijn de Clinical Queries echter moeilijker te vinden. Daarom is het handig om bepaalde ‘Clinical Queries’ op te nemen in ‘My NCBI’. Deze queries bevinden zich onder Browse Filters (mogelijkheid onder Search Filters)

Het is ook mogelijk speciale zoekfilters te creëeren, zoals b.v. het Cochrane highly sensitive filter voor RCT’s. Dit kan onder Custom Filters.

Controleer wel via ‘Run Filter” of het filter werkt en sla het daarna op.

Daarna moet je het filter nog activeren door het hokje aan te vinken. Dus je zou alle filters van de ‘Clinical study category’ kunnen opnemen en deze afhankelijk van het domein van de vraag kunnen activeren.

Zo heb je altijd alle filters bij de hand. De resultaten worden automatisch getoond (aan de rechterkant).

Reblog this post [with Zemanta]




PubMed® Redesign [2] News, webcast

4 10 2009

Since last week you can try the redesigned PubMed (see post). There is a link on the PubMed homepage which will connect to a preview version. The direct link to the preview is: http://preview.ncbi.nlm.nih.gov/pubmed.

Notably, the preview version is expected to run for at least two weeks after which the old PubMed will dissapear! (see NLM technical Bulletin, Sept 11) and Twitter). Since the playing time might be very short: start trying the new interface now!

Tried it? Did you fill in the poll: What do you think of the PubMed Redesign?

As announced in the NLM Technical Bulletin (Oct 1) and on Twitter, there will be webcasts Tuesday, Oct 6: 9:00* – 9:30 am, 11:00 – 11:30 am and Wednesday, Oct 7: 1:00  – 1:30 pm, 2:00 pm – 2:30 pm Eastern Time.
(*see here for the corresponding time  in your timezone).

You are advised to read the article, PubMed® Redesign, before you attend the webcast. Only the first 300 participants will be able to attend. However, the Webcasts will later appear at: http://www.nlm.nih.gov/bsd/disted/clinics/pmredesign09.html.

Want to keep uptodate?
Take an email alert or a RSS-feed to NLM technical Bulletin and/or follow @nnlmscr and  @ncbi_pubmed on Twitter.
Twitter Librarians  who may bring you news on the subject: @pfanderson, @shamsha, @alisha764, @uconnhealthlib, @mfenner and sometimes @laikas
Reblog this post [with Zemanta]




PubMed® Redesign is here… to try.

1 10 2009

30-9-2009 23-35-13 pubmed try the redisgned PubmedWe have been waiting months for it, it has been announced several times, we have seen previews, webinars, small changes were introduced over time, till suddenly, today (30-09-09) there was a bright button on the front page of PubMed inviting you to “Try the redesigned PubMed”.

You can click on the button or go directly to http://preview.ncbi.nlm.nih.gov/pubmed.

As a matter of fact @pfanderson already informed fellow tweeting librarians about the PubMed preview link the day before (see Tweet) and within minutes the entire Twitter Librarian community was buzzing about it (see the start of the discussion, taken from Eagledawg post on this subject). People thought that the link was not meant to be public, because it was picked up from a webinar, and no official announcement had been made. But today the new PubMed (first time I see the ®?) is live -although still optional-, accompanied by an official announcement of the redesign at NLM Technichal Bulletin (later followed by a post on Linkout in the PubMed Redesign).

Patricia has depicted the changes in a Powerpoint Slide (see post at Emerging Technologies Librarian: What I most want to be able to find in the new Pubmed.

My take on this:

  • While the front page looks “Functional, clear, ‘modern’” as  someone on Patricia’s blogpost said, I agree with David Rothman, that there are “TONS of wasted screen real estate on that front page”. Why is the search bar hidden at the top?
  • The buttons themselves are relatively easy to find and understand. Although some options like “PubMed Quickstart” are not always straightforward (mistaken for “easy search option” instead of HELP). But that is probably just a matter of getting used to the new design.
  • But what happens if you search: The Details-tab is no longer there and the History is gone. Yes, Limits, Preview/Index, History and Details tabs’ features have been consolidated in Advanced search (see techbull).
  • This means in practice that the front page only lends itself for performing one-search-at-the–time, without being able to check the Details-tab (only indirectly by going to Advanced Search). It needs little imagination to foresee what will happen. Users will type in (“Google”) terms, the combinations of which are inspired by the “Auto suggest” function. There is no way to check the mapping of the words, there is no way to combine MeSH and textwords (unless you know them by head). Basically this search page only lends itself to “quick and dirty searches”, the “One string only”-Google searches. The new PubMed Interface is all about “Serependity”. Some people may like that. I don’t (mostly…).
  • Once done it is easier to save the search or take and RSS-feed (but given the quality of the search…) .
  • No functionalities have gone, all there has been done is replacing the functions. But this can (and in my view) has implications for the functionality of PubMed itself,
  • Thus advanced searchers have to use the “Advanced Search”. But in contrast to the front page this one is full of limits, indexes and bars that should be wisely (and often not) applied. For people searching for evidence this site is not handy at all. In fact, I find it a real nuisance to use.
    I agree with Creaky: some 3rd party tools seem more adequate for beginners/simple searches. But for advanced searches I will move to OVID MEDLINE, for good. Alas I still have to teach my clients and students PubMed. It will be quite a task to see how that can be best done.

So I conclude:

“It is possible that I am about to preach to the choir, but I am going to come right out and say it anyway. I hate PubMed. I hate it with a burning passion. For a site that is as vital to scientific progress as PubMed is, their search engine is shamefully bad. It’s embarrassingly, frustratingly, painfully bad.”

Looks familiar? Anna Kushnir said that almost one and a half year ago... And Anna did get her way. Her ranting elicited a response of Dr. Lipman of the NCBI who reassured her “that a number of changes are underway that will make PubMed work better for her and many other users”. Pubmed is now “easy to use” for people like her. Will there come a PubMed that suits me too?


You may also want to read:

  • Kraftylibrarian: The PubMed redesign is here
  • Pubmed changes at the front door (2009/04/01/)
  • Advanced Neuritis in Pubmed (2009/03/08/)
  • Pubmed: past present and future – part iii: the future (2008/06/27/)
  • Pubmed: past present and future – part-ii/ the present (2008/06/15/)
  • Pubmed: past present and future – part-i/ the past(2008/06/11/)
  • Reblog this post [with Zemanta]





    UpToDate or Dynamed?

    5 07 2009

    Guest author: Shamsha Damani (@shamsha) ;
    Submission for the July Medlib’s Round

    Doctors and other healthcare providers are busy folks. They often don’t have time to go through all the primary literature, find the best evidence, critique it and apply it to their patients in real-time. This is where point-of-care resources shine and make life a bit easier. There are several such tools out there, but the two that I use on a regular basis are UpToDate and DynaMed. There are others like InfoPoems, ACP’s PIER, MD Consult and BMJ’s Point of Care. I often get asked which ones are the best to use and why. The librarian answer to this question: depends on what you are looking for! Not a fair answer I admit, so I wanted to highlight some pros and cons of UpToDate and DynaMed to help you better determine what route to take the next time you find yourself in need of a quick answer to a clinical question.

    UpToDate

    Pros:

    • Comprehensive coverage
    • Easy-to-read writing style
    • The introduction of grading the evidence is certainly very welcome!

    Cons:

    • Expensive
    • Conflict of interest policy a bit perplexing
    • Search feature could use a makeover
    • Remote access at a high premium
    • Not accessible via smart phones
    • They didn’t come to MLA’09 this year and medical librarians felt snubbed (ok, that is not a con, just an observation!)

    DynaMed

    Pros:

    • Bulleted format is easy to read
    • Remote access part of subscription
    • No conflict of interest with authors
    • A lot of the evidence is graded
    • Accessible on PDAs (iPhones and Blackberries included!)

    Cons:

    • The user interface is a bit 1990s and could use a makeover
    • The coverage is not as extensive yet, though they keep adding more topics

    A lot has been written about UpToDate and DynaMed, both in PubMed as well as on various blogs. Jacqueline also did a fabulous post of the evidence-based-ness of UpToDate not too long ago. I used to think that I should pick one and stick to it, but have recently found myself re-thinking this attitude. I think that we need to keep in mind that these are point-of-care tools and should not be utilized as one’s only source of information. Use the tool to get an idea about current evidence and combine it with your own clinical judgment when needed at point-of-care. If suspicious, look up the primary literature the good old way by using MEDLINE or other such databases. A point-of-care database will get you started; however, it is not meant to be a one-stop-shop.

    I can almost hear people saying: so which one do you prefer anyways? That’s like asking me if I prefer Coke or Pepsi. My honest answer: both! (databases as well as beverages!). So what is a busy clinician to do? If you have access to both (or more), spend some time playing with them and see which one you like. Everyone has a different searching and learning style and it is sometimes a matter of preference. DynaMed’s concise structure may be appealing to newbies, whereas seasoned clinicians may prefer UpToDate’s narrative approach. Based on my very unscientific observation of Twitter conversations, it appears that clinicians in general prefer UpToDate whereas librarians prefer DynaMed. Could this be because UpToDate markets heavily to clinicians and snubs librarians? Or could it be the price? Or could it be the age-old debate on what is evidence? I don’t know the answer, partly because I find it all a bit too political. I’ve seen healthcare providers often use Google or Wikipedia for medical answers, which is quite sad. If you are using either UpToDate or DynaMed (or another similar product), you have already graduated to the big leagues and are a true EBM player! So relax and don’t feel like you have to pick a side. I find myself using both on a regular basis; the degree of success I have with each can be gauged by my daily Twitter feed!

    Shamsha Damani





    10 + 1 PubMed Tips for Residents (and their Instructors)

    30 06 2009

    The next Grand Round, the weekly rotating carnival featuring the best medical blog posts, will be hosted by Edwin Leap, a practicing emergency physician. Because the  Grand Rounds are on June 30 -one day before July 1st, which is the traditional start of that thrilling and harrowing journey called ‘residency,’- Edwin decided to make the following theme: ‘What would you like to say to future physicians?’

    I’m sure doctors will give plenty advice on the skills that are most important (i.e., see here). But what advice can I give them? I’m not a doctor. I could give them some examples of “how not to behave”, but I’m sure that will be covered well by fellow patients, and probably also by blogging nurses (i.e. see the perfect Intern Survival Guide by Mother Jones RN).

    So I will stay with my expertise: searching. And to make it workable, I will restrict myself to PubMed, the platform that offers free  access to 18 million citations from MEDLINE and other life science journals. 18 million, that is a tremendous amount of literature! And that is one of the main problems: the sheer amount makes it very difficult to “pick the needle from the haystack”.

    Of course, literature searching is not a primary skill for doctors. It is far more important that a doctor is knowledgeable, handy, and a good communicator (!). But at one time or another, he/she has to look things up or wants to check whether current practice is the best way. Or at the very least, doctors have to stay on top of the best and  latest information. And that’s when they need to search for medical information.

    Below are some tips for beginning as well as more advanced PubMed searchers. Obviously, these are only tips, this post is no tutorial and I give but a few elaborate examples. However, I plan to show entire searches in future posts. Perhaps you can help me by sending in examples or asking questions/propose cases.

    Here are the 10 PubMed tips:

    Tip 1 : Look before you leap.
    Before even thinking of going to the PubMed site, consider whether this is the most obvious source to begin with. First decide whether you have a back- or a foreground question.
    A background question asks for general knowledge and/or “facts” (questions often starting with who, what, when, why, which). “How can one  diagnose appendicitis?” “Which treatments are available for advanced prostate cancer?”  These questions can be best answered by textbooks, handbooks and certain databases. UpToDate is usually perfect for these kind of questions. A question like “Which dose of drug X should I prescribe to a woman of 65 kg with disease Y.” looks very specific, but can generally be looked up in a Pharmacopeia (“general knowledge”). Some subjects are covered by in-house protocols or specialist guidelines.

    When you have a foreground question it is often better (especially for those “new” to the subject) to search evidence in aggregate or pre-filtered resources, like National Guideline Clearinghouse, the TRIP-database and/or the Cochrane Library. This will save you time, because it lowers the number needed to read: individual studies have been sought, selected, appraised and summarized for you.

    Besides PubMed there are also so called 3rd party Pubmed/MEDLINE tools, which can be handy for certain questions or approaches. I’m in the middle of writing about these tools, so keep in touch. Meanwhile you may want to read an excellent overview of many of these tools and more on the blog of Mike Cadogan: Medical search for physicians. Earlier I also wrote about the handy use of PubReminer and GoPubMed to analyze text words and MeSH-terms.

    Although very useful and intuitive, most of these 3rd party PubMed tools don’t have the power of PubMed and are not suitable for elaborate searches.

    Tip 2: A review article from PubMed.
    PubMed can be useful for quickly finding good reviews.
    Below is one such example. A few months ago, Bertalan Mesko (intern then) asked advice on twitter, because his professor had difficulties finding the cause of recurrent acute pancreatitis in a young adult. Considering this was a background question- I just did a quick and dirty search as follows:

    • Go to PubMed: www.pubmed.gov
    • Type acute pancreatitis in the search bar (pancreatitis may be ‘safer’, but will yield more results).
    • Click the Limits Tab and tick off the following options:
      • Facultative: Links to free full text (if you have no subscriptions/access to the medical library)
      • Facultative: Added to PubMed in the last 5 years (or read the first few hits)
      • Subsets: Core Clinical Journals
      • Type of article: Review
      • Tag-Terms: Title.
    • Click: Go

    28-6-2009 12-12-32 PubMed acute pancreatitis  sonder language restr

    So you search for acute pancreatitis in the title of review articles in core (English) clinical Journals. There are just 28 results in the last 5 years, including reviews in the Lancet and NEJM.

    28-6-2009 12-38-45 results acute pancreatitis kort 4 vd 28 its

    The Lancet review gives me a good suggestion:

    In most patients, acute pancreatitis is caused by gallstone obstruction or alcohol, and no genetic testing is needed. However, unexplained recurrent acute pancreatitis might be associated with known genetic mutations in the cationic trypsinogen gene protease serine 1 (PRSS1), SPINK1, or CFTR. Mutations in the PRSS1 gene are seen in most patients with hereditary pancreatitis. In the most frequent mutations, the function of trypsinogen is increased, causing premature enzyme activation and autolysis of acinar cells.

    Note that I didn’t limit on age and I didn’t add recurrent to the search, as I’m looking for a review that may discuss many forms of this disease in all age categories. Recurrent or young adult may not be mentioned in the abstract (nor in the MeSH), so I may miss important overviews if I add these terms to my search.

    If you get to many hits,  you may always narrow the search later.

    A similar approach has been used by drW to search for review articles on heparin induced thrombocytopenia (part 1 and 2).

    UpToDate is a good source as well, ..and clinical experience. Ves Dimov responded on Twitter that he had described a similar hereditary pancreatitis case on his blog.

    Note that at the end of the summer Limits will be under the Advanced Search.#

    Tip 3: PubMed is just one NCBI-database.
    As you may infer from the official web address of PubMed: http://www.ncbi.nlm.nih.gov/, PubMed is (just) one of the (freely available) databases of NCBI (National Center for Biotechnology Information) . If you click on the NCBI-logo (in PubMed) you reach the Entrez cross-database search page. Most databases are particularly suited for genetics, genomics and proteomics. Several of the residents I know are also involved in research  and may make ample use of GEO (gene expression database) and/or other databases.

    If you type for instance acute pancreatitis in the search bar, you see the hits per database, including the PubMed and MeSH database. In this case OMIM seems the most interesting of the genetic databases. OMIM is the “Online Mendelian Inheritance in Man” database. It contains full-text, referenced overviews with information on all known Mendelian disorders and over 12,000 genes and is intended for intended for use primarily by physicians and researchers.

    29-6-2009 2-58-03 NCBI acute pancreatitis

    There are 33 hits in OMIM which we could limit further (using the Limit Tab) to for instance the chromosome. Hits 3-5 describes the genes mentioned in the Lancet review and gives references to relevant studies. There is even an overview of labs performing certain tests (see for instance here)

    29-6-2009 23-53-39 OMIM 12

    In stead of going to Entrez, you can also directly search OMIM from the PubMed database (see Figure)

    30-6-2009 11-18-27 3x oMIM

    Tip 4: Looking up Citations
    One of the recent alterations to PubMed is that you can just type the title in PubMed’s search bar to find a specific article. You can also type in other specifications or an entire reference. But it doesn’t always work. When you type Lancet acute pancreatitis you get too many papers (if you would look for a primary study) but if you copy the following reference from Google: Frossard JL, et al. Acute pancreatitis Lancet 2008; 371(9607) you will get zero results. This is because different Journals have different reference-styles (order, initials, punctuation) and people often make mistakes while citing.

    Another possibility, much loved by librarians because of its versatility, is the Single Citation Matcher in the blue side bar. # You can fill in any field you like and some fields like “author name” have an auto-fill function.

    In this case I searched for the “Author name” Frossard JL (tick “only first author“) and the First page: 143.

    I get exactly 1 paper: the correct one.

    30-6-2009 1-06-47 SCM

    Tip 5: Saving your search and making alerts: RSS and MyNCBI
    It is important for a doctor to keep up with the new developments in your field. There are ample possibilities in PubMed. One is RSS. A previous post descibes how it can be created in PubMed.

    Another possibility is MyNCBI. Old fashioned? Not at all. In PubMed, I find it more useful and easier than RSS. You can find MyNCBI at the upper right or in the blue side bar#.

    You have to create a free account. Once you do that, you can save searches (single searches or set numbers, but NOT the entire Search History). You can immediately save a search after performing it [1] or you can left-click the set number in the History, in this case #14, and a pop-up with options appears [2]

    30-6-2009 2-02-47 save search

    Searches can be saved, and executed/adapted at later timepoints or can be used to create an alert. Alerts can be mailed at any frequency you like. If searches overlap it is good to combine them, so you don’t read the same items twice (or more).

    Other possibilities are: “Save Collections” (individual articles), make filters (see Tip 1) and share them.
    The Save function also works in some other NCBI-databases

    A nice gadget: under preferences, you can activate a highlight function: When logged in, the terms you search for are highlighted in the desired colors. That’s why acute pancreatitis and review are highlighted yellow in the PubMed search shown above.

    For more information see the FAQ

    30-6-2009 2-45-29 myncbi2

    Tip 6: Stop Googling PubMed: why you find too much or too little

    O.k. this is something you may not want to give up, because you’re from the Internet generation and you’re used to intuitive interfaces and searching by trial and error. You’re used to just take a glance at the first few hits out of thousands of records ranked by “relevance” , that exactly match the terms you entered.

    This is not what you should aim for in PubMed: finding a paper because the authors use exactly the same words as you search for – and looking at the first few hits (there is no ranking in PubMed, hits are shown chronologically) do not necessarily mean it is the most relevant to you.  It only matters if the study answers your question (for your particular patient), and if it is of good quality.

    Thus, don’t aim for wording similarities, aim to find the papers that provide you (and your particular patient) with the best answer.

    How do you do that?

    It depends on your question, but generally speaking it is not the best thing to type a whole sentence or the entire PICO in the search bar.

    Usually it is best to search per term and start with the most important term first and leave out the terms that do not really matter.

    So how would you search for the following question?

    Does  spironolactone (anti-androgenic) effectively reduce hirsutism in a female with PCOS? Is it safe and is it comparable to Cyproterone acetate?

    Some people type: PCOS hirsutism spironolactone treatment cyproterone acetate and add gender and age as well. This yields a few results which are on the topic, but yet you may miss the most relevant ones.

    A better way is to search for the two most important concepts: hirsutism AND spironolactone and to look for systematic reviews and RCT’s because these provide the best evidence (see TIP 9). If necessary PCOS can be added afterwards.

    Treatment is usually a superfluous term. It is (usually) better to look for RCT’s or -second best “cohort studies” (because these are the best study designs assessing effectiveness of interventions). Also take care not to apply unnecessary limits.

    Always ask yourself: is this word crucial? And does adding this word/limit reduce the chance that I find a relevant paper?

    Tip 7: Use Details to see how PubMed interpreted (mapped) your search

    Whether you use tip 6 or not, at the very least, check the translation of your search by clicking the Details Tab. Yes, your search is interpreted or ‘mapped’, didn’t you know? That is usually a good thing, because PubMed’s keywords (MeSH) are automatically found, if you use terms that PubMed considers as synonyms for certain MeSH. This can enhance your search, but sometimes the translation is either wrong or you didn’t use the correct word (according to PubMed).

    So if you check the search PCOS hirsutism spironolactone you will see that hirsutism and spironolactone are correctly mapped to a MeSH, whereas PCOS is not. Seeing this you must be alarmed, because it is very likely that there is a MeSH for such a common disease. The correct MeSH is polycystic ovary syndrome. But in this case you might as well leave PCOS from the search.

    30-6-2009 4-05-06 details pcosSometimes your term is wrongly translated. If you search for (early) mobilization (of patients), PubMed will translate this as: “metabolism”[MeSH Terms] (as well as “metabolism”[subheading], that is a qualification of a MeSH term). You can imagine that this may easily result in many irrelevant papers. Rather you should use MesH terms like early ambulation and/or the opposite: immobilization. (How, I will tell you in advanced Tips, to be published later)

    By taking ONE second to check Details you become aware of wrong translations and can do something about it. Exclude the term or modify the search. Or you can see that the translation is ok and leave it like that.

    Tip 8: MeSH or textwords?

    There are people who merely use MeSH and people who swear by textwords. I use them both.

    MeSH are keywords, added by indexers to the record. It would be a pity if you would miss relevant MeSH-terms, because this will inevitably lead to missing relevant articles.

    MeSH are incredibly useful for finding a group of diseases. Suppose I would like to search for the usefulness of exercise to lessen fatigue in cancer patients (no matter which cancer). If I just type cancer in the search bar, this term is not only translated into the MeSH neoplasms, but it is also automatically exploded, which means that all narrower terms (terms lower in hierarchy) are also searched for. Thus papers are found whether they are indexed with neoplasms, lymphoma or breast neoplasms.

    On the other hand, if you use only MeSH you will miss new non-indexed papers or ‘wrongly’ indexed papers, while some terms may not even have an appropriate MeSH.

    Therefore I usually use both MeSH and free textwords.

    In the above example it is sufficient to search for hirsutism AND spironolactone. By checking “Details” you know you’re searching for the right MeSH as well.

    If the MeSH is very different from the textwords you may search for both , thus: in case of early mobilization you may search:

    early ambulation[mh] OR immobilization[mh] OR early mobili* (* means that you truncate the term and find early mobilised/mobilized, moblisation(s) etc. =Note that when you use an asterisk there is no longer any mapping with the MeSH!!).

    Tip 9: Searching for Evidence: Clinical Queries or other search filters

    When u search for the best evidence, Clinical Queries may be very handy. These are prefab search filters that aim to find the best evidence.

    It is best to first search aggregate evidence by using the systematic review filter, which is really much broader because it also searches for reviews of clinical trials, evidence-based medicine and guidelines.

    You just can type some terms in the Search box, but I prefer to make a basic search in PubMed’s main page first (to check the terms) and to fill in the set number, i.e. #9, later. (see Figure)

    30-6-2009 13-33-33 Clinical Queries

    30-6-2009 13-41-52 SR spironolactoneYou only get 10 very relevant hits, including one synthesis of evidence in Clinical Evidence, several Cochrane Reviews and other systematic reviews. Since these are all very recent papers you may decide to stop here.
    If you like, you can check for individual trials as well by searching by Clinical Study Category (choose the default: therapy narrow and enter search #9 again). This gives 24 hits.

    One word of caution: Not all filters are that good. The Systematic Review Filter and the Narrow Therapy filter are quite good for a quick search though. Tip: you can adapt the filters yourselves.

    Tip 10: Search Logic (and Boolean operators)

    What do you think you search for if you type: hand OR arm AND foot?

    You probably mean to search for (hand or arm) AND foot, but Pubmed follows another logic, depending on the order of the words. In this case it puts (invisible) brackets round arm and foot, not hand or arm. Result: you find far more (irrelevant) articles, because you retrieve every(!) article using the word “hand” and a few extra with (arm and foot).

    29-6-2009 1-44-31 hand foot pubmedYou can keep it under control by placing the brackets yourselves.
    With complex searches I rather combine synonyms with OR and  concepts with AND using the history. It looks like this:

    30-6-2009 14-17-36 foot arm history(when you don’t add operators PubMed uses the AND-operator, thus #8 #9 means #8 AND #9).
    You can add another term to the search as well, or apply a clinical query or limit. The final search you can save in MyNCBI. It shows the search with the appropriate brackets when you execute it.

    Besides OR (synonyms) and AND (narrowing) you have the boolean operator NOT.

    Please, generally do not use NOT to get rid of articles that are irrelevant, but rather try to select positively. Why? Because by using NOT you might exclude relevant articles.

    Suppose you want to find articles about nosebleeds in children by using NOT adults. Then you also exclude articles about adults AND children.

    NOT can be very handy however to subtract searches from each other. Suppose you have screened 100 articles (#1) and you get a brilliant idea using another word, which gives set #5. You can go through 120 articles, but you can also subtract the two searches from each other: #5 NOT #1 : and you only have to check 20 instead of 12o records.

    Extra Tip (10+1): Use your library and librarian

    As an extra tip, this final and probably most useful tip.
    Follow library courses if you didn’t do so already during our internship, ask your librarian to help setting up a search for an automatic alert and to deduplicate results from different databases (i.e. MEDLINE and EMBASE) and ask the help of your librarian if you want to perform exhaustive or difficult searches or if you just want some advice. It is no shame to go to your librarian. We’re there for you.

    Let me end with a statement of a fellow librarian (Suzanne Bakker, freely adapted):

    “Doctors learn what a Hb-test is, but that doesn’t mean that they have to do the lab test themselves, each time a patient needs a test?! The same applies to searching. It is good that doctors learn the basic stuff, and understand some pitfalls, but they need not become information specialist”

    You don’t need to become an information specialist to become a very good doctor…

    ———————————————–

    #Some functionalities may move from the current page (tabs and blue side bar) to the advanced search this summer

    Note: Thanks to Edwin Leap who had the patience to await my post, while it was going out of hand and getting much bigger than intended…