A Filter for Finding “All Studies on Animal Experimentation in PubMed”

29 09 2010

ResearchBlogging.orgFor  an introduction to search filters you can first read this post.

Most people searching PubMed try to get rid of publications about animals. But basic scientists and lab animal technicians just want to find those animal studies.

PubMed has built-in filters for that: the limits. There is a limit  for “humans” and a limit for “animals”. But that is not good enough to find each and every article about humans, respectively animals. The limits are MeSH, Medical Subject Headings or index-terms and these are per definition not added to new articles, that haven’t been indexed yet. To name the main disadvantage…
Thus to find all papers one should at least search for other relevant MeSH and textwords (words in title and abstract) too.

A recent paper published in Laboratory Animals describes a filter for finding “all studies on animal experimentation in PubMed“, to facilitate “writing a systematic review (SR) of animal research” .

As the authors rightly emphasize, SR’s are no common practice in the field of animal research. Knowing what already has been done can prevent unnecessary duplication of animal experiments and thus unnecessary animal use. The authors have interesting ideas like registration of animal studies (similar to clinical trials registers).

In this article they describe the design of an animal filter for PubMed. The authors describe their filter as follows:

“By using our effective search filter in PubMed, all available literature concerning a specific topic can be found and read, which will help in making better evidencebased decisions and result in optimal experimental conditions for both science and animal welfare.”

Is this conclusion justified?

Design of the filter

Their filter is subjectively derived: the terms are “logically” chosen.

[1] The first part of the animal filter consists of only MeSH-terms.

You can’t use animals[mh] (mh=Mesh) as a search term, because MeSH are automatically exploded in PubMed. This means that narrower terms (lower in the tree) are also searched. If “Animals” were allowed to explode, the search would include the MeSH, “Humans”, which is at the end of one tree (primates etc, see Fig. below)

Therefore the MeSH-parts of their search consists of:

  1. animals [mh:noexp]: only articles are found that are indexed with “animals”, but not its narrower terms. Notably, this is identical to the PubMed Limit: animals).
  2. Exploded Animal-specific MeSH-terms not having humans as a narrow term, i.e. “fishes”[MeSH Terms].
  3. and non-exploded MeSH in those cases that humans occurred in the same branch. Like “primates”[MeSH Terms:noexp]
  4. In addition two other MeSH are used: “animal experimentation”[MeSH Terms] and “models, animal”[MeSH Terms]

[2] The second part of the search filter consist of terms in the title and abstract (command: [tiab]).

The terms are taken from relevant MeSH, two reports about animal experimentation in the Netherlands and in Europe, and the experience of the authors, who are experts in the field.

The authors use this string for non-indexed records (command: NOT medline[sb]). Thus this part is only meant to find records that haven’t (yet) been indexed, but in which (specific) animals are mentioned by the author in title or text. Synonyms and spelling variants have been taken into account.

Apparently the authors have chosen NOT to search for text words in indexed records only. Presumably it gives too much noise, to search for animals mentioned in non-indexed articles. However, the authors do not discuss why this was necessary.

This search string is extremely long. Partly because truncation isn’t used with the longer words: i.e. nematod*[tiab] instead of nematoda[Tiab] OR nematode[Tiab] OR nematoda[Tiab] OR nematode[Tiab] OR nematodes[Tiab]. Partly because they aim for completeness. However the usefulness of the terms as such hasn’t been verified (see below).

Search strategies can be freely accessed here.

Validation

The filter is mainly validated against the PubMed Limit “Animals”.

The authors assume that the PubMed Limits are “the most easily available and most obvious method”. However I know few librarians or authors of systematic reviews who would solely apply this so called ‘regular method’. In the past I have used exactly the same MeSH-terms (1) and the main text words (2) as included in their filter.

Considering that the filter includes the PubMed limit “Animals” [1.1] it does not come as a surprise that the sensitivity of the filter exceeds that of the PubMed limit Animals…

Still, the sensitivity (106%) is not really dramatic: 6% more records are found, the PubMed Limit “animals” is set as 100%.

Apparently records are very well indexed with the MeSH “animals”. Few true animal records are missed, because “animals” is a check tag. A check tag is a MeSH that is looked for routinely by indexers in every journal article. It is added to the record even if it isn’t the main (or major) point of an article.

Is an increased sensitivity of appr. 6% sufficient to conclude that this filter “performs much better than the current alternative in PubMed”?

No. It is not only important that MORE is found but to what degree the extra hits are relevant. Surprisingly, the authors ONLY determined SENSITIVITY, not specificity or precision.

There are many irrelevant hits, partly caused by the inclusion of animal population groups[mesh], which has some narrower terms that often not used for experimentation, i.e. endangered species.

Thus even after omission of animal population groups[mesh], the filter still gives hits like:

These are evidently NOT laboratory animal experiments and mainly caused by the inclusion invertebrates  like plankton.

Most other MeSH are not extremely useful either. Even terms as animal experimentation[mh] and models, animal[mh] are seldom assigned to experimental studies lacking animals as a MeSH.

According to the authors, the MeSH “Animals” will not retrieve studies solely indexed with the MeSH term Mice. However, the first records missed with mice[mesh] NOT animals[mh:noexp] are from 1965, when they apparently didn’t use “animals” as a check tag in addition to specific ‘animal’ MeSH.

Thus presumably the MeSH-filter can be much shorter and need only contain animal MeSH (rats[mh], mice[mh] etc) when publications older than 1965 are also required.

The types of vertebrate animals used in lab re...

Image via Wikipedia

Their text word string (2) is also extremely long.  Apart from the lack of truncation, most animal terms are not relevant for most searches. 2/3 of the experiments are done with rodents (see Fig). The other animals are often used for specific experiments (zebra-fish, Drosophila) or in another context, not related to animal experiments, such as:

swine flu, avian flu, milk production by cows, or allergy by milk-products or mites, stings by insects and bites by dogs and of course fish, birds, cattle and poultry as food, fetal calf serum in culture medium, but also vaccination with “mouse products” in humans. Thus most of the terms produce noise for most topics. An example below (found by birds[mesh] 🙂

On the other hand strains of mice and rats are missing from the search string: i.e. balb/c, wistar.

Extremely long search strings (1 page) are also annoying to use. However, the main issue is whether the extra noise matters. Because the filter is meant to find all experimental animal studies.

As Carlijn Hooijmans notices correctly, the filters are never used on their own, only in combination with topic search terms.

Hooijmans et al have therefore “validated” their filter with two searches. “Validated” between quotation marks because they have only compared the number of hits, thus the increase in sensitivity.

Their first topic is the use of probiotics in experimental pancreatitis (see appendix).

Their filter (combined with the topic search) retrieved 37 items against 33 items with the so called “regular method”: an increase in sensitivity of 21,1%.

After updating the search I got  38 vs 35 hits. Two of the 3 extra hits obtained with the broad filter are relevant and are missed with the PubMed limit for animals, because the records haven’t been indexed. They could also have been found with the text words pig*[tiab] or dog*[tiab]. Thus the filter is ok for this purpose, but unnecessary long. The MeSH-part of the filter had NO added value compared to animals[mh:noexp].

Since there are only 148 hits without the use of any filters, researchers could also use screen all hits. Alternatively there is a trick to safely exclude human studies:

NOT (humans[mh] NOT animals[mh:noexp])

With this double negation you exclude PubMed records that are indexed with humans[mh], as long as these records aren’t indexed with animals[mh:noexp] too. It is far “safer” than limiting for “animals”[mesh:noexp] only. We use a similar approach to ” exclude”  animals when we search for human studies.

This extremely simple filter yields 48 hits, finding all hits found with the large animal filter (plus 10 irrelevant hits).

Such a simple filter can easily be used for searches with relatively few hits, but gives too many irrelevant hits in case of  a high yield.

The second topic is food restriction. 9280 Records were obtained with the Limit: “Animals”, whereas this strategy combined with the complete filter retrieved 9650 items. The sensitivity in this search strategy was therefore 104%. 4% extra hits were obtained.

The MeSH-search added little to the search. Only 21 extra hits. The relevant hits were (again) only from before 1965.

The text-word part of the search finds relevant new articles, although there are quite some  irrelevant findings too, i.e. dieting and obtaining proteins from chicken.

4% isn’t a lot extra, but the aim of the researchers is too find all there is.

However, it is the question whether researchers want to find every single experiment or observation done in the animal kingdom. If I were to plan an experiment on whether food restriction lowers the risk for prostate cancer in a transgenic mice, need I know what the effects are of food restriction on Drosophila, nematodes, salmon or even chicken on whatever outcome? Would I like to screen 10,000 hits?

Probably most researchers would like separate filters for rodents and other laboratory animals (primates, dogs) and for work on Drosophila or fish. In some fields there might also be a need to filter clinical trials and reviews out.

Furthermore, it is not only important to have a good filter but also a good search.

The topic searches in the current paper are not ideal: they contain overlapping terms (food restriction is also found by food and restriction) and misses important MeSH (Food deprivation, fasting and the broader term of caloric restriction “energy intake” are assigned more often to records about food deprivation than caloric restriction).

Their search:

(“food restriction”[tiab] OR (“food”[tiab] AND “restriction”[tiab]) OR “feed restriction”[tiab] OR (“feed”[tiab] AND “restriction”[tiab]) OR “restricted feeding”[tiab] OR (“feeding”[tiab] AND “restricted”[tiab]) OR “energy restriction”[tiab] OR (“energy”[tiab] AND “restriction”[tiab]) OR “dietary restriction”[tiab] OR (dietary”[tiab] AND “restriction”[tiab]) OR “caloric restriction”[MeSH Terms] OR (“caloric”[tiab] AND “restriction”[tiab]) OR “caloric restriction”[tiab])
might for instance be changed to:

Energy Intake[mh] OR Food deprivation[mh] OR Fasting[mh] OR food restrict*[tiab] OR feed restrict*[tiab] OR restricted feed*[tiab] OR energy restrict*[tiab] OR dietary restrict*[tiab] OR  caloric restrict*[tiab] OR calorie restrict*[tiab] OR diet restrict*[tiab]

You do not expect such incomplete strategies from people who repeatedly stress that: most scientists do not know how to use PubMed effectively” and that “many researchers do not use ‘Medical Subject Headings’ (MeSH terms), even though they work with PubMed every day”…..

Combining this modified search with their animal filter yields 21920 hits instead of 10335 as found with their “food deprivation” search and their animal filter. A sensitivity of 212%!!! Now we are talking! 😉 (And yes there are many new relevant hits found)

Summary

The paper describes the performance of a subjective search filter to find all experimental studies performed with laboratory animals. The authors have merely “validated”  this filter against the Pubmed Limits: animals. In addition, they only determined sensitivity:  on average 7% more hits were obtained with the new animal filter than with the PubMed limit alone.

The authors have not determined the specificity or precision of the filter, not even for the 2 topics where they have applied the filter. A quick look at the results shows that the MeSH-terms other than the PubMed limit “animals” contributed little to the enhanced sensitivity. The text word part of the filter yields more relevant hits. Still -depending on the topic- there are many irrelevant records found, because  it is difficult to separate animals as food, allergens etc from laboratory animals used in experiments and the filter is developed to find every single animal in the animal kingdom, including poultry, fish, nematodes, flies, endangered species and plankton. Another (hardly to avoid) “contamination” comes from in vitro experiments with animal cells, animal products used in clinical trials and narrative reviews.

In practice, only parts of the search filter seem useful for most systematic reviews, and especially if these reviews are not meant to give an overview of all findings in the universe, but are needed to check if a similar experiment hasn’t already be done. It seems impractical if researchers have to make a systematic review, checking, summarizing and appraising  10,000 records each time they start a new experiment.

Perhaps I’m somewhat too critical, but the cheering and triumphant tone of the paper in combination with a too simple design and without proper testing of the filter asked for a critical response.

Credits

Thanks to Gerben ter Riet for alerting me to the paper. He also gave the tip that the paper can be obtained here for free.

References

  1. Hooijmans CR, Tillema A, Leenaars M, & Ritskes-Hoitinga M (2010). Enhancing search efficiency by means of a search filter for finding all studies on animal experimentation in PubMed. Laboratory animals, 44 (3), 170-5 PMID: 20551243

———————-

Advertisements




Thoughts on the PubMed Clinical Queries Redesign

7 07 2010

Added 2010-07-09:  It is possible to enter the set numbers again, but the results are not yet reliable. They are probably working on it.

Last Wednesday (June 30th 2010) the PubMed Clinical Queries were redesigned.

Clinical Queries are prefab search filters that enable you to find aggregate evidence (Systematic Reviews-filter) or articles in a certain domain (Clinical study category-filters: like diagnosis and therapy), as well as papers in the field of  Medical Genetics (not shown below).

This was how it looked:

Since there were several different boxes you had to re-enter your search each time you tried another filter.

Now the Clinical Queries page has been reconfigured with columns to preview the first five citations of the results for all three research areas.

So this is how it looks now (search= PCOS spironolactone cyproterone hirsutism (PubMed automatically connects with “AND”))

Click to enlarge

Most quick responses to the change are “Neat”, “improved”, “tightened up”…….

This change might be a stylistic improvement for those who are used to enter words in the clinical queries without optimizing the search. At least you see “what you get”, you can preview the results of 3 filters, and you can still see “all” results by clicking on “see all”.  However, if you want to see the all results of another filter, you still have to go back to the clinical queries again.

But… I was not pleased to notice that it is no longer possible to enter a set number (i.e. #9) in the clinical queries search bar.

….Especially since the actual change was just before the start of an EBM-search session. I totally relied on this feature….

  1. Laika (Jacqueline)
    laikas Holy shit. #Pubmed altered the clinical queries, so that I can’t optimize my search first and enter the setnumber in the clin queries later.
  2. Laika (Jacqueline)
    laikas Holy shit 2 And I have a search class in 15 minutes. Can’t prepare changes. I hate this #pubmed #fail
  3. Mark MacEachern
    markmac perfect timing (for an intrface chnge) RT @laikas Holy shit 2 And I have a search class in 15 min. Can’t prepare changes. #pubmed #fail

this quote was brought to you by quoteurl

Furthermore the clinical study category is now default on “therapy broad” instead of narrow. This means a lot more noise: the broad filter searches for (all) clinical trials, while the narrow filter is meant to find randomized controlled trials only.

Normally I optimize the search first before entering the final search set number into the clinical queries.(see  Tip 9 of  “10+1 Pubmed tips for residents and their instructors“).  For instance, the above search would not include PCOS (which doesn’t map to the proper MeSH and isn’t required) and cyproterone, but would consist of hirsutism AND spironolactone (both mapping to the appropriate MeSH).

The set number of the “optimized” search is then entered in the search box of the Systematic Review filter. This yields 9 more hits, including Cochrane systematic reviews. The narrow therapy filter gives more hits, that are more relevant as well (24).

The example that is shown in the NLM technical bulletin (dementia stroke) yields 142 systematic reviews and 1318 individual trials of which only the 5 most recent trials are shown. Not very helpful to doctors and scientists, IMHO.

Anyway, we “lost” a (roundabout) way- to optimize the search before entering it into the search box.

The preview of 3 boxes is o.k., the looks are o.k. but why is this functionality lost?

For the moment I decided to teach my class another option that I use myself: adding clinical queries to your personal NCBI account so that the filters show up each time you perform a search in PubMed ( this post describes how to do it).

It only takes some time to make NCBI accounts and to explain the procedure to the class, time you would like to save for the searches themselves  (in a 1-2 hr workshop). But it is the most practical solution.

We notified PubMed, but it is not clear whether they plan to restore this function.

Note: 2010-07-09:  It is possible to enter the set numbers again, but the results are not yet reliable. They are probably working on it.

Still, for advanced users, adding filters to your NCBI may be most practical.

——-

*re-entering spironolactone and hirsutism in the clinical queries is doable here, but often the search is more complex and different per filter. For instance I might add a third concept when looking for an individual trial.





Adding Methodological Filters to MyNCBI

26 11 2009

Idea: Arnold Leenders
Text: “Laika”

Methodological Search Filters can help to narrow down a search by enriching for studies with a certain study design or methodology. PubMed has build-in methodological filters, the so called Clinical Queries for domains (like therapy and diagnosis) and for evidence based papers (like theSystematic Review subset” in Pubmed). These searches are often useful to quickly find evidence on a topic or to perform a CAT (Critical Appraised Topic). More exhaustive searches require broader  filters not incorporated in PubMed. (See Search Filters. 1. An Introduction.).

The Redesign of PubMed has made it more difficult to apply Clinical Queries after a search has been optimized. You can still go directly to the clinical queries (on the front page) and fill in some terms, but we rather advise to build the strategy first, check the terms and combine your search with filters afterwards.

Suppose you would like to find out whether spironolactone effectively reduces hirsutism in a female with PCOS (see 10+ 1 Pubmed Tips for Residents and their Instructors, Tip 9). You first check that the main concepts hirsutism and spironactone are o.k. (i.e. they map automatically with the correct MeSH). Applying the clinical queries at this stage would require you to scroll down the page each time you use them.

Instead you can use filters in My NCBI for that purpose. My NCBI is your (free) personal space for saving searches, results, PubMed preferences, for creating automatic email alerts and for creating Search Filters.
The My NCBI-option is at the upper right of the PubMed page. You first have to create a free account.

To activate or create filters, go to [1] My NCBI and click on [2] Search Filters.

Since our purpose is to make filters for PubMed, choose [3] PubMed from the list of NCBI-databases.

Under Frequently Requested Filters you find the most popular Limit options. You can choose any of the optional filters for future use. This works faster than searching for the appropriate limit each time. You can for instance use the filter for humans to exclude animals studies.

The Filters we are going to use are under “Browse Filters”, Subcategory Properties….

….. under Clinical Queries (Domains, i.e. therapy) and Subsets (Systematic Review Filters)

You can choose any filter you like. I choose the Systematic Review Filter (under Subsets) and the Therapy/Narrow Filter under  Clinical Queries.

In addition you can add custom filters. For instance you might want to add a sensitive Cochrane RCT filter, if you perform broad searches. Click Custom Filters, give the filter a name and copy/paste the search string you want to use as filter.

Control via “Run Filter” if the Filter works (the number of hits are shown) and SAVE the filter.

Next you have to activate the filters you want to use. Note there is a limit of five 15 filters (including custom filters) that can be selected and listed in My Filters. [edited: July 5th, hattip Tanya Feddern-Bekcan]

Under  My Filters you now see the Filters you have chosen or created.

From now on I can use these filters to limit my search. So lets go to my original search in “Advanced Search”. Unfiltered, search #3 (hirsutism  AND spironolactone) has 197 hits.

When you click on the number of hits you arrive at the results page.
At the right are the filters with the number of results of your search combined with these filters (between brackets).

When you click at the Systematic Reviews link you see the 11 results, most of them very relevant. Filters (except the Custom Filters) can be appended to the search (and thus saved) by clicking the yellow + button.

Each time you do a search (and you’re logged in into My NCBI)  the filtered results are automatically shown at the right.

Clinical Queries zijn vaak handig als je evidence zoekt of een CAT (Critical Appraised Topic) maakt. In de nieuwe versie van PubMed zijn de Clinical Queries echter moeilijker te vinden. Daarom is het handig om bepaalde ‘Clinical Queries’ op te nemen in ‘My NCBI’. Deze queries bevinden zich onder Browse Filters (mogelijkheid onder Search Filters)

Het is ook mogelijk speciale zoekfilters te creëeren, zoals b.v. het Cochrane highly sensitive filter voor RCT’s. Dit kan onder Custom Filters.

Controleer wel via ‘Run Filter” of het filter werkt en sla het daarna op.

Daarna moet je het filter nog activeren door het hokje aan te vinken. Dus je zou alle filters van de ‘Clinical study category’ kunnen opnemen en deze afhankelijk van het domein van de vraag kunnen activeren.

Zo heb je altijd alle filters bij de hand. De resultaten worden automatisch getoond (aan de rechterkant).

Reblog this post [with Zemanta]




Training: ‘Getting The Best Out Of Search Filters’

24 01 2009

Information Specialists, other information professionals and researchers seeking more insight into the usefulness of search filters might be interested in the following training event:

YHEC Training Event: ‘Getting The Best Out Of Search Filters’

University of York, 26 February 09 or UK Cochrane Centre, Oxford, 04 March 09

This training event will explore how to identify, critically appraise and test out search filters, focusing on health and social care.

The training day presenters will be Julie Glanville (Project Director, Information Services, York Health Economics Consortium, University of York) and Carol Lefebvre (Senior Information Specialist at the UK Cochrane Centre).

By the end of the study day, participants will have:

  • An awareness of how to identify published search filters;
  • An understanding of the features of search filter design to be able to critically appraise search filters;
  • An awareness of the key issues to be considered in assessing the suitability of search filtesr for specific questions;
  • An understanding of the challenges of translating search filters between interfaces and databases.

For more information see here To book a place please click here.

An overview of other training events can be found here

Unfortunately, both of these courses are now fully booked. YHEC will be running a further course later in the year (perhaps in November), and they may be running a course in the Netherlands in due course as well. You can contact ab588@york.ac.uk to be put on a mailing list to be kept informed of these courses.





Search Filters. 1. An Introduction

22 01 2009

I’ll be writing a lot about search filters in the near future. Before I do, I think it would be useful to give an introduction.

First I want to stress that this series will not deal with Google, Twitter etc. search filters. Although I might write about such filters on another occasion, this series is about filters for biomedical bibliographic databases, such as MEDLINE (PubMed) or EMBASE.

Below is a short Dutch presentation on search filters I gave at a symposium on search filters in 2005.[1] (some slides don’t show well in Slideshare)

What are search filters? [1-5]bmj-filters

Search filters are predefined and pretested search queries designed to retrieve selections of records in specified electronic information sources. Usually they are created by librarians, but they can be run by clinicians and researchers as well.

Why are search filters useful?

It is increasingly difficult, especially for the busy clinician, to find the information he/she wants in a database like PubMed. Filters can help to narrow down the search. In this way you can reduce the number needed to read (i.e. in a quick search) and/or increase the number of relevant papers (i.e. for a systematic review).

Different classes of filters:

  • Subject vs Methodological Filters
  • Sensitive vs Specific Filters
    • Sensitive filters are broad filters, designed to find as much relevant papers as possible, often at the cost of much ‘noise’.
    • Specific filters are designed to find a small set of very relevant papers, with the risk of omission of a considerable number of relevant papers.
  • Short vs Long Filters
    Filters may be simple -even consisting of one single term- or may be complex. They can comprise keywords, text words or both.
  • Database and search-interface dependency
    Filters are usually designed for a specific database and interface. Not every filter that has been developed can be directly translated into another dat
    abase/platform because of different
    keywords terms, hierarchy of keywords, structure and commands. For instance in EMBASE Case Control Study (which is not strictly a controlled study) is a Narrower term of Controlled Study, together with Controlled Clinical Trial. In MEDLINE Case-Control Studies‘ is considered an epidemiologic study, whereas Controlled Clinical Trials is a Publication Type. Note that the MESH is in the plural form: ‘Case-Control Studies
    The OVID platform allows separate searching of the abstract field (command: .ab.), whereas PubMed has no separate command for this ([tiab], title and abstract). Adjacency searching and non-explosion of subheadings (qualifiers of a MeSH term) is also possible in OVID, but not in PubMed.
    controlled-clinical-trial-embase-medline-90
  • Time dependency
    The performance of a filter may change over time because other terms may prevail or database keywords may have been added, removed or changed. In PubMed for instance the MeSH Randomized controlled trial has been changed in the MeSH Randomized controlled trial as a topic.
  • Subjective vs Objective Filters.[1,2]
    • The 1st generation of filters are subjectively derived, based on the expertise of the searcher.
    • 2nd generation is also subjectively derived, but then tested and validated against a gold standard, i.e. a known set of relevant records, to determine the effectiveness of the filter at retrieving relevant records.
    • 3rd generation involve objective approaches to filter design (e.g. frequency analysis or logistic regression). Search filter is tested on an independent set of known relevant records (gold standard).
      Whether filters are broadly applicable to different clinical areas will depend on the choice of the golden standard (subject, publication year, size) and the presence and composition of an extern valididity standard (a set of records different from the records used to develop the filter, against which the developed filter is tested)

Performance measures (for 2nd and 3rd generation filters), Figure adapted from [9]

Sensitivity: The number of relevant records retrieved by the search filter as a proportion of the total number of records in the gold standard.(A/A+C)
Specificity: The number of records that are not relevant and are not retrieved as a proportion of the total number of records.(D/D+B)
Precision: (
or positive predictive value), fraction of returned positives that are true positives. (A/(A+B)

2x2-search-filter

Is there any difference between search filters and limits?

Not really, both are search terms that can be used to narrow the search. In PubMed however limits usually consist of ONE single MeSH-term (MEDLINE Subject Headings, i.e. key words assigned by MEDLINE-indexers), thus these limits will miss recent papers that have not been indexed by MEDLINE. Therefore it is often safer to use a broader filter or no limit at all.

For instance consider the search tinnitus AND behavioral treatment (set #1 in the Fig. below). This yields 175 hits.
Most people, if they want to find the best individual studies limit for RCT, i.e. the subject search is combined (ANDed) with “randomized controlled trial[Publication Type]”. This yields 23 hits (#2).
However if they would have combined ther search with the narrow therapy-filter (#3) they would have found 3 extra hits, two of which being RCT’s, but still in process and not indexed:

1: Weise C et al Biofeedback-based behavioral treatment for chronic tinnitus: results of a randomized controlled trial.J Consult Clin Psychol. 2008 Dec;76(6):1046-57.
2: Kaldo V, et al Internet versus group cognitive-behavioral treatment of distress associated with tinnitus: a randomized controlled trial.
Behav Ther. 2008 Dec;39(4):348-59. Epub 2008 Apr 20.

The narrow therapy filter has found these very recent articles, because it not only searches for randomized controlled trial[Publication Type], but also for randomized AND controlled AND trial in title and abstract.
The broad therapy filter (#4) searches for clinical trials in general and finds many clinical trials that are not RCT’s.
The narrow (#5) and broad (#6) Cochrane RCT-filters are highly sensitive search strategies meant to identify randomized controlled filters for a Cochrane Review (see 10).

Thus for a quick search for relevant papers search #3 (narrow therapy filter) is most optimal.

search-met-diverse-filters

References

  1. Limpens J [ppt] Introductie Zoekfilters 2005 (Dutch)
  2. Booth A (Scharr) [ppt] Quality Search Filters at http://www.le.ac.uk/li/lgh/library/ABooth.ppt
  3. Jenkins M. Evaluation of methodological search filters–a review. Health Info Libr J. 2004 Sep;21(3):148-63.
  4. Glanville J, Bayliss S, Booth A et al So many filters, so little time: the development of a search filter appraisal checklist.J Med Libr Assoc. 2008 Oct;96(4):356-61.
  5. UBC Health LiBrary wiki: Systematic_review_searching and Filters (ie.hedges)
  6. Haynes RB, Wilczynski N, McKibbon, KA et al. Developing optimal search strategies for detecting clinically sound studies in Medline. J Am Med Inform Assoc. 1994 Nov-Dec;1(6):447-58.
  7. PubMed Clinical Queries
  8. Systematic review subset in PubMed
  9. McKibbon, KA, Wilczynski, NL, Haynes, RB Retrieving randomized controlled trials from medline: a comparison of 38 published search filters. Health Info Libr J. 2008.
  10. Post: New Cochrane Handbook: Altered Search Policies.