Grey Literature: Time to make it systematic

Guest author: Shamsha Damani (@shamsha)

Grey literature is a term I first encountered in library school; I remember dubbing it “the-wild-goose-chase search” because it is time consuming, totally un-systematic, and a huge pain altogether. Things haven’t changed much in the grey literature arena, as I found out last week, when my boss asked me to help with the grey literature part of a systematic review.

Let me back up a bit and offer the official definition for grey literature by the experts of the Grey Literature International Steering Committee: “Information produced on all levels of government, academics, business and industry in electronic and print formats not controlled by commercial publishing i.e. where publishing is not the primary activity of the producing body.” Grey literature can include things such as policy documents, government reports, academic papers, theses, dissertations, bibliographies, conference abstracts/proceedings/papers, newsletters, PowerPoint presentations, standards/best practice documents, technical specifications, working papers and more! (Benzies et al 2006). So what is so time consuming about all this? There is no one magic database that will search all these at once. Translation: you have to search a gazillion places separately, which means you have to learn how to search each of these gazillion websites/databases separately. Now if doing searches for systematic reviews is your bread-and-butter, then you are probably scoffing already. But for a newbie like me, I was drowning big time.

After spending what seemed like an eternity to finish my search, I went back to the literature to see why inclusion of grey literature was so important. I know that grey literature adds to the evidence base and results in a comprehensive search, but it is often not peer-reviewed, and the quality of some of the documents is often questionable. So what I dug up was a bit surprising. The first was a Cochrane Review from 2007 titled “Grey literature in meta-analyses of randomized trials of health care interventions (review).” The authors concluded that not including grey literature in meta-analyses produced inflated results when looking at treatment effects. So the reason for inclusion of grey literature made sense: to reduce publication bias. Another paper published in the Bulletin of the World Health Organization concluded that grey literature tends to be more current, provides global coverage, and may have an impact on cost-effectiveness of various treatment strategies. This definitely got my attention because of the new buzzword in Washington: Comparative Effectiveness Research (CER). A lot of the grey literature is comprised of policy documents so it definitely has a big role to play in systematic reviews as well. However, the authors also pointed out that there is no systematic way to search the grey literature and undertaking such a search can be very expensive and time consuming. This validated my frustrations, but gave no solutions.

When I was struggling to get through my search, I was delighted to find a wonderful resource from the Canadian Agency for Drugs and Technologies in Health. They have created a document called “Grey Matters: A Practical Search Tool for Evidence-Based Medicine”, which is a 34-page checklist of many of the popular websites for searching grey literature, including a built-in documentation system. It was still tedious work because I had to search a ton of places, many resulting in no hits. But at least I had a start and a transparent way of documenting my work.

However, I’m still at a loss for why there are no official guidelines for librarians to search for grey literature. There are clear guidelines for authors of grey literature. Benzies and colleagues give compelling reasons for inclusion of grey literature in a systematic review, complete with a checklist for authors! Why not have guidelines for searching too? I know that every search would require different tools; but I think that a master list can be created, sort of like a must-search-these-first type of a list. It surely would help a newbie like me. I know that many libraries have such lists but they tend to be 10 pages long, with bibliographies for bibliographies! Based on my experience, I would start with the following resources the next time I encounter a grey literature search:

1. National Guideline Clearinghouse
2. Centre for Reviews and Dissemination
3. Agency for Healthcare Research and Quality (AHRQ)
4. Health Technology Assessment International (HTAI)
5. Turning Research Into Practice (TRIP)

Some databases like Mednar, Deep Dyve, RePORTer, OAIster, and Google Scholar also deserve a mention but I have not had much luck with them. This is obviously not meant to be an exhaustive list. For that, I present my delicious page: http://delicious.com/shamsha/greylit, which is also ever-growing.

Finally, a request for the experts out there: if you have any tips on how to make this process less painful, please share it here. The newbies of the world will appreciate it.

Shamsha Damani

Clinical Librarian

Related articles by Zemanta

• Grey Matters: a practical tool for evidence-based searching (openmedicine.ca)
• Three NEJM Perspective pieces on CER (doctorrw.blogspot.com)

Advertisement