P C R : Publish, Cite Cyagen and get a Reward of $100 or more !?

15 08 2015

Thursday I got an odd mail from Cyagen.

Cyagen offered me $100 or more in reward for citing their animal model services in an article. Even more surprising the reward was dependent on the impact factor of the journal I was publishing in.

Cyagen gives the following example:

If you published a paper in Science (IF = 30) and cite Cyagen Biosciences, you will be entitled to a voucher with a face value of $3,000 upon notification of the publication (PMID).

Thus the higher the impact factor the higher the reward.

CYAGEN 1 2015-08-15_14-51-18

Click to enlarge

First I thought this was link bait, because the mail was send from vresp.com. (spam reputation) and not cyagen.com.

But tweets from @doctorsdilemma and @PhytophthoraLab pointed me to this advertorial at Cyagen’s website.

It was real.

CYAGEN 2 2015-08-15_14-49-56

This doesn’t feel right, because it “smells like a conflict of interest”.

In clinical research we all know where conflicts of interest can lead to: biased results. Or according to Corollary 5 in the famous Iaonnides paper “Why Most Published Research Findings Are False”:

“The greater the financial and other interests and prejudices in a scientific field, the less likely the research findings are to be true.”

We expressed our concerns on Twitter but it took a while for Cyagen to respond to our concern (see Facebook-Q&A *).

Meanwhile Ben Goldacre from BadScience and Retraction Watch wrote a post about Cyagen’s “pay-for-citation program”.

Ben Goldacre emphasizes the COI-problem:

I would imagine that this is something journal editors will be interested in, and concerned by. We worry about “conflict of interest” a lot in science, and especially medicine: if someone has shares in a company, or receives money from it, and their publication discusses that company’s products, then this needs to be declared in the paper. If you get funding for a study then, again, you must declare it. If you have received payment for making an academic citation, then in my view this is clearly a source of funds that should be declared.

The most problematic point, according to Goldacre, is that (part of) the researchers “have received funds from Cyagen, in exchange for an academic citation.”  He keeps an archive of the 164 papers that cite Cyagen, so one can always check whether “any one of them has declared receiving money from the company”.
(because in his opinion they should)

Some of the commenters to Goldacres’ post go even further and state:

“It’s one thing to fund a study. It’s another thing entirely to pay someone to write what you want — that’s not science, it’s PR. You’re not being puritanical at all, this is corruption, plain and simple.”


“This is an old and very well researched problem. TV makers get paid to have the popular detective drive a certain make of car, scoff a certain brand of drink and so on. That this kind of advertising is not innocent and that it works is beyond doubt. The media have established firm rules about this, science journals need just copy them.”

Although Retraction Watch lets both sides be heard, the title of its post “Researchers, need $100? Just mention Cyagen in your paper!” suggests researchers are being bribed by Cyagen to mention its product anywhere in the paper.

I agree that the Cyagen incentive is a (very curious) marketing trick, but  it has nothing to do with ghostwriting or corruption and IMHO the authors of the 164 papers are not to blame and need not declare anything.

Let me explain why.

  • First, as everybody familiar with benchwork knows, authors are required to mention the brand and source of the used reagents and tools (including strains of mice). This is for transparency and reproducibility. There can be huge differences between products and it is important to know which enzyme, primer or mouse strain someone has used.
  • Authors only MENTION the products, reagents etc. that they have used. Mentioning a brand doesn’t mean an endorsement, although you probably wouldn’t choose a product if you thought it worked suboptimal.
    Indeed the first two papers on the Cyagen citations list both mentionCyagen only ONCE in the materials and methods where it belongsCYAGEN 3 2015-08-15_14-32-01
  • The product itself is not being tested, it is just being used, something else is being tested. (it would be different if the discount was for a diet supplement that was being tested)
  • The authors don’t receive money, they receive a discount on their next purchase. This is common practice in the lab when you are a regular customer.
  • You don’t write COI’s for getting (normal) discounts.

Still, I do think that this incentive is rather inappropriate and not a very wise thing to do (even from a marketing point of view).

It is not the discount that is problematic, neither is the mention of the product’s name and brand in the articles (as this is required).

It is the coupling of the discount (erroneously called “reward”) to “a mention” (erroneously called “citation”) that is not a proper thing to do and even more so, the coupling of the height of the discounts to the impact factor of a journal.

The idea behind it is that Cyagen not only wants to thank its customers, it wants to “increase the number of publications featuring Cyagen as a product or service provider to add strength to our company’s reputation and visible experience”

Of course, Cyagen’s reputation is not linearly dependent on the number of the publications and certainly not the citation score. Reputation depends on other things, like the quality of your product, client satisfaction and …  your reputatIon.

It now seems that the entire marketing campaign will just have the opposite effect: it will harm Cyagen’s reputation (and perhaps also that of the researchers).

My advice to Cyagen is to directly remove the incentive from their website and stop the unsolicited mailing.

And please, Cyagen, stop to defend your P(C)R-tactics, just admit it was a wrong approach and learn from it.

* Later I found out that the a Q&A on Facebook had been extracted from an interview a blogger was still in the process of doing with a product manager of Cyagen. Apparently social media etiquette is another thing Cyagen doesn’t master very well.

Update post:2015-08-16

Social Media in Clinical Practice by Bertalan Meskó [Book Review]

13 09 2013

How to review a book on Medical Social Media written by an author, who has learned you many Social Media skills himself?

Thanks to people like Bertalan Meskó, the author of the book concerned,  I am not a novice in the field of Medical Social Media.

But wouldn’t it be great if all newcomers in the medical social media field could benefit from Bertalan’s knowledge and expertise? Bertalan Meskó, a MD with a  Summa Cum Laude PhD degree in clinical genomics, has already shared his insights by posts on award-winning blog ScienceRoll, via Twitter and Webicina.com (an online service that curates health-related social media resources), by giving presentations and social media classes to medical students and physicians.

But many of his students rather read (or reread) the topics in a book instead of e-learning materials. Therefore Bertalan decided to write a handbook entitled “Social Media in Clinical Practice”.

This is the table of contents (for more complete overview see Amazon):

  1. Social media is transforming medicine and healthcare
  2. Using medical search engines with a special focus on Google
  3. Being up-to-date in medicine
  4. Community sites Facebook, Google+ and medical social networks
  5. The world of e-patients
  6. Establishing a medical blog
  7. The role of Twitter and microblogging in medicine
  8. Collaboration online
  9. Wikipedia and Medical Wikis
  10. Organizing medical events in virtual environments
  11. Medical smartphone and tablet applications
  12. Use of social media by hospitals and medical practices
  13. Medical video and podcast
  14. Creating presentations and slideshows
  15. E-mails and privacy concerns
  16. Social bookmarking
  17. Conclusions

As you can see, many social media tools are covered and in this respect the book is useful for everyone, including patients and consumers.

But what makes “Social Media in Clinical Practice” especially valuable for medical students and clinicians?

First, specific medical search engines/social media sites/tools are discussed, like (Pubmed [medical database, search engine], Sermo [Community site for US physicians], Medworm [aggregator of RSS feeds], medical smartphone apps and sources where to find them, Medical Wiki’s like Radiopaedia.
Scientific Social media sites, with possible relevance to physicians are also discussed, like Google Scholar and Wolphram Alpha.

Second, numerous medical examples are given (with links and descriptions). Often, examples are summarized in tables in the individual chapters (see Fig 1 for a random example ;) ). Links can also be found at the end of the book, organized per chapter.

12-9-2013 7-20-28 Berci examples of blogs

Fig 1. Examples represented in a Table

Third, community sites and non-medical social media tools are discussed from the medical prespective. With regard to community sites and tools like Facebook, Twitter, Blogs and Email special emphasis is placed on (for clinicians very important) quality, privacy and legacy concerns, for instance the compliance of websites and blogs with the HONcode (HON=The Health On the Net Foundation) and HIPAA (Health Insurance Portability and Accountability Act), the privacy settings in Facebook and Social Media Etiquette (see Fig 2).

12-9-2013 7-40-18 berci facebook patient

Fig. 2 Table from “Social Media in Clinical Practice” p 42

The chapters are succinctly written, well organized and replete with numerous examples. I specifically like the practical examples (see for instance Example #4).

12-9-2013 11-19-39 berci example

Fig 3 Example of Smartphone App for consumers

Some tools are explained in more detail, i.e. the anatomy of a tweet or a stepwise description how to launch a WordPress blog.
Most chapters end with a self test (questions),  next steps (encouraging to put the theory into practice) and key points.

Thus in many ways a very useful book for clinical practice (also see the positive reviews on Amazon and the review of Dean Giustini at his blog).

Are there any shortcomings, apart from the minimal language-shortcomings, mentioned by Dean?

Personally I find that discussions of the quality of websites concentrate a bit too much on the formal quality (contact info, title, subtitle etc)). True, it is of utmost importance, but quality is also determined by  content and clinical usefulness. Not all websites that are formally ok deliver good content and vice versa.

As a medical  librarian I pay particular attention to the search part, discussed in chapter 3 and 4.
Emphasis is put on how to create alerts in PubMed and Google Scholar, thus on the social media aspects. However searches are shown, that wouldn’t make physicians very happy, even if used as an alert: who wants a PubMed-alert for cardiovascular disease retrieving 1870195 hits? This is even more true for a the PubMed search “genetics” (rather meaningless yet non-comprehensive term).
More importantly, it is not explained when to use which search engine.  I understand that a search course is beyond the scope of this book, but a subtitle like “How to Get Better at Searching Online?” suggests otherwise. At least there should be hints that searching might be more complicated in practice, preferably with link to sources and online courses.  Getting too much hits or the wrong ones will only frustrate physicians (also to use the socia media tools, that are otherwise helpful).

But overall I find it a useful, clearly written and well structured practical handbook. “Social Media in Clinical Practice” is unique in his kind – I know of no other book that is alike-. Therefore I recommend it to all medical students and health care experts who are interested in digital medicine and social media.

This book will also be very useful to clinicians who are not very fond of social media. Their reluctance may change and their understanding of social medicine developed or enhanced.

Lets face it: a good clinician can’t do without digital knowledge. At the very least his patients use the internet and he must be able to act as a gatekeeper identifying and filtering thrustworty, credible and understandable information. Indeed, as Berci writes in his conclusion:

“it obviously is not a goal to transform all physicians into bloggers and Twitter users, but (..) each physician should find the platforms, tools and solutions that can assist them in their workflow.”

If not convinced I would recommend clinicians to read the blog post written at the the Fauquier ENT-blog (refererred to by Bertalan in chapter 6, #story 5) entiteld: As A Busy Physician, Why Do I Even Bother Blogging?

SM in Practice (AMAZON)

Book information: (also see Amazon):

  • Title: Social Media in Clinical Practice
  • Author: Bertalan Meskó
  • Publisher: Springer London Heidelberg New York Dordrecht
  • 155 pages
  • ISBN 978-1-4471-4305-5
  • ISBN 978-1-4471-4306-2 (eBook)
  • ISBN-10: 1447143051
  • DOI 10.1007/978-1-4471-4306-2
  • $37.99 (Sept 2013) (pocket at Amazon)

Friday Foolery #55 Entrance to the Maternity Ward

26 07 2013

This Picture speaks for itself, I suppose

Source: Facebook page of George Takei (link to picture)

26-7-2013 22-36-52 PUSH PUSH

Between the Lines. Finding the Truth in Medical Literature [Book Review]

19 07 2013

In the 1970s a study was conducted among 60 physicians and physicians-in-training. They had to solve a simple problem:

“If a test to detect a disease whose prevalence is 1/1000 has a false positive rate of 5 %, what is the chance that a person found to have a positive result actually has the disease, assuming that you know nothing about the person’s symptoms or signs?” 

Half of the “medical experts” thought the answer was 95%.
Only a small proportion, 18%, of the doctors arrived at the right answer of 2%.

If you are a medical expert who comes the same faulty conclusion -or need a refresher how to arrive at the right answer- you might benefit from the book written by Marya Zilberberg: “Between the Lines. Finding the Truth in Medical Literature”.

The same is true for a patient whose doctor thinks he/she is among the 95% to benefit form such a test…
Or for journalists who translate medical news to the public…
Or for peer reviewers or editors who have to assess biomedical papers…

In other words, this book is useful for everyone who wants to be able to read “between the lines”. For everyone who needs to examine medical literature critically from time to time and doesn’t want to rely solely on the interpretation of others.

I hope that I didn’t scare you off with the abovementioned example. Between the Lines surely is NOT a complicated epidemiology textbook, nor a dull studybook where you have to struggle through a lot of definitions, difficult tables and statistic formulas and where each chapter is followed by a set of review questions that test what you learned.

This example is presented half way the book, at the end of Part I. By then you have enough tools to solve the question yourself. But even if you don’t feel like doing the exact calculation at that moment, you have a solid basis to understand the bottomline: the (enormous) 93% gap (95% vs 2% of the people with a positive test are considered truly positive) serves as the pool for overdiagnosis and overtreatment.

In the previous chapters of Part I (“Context”), you have learned about the scientific methods in clinical research, uncertainty as the only certain feature of science, the importance of denominators, outcomes that matter and outcomes that don’t, Bayesian probability, evidence hierarchies, heterogeneous treatment effects (does the evidence apply to this particular patient?) and all kinds of biases.

Most reviewers prefer part I of the book. Personally I find part II (“Evaluation”) as interesting.

Part II deals with the study question, and study design, pros and cons of observational and interventional studies, validity, hypothesis testing and statistics.

Perhaps part II  is somewhat less narrative. Furthermore, it deals with tougher topics like statistics. But I find it very valuable for being able to critically appraise a study. I have never seen a better description of “ODDs”: somehow ODDs it is better to grasp if you substitute “treatment A” and “treatment B” for “horse A” and “horse B”, and substitute “death” for “loss of a race”.
I knew the basic differences between cohort studies, case control studies and so on, but I kind of never realized before that ODDs Ratio is the only measure of association available in a case-control study and that case control studies cannot estimate incidence or prevalence (as shown in a nice overview in table 4).

Unlike many other books about “the art of reading of medical articles”, “study designs” or “Evidence Based Medicine”, Marya’s book is easy to read. It is written at a conversational tone and statements are illustrated by means of current, appealing examples, like the overestimation of risk of death from the H1N1 virus, breast cancer screening and hormone replacement therapy.

Although I had printed this book in a wrong order (page 136 next to 13 etc), I was able to read (and understand) 1/3 of the book (the more difficult part II) during a 2 hour car trip….

Because this book is comprehensive, yet accessible, I recommend it highly to everyone, including fellow librarians.

Marya even mentions medical librarians as a separate target audience:

Medical librarians may find this book particularly helpful: Being at the forefront of evidence dissemination, they can lead the charge of separating credible science from rubbish.

(thanks Marya!)

In addition, this book may be indirectly useful to librarians as it may help to choose appropriate methodological filters and search terms for certain EBM-questions. In case of etiology questions words like “cohort”, “case-control”, “odds”, “risk” and “regression” might help to find the “right” studies.

By the way Marya Ziberberg @murzee at Twitter and she writes at her blog Healthcare etc.

p.s. 1 I want to apologize to Marya for writing this review more than a year after the book was published. For personal reasons I found little time to read and blog. Luckily the book lost none of its topicality.

p.s. 2 patients who are not very familiar with critical reading of medical papers might benefit from reading “your medical mind” first [1]. 

bwtn the lines

Amazon Product Details

Medpedia, the Medical Wikipedia, is Dead. And we Missed its Funeral…

12 07 2013

In a post about Wikipedia in 2009 I suggested that initiatives like Ganfyd or Medpedia, might be a solution to Wikipedia’s accuracy and credibility problems, because only health experts are allowed to edit or contribute to the content of these knowledge bases.

MedPedia is a more sophisticated platform than Ganfyd, which looks more like a simple medical encyclopedia. A similar online encyclopedia project with many medical topics, Google Knol, was discontinued by Google as of May 1, 2012.

But now it appears Medpedia may have followed Google KNOL into the same blind alley.

Medpedia was founded in 2007 [2a] by James Currier, an entrepreneur and investor [2b], and an early proponent of social media. He founded the successful Tickle in 1999, when the term Web 2.0 was coined, but not yet mainstream. And his list of  investments is impressive: Flickr, Branchout and Goodreads for instance.

On its homepage Medpedia was described as a “long term, worldwide project to evolve a new model for sharing and advancing knowledge about health, medicine and the body.”
It was developed in association with top medical schools and organizations such as Harvard, Stanford, American College of Physicians, and the NHS. Medpedia was running on the same software and under the same license as Wikipedia and aimed both at the public and  the experts. Contrary to Wikipedia only experts were qualified to contribute to the main content (although others could suggest changes and new topics). [3, 4 , 5, 6] In contrast to many other medical wikis, Medpedia featured a directory of medical editor profiles with general and Medpedia-specific information. This is far more transparent than wikis without individual author recognition [5].

Although promising, Medpedia never became a real success. Von Muhlen wrote in 1999 [4] that there were no articles reporting success metrics for Medpedia or similar projects. In contrast, Wikipedia remains immensely popular among patients and doctors.

Health 2.0 pioneers like E-Patient Dave (@ePatientDave) and Bertalan Meskó (@berci) saw Medpedia’s Achilles heel right from the start:

Bertalan Meskó at his blog Science Roll [7]:

We need Medpedia to provide reliable medical content? That’s what we are working on in Wikipedia.

I believe elitism kills content. Only the power of masses controlled by well-designed editing guidelines can lead to a comprehensive encyclopaedia.

E-patient Dave (who is a fierce proponent of participatory medicine where everyone, medical expert or not, works in partnership to produce accurate information), addresses his concern in his post

“Medpedia: Who gets to say what info is reliable?” [8]

The title says it all. In Dave’s opinion it is “an error to presume that doctors inherently have the best answer” or as Dave summarizes his concern: “who will vet the vetters?”

In addition, Clay Shirky noted that some Wikipedia entries like the biopsy-entry were far more robust than the Medpedia entries [9,10 ].

Ben Toth on the other hand found the Atrial Fibrillation-Medpedia item better than the corresponding Wikipedia page in some respects, but less up-to-date [11].

In her Medpedia review in the JMLA medical librarian Melissa Rethlefsen [5] concludes that “the content of Medpedia is varied and not clearly developed, lacks topical breadth and depth and that it is more a set of ideals than a workable reference source. Another issue is that Medpedia pages never ranked high, which means its content was hardly findable in today’s Google-centric world.

She concludes that for now (2009) “it means that Wikipedia will continue to be the medical wiki of choice”.

I fear that this will be forever, for Medpedia ceased to exist.

I noticed it yesterday totally by coincidence: both my Medpedia blog badge  and Mesko’s Webicina-“Medical Librarianship in Social Medicine”-wiki page were redirected to a faulty page.

I checked the Internet, but all I could find was a message at Wikipedia:

‘It appears that Medpedia is now closed but there is no information about it closing. Their Facebook and Twitter feeds are still open but they have not been updated in a few years. Their webpage now goes to a spam site.

I checked the Waybackmachine and found the “last sparks of life” at January 2013:

11-7-2013 23-57-49 waybackmachine medpedia

This morning I contacted Medpedia’s founder James Currier, who kindly and almost instantly replied to all my questions.

These are shown (with permission) in entirety below.


[me: ] I hope that you don’t mind that I use LinkedIn to ask you some questions about Medpedia.

{James:] I don’t mind at all!

Is Medpedia dead? And if so, why was it discontinued?

For now it is. We worked on it for 6 years, had a fantastic team of developers, had fantastic partners who supported us, had a fantastic core group of contributors like yourself, and I personally spent millions of dollars on it. In other words, we gave it a really good effort. But it never got the sort of scale it needed to become something important. So for the last two years, we kept looking for a new vision of what it could become, a new mission. We never found one, and it was expensive to keep running.
In the meantime, we had found a new mission that Medpedia could not be converted into, so we started a new company, Jiff, to pursue it. “Health Care in a Jiff” is the motto. Jiff continues the idea of digitizing healthcare, and making it simple and transparent for the individual, but goes after it in a very different way. More info about Jiff here: https://www.jiff.com and here https://www.jiff.com/static/newsJiff has taken our time and attention, and hopefully will produce the kinds of benefits we were hoping to see from Medpedia.

Why weren’t people informed and  was Medpedia quietly shut down?

We definitely could have done a better job with that! I apologize. We were under a tight time frame due to several things, such as people leaving the effort, technical issues around where the site was being hosted, and corporate and tax issues after 6 years of operating. So it was rushed, and we should have figured out a way to do a better job of communicating.

Couldn’t the redirection to the spam-site be prevented? And can you do something about it?

I didn’t know about that! I’ll look into it and find out what’s going on.*

Your LinkedIn profile says you’re still working for MedPedia. Why is that? Are there plans to make a new start, perhaps? And how?

Yes, I haven’t updated my LinkedIn profile in a while. I just made that change. We have no current plans to restart Medpedia. But we’re always looking for a new mission that can be self sustaining! Let me know if you have one.

And/or do you have (plans for) other health 2.0 initiatives?

Jiff is our main effort now, and there’s a wonderful CEO, Derek Newell running it.

I know you are a busy man, but I think it is important to inform all people who thought that Medpedia was a good initiative.

Thank you for saying you thought it was a good initiative. I did too! I just wish it had gotten bigger. I really appreciate your questions, and your involvement. Not all projects flourish, but we’ll all keep trying new ideas, and hopefully one will break out and make the big difference we hope for.

*somewhat later James gave an update about the redirection:

By the way, I asked about the redirect, and found out that that that page is produced by our registrar that holds the URL medpedia.com.

We wanted to put up the following message and I thought it was up:

“Medpedia was a great experiment begun in 2007.
Unfortunately, it never reached the size to be self sustaining, and it ceased operations in early 2013.
Thank you to all who contributed!”

I’m going to work again on getting that up!


I have one question left : what happened with all the materials the experts produced? Google Knol gave people time to export their contributions. Perhaps James Currier can answer that question too.

I also wonder why nobody noticed that Medpedia was shut down. Apparently it isn’t missed.

Finally I would like to thank all wo have contributed to this “experiment”. As a medical librarian, who is committed to providing reliable medical information, I still find it a shame that Medpedia didn’t work.

I wish James Currier all the best with his new initiatives.


  1. The Trouble with Wikipedia as a Source for Medical Information
    (https://laikaspoetnik.wordpress.com) (2009/09/14)
  2. [a] Medpedia and [b] James Currier , last edited at 6/30/13*  and 7/12/13 respectively (crunchbase.com)
  3. Laurent M.R. & Vickers T.J. (2009). Seeking Health Information Online: Does Wikipedia Matter?, Journal of the American Medical Informatics Association, 16 (4) 471-479. DOI:
  4. von Muhlen M. & Ohno-Machado L. (2012). Reviewing social media use by clinicians, Journal of the American Medical Informatics Association, 19 (5) 777-781. DOI:
  5. Rethlefsen M.L. (2009). Medpedia, Journal of the Medical Library Association : JMLA, 97 (4) 325-326. DOI:
  6. Medpedia: Reliable Crowdsourcing of Health and Medical Information (highlighthealth.com) (2009/7/24)
  7. Launching MedPedia: From the perspective of a Wikipedia administrator (scienceroll.com) (2009/2/20)
  8. Medpedia: Who gets to say what info is reliable? (e-patients.net/) (2009/2/20)
  9. Clay Shirky at MLA ’11 – On the Need for Health Sciences Librarians to Rock the Boat (mbanks.typepad.com) (2011
  10. Wikipedia vs Medpedia: The Crowd beats the Experts (http://blog.lib.uiowa.edu/hardinmd/2011/05/31
  11. Medpedia and Wikipedia (nelh.blogspot.nl) (2009/10/08)
  12. Jiff wants to do for employer wellness programs what WordPress did for blogs (medcitynews.com)
  13. Jiff Unveils Health App Development Platform, Wellness Marketplace (eweek.com)

No, Google Scholar Shouldn’t be Used Alone for Systematic Review Searching

9 07 2013

Several papers have addressed the usefulness of Google Scholar as a source for systematic review searching. Unfortunately the quality of those papers is often well below the mark.

In 2010 I already [1]  (in the words of Isla Kuhn [2]) “robustly rebutted” the Anders’ paper PubMed versus Google Scholar for Retrieving Evidence” [3] at this blog.

But earlier this year another controversial paper was published [4]:

“Is the coverage of google scholar enough to be used alone for systematic reviews?

It is one of the highly accessed papers of BMC Medical Informatics and Decision Making and has been welcomed in (for instance) the Twittosphere.

Researchers seem  to blindly accept the conclusions of the paper:

But don’t rush  and assume you can now forget about PubMed, MEDLINE, Cochrane and EMBASE for your systematic review search and just do a simple Google Scholar (GS) search instead.

You might  throw the baby out with the bath water….

… As has been immediately recognized by many librarians, either at their blogs (see blogs of Dean Giustini [5], Patricia Anderson [6] and Isla Kuhn [1]) or as direct comments to the paper (by Tuulevi OvaskaMichelle Fiander and Alison Weightman [7].

In their paper, Jean-François Gehanno et al examined whether GS was able to retrieve all the 738 original studies included in 29 Cochrane and JAMA systematic reviews.

And YES! GS had a coverage of 100%!


All those fools at the Cochrane who do exhaustive searches in multiple databases using controlled vocabulary and a lot of synonyms when a simple search in GS could have sufficed…

But it is a logical fallacy to conclude from their findings that GS alone will suffice for SR-searching.

Firstly, as Tuulevi [7] rightly points out :

“Of course GS will find what you already know exists”

Or in the words of one of the official reviewers [8]:

What the authors show is only that if one knows what studies should be identified, then one can go to GS, search for them one by one, and find out that they are indexed. But, if a researcher already knows the studies that should be included in a systematic review, why bother to also check whether those studies are indexed in GS?


Secondly, it is also the precision that counts.

As Dean explains at his blog a 100% recall with a precision of 0,1% (and it can be worse!) means that in order to find 36 relevant papers you have to go through  ~36,700 items.


Are the authors suggesting that researchers consider a precision level of 0.1% acceptable for the SR? Who has time to sift through that amount of information?

It is like searching for needles in a haystack.  Correction: It is like searching for particular hay stalks in a hay stack. It is very difficult to find them if they are hidden among other hay stalks. Suppose the hay stalks were all labeled (title), and I would have a powerful haystalk magnet (“title search”)  it would be a piece of cake to retrieve them. This is what we call “known item search”. But would you even consider going through the haystack and check the stalks one by one? Because that is what we have to do if we use Google Scholar as a one stop search tool for systematic reviews.

Another main point of criticism is that the authors have a grave and worrisome lack of understanding of the systematic review methodology [6] and don’t grasp the importance of the search interface and knowledge of indexing which are both integral to searching for systematic reviews.[7]

One wonders why the paper even passed the peer review, as one of the two reviewers (Miguel Garcia-Perez [8]) already smashed the paper to pieces.

The authors’ method is inadequate and their conclusion is not logically connected to their results. No revision (major, minor, or discretionary) will save this work. (…)

Miguel’s well funded criticism was not well addressed by the authors [9]. Apparently the editors didn’t see through and relied on the second peer reviewer [10], who merely said it was a “great job” etcetera, but that recall should not be written with a capital R.
(and that was about the only revision the authors made)

Perhaps it needs another paper to convince Gehanno et al and the uncritical readers of their manuscript.

Such a paper might just have been published [11]. It is written by Dean Giustini and Maged Kamel Boulos and is entitled:

Google Scholar is not enough to be used alone for systematic reviews

It is a simple and straightforward paper, but it makes its points clearly.

Giustini and Kamel Boulos looked for a recent SR in their own area of expertise (Chou et al [12]), that included a comparable number of references as that of Gehanno et al. Next they test GS’ ability to locate these references.

Although most papers cited by Chou et al. (n=476/506;  ~95%) were ultimately found in GS, numerous iterative searches were required to find the references and each citation had to be managed once at a time. Thus GS was not able to locate all references found by Chou et al. and the whole exercise was rather cumbersome.

As expected, trying to find the papers by a “real-life” GS search was almost impossible. Because due to its rudimentary structure, GS did not understand the expert search strings and was unable to translate them. Thus using Chou et al.’s original search strategy and keywords yielded unmanageable results of approximately >750,000 items.

Giustini and Kamel Boulos note that GS’ ability to search into the full-text of papers combined with its PageRank’s algorithm can be useful.

On the other hand GS’ changing content, unknown updating practices and poor reliability make it an inappropriate sole choice for systematic reviewers:

As searchers, we were often uncertain that results found one day in GS had not changed a day later and trying to replicate searches with date delimiters in GS did not help. Papers found today in GS did not mean they would be there tomorrow.

But most importantly, not all known items could be found and the search process and selection are too cumbersome.

Thus shall we now for once and for all conclude that GS is NOT sufficient to be used alone for SR searching?

We don’t need another bad paper addressing this.

But I would really welcome a well performed paper looking at the additional value of a GS in SR-searching. For I am sure that GS may be valuable for some questions and some topics in some respects. We have to find out which.


  1. PubMed versus Google Scholar for Retrieving Evidence 2010/06 (laikaspoetnik.wordpress.com)
  2. Google scholar for systematic reviews…. hmmmm  2013/01 (ilk21.wordpress.com)
  3. Anders M.E. & Evans D.P. (2010) Comparison of PubMed and Google Scholar literature searches, Respiratory care, May;55(5):578-83  PMID:
  4. Gehanno J.F., Rollin L. & Darmoni S. (2013). Is the coverage of Google Scholar enough to be used alone for systematic reviews., BMC medical informatics and decision making, 13:7  PMID:  (open access)
  5. Is Google scholar enough for SR searching? No. 2013/01 (blogs.ubc.ca/dean)
  6. What’s Wrong With Google Scholar for “Systematic” Review 2013/01 (etechlib.wordpress.com)
  7. Comments at Gehanno’s paper (www.biomedcentral.com)
  8. Official Reviewer’s report of Gehanno’s paper [1]: Miguel Garcia-Perez, 2012/09
  9. Authors response to comments  (www.biomedcentral.com)
  10. Official Reviewer’s report of Gehanno’s paper [2]: Henrik von Wehrden, 2012/10
  11. Giustini D. & Kamel Boulos M.N. (2013). Google Scholar is not enough to be used alone for systematic reviews, Online Journal of Public Health Informatics, 5 (2) DOI:
  12. Chou W.Y.S., Prestin A., Lyons C. & Wen K.Y. (2013). Web 2.0 for Health Promotion: Reviewing the Current Evidence, American Journal of Public Health, 103 (1) e9-e18. DOI:

Diane-35: Geen Reden tot Paniek!

12 03 2013

Dear english-speaking readers of this blog.

This post is about the anti-acne drug Diane-35 that (with other 3rd and 4th generation combined oral contraceptives (COCs)) has been linked to the deaths of several women in Canada, France and the Netherlands. Since there is a lot of media attention (and panic) in the Netherlands, the remainder of this post is in Dutch. Please write in the comments (or tweet) if you would like me to summarize the health concerns of these COCs in a separate English post.



Er is de laatste tijd nogal veel media-aandacht voor Diane-35. Het begon allemaal in Frankrijk, waar de Franse toezichthouder op geneesmiddelen ANSM Diane-35* eind januari van de markt wilde halen omdat in de afgelopen 25 jaar 4 vrouwen na het gebruik ervan waren overleden. In beroep werd dit afgewezen, waarna de ANSM de EMA (European Medicines Agency) verzocht om de veiligheid van Diane en 3e/4e generatie orale combinatie anticonceptiepillen (OAC) nog eens te onderzoeken.

In januari overleed ook een 21-jarige gebruikster van Diane-35 in Nederland. Met terugwerkende kracht ontving het Nederlandse Bijwerkingscentrum Lareb 97 meldingen van bijwerkingen van Diane-35. Hieronder waren 9 sterfgevallen uit 2011 en eerder.** Overigens werden ook sterfgevallen gemeld van vrouwen die vergelijkbare (3e en 4e generatie) orale anticonceptiemiddelen hadden gebruikt, zoals Yasmin.

Alle vrouwen zijn overleden aan bloedproppen in bloedvaten (trombose dan wel longembolie).  Totaal waren er  89 meldingen van bloedstolsels, wat altijd (ook zonder dodelijke afloop) een ernstige bijwerking is.

Aanleiding voor Canada en België, om ook in de statistieken te duiken. In Canada bleken sinds 2000 11 vrouwen die Diane-35 slikten te zijn overleden en in België zijn er sinds 2008 29 meldingen van trombose door het gebruik van de 3e/4e generatiepil (5 door Diane-35, geen doden)

Dit nieuws sloeg in als een bom. Veel mensen raakten in paniek. Of zijn boos op Bayer*, het CBG (College ter Beoordeling van Geneesmiddelen) en/of minister Schippers die het naar hun idee laten afweten. Op Twitter zie ik aan de lopende band tweets voorbijkomen als:

Diane-35 pil: heet deze zo omdat je er niet altijd de 35 jaar mee haalt?“.

Tonnen #rundvlees halen we uit de handel om wat #paardenvlees. Maar doden door de Diane 35 #pil doet de regering niks mee.

Oud Nieuws

Dergelijke reacties zijn sterk overdreven. Er is absoluut geen reden tot paniek.

Echter waar rook is, is vuur. Ook al gaat het hier om een klein brandje.

Maar wat betreft Diane-35 is die rook er al jaren…. Waarom roept men nu ineens: “Brand!”?

De meeste sterfgevallen zijn van vòòr dit jaar. Dat de Fransen authoriteiten nu zoveel daadkracht tonen komt waarschijnlijk omdat hen laksheid verweten werd bij recente schandalen met PIP-borstimplantaten en Mediator, dat meer dan 500 sterfgevallen veroorzaakt heeft. [Reuterszie ook het blog van Henk Jan Out]

Verder was allang bekend dat Diane-35 de kans op bloedstolsels verhoogde.

Niet alleen Diane-35

Men kan de risico’s van Diane-35 niet los zien van de risico’s van orale anticonceptiemiddelen (OAC’s) in het algemeen.

Diane-35 lijkt qua samenstelling erg op de 3e generatie OAC.  Het is echter uniek omdat het in plaats van een 3e generatie progestogeen cyproteronacetaat bevat. ‘De pil’ bevat levonorgestrel, dit is een 2e generatie progestogeen. Al de OAC combinatiepillen bevatten daarnaast (tegenwoordig) een lage dosering ethinylestradiol.

Zoals gezegd, is al jaren bekend dat alle OAC’s, dus ook ‘de pil’, de kans op bloedstolsels in bloedvaten licht verhogen[1,2,3]. Op zijn hoogst verhogen 2e generatie OAC’s (met levonorgestrel) die kans met een factor 4. Derde generatie pillen lijken die kans verder te verhogen. Met hoeveel precies, daarover verschillen de meningen. Voor wat beteft Diane-35, ziet de een géén tot nauwelijks effect [4], de ander een 1,5 [5]  tot 2 x [3] sterker effect.  Het totaalplaatje ziet er ongeveer als volgt uit:

7-3-2013 15-48-49 risico's pil

Absolute en relatieve kans op VTE (Veneuze trombo-embolie).
Uit: http://www.anticonceptie-online.nl/pil.htm

Risico’s in Perspectief

Een 1,5-2 x groter risico vergeleken met de “gewone pil”, lijkt een enorm groot risico. Dit zou ook een groot effect zijn als trombo-embolie vaak voorkwam. Stel dat 1 op de 100 mensen trombose krijgt per jaar, dan zouden op jaarbasis 2-4 op de 100 mensen trombose krijgen na de ‘pil’ en 3-8 mensen na Diane-35 of een 3e of 4e generatiepil. Dit is een groot absoluut risico. Dat risico zou je normaal niet nemen.

Maar trombo-embolie is zeldzaam. Het komt voor bij 5-10 op de 100.000 vrouwen per jaar. En totaal zal 1 op miljoen vrouwen daaraan overlijden. Dat is een heel minieme kans.Vier tot zes keer een kans van iets meer dan 0 blijft een kans van bijna 0. Dus in absolute zin, brengen Diane-35 en OAC’s weinig risico met zich mee.

Daarbij komt dat trombose niet direct door de pil veroorzaakt hoeft te zijn. Roken, leeftijd, (over)gewicht, erfelijke aanleg voor stollingsproblemen kunnen ook een (grote) rol spelen. Verder kunnen deze factoren samenspelen. Om deze reden worden OAC’s (ook de pil) afgeraden aan risicogroepen (oudere vrouwen die veel roken, aanleg voor trombose e.d.)

Het aantal bijwerkingendat mogelijk samenhangt met het gebruik van Diane-35, geeft eigenlijk aan dat dit een relatief veilig middel is.

Aanvaardbaar risico?

Om het nog meer in perspectief te plaatsen: zwangerschap geeft een 2x hoger risico op trombose dan Diane-35, en in de 12 weken na de bevalling is de kans nog weer 4-8 keer hoger dan in de zwangerschap (FDA). Toch zullen vrouwen het daarvoor niet laten om zwanger te worden. Het krijgen van een kind weegt meestal (impliciet) op tegen(kleine) risico’s (waarvan trombose er één is).

Men kan de (kans op) risico’s dus niet los zien van de voordelen. Als het voordeel hoog is zal men zelfs een zeker risico op de koop toe willen nemen (afhankelijk van de ernst van de aandoening en eht nut). Aan de andere kant wil je zelfs een heel klein risico niet lopen, als je geen baat hebt bij een middel of als er even goede, maar veiliger middelen zijn.

Maar mag de patiënte die overweging niet zelf met haar arts maken?

Geen plaats voor Diane-35  als anticonceptiemiddel

Diane-35 heeft een anticonceptiewerking, maar het is hiervoor niet (langer) geregistreerd. De laatste anticonceptie-richtlijn van de nederlandse huisartsen (NHG) uit 2010 [6] zegt expliciet dat er geen plaats meer is voor de pil met cyproteronacetaat. Dit omdat de gewone ‘pil’ even goed zwangerschap voorkomt én (iets) minder kans geeft op trombose als bijwerking. Dus waarom zou je een een potentieel hoger risico lopen, als dat niet nodig is? Helaas is de NHG-standaard minder expliciet over 2e en 3e generatie OAC’s.

In andere landen denkt men vaak net zo (In de VS is Diane-35 echter niet geregistreerd).

Dit zegt bijv de RCOG (Royal College of Obstetricians and Gynaecologist, UK) in hun evidence-based richtlijn die specifiek gaat over OAC’s en kans op trombose [1]

10-3-2013 17-27-40 RCOG CPA

Diane-35 als middel tegen ernstige acne en overbeharing.

Omdat het cyproteron acetaat in Diane-35 een sterk anti-androgene werking heeft kan het worden ingezet  bij ernstige acné en overbeharing (dat laatste met name bij vrouwen met PCOS, een gynecologische aandoening). Desgewenst kan het dan tevens dienst doen als anticonceptiemiddel: 2 vliegen in één klap.

Clinical Evidence, dat een heel mooie evidence based bron is die voor-en nadelen van behandelingen tegen elkaar afzet, concludeert dat middelen met cyproteron acetaat ondanks hun prima werking, bij ernstige overbeharing (bij PCOS) niet de voorkeur verdienen boven middelen als metformine. Het risico op trombose is in deze overweging meegenomen.[7]

3-3-2013 14-54-14 CLINical Evidence PCOS cyproterone acetate

Volgens een Cochrane Systematisch Review hielpen alle OAC’s wel bij acné, maar OAC’s met cyproteron leken wat effectiever dan pillen met 2e of 3e generatie progestogeen. De resultaten waren echter tegenstrijdig en de studies niet zo erg sterk.[8]

Sommigen concluderen op basis van dit Cochrane Review dat alle OAC’s even goed helpen en dat de gewone pil dus voorkeur verdient (zie bijv. dit recente artikel van Helmerhorst in de BMJ [2], en de NHG standaard acne [9]

Maar in de meest recente Richtlijn Acneïforme Dermatosen [10] van de Nederlandse Vereniging voor Dermatologie en Venereologie (NVDV) wordt er op basis van dezelfde evidence iets anders geconcludeerd: 10-3-2013 22-43-02 ned vereniging voor dermatologie

De Nederlandse dermatologen komen dus met een positieve aanbeveling van Diane-35 ten opzichte van andere anticonceptiemiddelen bij vrouwen die ook anticonceptie wensen. Nergens in deze richtlijn wordt expliciet gerefereerd aan trombose als mogelijke bijwerking.

Het voorschrijfbeleid in de praktijk.

Als Diane-35 niet als anticonceptiemiddel voorgeschreven wordt, en het wordt slechts bij ernstige vormen van acne of overbeharing gebruikt, hoe kan dit middel met een zo’n laag risico dan zo’n omvangrijk probleem worden? De doelgroep èn de kans op bijwerkingen is immers heel klein. En hoe zit het met 3e en 4e generatie OAC’s die niet eens bij acné voorgeschreven zullen worden? Daar zou de doelgroep nog kleiner moeten zijn.

De realiteit is dat de omvang van het probleem niet zozeer door het on-label gebruik komt maar, zoals Janine Budding al aangaf op haar blog Medical Facts door off-label voorschrijfgedrag, dus voor een  andere indicatie dan waarvoor het geneesmiddel is geregistreerd. In Frankrijk gebruikt de helft van de vrouwen die OAC’s gebruiken, de 3e en 4e generatie OAC: dat is ronduit buitensporig, en niet volgens de richtlijnen.

In Nederland slikkten ruim 161.000 vrouwen Diane-35 of een generieke variant met exact dezelfde werking. Ook veel Nederlandse en Canadese gebruiken Diane-35 en andere 3e en 4e generatie OAC’s puur anticonceptiemiddel. Voor een deel, omdat sommige huisartsen het ‘in de pen’ hebben of denken dat een meisje dan gelijk van haar puistjes afgeholpen is. Voor een deel omdat, in  Nederland en  Frankrijk, Diane-35 vergoed wordt en de gewone pil niet. Er is, zeker in Frankrijk, een run op een ‘gratis’ pil.

Online bedrijven spelen mogelijk ook een rol. Deze lichten vaak niet goed voor. Eén zo’n bedrijf (met gebrekkige info over Diane op hun website) gaat zelfs zover het twitter account @diane35nieuws te creeeren als dekmantel voor online pillenverkoop.

Wat nu?

Hoewel de risico’s van Diane-35 allang bekend waren en gering lijken te zijn, en bovendien vergelijkbaar met die van de 3e en 4e generatie  OAC’s, is er een massaal verzet tegen Diane-35 op gang gekomen, die niet meer te stuiten lijkt. Niet de experts, maar de media en de politiek lijken de discussie te voeren.Erg verwarrend en soms misleidend voor de patiënt.

Mijn inziens is het besluit van de Nederlandse huisartsen, gynecologen en recent ook de dermatologen om Diane-35 voorlopig niet voor te schrijven aan nieuwe patiënten tot de autoriteiten een uitspraak hebben gedaan over de veiligheid***, gezien de huidige onrust, een verstandige.

Wat niet verstandig is om zomaar met de Diane-35 pil te stoppen. Overleg altijd eerst met uw arts wat voor u de beste optie is.

In 1995 heeft een vergelijkbare reactie op waarschuwingen over de tromboserisico’s van bepaalde OAC’s geleid tot een ware “pil scare”: vrouwen gingen massaal over op een andere pil of stopten er in het geheel mee. Gevolg: een piek aan ongewenste zwangerschappen (met overigens een veel hogere kans op trombose) en abortussen. Conclusie destijds [10]:

“The level of risk should, in future, be more carefully assessed and advice more carefully presented in the interests of public health.”

Kennelijk is deze les aan Nederland en Frankrijk voorbijgegaan.

Hoewel ik denk dat Diane-35 maar voor een beperkte groep echt zinvol is boven de bestaande middelen, is het te betreuren dat op basis van ongefundeerde reacties, patiënten straks mogelijk zelf geen keuzevrijheid meer hebben. Mogen zij zelf de balans tussen voor-en nadelen bepalen?

Het is begrijpelijk (maar misschien niet zo heel professioneel), dat dermatologen nogal gefrustreerd reageren, nu een bepaalde groep patienten tussen wal en schip raakt. Tevens moet men niet op basis van evidence en argumenten, maar onder druk van media en politiek, tot een ander beleid overgaan.

11-3-2013 23-27-37 reactie dermatologen 2

Want laten we wel wezen, sommige dermatologische en gynecologische patiënten hebben wel baat bij Diane-35.


En tot slot een prachtige reactie van een acne-patiënte op een  blog post van Ivan Wolfers. Zij vat de essentie in enkele zinnen samen. Net als bovenstaande dames, een patient die zeer weloverwogen met haar arts beslissingen neemt op basis van de bestaande info.

Zoals het zou moeten…

12-3-2013 0-57-27 reactie patient


* Diane-35 wordt geproduceerd door Bayer. Het staat ook bekend als Minerva, Elisa en in buitenland bijvoorbeels als Dianette. Er zijn verder ook veel merkloze preparaten met dezelfde samenstelling.

**Inmiddels heeft zijn er nog 4 dodelijke gevallen na gebruik van Diane-35 in Nederland bijgekomen (Artsennet, 2013-03-11)

***Hopelijk wordt de gewone pil dan ook in de vergelijking meegenomen. Dit is wel zo eerlijk: het gaat immers om een vergelijking.


  • Venous Thromboembolism and Hormone Replacement Therapy – Green-top Guide line 40 (2010) Royal College of Obstetricians and Gynaecologists, 


  • Helmerhorst F.M. & Rosendaal F.R. (2013). Is an EMA review on hormonal contraception and thrombosis needed?, BMJ (Clinical research ed.), PMID:
  • van Hylckama Vlieg A., Helmerhorst F.M., Vandenbroucke J.P., Doggen C.J.M. & Rosendaal F.R. (2009). The venous thrombotic risk of oral contraceptives, effects of oestrogen dose and progestogen type: results of the MEGA case-control study., BMJ (Clinical research ed.), PMID:
  • Spitzer W.O. (2003) Cyproterone acetate with ethinylestradiol as a risk factor for venous thromboembolism: an epidemiological evaluation., Journal of obstetrics and gynaecology Canada : JOGC = Journal d’obstétrique et gynécologie du Canada : JOGC, PMID:
  • Martínez F., Ramírez I., Pérez-Campos E., Latorre K. & Lete I. (2012) Venous and pulmonary thromboembolism and combined hormonal contraceptives. Systematic review and meta-analysis., The European journal of contraception & reproductive health care : the official journal of the European Society of Contraception, PMID:
  • NHG-Standaard Anticonceptie 2010 Anke Brand, Anita Bruinsma, Kitty van Groeningen, Sandra Kalmijn, Ineke Kardolus, Monique Peerden, Rob Smeenk, Suzy de Swart, Miranda Kurver, Lex Goudswaard.

  • Cahill D. (2009). PCOS., Clinical evidence, PMID:
  • Arowojolu A.O., Gallo M.F., Lopez L.M. & Grimes D.A. (2012). Combined oral contraceptive pills for treatment of acne., Cochrane database of systematic reviews (Online), PMID:
  • Kertzman M.G.M., Smeets J.G.E., Boukes F.S. & Goudswaard A.N. [Summary of the practice guideline ‘Acne’ (second revision) from the Dutch College of General Practitioners]., Nederlands tijdschrift voor geneeskunde, PMID:
  • Richtlijn Acneïforme Dermatosen, © 2010, Nederlandse Vereniging voor Dermatologie en Venereologie (NVDV)
  • Furedi A. The public health implications of the 1995 ‘pill scare’., Human reproduction update, PMID:


Get every new post delivered to your Inbox.

Join 3,099 other followers