This post is update after a discussion at Twitter with @animalevidence who pointed me at a great blog post at Speaking of Research ([19], a repost of [20], highlighting the shortcomings of the current study using just one single inbred strain of mice (C57Bl6) [2013-02-26]. Main changes are in blue
A recent paper published in PNAS [1] caused quite a stir both inside and outside the scientific community. The study challenges the validity of using mouse models to test what works as a treatment in humans. At least this is what many online news sources seem to conclude: “drug testing may be a waste of time”[2], “we are not mice” [3, 4], or a bit more to the point: mouse models of inflammation are worthless [5, 6, 7].
But basically the current study looks only at one specific area, the area of inflammatory responses that occur in critically ill patients after severe trauma and burns (SIRS, Systemic Inflammatory Response Syndrome). In these patients a storm of events may eventually lead to organ failure and death. It is similar to what may occur after sepsis (but here the cause is a systemic infection).
Furthermore the study only uses one single approach: it compares the gene response patterns in serious human injuries (burns, trauma) and a human model partially mimicking these inflammatory diseases (human healthy volunteers receiving a low dose endotoxin) with the corresponding three animal models (burns, trauma, endotoxin).
And, as highlighted by Bill Barrington of “Understand Nutrition” [8], the researchers have only tested the gene profiles in one single strain of mice: C57Bl6 (B6 for short). If B6 was the only model used in practice this would be less of a problem. But according to Mark Wanner of the Jackson Laboratory [19, 20]:
It is now well known that some inbred mouse strains, such as the C57BL/6J (B6 for short) strain used, are resistant to septic shock. Other strains, such as BALB and A/J, are much more susceptible, however. So use of a single strain will not provide representative results.
The results in itself are very clear. The figures show at a glance that there is no correlation whatsoever between the human and B6 mouse expression data.
Seok and 36 other researchers from across the USA looked at approximately 5500 human genes and their mouse analogs. In humans, burns and traumatic injuries (and to a certain extent the human endotoxin model) triggered the activation of a vast number of genes, that were not triggered in the present C57Bl6 mouse models. In addition the genomic response is longer lasting in human injuries. Furthermore, the top 5 most activated and most suppressed pathways in human burns and trauma had no correlates in mice. Finally, analysis of existing data in the Gene Expression (GEO) Database showed that the lack of correlation between mouse and human studies was also true for other acute inflammatory responses, like sepsis and acute infection.
This is a high quality study with interesting results. However, the results are not as groundbreaking as some media suggest.
As discussed by the authors [1], mice are known to be far more resilient to inflammatory challenge than humans*: a million fold higher dose of endotoxin than the dose causing shock in humans is lethal to mice.* This, and the fact that “none of the 150 candidate agents that progressed to human trials has proved successful in critically ill patients“ already indicates that the current approach fails.
[This is not entirely correct the endotoxin/LPS dose in mice is 1000–10,000 times the dose required to induce severe disease with shock in humans [20] and mice that are resilient to endotoxin may still be susceptible to infection. It may well be that the endotoxin response is not a good model for the late effects of sepsis]
The disappointing trial results have forced many researchers to question not only the usefulness of the current mouse models for acute inflammation [9,10; refs from 11], but also to rethink the key aspects of the human response itself and the way these clinical trials are performed [12, 13, 14]. For instance, emphasis has always been on the exuberant inflammatory reaction, but the subsequent immunosuppression may also be a major contributor to the disease. There is also substantial heterogeneity among patients [13-14] that may explain why some patients have a good prognosis and others haven’t. And some of the initially positive results in human trials have not been reproduced in later studies either (benefit of intense glucose control and corticosteroid treatment) [12]. Thus is it fair to blame only the mouse studies?
It is true that animal studies, including rodent studies, have their limitations. But what are the alternatives? In vitro studies are often even more artificial, and direct clinical testing of new compounds in humans is not ethical.
Obviously, the final proof of effectiveness and safety of new treatments can only be established in human trials. No one will question that.
A lot can be said about why animal studies often fail to directly translate to the clinic [15]. Clinical disparities between the animal models and the clinical trials testing the treatment (like in sepsis) are one reason. Other important reasons may be methodological flaws in animal studies (i.e. no randomization, wrong statistics) and publication bias: non-publication of “negative” results appears to be prevalent in laboratory animal research.[15-16]. Despite their shortcomings, animal studies and in vitro studies offer a way to examine certain aspects of a process, disease or treatment.
In summary, this study confirms that the existing (C57Bl6) mouse model doesn’t resemble the human situation in the systemic response following acute traumatic injury or sepsis: the genomic response is entirely different, in magnitude, duration and types of changes in expression.
The findings are not new: the shortcomings of the mouse model(s) were long known. It remains enigmatic why the researchers chose only one inbred strain of mice, and of all mice only the B6-strain, which is less sensitive to endotoxin, and only develop acute kidney injury (part of organ failure) at old age (young mice were used) [21]. In this paper from 2009 (!) various reasons are given why the animal models didn’t properly mimic the human disease and how this can be improved. The authors stress that:
“the genetically heterogeneous human population should be more accurately represented by outbred mice, reducing the bias found in inbred strains that might contain or lack recessive disease susceptibility loci, depending on selective pressures.”
Both Bill Barrington [8] and Mark Wanner [18,19] propose the use of “diversity outbred cross or collaborative cross mice that provide additional diversity.” Indeed, “replicating genetic heterogeneity and critical clinical risk factors such as advanced age and comorbid conditions (..) led to improved models of sepsis and sepsis-induced AKI (acute kidney injury).
The authors of the PNAS paper suggest that genomic analysis can aid further in revealing which genes play a role in the perturbed immune response in acute inflammation, but it remains to be seen whether this will ultimately lead to effective treatments of sepsis and other forms of acute inflammation.
It also remains to be seen whether comprehensive genomic characterization will be useful in other disease models. The authors suggest for instance, that genetic profiling may serve as a guide to develop animal models. A shotgun analyses of gene expression of thousands of genes was useful in the present situation, because “the severe inflammatory stress produced a genomic storm affecting all major cellular functions and pathways in humans which led to sufficient perturbations to allow comparisons between the genes in the human conditions and their analogs in the murine models”. But rough analysis of overall expression profiles may give little insight in the usefulness of other animal models, where genetic responses are more subtle.
And predicting what will happen is far less easy that to confirm what is already known….
NOTE: as said the coverage in news and blogs is again quite biased. The conclusion of a generally good Dutch science news site (the headline and lead suggested that animal models of immune diseases are crap [6]) was adapted after a critical discussion at Twitter (see here and here), and a link was added to this blog post). I wished this occurred more often…. In my opinion the most balanced summaries can be found at the science-based blogs: ScienceBased Medicine [11] and NIH’s Director’s Blog [17],whereas “Understand Nutrition” [8] has an original point of view, which is further elaborated by Mark Wanner atSpeaking of Research [19] and Genetics and your health Blog [20]
References
Seok, J., Warren, H., Cuenca, A., Mindrinos, M., Baker, H., Xu, W., Richards, D., McDonald-Smith, G., Gao, H., Hennessy, L., Finnerty, C., Lopez, C., Honari, S., Moore, E., Minei, J., Cuschieri, J., Bankey, P., Johnson, J., Sperry, J., Nathens, A., Billiar, T., West, M., Jeschke, M., Klein, M., Gamelli, R., Gibran, N., Brownstein, B., Miller-Graziano, C., Calvano, S., Mason, P., Cobb, J., Rahme, L., Lowry, S., Maier, R., Moldawer, L., Herndon, D., Davis, R., Xiao, W., Tompkins, R., , ., Abouhamze, A., Balis, U., Camp, D., De, A., Harbrecht, B., Hayden, D., Kaushal, A., O’Keefe, G., Kotz, K., Qian, W., Schoenfeld, D., Shapiro, M., Silver, G., Smith, R., Storey, J., Tibshirani, R., Toner, M., Wilhelmy, J., Wispelwey, B., & Wong, W. (2013). Genomic responses in mouse models poorly mimic human inflammatory diseases Proceedings of the National Academy of Sciences DOI: 10.1073/pnas.1222878110
Wiersinga WJ (2011). Current insights in sepsis: from pathogenesis to new treatment targets. Current opinion in critical care, 17 (5), 480-6 PMID: 21900767
Khamsi R (2012). Execution of sepsis trials needs an overhaul, experts say. Nature medicine, 18 (7), 998-9 PMID: 22772540
Hotchkiss RS, Coopersmith CM, McDunn JE, & Ferguson TA (2009). The sepsis seesaw: tilting toward immunosuppression. Nature medicine, 15 (5), 496-7 PMID: 19424209
van der Worp, H., Howells, D., Sena, E., Porritt, M., Rewell, S., O’Collins, V., & Macleod, M. (2010). Can Animal Models of Disease Reliably Inform Human Studies? PLoS Medicine, 7 (3) DOI: 10.1371/journal.pmed.1000245
ter Riet, G., Korevaar, D., Leenaars, M., Sterk, P., Van Noorden, C., Bouter, L., Lutter, R., Elferink, R., & Hooft, L. (2012). Publication Bias in Laboratory Animal Research: A Survey on Magnitude, Drivers, Consequences and Potential Solutions PLoS ONE, 7 (9) DOI: 10.1371/journal.pone.0043404
Warren, H. (2009). Editorial: Mouse models to study sepsis syndrome in humans Journal of Leukocyte Biology, 86 (2), 199-201 DOI: 10.1189/jlb.0309210
Doi, K., Leelahavanichkul, A., Yuen, P., & Star, R. (2009). Animal models of sepsis and sepsis-induced kidney injury Journal of Clinical Investigation, 119 (10), 2868-2878 DOI: 10.1172/JCI39421
Two weeks ago there was a hot debate among Dutch Tweeps on “bad science, bad science journalism and bad science communication“. This debate was started and fueled by different Dutch blog posts on this topic.[1,4-6]
A controversial post, with both fierce proponents and fierce opposition was the post by Daniel Lakens [1], an assistant professor in Applied Cognitive Psychology.
I was among the opponents. Not because I don’t like a new fresh point of view, but because of a wrong reasoning and because Daniel continuously compares apples and oranges.
Since Twitter debates can’t go in-depth and lack structure and since I cannot comment to his Google sites blog, I pursue my discussion here.
The title of Daniels post is (freely translated, like the rest of his post):
The University press office send out a press release [5 ], which was picked up by news media [4,6] and one of the Dutch authors of this study, Loes Keijsers, tweeted enthusiastically: ”Want to know whether a 16 year old adult will suffer from delinquency, than look at his music taste at age 12!”
Weten of een jongeren met 16 jaar aan winkeldiefstal doet? Kijk dan naar muziekvoorkeur op 12 jaar! Nieuwe studie @FSW_UU @UniUtrecht
According to Hans, Loes could have easily broadcasted (more) balanced tweets, like “Music preference doesn’t predict shoplifting” or“12 year olds who like Bach keep quiet about shoplifting when 16.” But even then, Hans argues, the tweets wouldn’t have been scientifically underpinned either.
In column style Hans explains why he thinks that the study isn’t methodologically strong: no absolute numbers are given; 7 out of 11 (!) music styles are positively associated with delinquency, but these correlations are not impressive: the strongest predictor (Gothic music preference) can explain no more than 9% of the variance in delinquent behaviour, which can include anything from shoplifting, vandalism, fighting, graffiti spraying, switching price tags. Furthermore the risks of later “delinquent” behavior are small: on a scale 1 (never) to 4 (4 times or more) the mean risk was 1,12. Hans also wonders whether it is a good idea to monitor kids with a certain music taste.
Thus Hans concludes “this study isn’t good science”. Daniel, however, concludes that Hans’ writing is not good science journalism.
First Daniel recalls he and other PhD’s took a course on how to peer review scientific papers. On basis of their peer review of a (published) article 90% of the students decided to reject it. The two main lessons learned by Daniel were:
It is easy to critize a scientific paper and grind it down. No single contribution to science (no single article) is perfect.
New scientific insights, although imperfect, are worth sharing, because they help to evolve science. *¹
According to Daniel science jounalists often make the same mistakes as the peer reviewing PhD-students: critisizing the individuel studies without a “meta-view” on science.
Peer review and journalism however are different things (apples and oranges if you like).
Peer review (with all its imperfections) serves to filter, check and to improve the quality of individual scientific papers (usually) before they are published [10]. My papers that passed peer review, were generally accepted. Of course there were the negative reviewers, often the ignorant ones, and the naggers, but many reviewers had critique that helped to improve my paper, sometimes substantially. As a peer reviewer myself I only try to separate the wheat from the chaff and to enhance the quality of the papers that pass.
Science journalism also has a filter function: it filters already peer reviewed scientific papers* for its readership, “the public” by selecting novel relevant science and translating the scientific, jargon-laded language, into language readers can understand and appreciate. Of course science journalists should put the publication into perspective (call it “meta”).
Surely the PhD-students finger exercise resembles the normal peer review process as much as peer review resembles science journalism.
I understand that pure nitpicking seldom serves a goal, but this rarely occurs in science journalism. The opposite, however, is commonplace.
Daniel disapproves Hans van Maanen’s criticism, because Hans isn’t “meta” enough. Daniel: “Arguing whether an effect size is small or mediocre is nonsense, because no individual study gives a good estimate of the effect size. You need to do more research and combine the results in a meta-analysis”.
Apples and oranges again.
Being “meta” has little to do with meta-analysis. Being meta is … uh … pretty meta. You could think of it as seeing beyond (meta) the findings of one single study*.
A meta-analysis, however, is a statistical technique for combining the findings from independent, but comparable (homogeneous) studies in order to more powerfully estimate the true effect size (pretty exact). This is an important, but difficult methodological task for a scientist, not a journalist. If a meta-analysis on the topic exist, journalists should take this into account, of course (and so should the researchers). If not, they should put the single study in broader perspective (what does the study add to existing knowledge?) and show why this single study is or is not well done?
Daniel takes this further by stating that “One study is no study” and that journalists who simply echo the press releases of a study ànd journalists who just amply criticizes only single publication (like Hans) are clueless about science.
Apples and oranges! How can one lump science communicators (“media releases”), echoing journalists (“the media”) and critical journalists together?
I see more value in a critical analysis than a blind rejoicing of hot air. As long as the criticism guides the reader to appreciate the study.
And if there is just one single novel study, that seems important enough to get media attention, shouldn’t we judge the research on its own merits?
Then Daniel asks himself: “If I do criticize those journalists, shouldn’t I criticize those scientists who published just a single study and wrote a press release about it? “
His conclusion? “No”.
Daniel explains: science never provides absolute certainty, at the most the evidence is strong enough to state what is likely true. This can only be achieved by a lot of research by different investigators.
Therefore you should believe in your ideas and encourage other scientists to pursue your findings. It doesn’t help when you say that music preference doesn’t predict shoplifting. It does help when you use the media to draw attention to your research. Many researchers are now aware of the “Music Marker Theory”. Thus the press release had its desired effect. By expressing a firm belief in their conclusions, they encourage other scientists to spend their sparse time on this topic. These scientists will try to repeat and falsify the study, an essential step in Cumulative Science.At a time when science is under pressure, scientists shouldn’t stop writing enthusiastic press releases or tweets.
The latter paragraph is sheer nonsense!
Critical analysis of one study by a journalist isn’t what undermines the public confidence in science. Rather it’s the media circus, that blows the implications of scientific findings out of proportion.
As exemplified by the hilarious PhD Comic below research results are propagated by PR (science communication), picked up by media, broadcasted, spread via the internet. At the end of the cycle conclusions are reached, that are not backed up by (sufficient) evidence.
PhD Comics – The news Cycle
Daniel is right about some things. First one study is indeed no study, in the sense that concepts are continuously tested and corrected: falsification is a central property of science (Popper). He is also right that science doesn’t offer absolute certainty (an aspect that is often not understood by the public). And yes, researchers should believe in their findings and encourage other scientists to check and repeat their experiments.
Though not primarily via the media. But via the normal scientific route. Good scientists will keep track of new findings in their field anyway. Suppose that only findings that are trumpeted in the media would be pursued by other scientists?
And authors shouldn’t make overstatements. They shouldn’t raise expectations to a level which cannot be met. The Dutch study only shows weak associations. It simply isn’t true that the Dutch study allows us to “predict” at an individual level if a 12 year old will “act out” at 16.
This doesn’t help lay-people to understand the findings and to appreciate science.
The idea that media should just serve to spotlight a paper, seems objectionable to me.
Going back to the meta-level: what about the role of science communicators, media, science journalists and researchers?
Now what about Daniel criticism of van Maanen? In my opinion, van Maanen is one of those rare critical journalists who serve as an antidote against uncritical media diarrhea (see Fig above). Comparable to another lone voice in the media: Ben Goldacre. It didn’t surprise me that Daniel didn’t approve of him (and his book Bad Science) either [11].
Does this mean that I find Hans van Maanen a terrific science journalist? No, not really. I often agree with him (i.e. see this post [12]). He is one of those rare journalists who has real expertise in research methodology . However, his columns don’t seem to be written for a large audience: they seem too complex for most lay people. One thing I learned during a scientific journalism course, is that one should explain all jargon to one’s audience.
Personally I find this critical Dutch blog post[13] about the Music Marker Theory far more balanced. After a clear description of the study, Linda Duits concludes that the results of the study are pretty obvious, but that the the mini-hype surrounding this research is caused by the positive tone of the press release. She stresses that prediction is not predetermination and that the musical genres are not important: hiphop doesn’t lead to criminal activity and metal not to vandalism.
The study got widely covered by the media. The NRC, for which the main author Martijn Katan works as a science columnist, columnist, spent two full (!) pages on the topic -with no single critical comment-[3].
As if this wasn’t enough, the latest column of Katan again dealt with his article (text freely available at mkatan.nl)[4].
I found Katan’s column “Col hors Catégorie” [4] quite arrogant, especially because he tried to belittle a (as he called it) “know-it-all” journalist who criticized his work in a rivaling newspaper. This wasn’t fair, because the journalist had raised important points [5, 1] about the work.
The piece focussed on the long road of getting papers published in a top journal like the NEJM.
Katan considers the NEJM as the ”Tour de France” among medical journals: it is a top achievement to publish in this paper.
Katan also states that “publishing in the NEJM is the best guarantee something is true”.
I think the latter statement is wrong for a number of reasons.*
First, most published findings are false[6]. Thus journals can never “guarantee” that published research is true.
Factors that make it less likely that research findings are true include a small effect size, a greater number and lesser preselection of tested relationships, selective outcome reporting, the “hotness” of the field (all applying more or less to Katan’s study, he also changed the primary outcomes during the trial[7]), a small study, a great financial interest and a low pre-study probability (not applicable) .
It is true that NEJM has a very high impact factor. This is a measure for how often a paper in that journal is cited by others. Of course researchers want to get their paper published in a high impact journal. But journals with high impact factors often go for trendy topics and positive results. In other words it is far more difficult to publish a good quality study with negative results, and certainly in an English high impact journal. This is called publication bias (and language bias) [8]. Positive studies will also be more frequently cited (citation bias) and will more likely be published more than once (multiple publication bias) (indeed, Katan et al already published about the trial [9], and have not presented all their data yet [1,7]). All forms of bias are a distortion of the “truth”.
(This is the reason why the search for a (Cochrane) systematic review must be very sensitive [8] and not restricted to core clinical journals, but even include non-published studies: for these studies might be “true”, but have failed to get published).
Indeed, the group of Ioannidis just published a large-scale statistical analysis[10] showing that medical studies revealing “very large effects” seldom stand up when other researchers try to replicate them. Often studies with large effects measure laboratory and/or surrogate markers (like BMI) instead of really clinically relevant outcomes (diabetes, cardiovascular complications, death)
Importantly, the NEJM has the highest proportion of trials (RCTs) with sole industry support(35% compared to 7% in the BMJ) [12] . On several occasions I have discussed these conflicts of interests and their impact on the outcome of studies ([13, 14; see also [15,16] In their study, Gøtzsche and his colleagues from the Nordic Cochrane Centre [12] also showed that industry-supported trials were more frequently cited than trials with other types of support, and that omitting them from the impact factor calculation decreased journal impact factors. The impact factor decrease was even 15% for NEJM(versus 1% for BMJ in 2007)! For the journals who provided data, income from the sales of reprints contributed to 3% and 41% of the total income for BMJ and The Lancet.
A recent study, co-authored by Ben Goldacre (MD & science writer) [17] confirms that funding by the pharmaceutical industry is associated with high numbers of reprint orders. Again only the BMJ and the Lancet provided all necessary data.
Finally and most relevant to the topic is a study [18], also discussed at Retractionwatch[19], showing that articles in journals with higher impact factors are more likely to be retracted and surprise surprise, the NEJM clearly stands on top. Although other reasons like higher readership and scrutiny may also play a role [20], it conflicts with Katan’s idea that ”publishing in the NEJM is the best guarantee something is true”.
I wasn’t aware of the latter study and would like to thank drVes and Ivan Oranski for responding to my crowdsourcing at Twitter.
@laikas re: "NEJM publishing is the best guarantee something is true" - Incorrect, of course. Just look at the retractions, etc.
de Ruyter JC, Olthof MR, Seidell JC, & Katan MB (2012). A trial of sugar-free or sugar-sweetened beverages and body weight in children. The New England journal of medicine, 367 (15), 1397-406 PMID: 22998340
de Ruyter JC, Olthof MR, Kuijper LD, & Katan MB (2012). Effect of sugar-sweetened beverages on body weight in children: design and baseline characteristics of the Double-blind, Randomized INtervention study in Kids. Contemporary clinical trials, 33 (1), 247-57 PMID: 22056980
Pereira, T., Horwitz, R.I., & Ioannidis, J.P.A. (2012). Empirical Evaluation of Very Large Treatment Effects of Medical InterventionsEvaluation of Very Large Treatment Effects JAMA: The Journal of the American Medical Association, 308 (16) DOI: 10.1001/jama.2012.13444
Lundh, A., Barbateskovic, M., Hróbjartsson, A., & Gøtzsche, P. (2010). Conflicts of Interest at Medical Journals: The Influence of Industry-Supported Randomised Trials on Journal Impact Factors and Revenue – Cohort Study PLoS Medicine, 7 (10) DOI: 10.1371/journal.pmed.1000354
Lexchin, J. (2003). Pharmaceutical industry sponsorship and research outcome and quality: systematic review BMJ, 326 (7400), 1167-1170 DOI: 10.1136/bmj.326.7400.1167
Smith R (2005). Medical journals are an extension of the marketing arm of pharmaceutical companies. PLoS medicine, 2 (5) PMID: 15916457 (free full text at PLOS)
Handel, A., Patel, S., Pakpoor, J., Ebers, G., Goldacre, B., & Ramagopalan, S. (2012). High reprint orders in medical journals and pharmaceutical industry funding: case-control study BMJ, 344 (jun28 1) DOI: 10.1136/bmj.e4212
Fang, F., & Casadevall, A. (2011). Retracted Science and the Retraction Index Infection and Immunity, 79 (10), 3855-3859 DOI: 10.1128/IAI.05661-11
Agrawal A, & Sharma A (2012). Likelihood of false-positive results in high-impact journals publishing groundbreaking research. Infection and immunity, 80 (3) PMID: 22338040
——————————————–
* Addendum: my (unpublished) letter to the NRC
Tour de France.
Nadat het NRC eerder 2 pagina’ s de loftrompet over Katan’s nieuwe studie had afgestoken, vond Katan het nodig om dit in zijn eigen column dunnetjes over te doen. Verwijzen naar je eigen werk mag, ook in een column, maar dan moeten wij daar als lezer wel wijzer van worden. Wat is nu de boodschap van dit stuk “Col hors Catégorie“? Het beschrijft vooral de lange weg om een wetenschappelijke studie gepubliceerd te krijgen in een toptijdschrift, in dit geval de New England Journal of Medicine (NEJM), “de Tour de France onder de medische tijdschriften”. Het stuk eindigt met een tackle naar een journalist “die dacht dat hij het beter wist”. Maar ach, wat geeft dat als de hele wereld staat te jubelen? Erg onsportief, omdat die journalist (van Maanen, Volkskrant) wel degelijk op een aantal punten scoorde. Ook op Katan’s kernpunt dat een NEJM-publicatie “de beste garantie is dat iets waar is” valt veel af te dingen. De NEJM heeft inderdaad een hoge impactfactor, een maat voor hoe vaak artikelen geciteerd worden. De NEJM heeft echter ook de hoogste ‘artikelterugtrekkings’ index. Tevens heeft de NEJM het hoogste percentage door de industrie gesponsorde klinische trials, die de totale impactfactor opkrikken. Daarnaast gaan toptijdschriften vooral voor “positieve resultaten” en “trendy onderwerpen”, wat publicatiebias in de hand werkt. Als we de vergelijking met de Tour de France doortrekken: het volbrengen van deze prestigieuze wedstrijd garandeert nog niet dat deelnemers geen verboden middelen gebruikt hebben. Ondanks de strenge dopingcontroles.
The second Continuing Education Course (CEC) I followed was given byTiina Heino and Katri Larmo of the Terkko Meilahti Campus Library at the University of Helsinki in Finland.
The full title of the course was Visibility and impact – library’s new role: How the library can support the researcher to get visibility and generate impact to researcher’s work. You can read the abstract here.
The hands-on workshop mainly concentrated on the social bookmarking sites Connotea, Mendeley and Altmetric.
Furthermore we got information on CiteULike, ORCID, Faculty of 1000 Posters and Pinterest. Also services developed in Terkko, such as ScholarChart and TopCited Articles, were shortly demonstrated.
What I especially liked in the hands on session is that the tutors had prepared a wikispace with all the information and links on the main page ( https://visibility2012.wikispaces.com) and a separate page for each participant to edit (here is my page). You could add links to your created accounts and embed widgets for Mendeley.
There was sufficient time to practice and try the tools. And despite the great number of participants there was ample room for questions (& even for making a blog draft ).
The main message of the tutors is that the process of publishing scientific research doesn’t end at publishing the article: it is equally important what happens after the research has been published. Visibility and impact in the scientific community and in the society are crucial for making the research go forward as well as for getting research funding and promoting the researcher’s career. The Fig below (taken from the presentation) visualizes this process.
The tutors discussed ORCID, Open Researcher and contributor ID, that will be introduced later this year. It is meant to solve the author name ambiguity problem in scholarly communication by central registry of unique identifiers for each author (because author names can’t be used to reliably identify all scholarly author). It will be possible for authors to create, manage and share their ORCID record without membership fee. For further information see several publications and presentations by Martin Fenner. I found this one during the course while browsing Mendeley.
Once published the author’s work can be promoted using bookmarking tools, like CiteULike, Connotea and Mendeley. You can easily registrate for Connotea and Mendeley using your Facebook account. These social bookmarking tools are also useful for networking, i.e. to discover individuals and groups with the same field of interest. It is easy to synchronize your Mendeley with your CiteULike account.
Mendeley is available in a desktop and a web version. The web version offers a public profile for researchers, a catalog of documents, and collaborative groups (the cloud of Mendeley). The desktop version of Mendeley is specially suited for reference management and organizing your PDF’s. That said Mendeley seems most suitable for serendipitous use (clicking and importing a reference you happen to see and like) and less useful for managing and deduplicating large numbers of records, i.e. for a systematic review.
Also (during the course) it was not possible to import several PubMed records at once in either CiteULike or Mendeley.
What stroke me when I tried Mendeley is that there were many small or dead groups. A search for “cochrane” for instance yielded one large group Cochrane QES Register, owned by Andrew Booth, and 3 groups with one member (thus not really a group), with 0 (!) to 6 papers each! It looks like people are trying Mendeley and other tools just for a short while. Indeed, most papers I looked up in PubMed were not bookmarked at all. It makes you wonder how widespread the use of these bookmarking tools is. It probably doesn’t help that there are so many tools with different purposes and possibilities.
Another tool that we tried was Altmetric. This is a free bookmarklet on scholarly articles which allows you to track the conversations around scientific articles online. It shows the tweets, blogposts, Google+ and Facebook mentions, and the numbers of bookmarks on Mendeley, CiteULike and Connotea.
Indeed altmetrics did correctly refer to my blog (even to 2 posts).
I liked altmetrics*, but saying that it is suitable for scientific metrics is a step too far. For people interested in this topic I would like to refer -again- to a post of Martin Fenner on altmetrics (in general). He stresses that “usage metrics” has its limitations because of its proness to “gaming” (cheating).
But the current workshop didn’t address the shortcomings of the tools, for it was meant as a first practical acquaintance with the web 2.0 tools.
For the other tools (Faculty of 1000 Posters, Pinterest) and the services developed in Terkko, such as ScholarChart and TopCited Articles, see the wikipage and the presentation:
*Coincidentally I’m preparing a post on handy chrome extensions to look for tweets about a webpage. Altmetric is another tool which seems very suitable for this purpose
But now it appears that there is an entire list of “Predatory, Open-Access Publishers”. This list was created by Jeffrey Beall, academic librarian at the University of Colorado Denver. He just updated the list for 2012 here (PDF-format).
According to Jeffrey predatory, open-access publishers
are those that unprofessionally exploit the author-pays model of open-access publishing (Gold OA) for their own profit. Typically, these publishers spam professional email lists, broadly soliciting article submissions for the clear purpose of gaining additional income. Operating essentially as vanity presses, these publishers typically have a low article acceptance threshold, with a false-front or non-existent peer review process. Unlike professional publishing operations, whether subscription-based or ethically-sound open access, these predatory publishers add little value to scholarship, pay little attention to digital preservation, and operate using fly-by-night, unsustainable business models.
Jeffrey recommends not to do business with the following (illegitimate) publishers, including submitting article manuscripts, serving on editorial boards, buying advertising, etc. According to Jeffrey, “there are numerous traditional, legitimate journals that will publish your quality work for free, including many legitimate, open-access publishers”.
(For sake of conciseness, I only describe the main characteristics, not always using the same wording; please see the entire list for the full descriptions.)
PAGEPressmany dead links, a prominent link to PayPal
Versita Open paid subscription for print form. ..unclear business model
An asterisk (*) indicates that the publisher is appearing on this list for the first time.
How complete and reliable is this list?
Clearly, this list is quite exhaustive. Jeffrey did a great job listing many dodgy OA journals. We should watch (many) of these OA publishers with caution. Another good thing is that the list is updated annually.
Personally, I would have preferred a distinction between real bogus or spammy journals and journals that seem to have “too many journals to properly handle” or that ask (too much ) money for subscription/from the author. The scientific content may still be good (enough).
Furthermore, I would rather see a neutral description of what is exactly wrong about a journal. Especially because “Beall’s list” is a list and not a blog post (or is it?). Sometimes the description doesn’t convince me that the journal is really bogus or predatory.
Examples of subjective portrayals:
Dove Press: This New Zealand-based medical publisher boasts high-quality appearing journals and articles, yet it demands a very high author fee for publishing articles. Its fleet of journals is large, bringing into question how it can properly fulfill its promise to quickly deliver an acceptance decision on submitted articles.
Libertas Academia ”The tag line under the name on this publisher’s page is “Freedom to research.” It might better say “Freedom to be ripped off.”
Hindawi .. This publisher has way too many journals than can be properly handled by one publisher, I think (…)
I do like funny posts, but only if it is clear that the post is intended to be funny. Like the one by Alan Dove PhD about JCBBR.
JCBBR is dedicated to increasing the depth of research across all areas of this subject.
Translation: we’re launching a new journal for research that can’t get published anyplace else.
The journal welcomes the submission of manuscripts that meet the general criteria of significance and scientific excellence in this subject area.
We’ll take pretty much any crap you excrete.
Hattip: Catherine Arnott Smith, PhD at the MedLib-L list.
One of the TwitJC organizers, Fi Douglas (@fidouglas on Twitter), argued that using paid-for journals would defeat the objective that #TwitJC is open to everyone. I can imagine that fee-based articles could set a too high threshold for many doctors. In addition, I sympathize with promoting OA.
But is there evidence for my theory that selecting FFT papers might lead to bias?
Lets first look at the extent of the problem. Which percentage of papers do we miss by limiting for free-access papers?
A survey in PLOS by Björk et al [1] found that one in five peer reviewed research papers published in 2008 were freely available on the internet. Overall 8,5% of the articles published in 2008 (and 13,9 % in Medicine) were freely available at the publishers’ sites (gold OA). For an additional 11,9% free manuscript versions could be found via the green route: i.e. copies in repositories and web sites (7,8% in Medicine). As a commenter rightly stated, the lag time is also important, as we would like to have immediate access to recently published research, yet some publishers (37%) impose an access-embargo of 6-12 months or more. (these papers were largely missed as the 2008 OA status was assessed late 2009).
PLOS 2009
The strength of the paper is that it measures OA prevalence on an article basis, not on calculating the share of journals which are OA: an OA journal generally contains a lower number of articles.
The authors randomly sampled from 1.2 million articles using the advanced search facility of Scopus. They measured what share of OA copies the average researcher would find using Google.
Another paper published in J Med Libr Assoc (2009) [2], using similar methods as the PLOS survey examined the state of open access (OA) specifically in the biomedical field. Because of its broad coverage and popularity in the biomedical field, PubMed was chosen to collect their target sample of 4,667 articles. Matsubayashi et al used four different databases and search engines to identify full text copies. The authors reported an OA percentage of 26,3 for peer reviewed articles (70% of all articles), which is comparable to the results of Björk et al. More than 70% of the OA articles were provided through journal websites. The percentages of green OA articles from the websites of authors or in institutional repositories was quite low (5.9% and 4.8%, respectively).
In their discussion of the findings of Matsubayashi et al, Björk et al. [1] quickly assessed the OA status in PubMed by using the new “link to Free Full Text” search facility. First they searched for all “journal articles” published in 2005 and then repeated this with the further restrictions of “link to FFT”. The PubMed OA percentages obtained this way were 23,1 for 2005 and 23,3 for 2008.
This proportion of biomedical OA papers is gradually increasing. A chart in Nature’s News Blog [9] shows that the proportion of papers indexed on the PubMed repository each year has increased from 23% in 2005 to above 28% in 2009. (Methods are not shown, though. The 2008 data are higher than those of Björk et al, who noticed little difference with 2005. The Data for this chart, however, are from David Lipman, NCBI director and driving force behind the digital OA archive PubMed Central).
Again, because of the embargo periods, not all literature is immediately available at the time that it is published.
In summary, we would miss about 70% of biomedical papers by limiting for FFT papers. However, we would miss an even larger proportion of papers if we limit ourselves to recently published ones.
Of course, the key question is whether ignoring relevant studies not available in full text really matters.
Reinhard Wentz of the Imperial College Library and Information Service already argued in a visionary 2002 Lancet letter[3] that the availability of full-text articles on the internet might have created a new form of bias: FUTON bias (Full Text On the Net bias).
Wentz reasoned that FUTON bias will not affect researchers who are used to comprehensive searches of published medical studies, but that it will affect staff and students with limited experience in doing searches and that it might have the same effect in daily clinical practice as publication bias or language bias when doing systematic reviews of published studies.
Wentz also hypothesized that FUTON bias (together with no abstract available (NAA) bias) will affect the visibility and the impact factor of OA journals. He makes a reasonable cause that the NAA-bias will affect publications on new, peripheral, and under-discussion subjects more than established topics covered in substantive reports.
The study of Murali et al [4] published in Mayo Proceedings 2004 confirms that the availability of journals on MEDLINE as FUTON or NAA affects their impact factor.
Of the 324 journals screened by Murali et al. 38.3% were FUTON, 19.1% NAA and 42.6% had abstracts only. The mean impact factor was 3.24 (±0.32), 1.64 (±0.30), and 0.14 (±0.45), respectively! The authors confirmed this finding by showing a difference in impact factors for journals available in both the pre and the post-Internet era (n=159).
Murali et al informally questioned many physicians and residents at multiple national and international meetings in 2003. These doctors uniformly admitted relying on FUTON articles on the Web to answer a sizable proportion of their questions. A study by Carney et al (2004) [5] showed that 98% of the US primary care physicians used the Internet as a resource for clinical information at least once a week and mostly used FUTON articles to aid decisions about patient care or patient education and medical student or resident instruction.
Murali et al therefore conclude that failure to consider FUTON bias may not only affect a journal’s impact factor, but could also limit consideration of medical literature by ignoring relevant for-fee articles and thereby influence medical education akin to publication or language bias.
This proposed effect of the FFT limit on citation retrieval for clinical questions, was examined in a more recent study (2008), published in J Med Libr Assoc [6].
Across all 4 questions based on a research agenda for physical therapy, the FFT limit reduced the number of citations to 11.1% of the total number of citations retrieved without the FFT limit in PubMed.
Even more important, high-quality evidence such as systematic reviews and randomized controlled trials were missed when the FFT limit was used.
For example, when searching without the FFT limit, 10 systematic reviews of RCTs were retrieved against one when the FFT limit was used. Likewise when searching without the FFT limit, 28 RCTs were retrieved and only one was retrieved when the FFT limit was used.
The proportion of missed studies (appr. 90%) is higher than in the studies mentioned above. Possibly this is because real searches have been tested and that only relevant clinical studies have been considered.
The authors rightly conclude that consistently missing high-quality evidence when searching clinical questions is problematic because it undermines the process of Evicence Based Practice. Krieger et al finally conclude:
“Librarians can educate health care consumers, scientists, and clinicians about the effects that the FFT limit may have on their information retrieval and the ways it ultimately may affect their health care and clinical decision making.”
It is the hope of this librarian that she did a little education in this respect and clarified the point that limiting to free full text might not always be a good idea. Especially if the aim is to critically appraise a topic, to educate or to discuss current best medical practice.
References
Björk, B., Welling, P., Laakso, M., Majlender, P., Hedlund, T., & Guðnason, G. (2010). Open Access to the Scientific Journal Literature: Situation 2009 PLoS ONE, 5 (6) DOI: 10.1371/journal.pone.0011273
Matsubayashi, M., Kurata, K., Sakai, Y., Morioka, T., Kato, S., Mine, S., & Ueda, S. (2009). Status of open access in the biomedical field in 2005 Journal of the Medical Library Association : JMLA, 97 (1), 4-11 DOI: 10.3163/1536-5050.97.1.002
WENTZ, R. (2002). Visibility of research: FUTON bias The Lancet, 360 (9341), 1256-1256 DOI: 10.1016/S0140-6736(02)11264-5
Murali NS, Murali HR, Auethavekiat P, Erwin PJ, Mandrekar JN, Manek NJ, & Ghosh AK (2004). Impact of FUTON and NAA bias on visibility of research. Mayo Clinic proceedings. Mayo Clinic, 79 (8), 1001-6 PMID: 15301326
Carney PA, Poor DA, Schifferdecker KE, Gephart DS, Brooks WB, & Nierenberg DW (2004). Computer use among community-based primary care physician preceptors. Academic medicine : journal of the Association of American Medical Colleges, 79 (6), 580-90 PMID: 15165980
Krieger, M., Richter, R., & Austin, T. (2008). An exploratory analysis of PubMed’s free full-text limit on citation retrieval for clinical questions Journal of the Medical Library Association : JMLA, 96 (4), 351-355 DOI: 10.3163/1536-5050.96.4.010
The decision of the editors was based on the failure of at least 10 other studies to confirm these findings and on growing support that the results were caused by contamination. When the authors refused to retract their paper, Science issued an Expression of Concern [2].
In my opinion retraction is premature. Science should at least await the results of two multi-center studies, that were designed to confirm or disprove the results. These studies will continue anyway… The budget is already allocated.
Furthermore, I can’t suppress the idea that Science asked for a retraction to exonerate themselves for the bad peer review (the paper had serious flaws) and their eagerness to swiftly publish the possibly groundbreaking study.
And what about the other studies linking the XMRV to ME/CFS or other diseases: will these also be retracted?
And what happens in the improbable case that the multi-center studies confirm the 2009 paper? Would Science republish the retracted paper?
Thus in my opinion, it is up to other scientists to confirm or disprove findings published. Remember that falsifiability was Karl Popper’s basic scientific principle. My conclusion was that “fraud is a reason to retract a paper and doubt is not”.
This is my opinion, but is this opinion shared by others?
When should editors retract a paper? Is fraud the only reason? When should editors issue a letter of concern? Are there guidelines?
Schekman considers it “an unusual situation to retract a paper even if the original findings in a paper don’t hold up: it’s part of the scientific process for different groups to publish findings, for other groups to try to replicate them, and for researchers to debate conflicting results.”
I don’t have any hard numbers on how often journals ask scientists to retract a paper, only my sense that it is very rare. Author retractions are more frequent, but I’m only aware of a handful of those in a year. I can recall a few other cases in which the authors were asked to retract a paper, but in those cases scientific fraud was involved. That’s not the case here. I don’t believe there is a standard policy that enumerates how such decisions are made; if they exist they are not public.
However, there is a Guideline for editors, the Guidance from the Committee on Publication Ethics (COPE) (PDF) [5]
Ivanoranski, of the great blog Retraction Watch, linked to it when we discussed reasons for retraction.
With regard to retraction the COPE-guidelines state that journal editors should consider retracting a publication if:
they have clear evidence that the findings are unreliable, either as a result of misconduct (e.g. data fabrication) or honest error (e.g. miscalculation or experimental error)
the findings have previously been published elsewhere without proper crossreferencing, permission or justification (i.e. cases of redundant publication)
it constitutes plagiarism
it reports unethical research
According to the same guidelines journal editors should consider issuing an expression of concern if:
they receive inconclusive evidence of research or publication misconduct by the authors
there is evidence that the findings are unreliable but the authors’ institution will not investigate the case
they believe that an investigation into alleged misconduct related to the publication either has not been, or would not be, fair and impartial or conclusive
an investigation is underway but a judgement will not be available for a considerable time
Thus in the case of the Science XMRV/CSF paper an expression of concern certainly applies (all 4 points) and one might even consider a retraction, because the results seem unreliable (point 1). But it is not 100% established that the findings are false. There is only serious doubt……
The guidelines seem to leave room for separate decisions. To retract a paper in case of plain fraud is not under discussion. But when is an error sufficiently established ànd important to warrant retraction?
Apparently retractions are on the rise. Although still rare (0.02% of all publications by the late 2000s) there has been a tenfold increase in retractions compared to the early 1980s (see review at Scholarly Kitchen [6] about two papers: [7] and [8]). However it is unclear whether increasing rates of retraction reflect more fraudulent or erroneous papers or a better diligence. The first paper [7] also highlights that, out of fear of litigation, editors are generally hesitant to retract an article without the author’s permission.
At the blog Nerd Alert they give a nice overview [9] (based on Retraction Watch, but then summarized in one post ) . They clarify that papers are retracted for “less dastardly reasons then those cases that hit the national headlines and involve purposeful falsification of data”, such as the fraudulent papers of Andrew Wakefield (autism caused by vaccination). Besides the mistaken publication of the same paper twice, data over-interpretation, plagiarism and the like, the reason can also be more trivial: ordering the wrong mice or using an incorrectly labeled bottle.
Still, scientist don’t unanimously agree that such errors should lead to retraction.
Drug Monkeyblogs about his discussion [10] with @ivanoransky over a recent post at Retraction Watch, which asks whether a failure to replicate a result justifies a retraction [11]“. Ivanoransky presents a case, where a researcher (B) couldn’t reproduce the findings of another lab (A) and demonstrated mutations in the published protein sequence that excluded the mechanism proposed in A’s paper. This wasn’t retracted, possibly because B didn’t follow the published experimental protocols of A in all details. (reminds me of the XMRV controversy).
Drugmonkeysays (quote): (cross-posted at Scientopia here – hmmpf isn’t that an example of redundant publication?)
“I don’t give a fig what any journals might wish to enact as a policy to overcompensate for their failures of the past. In my view, a correction suffices” (provided that search engines like Google and PubMed make clear that the paper was in fact corrected).
Drug Monkey has a point there. A clear watermark should suffice.
However, we should note that most papers are retracted by authors, not the editors/journals, and that the majority of “retracted papers” remain available. Just 13.2% are deleted from the journal’s website. And 31% are not clearly labelled as such.
Summary of how the naïve reader is alerted to paper retraction (from Table 2 in [7], see: Scholarly Kitchen [6])
Watermark on PDF (41.1%)
Journal website (33.4%)
Not noted anywhere (31.8%)
Note appended to PDF (17.3%)
PDF deleted from website (13.2%)
My conclusion?
Of course fraudulent papers should be retracted. Also papers with obvious errors that invalidate the conclusions.
However, we should be extremely hesitant to retract papers that can’t be reproduced, if there is no undisputed evidence of error.
All retracted papers (and papers with major deficiencies and shortcomings) should be clearly labeled as such (as Drugmonkey proposed, not only at the PDF and at the Journal website, but also by search engines and biomedical databases).
Or lets hope, with Biochembelle [15], that the future of scientific publishing will make retractions for technical issues obsolete (whether in the form of nano-publications [16] or otherwise):
One day the scientific community will trade the static print-type approach of publishing for a dynamic, adaptive model of communication. Imagine a manuscript as a living document, one perhaps where all raw data would be available, others could post their attempts to reproduce data, authors could integrate corrections or addenda….
Wager E, Williams P. Why and how do journals retract articles? An analysis of Medline retractions 1988-2008. J Med Ethics. 2011 Apr 12. [Epub ahead of print]
Steen RG. Retractions in the scientific literature: is the incidence of research fraud increasing? J Med Ethics. 2011 Apr;37(4):249-53. Epub 2010 Dec 24.
Greenfield also happened to have a minor side job as the editor-in-chief of Elsevier’s Surgery News. Surgery News is not a peer-reviewed journal, but what Greenfield later defines as a monthly throw-away newspaper (of the kind Elsevier produces a lot).
As an-editor-in chief Greenfield wrote open editorials (opinion pieces) for Surgery News. He found a very suitable theme for the February issue: Valentine’s day.
Valentine’s Day is about love, and the editorial was about romantic gut feeling possibly having a physiological basis. In other words, the world of sexual chemical signals that give you butterflies-feelings. The editorial jumps from mating preferences of fruit flies, stressed female rotifers turning into males and synchronization of menstrual cycles of women who live together, to a study suggesting that “exposure” to semen makes female college students less depressed. All 4 topics are based on scientific research, published in peer review papers.
Valentines Day asks for giving this “scientific” story a twist, so he concludes the editorial as follows:
“So there’s a deeper bond between men and women than St. Valentine would have suspected, and now we know there’s a better gift for that day than chocolates.”
Now, everybody knows that that conclusion ain’t supported by the data.
This would have required at least a double-blind randomized trial, comparing the mood-enhancing effects of chocolate compared to ……. (yikes!).
Just joking, of course…., similar as dear Lazar was trying to be funny….
No, the editorial wasn’t particularly funny.
And somehow it isn’t pleasant to think of a man’s love fluid wrapped in a ribbon and a box with hearts, while you expect some chocolates. Furthermore it suggests that sperm is something a man just gives/donates/injects, not a resultant of mutual love.
However this was the opposite of what Greenfield had in mind:
Thus the man just tried to “Amuse his readers” and highlight research on “some fascinating new findings related to semen.”
I would have appreciated a more subtle ending of the editorial, but I would take no offense.
….Unlike many of his fellow female surgeons. The Women in Surgery Committee and the Association of Women Surgeons considered his editorial as “demeaning to women” (NY-Times).
He offered his sincere apologies and resigned as Editor-in-Chief of the paper. The publication was retracted. As a matter of fact the entire February issue of Surgery News was taken off the ACS-website. Luckily, Retraction Watch published the editorial in its entirety.
Greenfield’s apologies weren’t enough, women surgeons brought the issue to the Board of Regents, who asked him to resign, which he eventually did.
A few weeks later he wrote a resentful letter. This is not a smart thing to do, but is understandable for several reasons. First, he didn’t he mean to be offensive and made his apologies. Second, he has an exemplary career as a longtime mentor and advocate of women in surgery. Third, true reason for his resign wasn’t the implicit plead for unprotected sex, but rather that the editorial reflected “a macho culture in surgery that needed to change.”Fourth, his life is ruined over something trivial.
Why can’t one write a lighthearted opinion-piece at Valentine’s day without getting resigned? Is it because admitting that “the “bond between men and women” is natural and runs deep” is one of those truths you cannot utter (Paul Rahe).
Is this perhaps typically American?
Elmar Veerman (Dutch Journalist, science editor at VPRO) comments at at Retraction Watch:
(…) Frankly, I don’t see the problem. I find it rather funny and harmless. Perhaps because I’m from Europe, where most people have a more relaxed attitude towards sex. Something like ‘nipplegate’ could never happen here (a nipple on tv, so what). (…)I have been wondering for years why so many Americans seem to think violence is fine and sex is scary.
Not only female surgeons object to the editorial. Well-known male (US) surgeons “fillet” the editorial at their blogs: Jeffrey Parks at Buckeye Surgeon ( 1 and 2), Orac Knows at Respectful Insolence (1 and 2) and Skeptical Scalpel (the latter quite mildly).
Jeffrey and Orac do not only think the man is humorless and a sexist, but also that the science behind the mood-enhancing aspects of semen is crap.
Although Jeffrey only regards “The “science” a little suspect as per Orac.”…. Because of course: “Orac knows.”
Orac exaggerates what Greenfield has said in the “breathtakingly inappropriate and embarrassing article for Surgery News”, as he calls it. [1]: ”Mood-enhancing effects of semen” becomes in Orac’s words “the cure for female depression“ and ”a woman needs a man to inject his seed into her in order to be truly happy“.
Of course, it is not fair to twist words this way.
I didn’t look at the original articles and I don’t really expect much of it. However, it just shows the Gallop study is not the only study, linking semen to positive health effects.
Assuming Greenfield had more than a joke in mind, and wanted to reflect on the state of art of health aspects of semen, it surprises me that he didn’t dig any further than this article from 2002.
Mentioned in a comment somewhere, but I can’t track it down.
If Greenfield used Scientific American as a source he should have read it all to the end, where the author states: “I bid adieu, please accept, in all sincerity, my humblest apologies for what is likely to be a flood of bad, off-color jokes—men saying, “I’m not a medical doctor, but my testicles are licensed pharmaceutical suppliers” and so on—tracing its origins back to this innocent little article. Ladies, forgive me for what I have done.”
Peer review is never funny, you think.
It is hard to review papers, especially when they are poorly written. From the author’s point of view, it is annoying and frustrating to see a paper rejected on basis of comments of peer reviewers, who either don’t understand the paper or thwart you in your attempts to get the paper published, for instance because you are a competitor in the field.
Still, from a (great) distance the peer review process can be funny… in some respects.
Read for instance a collection of memorable quotes from peer review critiques of the past year in Environmental Microbiology (EM does this each December). Here are some excerpts:
Done! Difficult task, I don’t wish to think about constipation and faecal flora during my holidays!
This paper is desperate. Please reject it completely and then block the author’s email ID so they can’t use the online system in future.
It is sad to see so much enthusiasm and effort go into analyzing a dataset that is just not big enough.
The abstract and results read much like a laundry list.
.. I would suggest that EM is setting up a fund that pays for the red wine reviewers may need to digest manuscripts like this one.
I have to admit that I would have liked to reject this paper because I found the tone in the Reply to the Reviewers so annoying.
I started to review this but could not get much past the abstract.
This paper is awfully written. There is no adequate objective and no reasonable conclusion. The literature is quoted at random and not in the context of argument…
Stating that the study is confirmative is not a good start for the Discussion.
I suppose that I should be happy that I don’t have to spend a lot of time reviewing this dreadful paper; however I am depressed that people are performing such bad science.
Preliminary and intriguing results that should be published elsewhere.
Reject – More holes than my grandad’s string vest!
The writing and data presentation are so bad that I had to leave work and go home early and then spend time to wonder what life is about.
Very much enjoyed reading this one, and do not have any significant comments. Wish I had thought of this one.
This is a long, but excellent report. [...] It hurts me a little to have so little criticism of a manuscript.
More seriously, the Top 20 Reasons (Negative Comments) Written by the Reviewers Recommending Rejection of 123 Medical Education Manuscripts can be found at Academic Medicine (vol 76, no . 9 / 2 0 0 1). The top 5 is:
Statistics: inappropriate, incomplete, or insufficiently described, etc. 11.2 %
Overinterpretation of the results 8.7 %
Inappropriate, suboptimal, insufficiently described instrument 7.3%
Sample too small or biased 5.6 %
Text difficult to follow, to understand 3.9%
Neuroskeptic describes 9 types of review decisions in the The Wheel of Peer Review. Was your paper reviewed by “Bee-in-your-Bonnet” or by “Cite Me, Me, Me!”
The subjects you propose for a series of Mathematical and Metaphysical Essays are so very profound, that there is perhaps not a single subscriber to our Journal who could follow them.
Responses to a rejection are also of all ages. See this video anno 1945 (yes this scene has been used tons of times for other purposes)
Yet if you are still unsuccessful and want a definitive decision rendered within hours of submission you can always send your paper to the Journal of Universal Rejection.
Almost a year ago I started a new series Kaleidoscope, with a “kaleidoscope” of facts, findings, views and news gathered over the last 1-2 weeks.
It never got beyond the first edition. Perhaps the introduction of this Kaleidoscope was to overwhelming & dazzling: lets say it was very rich in content. Or as Andrew Spong tweeted: “Part cornucopia, part cabinet of wonders, it’s @laikas Kaleidoscope 2009 wk 47″
This is a reprise in a (somewhat) “shorter” format. Lets see how it turns out.
This edition will concentrate on Social Media (Blogging, Twitter Google Wave). I fear that I won’t keep my promise, if I deal with more topics.
Medical Grand Rounds and News from the Blogosphere
Life in the Fast Lane is the host of this weeks Grand Rounds. This edition is truly terrific, if not terrifying. Not only does it contain “killer posts”, each medblogger has also been coupled to its preferred deadly Aussie critter.
Want to know how a full time ER-doctor/educator/textbook author/blogger/editor /health search engine director manages to complete work-related tasks …when the kids are either at school or asleep(!), then read this recent interview with Mike Cadogan, the founder of Life in the Fast Lane.
And certainly don’t forget to submit your post related to medical information to the MedLibs Round (about medical information) here. More details can be found at Laika’s MedLibLog and at Highlight Health, the host of the upcoming Edition. (sorry, writing this post took longer than I thought: you have one day left for submission)
Dr Shock of the blog with the same name advises us to submit good quality, easy-to-understand posts dealing with science, environment or medicine to Scientia Pro Publicavia the blog carnival submission form.
There is a new on-line science blogging community – Scientopia, till now mostly consisting of bloggers who left Scienceblogs after (but not because of) Pepsigate. New members can only be added to the collective by invitation (?). Obviously, pepsi-researchers will not be invited, but it remains to be seen who will… Hopefully it doesn’t become an elitist club. Virginia Heffernan (NY-Times)has an outspoken opinion about the (ex-) sciencebloggers, illustrated by this one-liner
“ScienceBlogs has become Fox News for the religion-baiting, peak-oil crowd.”
Although I don’t appreciate the ranting-style of some of the blogs myself (the sub-“South Park” blasphemy style of PZ Myers, as Virginia puts it). I don’t think most Scienceblogs deserve to be labelled as “preoccupied with trivia, name-calling and saber rattling”.
See balanced responses at: Neurodojo, Neuron Culture & Neuroanthropology (anything with neuro- makes sense, I guess).
Want to understand more about ScienceBlogs and why it was such a terrific community, then read Bora Z’s (rather long) ScienceBlog farewell post.
Oh.. and there is yet another new science blogging platform: http://www.labspaces.net/, that has evolved from a science news aggregator . It looks slick.
Social Media
Speaking about Twitter, did you know that Twitter reached its 20 billionth tweet over the weekend, a milestone that came just a few months after hitting the 10 billion tweet mark!? (read more in the Guardian)
Well and if you have no idea WHAT THE FUCK IS MY SOCIAL MEDIA “STRATEGY”? you might click the link to get some (new) ideas. You probably need to refresh the site a couple of times to find the right answer.
“Especially in medicine, we’re using so many different resources, including all the syllabuses and slides. I’m able to pull them up and search them whenever I need to. It’s a fantastic idea.”
Good news for Facebook friends: VoIP giant Vonage has just introduced a new iPhone, iPod touch and Android app that allows users to call their Facebook friends for free (Mashable).
Votes down for the social media site Digg.com: an undercover investigation has exposed that a group of influential conservative members were involved in censorship, deliberately trying to ban progressives, by “burying them” (voting down), which effectively means these progressives don’t get enough “digs” to reach the front page where most users spend their time.
A very strange move: a journal has changed a previously stated conclusion of a previously published paper after a Reuters Health story about serious shortcomings in the report. Read more about it at Gary Schwitzer’s HealthNewsReview Blog.
Finally for the EBM-addicts among us: The Center of Evidence Based Medicine released a new (downloadable) Levels of Evidence Table. At the CEBM-blog they stress that hierarchies of evidence have been somewhat inflexibly used, but are essentially a heuristic, or short-cut to finding the likely best evidence. At first sight the new Table looks simpler, and more easy to use.
Welcome to this months edition of MedLib’s Round, a blog carnival of “excellent blog posts in the field of medical information”.
This round is a little belated, because of late submissions and my absence earlier this week.
But lets wait no longer …..!
Peer Review, Impact Factors & Conflict of Interest
Walter JessenatHighlight HEALTHwrites about the NIH Peer Review process. Included is an interesting video, that provides an inside look at how scientists from across the US review NIH grant applications for scientific and technical merit. These scientists do seem take their job seriously.
But what about peer review of scientific papers? Richard Smith, doctor, former editor of the BMJ and a proponent of open access publishing, wrote a controversial post at the BMJ Groups Blog called “scrap peer review and beware of “top journals“. Indeed the “top journals” publish the sexy stuff, whereas evidence comprises both the glamorous and the unglamorous. But is prepublication peer review really that bad and should we only filter afterwards?
In a thoughtful post at his Nature blog Confessions of a (former) Lab Ratanother Richard (Grant) argues that although peer review suffers terribly from several shortcomings it is still required. Richard Grant also clears up one misconception:
Peer review, done properly, might guarantee that work is done correctly and to the best of our ability and best intentions, but it will not tell you if a particular finding is right–that’s the job of other experimenters everywhere; to repeat the experiments and to build on them.
At Scholarly Kitchen(about what is hot and cooking in scholarly publishing) theydon’t think “peer review“ is a clear concept, since the list of ingredients differ per journal and article. Read their critical analysis and suggestions for improvement of the standard recipe here.
An interesting view is presented by David Crotty atScholarly Kitchen. He states that it is “hypocritical for ScienceBlog’s bloggers to have objected so strenuously: “ScienceBlogs has never been a temple of purity, free of bias or agenda.” Furthermore the bloggers enjoy more traffic and a fee for being a scienceblogger, and promote their “own business” too. David finds it particularly ironic that these complaints come from the science blogosphere, which has regularlybeena bastionof support for the post-publication review philosophy. Read more here.
Indeed according to a note of Scienceblog at the disappeared blog their intention was “to engage industry in pursuit of science-driven social change”, although it was clearly not the right way.
The partiality of business, including pharma, makes it’s presence in and use of Social Media somewhat tricky. Still it is important for pharma to get involved in web2.0. Interested in a discussion on this topic? Than follow the tags #HCSM (HealthCare Social Media) and #HCSMEU (Europe) on Twitter. Andrew Spong, has launched an open wiki, where you can read all about #HCSMEU.
The value of journal impact factorsis also debatable. In the third part of the series “Show me the evidence” Kathleen Crea at EBM and Clinical Support Librarians @ UCHCstarts with an excerpt of an article with the intriguing title “The Top-Ten in Journal Impact Factor Manipulation”:
“The assumption that Impact Factor (IF) is a number absolutely proportional to science quality has led to misuses beyond the index’s original scope, even in the opinion of its devisor.”
The post itself (Teaching & Learning in Medicine, Research Methodology, Biostatistics: Show Me the Evidence (Part 3)b) is not so much about evidence, but offers a wealth of information about journal impact factors, comparisons of sites for citation analysis, and some educational materials for teaching others about citation analysis. Not only are Journal Citation Reports and SCOPUS discussed, but also the Eigenfactor,h-index and JANE.
Perhaps we need another system of publishing and peer review? Will the future be to publish triplets and peer review thesevia Twitter by as many reviewers as possible? Read about this proposal of Barend Mons (of the same group that created JANE) at this blog. Here you can also find a critical review of an article comparing Google Scholar and PubMed for retrieving evidence.
Social Media, Blogs & Web 2.0 tools
There are several tools to manage the scientific articles, like CiteULike and Mendeley. At his blogGobbledygookMartin Fennerdiscusses the pros and cons of a new web-based tool specifically for discussing papers in Journal Clubs: JournalFire
Heidi Allen atHeidi Allen Digital Strategystarted a discussionon the meaning of social-medicine for Physicians. The link to the original submission doesn’t work right now, but if you follow this link you see several posts on social-medicine, including “Physicians in Social Media”, where 3 well-known physicians give their view on the meaning of social-medicine.
The post of Guus van den Brekel at DigiCMB with the inspiring title Discovering new seas of knowledge partly goes about the seas of knowledge gained at the EAHIL2010 (European Association for Health Information and Libraries) meeting, with an overview of many sessions, and materials when possible. And I should stress when possible, because the other part of the post is about the difficulty of obtaining access to this sea of knowledge. Guus wonders:
In this age of Open Access, web 2.0 and the expectancy of the “users” -being us librarians (…) one would assume that much (if not all) is freely available via Conferences websites and/or social media. Why then do I find it hard to find the extra info about those events, including papers and slides and possibly even webcasts? Are we still not into the share-mode and overprotective to one’s own achievements(….)
Guus makes a good point,especially in this era, when not all of us are able to go and visit far away places. Luckily we have Guus who did a good job of compiling as much material as possible.
Nikki Dettmarpresents two posts on Electronic Health Records at Eagledawg.net, inspired by a recent Medical Library Association meeting that included a lot about electronic health records (EHRs). In the first part “Electronic Health Records: Not All About the Machine” she mentions the launch of an OpenNotes study that ”evaluates the impact on both patients and physicians of sharing, through online medical record portals, the comments and observations made by physicians after each patient encounter.” The second post is entitled “a snapshot of ephemeral chaos“. And yes the title says it all.
Alisha764′s Blogcelebrated its 1 year anniversary in February.Alisha Miles aim for the next year is to not only post more but to focus on hospital libraries including her experience as a hospital librarian. Excellent idea, Alisha! I liked the post Rounding: A solo medical librarian’s perspective with several practical tips if you join the round as a librarian. I hope you can find time to write more like this, Alisha!
Our next host is Walter Jessenat Highlight HEALTH. You can already start submitting the link to a (relevant) post you have written here.
“Libraries and journals articles as we know them will cease to exists” said Barend Mons at the symposium in honor of our Library 25th Anniversary (June 3rd). “Possibly we will have another kind of party in another 25 years”…. he continued, grinning.
What he had to say the next half hour intrigued me. And although I had no pen with me (it was our party, remember), I thought it was interesting enough to devote a post to it.
I’m basing this post not only on my memory (we had a lot of Italian wine at the buffet), but on an article Mons referred to [1], a Dutch newspaper article [2]), other articles [3-6] and Powerpoints [7-9] on the topic.
This is a field I know little about, so I will try to keep it simple (also for my sake).
Mons started by touching on a problem that is very familiar to doctors, scientists and librarians: information overload by a growing web of linked data. He showed a picture that looked like the one at the right (though I’m sure those are Twitter Networks).
As he said elsewhere [3]:
(..) the feeling that we are drowning in information is widespread (..) we often feel that we have no satisfactory mechanisms in place to make sense of the data generated at such a daunting speed. Some pharmaceutical companies are apparently seriously considering refraining from performing any further genome-wide association studies (… whole genome association –…) as the world is likely to produce many more data than these companies will ever be able to analyze with currently available methods .
With the current search engines we have to do a lot of digging to get the answers [8]. Computers are central to this digging, because there is no way people can stay updated, even in their own field.
However, computers can’t deal with the current web and the scientific information as produced in the classic articles (even the electronic versions), because of the following reasons:
Homonyms. Words that sound or are the same but have a different meaning. Acronyms are notorious in this respect. Barend gave PSA as an example, but, without realizing it, he used a better example: PPI. This means Protein Pump Inhibitor to me, but apparently Protein Protein Interactions to him.
Redundancy. To keep journal articles readable we often use different words to denote the same. These do not add to the real new findings in a paper. In fact the majority of digital information is duplicated repeatedly. For example “Mosquitoes transfer malaria”, is a factual statement repeated in many consecutive papers on the subject.
The connection between words is not immediately clear (for a computer). For instance, anti-TNF inhibitors can be used to treat skin disorders, but the same drugs can also cause it.
Data are not structured beforehand.
Weight: some “facts” are “harder” than others.
Not all data are available or accessible. Many data are either not published (e.g. negative studies), not freely available or not easy to find. Some portals (GoPubmed, NCBI) provide structural information (fields, including keywords), but do not enable searching full text.
Data are spread. Data are kept in “data silos” not meant for sharing [8](ppt2). One would like to simultaneously query 1000 databases, but this would require semantic web standards for publishing, sharing and querying knowledge from diverse sources…..
In a nutshell, the problem is as Barend put it: “Why bury data first and then mine it again?” [9]
Homonyms, redundancy and connection can be tackled, at least in the field Barend is working in (bioinformatics).
Different terms denoting the same concept (i.e. synonyms) can be mapped to a single concept identifier (i.e. a list of synonyms), whereas identical terms used to indicate different concepts (i.e. homonyms) can be resolved by a disambiguation algorithm.
The shortest meaningful sentence is a triplet: a combination of subject, predicate and object. A triplet indicates the connection and direction. ”Mosquitoes cause/transfer malaria” is such a triplet, where mosquitoes and malaria are concepts. In the field of proteins: “UNIPROT 05067 is a protein” is a triplet (where UNIPROT 05067 and protein are concepts), as are: “UNIprotein 05067 is located in the membrane” and “UNIprotein 0506 interacts with UNIprotein 0506″[8]. Since these triplets (statements) derive from different databases, consistent naming and availability of information is crucial to find them. Barend and colleagues are the people behind Wikiproteins, an open, collaborative wiki focusing on proteins and their role in biology and medicine [4-6].
Concepts and triplets are widely accepted in the world of bio-informatics. To have an idea what this means for searching, see the search engineQuertle, which allows semantic search of PubMed & full-text biomedical literature, automatic extraction of key concepts; Searching for ESR1 $BiologicalProcess will search abstracts mentioning all kind of processes where ESR1 (aka ERα, ERalpha, EStrogen Receptor 1) are involved. The search can be refined by choosing ‘narrower terms’ like “proliferation” or “transcription”.
The new aspects is that Mons wants to turn those triplets into (what he calls) nano-publications. Because not every statement is as ‘hard’, nano-publications are weighted by assigning numbers from 0 (uncertain) to 1 (very certain). The nano-publication “mosquitoes transfer malaria” will get a number approaching 1.
Such nano-publications offer little shading and possibility for interpretation and discussion. Mons does not propose to entirely replace traditional articles by nano-publications. Quote [3]:
While arguing that research results should be available in the form of nano-publications, are emphatically not saying that traditional, classical papers should not be published any longer. But their role is now chiefly for the official record, the “minutes of science” , and not so much as the principle medium for the exchange of scientific results. That exchange, which increasingly needs the assistance of computers to be done properly and comprehensively, is best done with machine-readable, semantically consistent nano-publications.
According to Mons, authors and their funders should start requesting and expecting the papers that they have written and funded to be semantically coded when published, preferably by the publisher and otherwise by libraries: the technology exists to provide Web browsers with the functionality for users to identify nano-publications, and annotate them.
Like the wikiprotein-wiki, nano-publications will be entirely open access. It will suffice to properly cite the original finding/publication.
In addition there is a new kind of “peer review”. An expert network is set up to immediately assess a twittered nano-publication when it comes out, so that the publication is assessed by perhaps 1000 experts instead of 2 or 3 reviewers.
On a small-scale, this is already happening. Nano-publications are send as tweets to people like Gert Jan van Ommen (past president of HUGO and co-author of 5 of my publications (or v.v.)) who then gives a red (don’t believe) or a green light (believe) via one click on his blackberry.
As Mons put it, it looks like a subjective event, quite similar to “dislike” and “like” in social media platforms like Facebook.
Barend often referred to a PLOS ONE paper by van Haagen et al [1], showing the superiority of the concept-profile based approach not only in detecting explicitly described PPI’s, but also in inferring new PPI’s.
[You can skip the part below if you're not interested in details of this paper]
Van Haagen et al first established a set of a set of 61,807 known human PPIs and of many more probable Non-Interacting Protein Pairs (NIPPs) from online human-curated databases (and NIPPs also from the IntAct database).
For the concept-based approach they used the concept-recognition software Peregrine, which includes synonyms and spelling variations of concepts and uses simple heuristics to resolve homonyms.
This concept-profile based approach was compared with several other approaches, all depending on co-occurrence (of words or concepts):
Word-based direct relation. This approach uses direct PubMed queries (words) to detect if proteins co-occur in the same abstract (thus the names of two proteins are combined with the boolean ‘AND’). This is the simplest approach and represents how biologists might use PubMed to search for information.
Concept-based direct relation (CDR). This approach uses concept-recognition software to find PPIs, taking synonyms into account, and resolving homonyms. Here two concepts (h.l. two proteins) are detected if they co-occur in the same abstract.
STRING. The STRING database contains a text mining score which is based on direct co-occurrences in literature.
The results show that, using concept profiles, 43% of the known PPIs were detected, with a specificity of 99%, and 66% of all known PPIs with a specificity of 95%. In contrast, the direct relations methods and STRING show much lower scores:
Word-based
CDR
Concept profiles
STRING
Sensitivity at spec = 99%
28%
37%
43%
39%
Sensitivity at spec = 95%
33%
41%
66%
41%
Area under Curve
0.62
0.69
0.90
0.69
These findings suggested that not all proteins with high similarity scores are known to interact but may be related in another way, e.g.they could be involved in the same pathway or be part of the same protein complex, but do not physically interact. Indeed concept-based profiling was superior in predicting relationships between proteins potentially present in the same complex or pathway (thus A-C inferred from concurrence protein pairs A-B and B-C).
Since there is often a substantial time lag between the first publication of a finding, and the time the PPI is entered in a database, a retrospective study was performed to examine how many of the PPIs that would have been predicted by the different methods in 2005 were confirmed in 2007. Indeed, using concept profiles, PPIs could be efficiently predicted before they enter PPI databases and before their interaction was explicitly described in the literature.
The practical value of the method for discovery of novel PPIs is illustrated by the experimental confirmation of the inferred physical interaction between CAPN3 and PARVB, which was based on frequent co-occurrence of both proteins with concepts like Z-disc, dysferlin, and alpha-actinin. The relationships between proteins predicted are broader than PPIs, and include proteins in the same complex or pathway. Dependent on the type of relationships deemed useful, the precision of the method can be as high as 90%.
In line with their open access policy, they have made the full set of predicted interactions available in a downloadable matrix and through the webtool Nermal, which lists the most likely interaction partners for a given protein.
According to Mons, this framework will be a very rich source for new discoveries, as it will enable scientists to prioritize potential interaction partners for further testing.
Barend Mons started with the statement that nano-publications will replace the classic articles (and the need for libraries). However, things are never as black as they seem.
Mons showed that a nano-publication is basically a “peer-reviewed, openly available” triplet. Triplets can be effectively retrieved ànd inferred from available databases/papers using a concept-based approach.
Nevertheless, effectivity needs to be enhanced by semantically coding triplets when published.
What will this mean for clinical medicine? Bioinformatics is quite another discipline, with better structured and more straightforward data (interaction, identity, place). Interestingly, Mons and van Haage plan to do further studies, in which they will evaluate whether the use of concept profiles can also be applied in the prediction of other types of relations, for instance between drugs or genes and diseases. The future will tell whether the above-mentioned approach is also useful in clinical medicine.
Implementation of the following (implicit) recommendations would be advisable, independent of the possible success of nano-publications:
Less emphasis on “publish or perish” (thus more on the data themselves, whether positive, negative, trendy or not)
Better structured data, partly by structuring articles. This has already improved over the years by introducing structured abstracts, availability of extra material (appendices, data) online and by guidelines, such as STARD (The Standards for Reporting of Diagnostic Accuracy)
Open Access
Availability of full text
Availability of raw data
One might argue that disclosing data is unlikely when pharma is involved. It is very hopeful therefore, that a group of major pharmaceutical companies have announced that they will share pooled data from failed clinical trials in an attempt to figure out what is going wrong in the studies and what can be done to improve drug development (10).
Unfortunately I don’t dispose of Mons presentation. Therefore two other presentations about triplets, concepts and the semantic web.
&
References
van Haagen HH, ‘t Hoen PA, Botelho Bovo A, de Morrée A, van Mulligen EM, Chichester C, Kors JA, den Dunnen JT, van Ommen GJ, van der Maarel SM, Kern VM, Mons B, & Schuemie MJ (2009). Novel protein-protein interactions inferred from literature context. PloS one, 4 (11) PMID: 19924298
Twitteren voor de wetenschap, Maartje Bakker, Volskrant (2010-06-05) (Twittering for Science)
Mons, B., Ashburner, M., Chichester, C., van Mulligen, E., Weeber, M., den Dunnen, J., van Ommen, G., Musen, M., Cockerill, M., Hermjakob, H., Mons, A., Packer, A., Pacheco, R., Lewis, S., Berkeley, A., Melton, W., Barris, N., Wales, J., Meijssen, G., Moeller, E., Roes, P., Borner, K., & Bairoch, A. (2008). Calling on a million minds for community annotation in WikiProteins Genome Biology, 9 (5) DOI: 10.1186/gb-2008-9-5-r89
(ppt1) SWAT4LS 2009Semantic Web Applications and Tools for Life Sciences http://www.swat4ls.org/
Amsterdam, Science Park, Friday, 20th of November 2009
Do you ever use Wikipedia? I do and so do many other people. It is for free, easy to use, and covers many subjects.
But do you ever use Wikipedia to look up scientific or medical information? Probably everyone does so once in a while. Dave Munger(Researchblogging) concluded a discussion on Twitter as follows:
“Wikipedia’s information quality is better than any encyclopedia, online or off. And, yes, it’s also easy to use”.
Wikipedia is an admirable initiative. It is a large online collaborative, multilingual encyclopedia written by contributors around the world.
But the key question is whether you can rely on Wikipedia as the sole source for medical, scientific or even popular information.
Well, you simply can’t and here are a few examples/findings to substantiate this point.
RANKING AND USE
E-patients
When you search for diabetes in Google (EN), Wikipedia’s entry about diabetes ranks second, below the American Diabetes Association Home Page. A recent study published in the J Am Med Inform Assoc [1] confirms what you would expect: that the English Wikipediais a prominent source of online healthinformation. Wikipedia ranked among the first ten results in more than 70%of search engines and health-keywords tested, and outranked other sources in case of rare disease-related keywords. Wikipedia’s articles were viewed more frequently than the corresponding MedlinePlus Topic pages. This corroborates another study that can be downloaded from the internet here [10]. This study by Envision Solutions, LLC, licensed under the Creative Commons License, concluded that the exposure of Internet user’s to health-related user-generated media (UGM) is significant, Wikipedia being the most reference resource on Google and Yahoo.
According to the Pew Internet & American Life Project*, 10 million US adults search online for information on health each day [1]. Most (66%) begin their research on a search engine like Yahoo or Google. In addition, Americans are saying that the information they find on the Internet is having an impact. According to Pew, “53% of health seekers report that their most recent health information session [influenced] how they take care of themselves or care for someone else.” In addition, 56% say the information they find online has boosted their confidence in their healthcare decision-making abilities.
In 2000, 46% of American adults had access to the internet, 5% of U.S. households had broadband connections, and 25% of American adults looked online for health information. Now, 74% of American adults go online, 57% of American households have broadband connections, and 61% of adults look online for health information.
Thus a lot of people look online for health care questions and are more inclined to use highly ranked sources.
This is not unique for Health topics but is a general phenomenon, i.e. see this mini-study performed by a curious individual: 96.6% of Wikipedia Pages Rank in Google’s Top 10 [12]. The extreme high traffic to Wikipedia due to search referrals has even been been denounced by SEO-people (see here) [13]: if you type “holiday” Wikipedia provides little value when ranking in the top 10: everybody knows what a holiday is
A nightmare for most educators in the curriculum is that students rely on UGM or Web 2.0 sites as a source of medical information. Just walk along medical students as they work behind their computers and take a quick glance at the pages they are consulting. These webpages often belong to the above category.
‘Many students said I know I shouldn’t but….’ and then qualified that they used Wikipedia first because it was easy to understand, they felt it was reasonably reliable, and accessible. One student used it to search directly from her phone when on placement..
50% of the doctors use it!
But these are only medical students. Practicing doctors won’t use Wikipedia to solve their clinical questions, because they know where to find reliable medical information.
A recent qualitative study published in Int J Med Inform [2]examined the “Web 2.0″ use by 35 junior physicians in the UK. Diaries and interviews encompassing 177 days of internet use or 444 search incidents, analyzed via thematic analysis. Although concepts are loosely defined (Web 2.0, internet and UMG are not properly defined, i.e. Google is seen as a web 2.0 tool (!) [see Annemarie's critical review [16] the results clearly show that 89% of these young physicians use at least one “Web 2.0 tool” (including Google!) in their medical practice, with 80% (28/35) reporting the use of wikis. The visit of wiki’s is largely accounted for by visits to Wikipedia: this was the second most commonly visited site, used in 26% (115/44) of cases and by 70% (25/35) of all physicians.Notably, only one respondent made regular contribution to a medical wiki site.
The main motivation for using the Internet for information seeking was the accessibility and ease of use over other tools (like textbooks), the uptodateness, the broad coverage and the extras such as interactive immages. On the other hand most clinicians realized that there was a limitation in the quality or usefulness of information found. It is reassuring that most doctors used UGM like Wikipedia for background or open questions, to fulfill the need for more in depth knowledge on a subject, or to find information for patients, not for immediate solving of clinical questions.
Apparently some doctors also heavily rely on Wikipedia that they refer to Wikipedia articles in publications (see the Int. J Cardiol. PubMed [3] abstract below)!!
WHY WIKIPEDIA IS NOT (YET) A TRUSTWORTHY AND HIGH QUALITY HEALTH SITE
Whether the common use of Wikipedia by e-patient, medical students and doctors is disadvantageous depends on the quality and the trustworthiness of the Wikipedia articles, and that is in its turn dependent on who writes the articles.
Basically, the strength of Wikipedia is it weakness: anyone can write anything on any subject, and anyone can edit it, anonymously.
Negative aspects include its coverage (choice of subjects but also the depth of coverage), the “overlinking”, the sometimes frustating interactions between authors and editors, regularly leading to (often polite) “revision wars“, but above all the lack of ‘expert’ authors or peer review. This may result in incomplete, wrong or distorted information.
Positive aspects are its accessibility, currency, availability in many languages, and the collective “authorship” (which is an admirable concept).
Differences in the interests and attention of Wikipedia’s editors mean that some areas, in the traditional sciences, for example, are better covered than others. (…) Overall, we found that the degree to which Wikipedia is lacking depends heavily on one’s perspective. Even in the least covered areas, because of its sheer size, Wikipedia does well, but since a collection that is meant to represent general knowledge is likely to be judged by the areas in which it is weakest, it is important to identify these areas and determine why they are not more fully elaborated. It cannot be a coincidence that two areas that are particularly lacking on Wikipedia—law and medicine—are also the purview of licensed experts.
It is not unexpected though that Wikipedia’s topical coverage is driven by the interests of its users.
The actual coverage of medical subjects may vary greatly. In one study [5], abstract-format, 2007) Wikipedia entries were screened for the most commonly performed inpatient surgical procedures in the U.S. Of the 39 procedures, 35 were indexed on Wikipedia. 85.7% of these articles were deemed appropriate for patients. All 35 articles presented accurate content, although only 62.9% (n=22) werefree of critical omissions. Risks of the procedures were significantly underreported. There was a correlation between an entry’s quality and how often it was edited.
This study suggests that Wikipedia may be a useful point of engagement for consumers looking for drug information, but that it should be supplementary to, rather than the sole source of, drug information. This is due, in part, to our findings that Wikipedia has a more narrow scope, is less complete, and has more errors of omission versus the comparator database. Consumers relying on incomplete entries for drug information risk being ill-informed with respect to important safety features such as adverse drug events, contraindications, drug interactions, and use in pregnancy.
These errors of omission may prove to be a substantial and largely hidden danger associated with exclusive use of
user-edited drug information sources.
Alternatively, user-edited sites may serve as an effective means of disseminating drug information and are promising as a means of more actively involving consumers in their own care. However, health professionals should not use user-edited sites as authoritative sources in their clinical practice, nor should they recommend them to patients without knowing the limitations and providing sufficient additional information and counsel…
Not Evidence Based
German researches found [7], not surprisingly, that Wikipedia (as well as two major German statutory health insurances):
“…failed to meet relevant criteria, and key information such as the presentation of probabilities of success on patient-relevant outcomes, probabilities of unwanted effects, and unbiased risk communication was missing. On average items related to the objectives of interventions, the natural course of disease and treatment options were only rated as “partially fulfilled”. (..) In addition, the Wikipedia information tended to achieve lower comprehensibility. In conclusion(..) Wikipedia (..) does not meet important criteria of evidence-based patient and consumer information though…”
Wrong, misleading, inaccurate
All above studies point at the incompleteness of Wikipedia. Even more serious is the fact that some of the Wikipedia addings are wrong or misleading. Sometimes on purpose. The 15 biggest wikipedia blunders [23] include the death announcements of Ted Kennedy (when he was still alive), Robert Byrd and others. Almost hilarious are the real time Wikipedia revisions after the presumed death of Kennedy and the death of Ken Lay (suicide, murde, heart attack? [24).
In the field of medicine, several drug companies have been caught altering Wikipedia entries. The first drug company messing with Wikipedia was AstraZeneca. References claiming that Seroquel allegedly made teenagers “more likely to think about harming or killing themselves” were deleted by a user of a computer registered to the drug company [25], according to Times [26]. Employees of Abbott Laboratories have also been altering entries to Wikipedia to “eliminate information questioning the safety of its top-selling drugs.”(See WSJ-blog [27] , brandweeknrx.com [28], and recently Kevin MD[29])
These are “straightforward” examples of fraudulent material. But sometimes the Wikipedia articles are more subtly colored by positive or negative bias.
Take for instance the English entry on Evidence Based Medicine (in fact the reason why I started this post). Totally open-minded I checked the entry, which was automatically generated in one of my posts by Zemanta. First I was surprised by the definition of EBM:
Evidence-based medicine (EBM) aims to apply the best available evidence gained from the scientific method to medical decision making. It seeks to assess the quality of evidence of the risks and benefits of treatments (including lack of treatment).
instead of the usually cited Sacket-definition (this is only cited at the end of the paper):
“the practice of evidence based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research”
In short, the whole article lacks cohesion: the definitions of EBM are not correct, there is too much emphasis on not directly relevant information (4 ways to grade the evidence and 3 statistical measures), the limitations are overemphasized (cf. chapter 7 with 6 in the Figure below) and put out of perspective.
Apparently this has also been noted by Wikipedia, because there is a notice on the Evidence Based Medicine Page saying:
This article has been nominated to be checked for its neutrality. Discussion of this nomination can be found on the talk page. (May 2009)
Now why in earth would some advocate of CAM write the Wikipedia EBM-entry? I can think of 4 (not mutually exclusive) reasons:
When you’re an EBM-nonbeliever or opponent this is THE chance to misinform readers about EBM (to the advantage of CAM).
The author was invited to write this entry.
No EBM-specialist or epidemiologist is willing to write the entry, or to write for Wikipedia in general (perhaps because they find Wikipedia lacks trustworthiness?)
EBM specialists/epidemiologists are not “allowed”/hindered to make major amendments to the text, let alone rewrite it.
According to Mr Naturopath point 2 is THE reason he wrote this article. Now the next question is “exactly by whom was he invited?” But the TALK-page reveals that Mr Naturapath makes it a tough job for other, better qualified writers, to edit the page (point 4). To see how difficult it is for someone to re-edit a page, please see the TALK-page. In fact, one look at this page discourages me from ever trying to make some amendments to any Wikpedia text.
SOLUTIONS?
Changes to Wikipedia’s organization
Wikipedia has long grasped that its Achilles heel is the free editability (see for instance this interview with Wikipedia’s founder [30]). Therefore, “WikiProjects” was initiated to help coordinate and organize the writing and editing of articles on a certain topic, as well as “Citizendium” which is an English-languagewiki-based freeencyclopedia project aimed to improve the Wikipedia model by providing a “reliable” encyclopedia. “It hopes to achieve this by requiring all contributors to use their real names, by strictly moderating the project for unprofessional behavior, by providing what it calls “gentle expert oversight” of everyday contributors, and also through its “approved articles,” which have undergone a form of peer-review by credentialed topic experts and are closed to real-time editing.”
Starting this fall Wikipedia will launch an optional feature called “WikiTrust” will color code every word of the encyclopedia based on the reliability of its author and the length of time it has persisted on the page: Text from questionable sources starts out with a bright orange background, while text from trusted authors gets a lighter shade.
The Wikipedia EBM article is within the scope of these two projects, and this is good news.However, Wikipedia still clings to the idea that: “Everyone is welcome to join in this endeavor (regardless of medical qualifications!).” In my opinion, it would be better if Wikipedia gave precedence to experts instead of hobbyists/ people from another field, because the former can be expected to know what they are talking about. It is quite off-putting for experts to contribute. See this shout-out:
Who are these so-called experts who will qualify material? From what I’ve seen so far, being an academic expert in a particular field hardly protects one from edit wars–Julie and 172 are two primary examples of this. Meanwhile, the only qualification I have seen so far is that they have a B.A. Gimme a friggin’ break! (and before I get accused of academic elitism, I make it known that I dropped out of college and spend an inordinate amount of time at work correcting the BS from the BAs, MAs, and PhDs).
While anyone can still edit entries, the site is testing pages that require changes to be approved by an experienced Wikipedia editor before they show up, the so called Flagged protection and patrolled revisions. (see Wikimedia) This proposal is only for articles that are currently under normal mechanisms of protection (i.e. the Obama-article cannot be edited by a newcomer).
Although this seems logic, it is questionable whether “experienced” editors are per definition better qualified than newcomers. A recent interesting analysis of the Augmented Social Cognition group [31], (cited in the Guardian [32]) shows a slowdown in growth of Wikipedia activity, with the activity slightly declining in all classes of editors except for the highest-frequency class of editors (1000+ edits). Here is an increase in their monthly edits.
In addition the study shows growing resistance from the Wikipedia community to new content. The total percentage of reverted edits increased steadily over the years, but more interestingly, low-frequency or occasional editors experienced a visibly greater resistance compared to high-frequency editors . Together this points at a growing resistance from the Wikipedia community to new content, especially when the edits come from occasional editors.
This is more or less in line with an earlier finding [9] showing that Wikipedia members feel more comfortable expressing themselves on the net than off-line and scored lower on agreeableness and openness compared to non-Wikipedians, a finding that was interpreted as consistent with the possibility that contributing to Wikipedia serves mainly egocentric motives.
A less noncommital approach is the demand to “Publish in Wikipedia or perish”, as described in Nature News[9]. “Anyone submitting to a section of the journal RNA Biology will, in the future, be required to also submit a Wikipedia page that summarizes the work. The journal will then peer review the page before publishing it in Wikipedia.” The project is described in detail here [10] and the wiki can be viewed here
Wiki’s for experts.
One possible solution is that scientist and medica experts contribute to wiki’s other than the Wikipedia. One such wiki is the wiki-surgery [5]. PubDrug, RxWiki , WikiProteins [11] and Gene Wiki [12] are other examples. In general, scientists are more inclined to contribute to these specialists wiki’s, that have oversight and formal contributions by fellow practitioners (this is also true for the RNA-wiki)
A medical Wikipedia
Yet another solution is a medical wikipedia, such as Ganfyd or Medpedia . Ganfyd is written by medical professionals. To qualify to edit or contribute to the main content of Medpedia approved editors must have an M.D., D.O., or Ph.D. in a biomedical field. Others, however, may contribute by writing in suggestions for changes to the site using the “Make a suggestion” link at the top of each page. Suggestions are reviewed by approved editors. Whether these medical wikipedias will succeed will depend on the input of experts and their popularity: to what extent will they be consulted by people with health questions?
I would like to end with a quote from Berci during twitterview (link in Wikipedia):
@Berci : @diariomedico And as Wikipedians say, Wikipedia is the best source to start with in your research, but should never be the last one. #DM1 9 months ago
REFERENCES
Scientific Articles
Laurent, M., & Vickers, T. (2009). Seeking Health Information Online: Does Wikipedia Matter? Journal of the American Medical Informatics Association, 16 (4), 471-479 DOI: 10.1197/jamia.M3059
Hughes, B., Joshi, I., Lemonde, H., & Wareham, J. (2009). Junior physician’s use of Web 2.0 for information seeking and medical education: A qualitative study International Journal of Medical Informatics, 78 (10), 645-655 DOI: 10.1016/j.ijmedinf.2009.04.008
Lee, C., Teo, C., & Low, A. (2009). Fulminant dengue myocarditis masquerading as acute myocardial infarction International Journal of Cardiology, 136 (3) DOI: 10.1016/j.ijcard.2008.05.023
Halavais, A., & Lackaff, D. (2008). An Analysis of Topical Coverage of Wikipedia Journal of Computer-Mediated Communication, 13 (2), 429-440 DOI: 10.1111/j.1083-6101.2008.00403.x
Devgan, L., Powe, N., Blakey, B., & Makary, M. (2007). Wiki-Surgery? Internal validity of Wikipedia as a medical and surgical reference Journal of the American College of Surgeons, 205 (3) DOI: 10.1016/j.jamcollsurg.2007.06.190
Clauson, K., Polen, H., Boulos, M., & Dzenowagis, J. (2008). Scope, Completeness, and Accuracy of Drug Information in Wikipedia Annals of Pharmacotherapy, 42 (12), 1814-1821 DOI: 10.1345/aph.1L474 (free full text)
Mühlhauser I, & Oser F (2008). [Does WIKIPEDIA provide evidence-based health care information? A content analysis] Zeitschrift fur Evidenz, Fortbildung und Qualitat im Gesundheitswesen, 102 (7), 441-8 PMID: 19209572
Amichai–Hamburger, Y., Lamdan, N., Madiel, R., & Hayat, T. (2008). Personality Characteristics of Wikipedia Members CyberPsychology & Behavior, 11 (6), 679-681 DOI: 10.1089/cpb.2007.0225
Daub, J., Gardner, P., Tate, J., Ramskold, D., Manske, M., Scott, W., Weinberg, Z., Griffiths-Jones, S., & Bateman, A. (2008). The RNA WikiProject: Community annotation of RNA families RNA, 14 (12), 2462-2464 DOI: 10.1261/rna.1200508
Mons, B., Ashburner, M., Chichester, C., van Mulligen, E., Weeber, M., den Dunnen, J., van Ommen, G., Musen, M., Cockerill, M., Hermjakob, H., Mons, A., Packer, A., Pacheco, R., Lewis, S., Berkeley, A., Melton, W., Barris, N., Wales, J., Meijssen, G., Moeller, E., Roes, P., Borner, K., & Bairoch, A. (2008). Calling on a million minds for community annotation in WikiProteins Genome Biology, 9 (5) DOI: 10.1186/gb-2008-9-5-r89
Huss, J., Orozco, C., Goodale, J., Wu, C., Batalov, S., Vickers, T., Valafar, F., & Su, A. (2008). A Gene Wiki for Community Annotation of Gene Function PLoS Biology, 6 (7) DOI: 10.1371/journal.pbio.0060175
Other Publications, blogposts (numbers in text need to be adapted)
Welcome to the first edition of MedLib’s Round, a blog carnival of the “best blog posts in the field of medical librarianship”.
Starting a new blog carnival is often difficult. You have to recruit bloggers, who want to participate by submitting blogposts and/or hosting future editions. (see this older post on Scienceroll - Thanks @hleman).
I didn’t sound out people to find if they were interested, but just gave it a try. — Therefore, I was very pleased that the idea was so enthusiastically received by many medical librarians ànd physicians from all over the world. Emergency physician Mike Cadogan (@sandnsurf) of Life in the First Lane already added the MedLib’s Round to his listing of Blogs Rankings and Rounds before it had even started.
Blog carnivals are meant to spread the word not only about established, but also about new bloggers. I’m therefore delighted that several librarians were inspired to (re)start blogging.
Shamsha Damani(@shamsha) accepted the invitation to become a guest writer on this blog to be able to submit a post (see below).
Alisha Miles (@alisha764) who start tweeting in Januari started her own blog Alisha 764 with the post “I am a Tree” saying: “I am no longer a mushroom, I am now a tree. Thank you to all of the other librarians’ posts & tweets that inspired me to start this blog.” Which clearly refers to the comment of @sandnsurf to the blogpost “What I learned in 2008 (about Web 2.0)“: “the most important thing is that you are actually a tree in this ecosystem, you are out there experimenting, thinking and trying to drive the revolution further…Most of my colleagues are still mushrooms…“
Thus apart from being a post-aggregator, a blog carnival can also inspire people with similar interests and connect them. From my own experience I know you can feel lonely as a blogger. So please take a look at the above mentioned blogs/twitter accounts and help them to flourish into full grown trees, so we can all enjoy their fruits (and vice versa).
AND NOW FOR…..THE FIRST MEDLIB’S ROUND
The MedLib’s Round is about medical librarian stuff. This field is much broader than searching PubMed or interlibrary loaning; it is related to all stages in the publication and medical information cycles (searching, citing, managing, writing, publishing, social networking).
This carnival covers many facets of that cycle.
SEARCHING THE WEB
For medical librarians searching is an important facet of their job. There are different sources to search, including “the World Wide Web” and bibliographic databases like PubMed.
Hope Leman of AltSearchEngines has compiled a list of Top 10 Health Search Engines of 2008. She urges all those interested in medical search to give these tools a spin. Her Top 10 bares great resemblance to the Top 8 Bedside Health Search Engines 2008 of @sandnsurf (Mike Cadogan), indicating that the same engines are appreciated and used by physicians as well. GoPubMedranks 2 in both lists. According to Hope “GoPubMed is a useful complement to PubMed proper, particularly to determine who the leading authorities are on particular topics. For further details on how to use GoPubMed see an earlier post of Mikeand several posts of David Rothman (here and here).
Guus van den Brekel of DigiCMB, who just won the Alliance Virtual Library Golden Leaf Awards 2009 (Second Life), told me that PubMed is by far the most frequently used search database by the hospital staff and students of the University Medical Center Groningen, where he works. In 2007, EVERY 2 MINS somebody used the Pubmed link, and every 30 seconds somebody clicked the SFX-link resolver in PubMed. Guus believes that such a tool needs to be published to as many platforms as possible, and in any format the patrons would like them. So far a Toolbar, Widget, HTML-box, OpenSearch pretty much covers that wish. The Widgets can be found at PubMed Search & News Widget
PubMed has introduced (or rather continuously introduces) several changes, that have been amply discussed here. Major changes include the Advanced Search, the citation sensor and the way terms typed in the search bar are translated. Non-librarians often don’t know that PubMed automatically maps the words, but the way this is done has changed, i.e. multi-term words are split. In her post Mapping door PubMed, written in Dutch and English, de Bibliothekershows that this altered mapping can have both unexpected positive and negative effects, and that it is always important to check the Details Tab.
Among the things that Nicole Dettmar (Eagle Dawg) of the Eagle Dawg Blog addresses at her post Eagle Dawg Blog: Hidden in the Bookshelf: PubMed & Discovery Initiative is the new Discovery Initiative of the NCBI, which is an effort to make the full potential of the NCBI Web services and underlying databases more available to users. Nicole gives various interesting links, which will tell you more about the upcoming changes.
MANAGING INFORMATION AND REFERENCES
Like many of her colleagues medical librarian Anne WelshFirst Person Narrative noticed clinicians prefer to perform one word Google-style searches (hé, does that sound familiar!). However, realizing that her medical library “expert opinion” was based on nothing more than a series of anecdotes, Anne decided to have a fish around for research on clinicians’ search strategies and information needs. Curious about the outcome? Then read the summary of the evidence in her well written research blogging post “Limiting the Dataset“.
Indeed it is hard to keep up with the literature. Apart from specific (often Google-style searches), most clinicians also try to read a few interesting journals, for instance the BMJ and the Lancet. Instead of going to the library it is also possible to take an email alert or a RSS feed to the journals of your choice. You can generate custom RSS feeds in PubMed for you favorite search and/or Journal, but this is a kind of cumbersome procedure for most people not used to it (see for instance my earlier post in Dutch and this post of David Rothman – a must-read for people not acquainted with the use of RSS for this purpose). Physician and medicine2.0 pioneer Ves Dimov of the Clinical Cases and Images – Blog has another solution to set up a RSS feed to journals, which I found astonishing simple and pretty awesome, because of the conveniently arrangement of the results. All you need is a free Google account to create Your Own “Medical Journal” with iGoogle Personalized Page. Want to know how it works, then please read his easy-to-follow post, which he has specially updated for this occasion. Ves has also included some ready made RSS feeds of the “Big Five” medical journals (NEJM, JAMA, BMJ, Lancet and Annals) plus 2-3 subspecialty journals as well as several podcasts in iGoogle.
Now, once you have the PDF’s of the papers you like you would like to store them in a handy way.Another physician, the Dutch psychiatristDr Shock MD PhD with a very eloguent blog of the same name, explores the use of Mendeley, a free social software for managing and sharing research papers and a Web 2.0 site for discovering research trends and connecting to like-minded academics (see Mendeley Manage Share and Discover Research Papers). Dr. Shock didn’t make up his mind yet whether he prefers Mendeley or Labmeeting (described in another post) as an online library. But offline he uses Sente, which he finds absolutely perfect. A chimera between Sente and one of the other tools would be his ideal management system.
PUBLISHING
Michelle Kraft of The Krafty Librarian was totally blown away by a presentation on Interactive Science Publishingat PSP 2009 Annual Conference (where she also gave a presentation herself). I didn’t know what interactive science publishing really meant, but Michelle can illustrate things so well, that you can readily imagine it all. This was needed as I could not access the examples she referred to without the risk of my computer becoming too slow or worse. But I understand from Michelle that it is a revolutionary new method of viewing online journals, although there are some answers to be addressed as well (see her post)
Imagine having the “PDF” of an article on congenital heart defects and be able to hear the heart sounds plus the video recording of the heart. The video would be more than just a snippet, it would be the entire video sectioned into “chapters” referenced within the various areas of the article. So while you are reading the article you can click on the link within the text referencing the image, sound, etc. and the image immediately jumps to that section the video. Imagine the data behind a large randomized controlled trial available in its entirety to all readers to be manipulated, reused, and viewed.
Another new publishing format is discussed by Shamsha Damani (@shamsha) on this blog (see: “How to make EBM easy to swallow“). Shamsha informs us that the BMJ will be publishing two summaries for each research article published. One called BMJ PICO, prepared by the authors, breaks down the article into the different EBM elements. The other called Short Cuts is written by BMJ itself. Here she hopes BMJ will shine, providing an easy to follow unbiased view of the article. Indeed, it would be very welcomed if more papers were in the ready-appraised-format, similar as found in the ACP-Journal Club. However, in the BMJ, it is the PICO-format written by the authors themselves which has the EBM structure, and is most preferred by the readers. According to some (including me) the Short Cuts are a bit woolly. Or as Shamsha says: “Personally I think it would have been better to have the BMJ reviewers write the PICO format, and do a bit more thorough critiquing”.
SOCIAL MEDIA & NLM, GOVERNMENTAL ORGANIZATIONS AND MEDICAL LIBRARIANS
In the same blogpost as mentioned above@Eagledawg mentions that the recent introduction of the #pubmed tag in Twitter (with the aim that you can later search for messages with this tag, see real time results here) led to various responses, which are not really appreciated as useful by the NLM because of the extreme short length of the tweets (140 characters including tag). It strikes Nicole that the NLM is not present on twitter (in contrast to the FDA and the CDC,also see a post of David Rothman). A good example of how the government could use using social media to respond to citizens is given by Andrew Wilson, a member of the recently introduced social-media team of the Department of Health and Human Service, who responded to the peanut-butter-and-salmonella recall issue on Twitter.
And what about the presence of the abovementioned contributors to this first Grand Round? Without exception they are all on Twitter and all but one use it on a regular basis. Now, assuming that most medical librarians aren’t on Twitter, doesn’t tell that something about this group? I wonder if Twitter presence is not the main reason for the swift start of this First MedLib’s Round.
That’s it for this edition.
I hope you enjoyed this first MedLib’s Round.
I surely enjoyed reading the many interesting and good quality posts that were submitted.
).
At “
![Reblog this post [with Zemanta]](http://img.zemanta.com/reblog_b.png?x-id=3cbb86cf-fc24-4a7e-9e44-8a5eccca3917)
Recent Comments