Internet Sources & Blog Posts in a Reference List? Yes or No?

13 02 2011

A Dutch librarian asked me to join a blog carnival of Dutch Librarians. This carnival differs from medical blog carnivals (like the Grand Rounds and “Medical Information Matters“) in its approach. There is one specific topic which is discussed at individual blogs and summarized by the host in his carnival post.

The current topic is “Can you use an internet source”?

The motive of the archivist Christian van der Ven for starting this discussion was the response to a post at his blog De Digitale Archivaris. In this post he wondered whether blog posts could be used by students writing a paper. It struck him that students rarely use internet sources and that most teachers didn’t encourage or allow to use these.

Since I work as a medical information specialist I will adapt the question as follows:

“Can you refer to an internet source in a biomedical scientific article, paper, thesis or survey”?

I explicitly use “refer to” instead of “use”. Because I would prefer to avoid discussing “plagiarism” and “copyright”. Obviously I would object to any form of uncritical copying of a large piece of text without checking it’s reliability and copyright-issues (see below).

Previously, I have blogged about the trouble with Wikipedia as a source for information. In short, as Wikipedians say, Wikipedia is the best source to start with in your research, but should never be the last one (quote from @berci in a twitterinterview). In reality, most students and doctors do consult Wikipedia and dr. Google (see here and here). However, they may not (and mostly should not) use it as such in their writings. As I have indicated in the earlier post it is not (yet) a trustworthy source for scientific purposes.

But Internet is more than Wikipedia and random Googling. As a matter of fact most biomedical information is now in digital form. The speed at which biomedical knowledge is advancing is tremendous. Books are soon out of date. Thus most library users confine themselves to articles in peer-reviewed scientific papers or to datasets (geneticists). Generally my patrons search the largest freely available database PubMed to access citations in mostly peer-reviewed -and digital- journals. These are generally considered as (reliable)  internet sources. But they do not essentially differ from printed equivalents.

However there are other internet sources that provide reliable or useful information. What about publications by the National Health Council, an evidence based guideline by NICE and/or published evidence tables? What about synopses (critical appraisals) such as published by DARE, like this one? What about evidence summaries by Clinical Evidence like, this one? All excellent, evidence based, commendable online resources. Without doubt these can be used as a reference in a paper. Thus there is no clearcut answer to the abovementioned question. Whether an internet source should be used as a reference in a paper is dependent on the following:

  1. Is the source relevant?
  2. Is the source reliable?
  3. What is the purpose of the paper and the topic?

Furthermore it depends on the function of the reference (not mutually exclusive):

  1. To give credit
  2. To add credibility
  3. For transparency and reproducibility
  4. To help readers find further information
  5. For illustration (as an example)

Lets illustrate this with a few examples.

  • Students who write an overview on a medical topic can use any relevant reference, including narrative reviews, UpToDate and other internet sites if appropriate .
  • Interns who have to prepare a CAT (critically appraised topic) should refer to 2-3 papers, providing the highest evidence (i.e. a systematic review and/or randomized controlled trial).
  • Authors writing systematic reviews only include high quality primary studies (except for the introduction perhaps). In addition they should (ideally) check congress abstracts, clinical trial registers (like, or actual raw data (i.e. produced by a pharmaceutical company).
  • Authors of narrative reviews may include all kinds of sources. That is also true for editorials, primary studies or theses. Reference lists should be as accurate and complete as possible (within the limits posed by for instance the journal).

Blog, wikis, podcasts and tweets.
Papers can also refer to blog posts, wikis or even tweets (there is APA guidance how to cite these). Such sources can merely be referred to because they serve as an example (articles about social media in Medicine for instance, like this recent paper in Am Pharm Assoc that analyzes pharmacy-centric blogs.

Blog posts are usually seen as lacking in factual reliability. However, there are many blogs, run by scientists, that are (or can be) a trustworthy source. As a matter of fact it would be inappropriate not to cite these sources, if  the information was valuable, useful and actually used in the paper.
Some examples of excellent biomedical web 2.0 sources.

  • The Clinical Cases and Images Blog of Ves Dimov, MD (drVes at Twitter), a rich source of clinical cases. My colleague once found the only valuable information (a rare patient case) at Dr Ves’ blog, not in PubMed or other regular sources. Why not cite this blog post, if this patient case was to be published?
  • is an aggregator of expert blogposts about peer-reviewed research. There are many other high quality scientific blogging platforms like Scientopia, the PLOSblogs etc. These kind of blogs critically analyse peer reviewed papers. For instance this blog post by Marya Zilberberg reveals how a RCT stopped early due to efficacy can still be severely flawed, but lead to a level one recommendation. Very useful information that you cannot find in the actual published study nor in the evidence based guideline
  • An example of an excellent and up-to-date wiki is the open HLWIKI (maintained by Dean Giustini, @giustini at Twitter) with entries about health librarianship, social media and current information technology topics, having over 565+ pages of content since 2006! It has a very rich content with extensive reference lists and could thus be easily used in papers on library topics.
  • Another concept is (an initiative of Jean Claude Bradley, discussed in a previous post. This is not only a wiki but also an open notebook, where actual primary scientific data can be found. Very impressive.
  • There is also WikiProteins (part of a conceptwiki), an open, collaborative wiki  focusing on proteins and their role in biology and medicine.

I would like to end my post with two thoughts.

First the world is not static. In the future scientific claims could be represented as formal RDF statements/triplets  instead of or next to the journal publications as we know them (see post on nanopublications). Such “statements” (already realized with regard to proteins and genes) are more easily linked and retrieved. In effect, peer review doesn’t prevent fraud, misrepresentation or overstatements.

Another side of the coin in this “blogs as an internet source”-dicussion is whether the citation is always appropriate and/or accurate?

Today a web page (, evidently meant for education of residents, linked to one of my posts. Almost the entire post was copied including a figure, but the only link used was one of my tags EBM (hidden in the text).  Even worse, blog posts are sometimes mentioned to give credit to disputable context. I’ve mentioned the tactics of Organized Wisdom before. More recently a site called links out of context to one of my blog post. Given the recent revelation of fraudulent anti-vaccine papers, I’m not very happy with that kind of “attribution”.

Related Articles

About these ads



17 responses

13 02 2011
Wilma van den Brink

Thank you so much. Interesting reading!! I am going to refer to this post on @followalibrary on twitter as well!

13 02 2011
Tweets that mention Internet Sources & Blog Posts in a Reference List? Yes or No? « Laika's MedLibLog --

[...] This post was mentioned on Twitter by Laika (Jacqueline), Laika (Jacqueline) and Wilma , Followalibrary. Followalibrary said: Interesting perspective! -> Blogging: Internet Sources & Blog Posts in a Reference List? Yes or No? [...]

13 02 2011

Excellent post, it got me wondering. How do you deal with the internet as a “fleeting” medium?
Wiki-articles can be changed. What if you refer to a wiki-article to make a case, and someone makes some changes to the content afterwards.
Blog posts or even an entire blog can be removed overnight. All it takes is to forget to pay for your domain name. How do you deal with broken links in a reference list?
Should you take screenshots of every online resource you use? (Or save the webpage as a PDF)

14 02 2011

@Wilma, thanks for your comment – and for your invitation!

@Johan, that is an important issue that I didn’t directly address. I did mention, that there are APA publications guidelines ((, also for electronic media citations. The APA Style Blog gives specific examples or goes into details where the APA guidelines do not. In case of a wiki-article (or other sources that may change over time) one has to add the retrieval date. Usually you do not quot an entire entry but use a wiki as a starting point. I think I would refer to a wiki if it had been crucial (more like an acknowledgement, not a quote)

With respect to broken links: the APA style doesn’t allow for hyperlinks in manuscripts ( This is different for most wikis and blogs, like mine who do hyperlink to other sources. But here we are mainly talking about citing of blogs and other internet sources in scientific papers.

Of course these measures are no real solutions for changed or deleted content. But it is the best way to refer to the page/post/information when it existed. If the original information is crucial, you can make screenshots, PDF’s or copies. That can be very important in certain situations i.e. when people deny that they have written something like in the case of Clinical Reader. Luckily there were screendumps around. You don’t publish these in official research papers though.

The problem of changing electronic references had been addressed by some of my Dutch carnival colleagues. See for instance the comments in the Call for submissions post by Christian ven der Ven. One very practical advice is to use the archive of the Wayback Machine to find the original versions (

Others nuance the difference with classic sources. New book editions, reports, newspapers can also change over time.

14 02 2011

Enjoyed the post as it nicely summarized many of the issues related to citing internet sources – especially when they exist exclusively online.

The problem of digital permanence (aka URL attrition, link rot, link decay) identified in the post and comments has been addressed to a degree by certain publishers (e.g., BioMedCentral) and journals (e.g., JMIR) via archiving services like WebCite ( URL attrition for references in journals has been demonstrated in studies at rates “ranging from 3.7 per cent after three months to 41.7 per cent after four years and span a variety of journal types” ( In addition to the decay itself, problems with archiving services related to copyright remain unresolved.

My answer to the blog post title question is the definitive: it depends.


14 02 2011

Thanks for the very useful information and additional links, kevin.

14 02 2011
Jean-Claude Bradley

Excellent post! For an example of a peer-reviewed paper using lab notebook pages and blog posts as references see:

14 02 2011

@Jean Claude Indeed, this is a very good example that lab notebook pages and blog posts can be used to support claims made in a peer reviewed article (quot from your blog post).

Reading your comment I suddenly remembered that Attila Chordash (@attilacsordas on Twitter) did a live thesis building blogxperiment. William Gunn (@MrGunn at Twitter, now working for Mendeley) did the same. Both wrote their dissertation via their blog to gain increased exposure and feedback.

Good examples to illustrate the point that blogposts and notebooks can be used as a source, but (still) quite unique in its kind.

14 02 2011
Jean-Claude Bradley

Nice examples – I also had a Masters student write her thesis on a wiki around the same time

14 02 2011
Mr. Gunn

This question got a large amount of discussion some years ago, and since a consensus has more or less been reached. All the major citation styles have a specific format for citing internet sources, some more useful than others. I wrote a post about this a few years back:

I’m sad to report that it doesn’t appear that the most useful distinction – static vs. dynamic content – has been factored by the folks who maintain the citation styles, but that’s not entirely surprising, as they’re not technical types anyways (which begs the question of why we’re still listening to them as authorities, but I digress…)

At any rate, blog posts have permalinks, and this is the canonical way to cite a blog post. Anyone who says otherwise is clueless. Likewise, a fundamental component of a wiki is the revision history, so you can actually cite the version of the page as you saw it, without having to mention the day you read it or save a snapshot of the page or anything like that. It’s built in. In general, many databases allow you to give a URL that specifies the atomic bit of content you’re referring to. Just mentioning the name of the database, as some citation styles suggest, is pretty much useless.

Not to go on a rant any longer… Yes, there does exist content on the internet worthy of citing in research papers (which will themselves be on the internet).

14 02 2011

Thanks for this very clarifying rant! And the useful link.
You were ahead of your time, I presume.

I think that blogs and wikis are less accepted in the world where I come from, the EBM-world.
Earlier I wrote about this EBM-web 2.0 split:

14 02 2011

I asked several scientists at Twitter to comment to my question whether blogs belong in a reference list:

@MrGunn (see comment above) answered: (!/mrgunn/statuses/37239598287425536)

@boraz answered: (!/BoraZ/statuses/37248622907240448).

The shortlinks link to the following blogposts by bora at the famous “A blog around the clock” (his previous blog)

A quote in the latter post:

This post of mine was cited in a review paper and this post was cited in an Editorial. And that’s just me. So, citing blogs is not such a new thing any more.

That does explain Mr Gunns response.

16 02 2011
Keith Grimaldi

Hi – I can’t see how we cannot quote internet sources, it’s clear that there is useful information that deserves to be included in subsequent work and articles. There are the obvious problems which are mentioned, sometimes in the future the work referred to will be lost forever, other times the links will change, hopefully, for the majority, the cited work will be stable.

The author citing the internet source has a responsibility to make it as easy as possible to find the work. If it is no longer on the internet there is not much to do, but in general, the citation should include a title & author, not just the URL. Often the link will change, the title will allow the work to be found.

9 03 2011
Researchers And Social Media: Uptake Increases When Obvious Benefits Result « The Scholarly Kitchen « digitalcollaboration

[...] Internet Sources & Blog Posts in a Reference List? Yes or No? ( [...]

9 03 2011
Multi-Author Medical Blogs – At the End it is all about Credibility « Laika's MedLibLog

[...] Internet Sources & Blog Posts in a Reference List? Yes or No? ( [...]

6 04 2011
Medicine 2.0 Blog Carnival | Medicina Familiar

[...] Internet Sources & Blog Posts in a Reference List? Yes or No? ( [...]

8 05 2011
3rd Call for Submissions for “Medical Information Matters”: Tools for Searching the Biomedical Literature « Laika's MedLibLog

[...] Internet Sources & Blog Posts in a Reference List? Yes or No? ( [...]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Get every new post delivered to your Inbox.

Join 607 other followers

%d bloggers like this: