An Educator by Chance

13 10 2010

The topic of the oncoming edition of the blog carnivalMedical Information Matters“, hosted by Daniel Hooker, is close to my heart.

Daniel at his call for submissions post:

I’d love to see posts on new things you’re trying out this year: new projects, teaching sessions, innovative services. Maybe it’s something tried and true that you’d like to reflect on. And this goes for anyone starting out fresh this term, not just librarians!

When I started as a clinical librarian 5 years ago, I mainly did search requests. Soon I also gave workshops as part of evidence based practice courses.

Our library gave the normal library courses PubMed, Reference Manager etc. We did little extra for medical students. There was a library introduction at the beginning and a PubMed training at the end of the curriculum.

Thus, when the interns had to do a CAT (Critically Appraised Topic), they had to start from SCRATCH 😉 : learn the PICO, domains, study types, searching the various databases.  After I gave  a dozen or so 1-hour long introductions to consecutive interns, repeating the same things over and over, I realized this was an ineffective use of time. So I organized a monthly CAT-introduction with a computer workshop. After this introduction I helped interns with their specific CAT, if necessary.

This course is appreciated very much and  interns usually sigh: “why didn’t we learn this before?! If we had known this…”, etcetera.

Thus we, librarians, were very enthusiastic when we got more time in the newly organized curriculum.

We made e-learning modules for the first year, two for the second year, a Pubmed-tutorial, and a computer workshop (150 min!). In the 4th year we grade the CATs.

The e-learning modules costed me tons of time. If you read the post “How to become a big e-learning nerd by mistake” at Finite Attention Span you understand why.

We used a system that was designed for exams. On my request the educational department embed the system in a website, so students could go back and forth. Lacking any good books on the topic, students should also be able to reread the text and print whatever they liked.

I was told that variation was important. Thus I used each and every of the 10 available question types. Drop down menus, clickable menus, making right pairs of terms etc. Ooh and I loved the one I used for PICO’s, where you could drag words in a sentence to the P, I, C or O. Wonderful.

Another e-learning module consisted largely of Adobe Captivate movies. As  described in the above mentioned post:

Recognise that you are on a learning curve. First of all, it is vital that your software does not always remind you to save individual files before closing the program. It is especially helpful if you can demonstrate this three times inside a week, so that you end up losing the equivalent of about two days’ work: this will provide you with a learning experience that is pretty much optimised.

Swear. Vigorously.

Become a virtuoso of the panic-save, performing Ctrl+S reflexively in your sleep, every three minutes (…)

Correcting the callouts and highlight boxes and animation timings so they don’t look like they were put together by committee is complicated. Also, writing really clear, unambiguous copy takes time.

It sounds familiar. It also regularly happened to me that I started with the wrong resolution. Then I heard afterwards: “Sorry, we can only use 800×600.”

But workshops are also time-consuming. Largely because the entire librarian staff is needed to run 30 workshops within a month (we have 350 students per year). Of course it didn’t end with those workshops. I had to make the lesson plan materials, had to instruct the tutors, make the time tables, the attendance lists and then put the data into an excel sheet again. I love it!

The knowledge is tested by exams. This year I had to make the questions myself -and score them too (luckily with help of one or two colleagues). Another time buster. The CATs had to be scored as well.

But it is worth all the pain and effort, isn’t it?

Students are sooo glad they learned all about EBM, CATS, scientific literature and searching…

Well, duh, not really.

Some things I learned in the meantime

  1. Medical students don’t give a da do not care much about searching and information literacy.
  2. Medical students don’t choose that study for nothing. They want to become doctors, not librarians.
  3. At the time we give the courses, the students not really need it. Unlike the interns, they do not need to present a CAT, shortly.
  4. Most of our work is undone by the influence of peers or tutors that learn the students all kind of “tricks” that aren’t.
  5. It is hard to make good exams. If the reasoning isn’t watertight, students will find it. And protest against it.
  6. …. Because even more important than becoming a doctor is their desire to pass the exams
  7. If the e-learning isn’t compulsory, it won’t be done.
  8. You can’t  test information literacy by multiple choice questions. It is “soft” knowledge, more a kind of approach or reasoning. Similarly PICO’s are seldom 100% wrong or right. The value of PICO-workshops lies in the discussions.
  9. The students just started their study. They’re mostly teens. These kids will have a completely other attitude after 4 years (no longer yelling, joking, mailing, Facebook-ing, or at least they are likely to stop after you ask).
  10. Education is something I did by chance. I just do it “in addition to my normal work”, i.e. in the same time.
  11. Even more important, I’m a beginner and have had no specific training. So I have to learn it the hard way.

Let me give some examples.

This year I wanted to update one of my modules. I had to, because practically all interfaces have changed the last two years (Think about PubMed for instance).

I made an appointment with the education department, because they had helped me enormously before.

Firstly I noticed that my name had been replaced by those of 3 people who hadn’t done anything (at least with regard to this particular e-learning course). Perhaps not so relevant here. But the first red flag…

The module was moved to another system. It looked much nicer, but apparently only allowed a few of those 10 types of questions. The drag and drop questions, I was so fond of, were replaced by irritating drop down menus. With the questions I made, it didn’t make sense.

The movies couldn’t be plaid fast forward, back or be stopped.

And the girl who I spoke to, a medical student herself, couldn’t disguise her dislike of the movies. First she didn’t like the call-outs and highlight boxes, she rather liked a voice (me speaking, deleting the laborious call-outs ?!). Then she said the videos were endless and it was nicer when the students could try it themselves (which was in fact the assignment). She ignored my suggestion that Adobe is suitable for virtual online training.

Then someone next to her said: Do you know “Snag-it”, you can make movies with that too!?

Do I know Snag-it? Yes I do. I even bought it for my home computer. But Snag-it is nowhere near Adobe Captivate, at least regarding call-outs and assembly. I almost mentioned Camtasia, which is from the same company as Snag-it, but more suitable for this job.

Then the girl said the movies were only meant to show “where to press the buttons”, which I repeatedly denied: those movies were meant to highlight the value of the various sources. She also suggested that I should do some usability testing, not on my colleagues, but on the students.

Funny how insights can change over times. The one who helped me considered it one of the best tutorials.

While talking to her, it stroke me that the movies were taking very long and I wondered whether each single call-out saying “press this” was functional. Perhaps she was right in a way. Perhaps some movies should be changed into plain screenshots (which I had tried to avoid, because they were so annoying Powerpoint like). If my aim wasn’t that students learned which button to press, why show it all the time?? (perhaps because Adobe shows every mouse click, it is so easy to keep it in..)

It is a long way to develop something that is educative, effective and not boring….

But little by little we can make things better.

Last year one of the coordinators proposed not to take an exam the first year but give an assignment. The students had to search for an original study on a topic in PubMed (2nd semester) and write a summary about it (3rd semester). The PubMed tutorial became compulsory, but the two Q & A sessions (with computers) were voluntary. Half of the students came to those sessions. And the atmosphere was very good. Most students really wanted to find a good study (you could only claim an article once). Some fished whether the answers were worth the full 4 points and what they had to do to get it. The quality of the searches and the general approach were quite good.

In good spirits I will start with updating the other modules. The first should be finished in a few days. That is… if they didn’t move this module to the next semester, as the catalog indicates.

That would be a shame, because then I have to change all the cardiology examples into pulmonology examples.

Gosh!…. No!!

Credits

The title is inspired by the  post “How to become a big e-learning nerd by mistake”.
Thanks to Annemarie Cunningham (@amcunningham on Twitter) for alerting me to it.

Related Articles

Advertisements




Fighting “Powerpoint-Death” by Science, Prezi or…?!

24 08 2010
Audience response radio frequency keypad with ...

Image via Wikipedia

Recently Kevin Clauson [@kevinclauson] made another great presentation, called the “Science behind Engaging Students in Class”. The presentation focuses on the use of “clickers” or an audience response system (ARS) to engage the audience (here mainly students in a class). It is an expanded version of “How to Fight Lecturalgia“.

First Kevin asks the audience questions about their knowledge/use of ARS -using the ARS system, of course-. Next he goes more deeply into the need to engage the audience (attention span, boredom) and then he addresses the successes and pitfalls of ARS.

Each statement has a scientific underpinning, and a reference to it.

For instance, one conclusion is that use of ARS improves performance with analytical type exam questions, but not with memorization exam questions.

Kevin stresses that ARS  it is just another tool, albeit a powerful one.

Of course you have to avoid the usual presentation-killer aspects of PowerPoint, like including too many slides, bullets and data, as so wonderfully illustrated by the famous “Dead by Powerpoint” presentation. I included it below, in case you’ve never seen it.

It also contains recommendations how to  improve your PowerPoint.

This following video elaborates on the same theme. It is called “Life after Death by PowerPoint”. Not an appropriate name, because it only magnifies Powerpoints killer-points. You might enjoy it though (if you can put up with the canned laugh).

Below is another presentation about the science of presentations. And although I noticed little science in it, I did find the CEO-presentation interesting because it discusses the use of live tweeting (and blogging) to give “contagious talks” (you know Twitter going “viral”).





The best moment teaching EBM-searching skills?

6 04 2008

When you are a (future) doctor you will obviously need to look for publications at one stage or another. PubMed is the place to look for relevant medical papers. Usually medical students begin to feel the urge to learn the ins and outs of PubMed (and searching in general) once they do their scientific training (4th year) or their internship, especially when they have to perform a CAT, critically appraised topic. Then it turns out their superficial knowledge of PubMed is one of the main hurdles. They find too many hits or too few and/or miss the relevant ones.

To help them I started a monthly class of 2 hours in which I learn interns (at the dept. Gynaecology) the basics of EBM, at least the first two steps: constructing a well answerable question using the PICO method (including defining the domain/levels of evidence) and finding the evidence in PubMed as well as in aggregate resources. (these two steps are called EBS or evidence based searching). Interns are asked to prepare 4 questions, all based on previous CATs. The first question is answered during an interactive power point presentation (first hour), the other 3 are practised ‘hands on’. If needed I give them personal aftercare.

It is a highly appreciated course, and it helped to improve the quality of the CATs. So that’s very encouraging.

I often get the same feedback from the user surveys:

  • well structured and informative
  • why didn’t we get this earlier?
  • too much information at once (especially at the end of the day)

To meet their wish my colleagues started a short introduction in PubMed prior to this ‘advanced’ class. As a result, the students are better acquainted with PubMed and we can delve more in depth into the subject. Last session they even prepared all questions. I wasn’t aware and asked one of the students (quite disappointed) why he still put the words in one string in the search bar instead of looking up each word separately and checking whether the words mapped correctly to the appropriate MeSH. He replied: “But I already did this at home. I checked out all the words.” showing his notes. And I must admit his search was quite good. So I was very satisfied with this group of students.

But the feedback remains the same. well structured and informative – why didn’t we get this earlier? – too much information at once. (especially at the end of the day)

Thus one would be inclined to think there is a need to teach students earlier on.

Now coincidently, a new curriculum has started in our academic hospital, in which EBM is incorporated into the clinical modules. The 1st year students learn about information resources and study designs. In the 2nd year they learn the basics of PubMed, EBM, PICO’s, Evidence Based Searching and Systematic Reviews.

Our library is involved in the educational process with respect to information resources, PICO’s and searching. Most of the teaching is in the form of e-learning (Dutch: COO, computer ondersteunend onderwijs) using the QMP (question mark perception) system, which is basically designed to test knowledge.

We have made a tutorial for PubMed (a-basic-learn-the-buttons-and-MeSH-course) and I prepared an e-learning module on PICO’s, study-designs and aggregate evidence, for the Cardiology block. This took me 6 weeks! It was reasonably well received by the students… That is, who bothered to give feedback.

During the course “Pulmonology” (february/march) we gave 30 “Finding the Evidence Search Workshops” to 6-12 students.
I had quite high expectations, since in theory these students should have a good theoretical basis (considering the earlier e-learning tutorials).

However their knowledge was quite disappointing, and even more so were their motivation and attitude. They were just a bunch of kids, most of them not very interested in PubMed, searching, EBM or whatsoever. They were often giggling and chatting, which I find rather distracting, or were passive, silent and gazing, which is even more distracting. And when I took a glimpse at their screens I often saw g-mail and unfamiliar colourful sites instead of PubMed.

I wondered at what point these students would pupate and transform into the butterflies called interns? And at this stage I couldn’t imagine them sitting on my bedside as a doctor I would trust unconditionally.

Was it really this bad? No, I’m a bit exaggerating. When I sound them out it appeared that they find the scientific methodology courses to fragmented, too basic and not the core of their study: firstly they want to pass their exams and secondly they want to become a doctor(!), not a scientist nor a librarian. I suppose E-learning and tutorials are not the ideal tools, not even for the computer generation. E-learning has to be dosed and is not as inspiring as a good tutor (at least that is what I think).

Anyway after one hour yawning, sighing and bewildered looks and after a much needed coffee break with cookies (a brilliant move of two of my collegues) I got the impression the penny finally dropped. Some students mumbled: “Mmm, I think I come to understand it” others smiled and uttered “Yes!” and the remaining questions were answered rather swiftly by most students. It even turned out that some of the glossy sites I had seen were on-line medical dictionaries, they used to look up the correct terms. Yes, this young generation is capable of multitasking.

If these courses were evaluated the same way as the above mentioned CAT-course, I guess the outcome would be as follows:

  • not particularly interesting
  • why do we have to learn this now? can’t it wait?
  • too much information….

We still need to find the ideal timing for these courses and also a better dosing. The best timing is when they need it the most, I suppose. The students who absorbed the information best were those who needed the information right now or found out that needed it before (i.e. they now realized that their previous searches were far from ideal). The form is also something to workat. Especially the e-learning modules should be better integrated into the clinical blocks. It is not sufficient to tune in with the subject. For students to appreciate and retain information, searching skills need to be taught in tandem with assignments. Students need to see the relevance of what they learning.