MSc in Evidence-Based Health Care

Submitted by amandab on 2 March 2009 - 11:26pm.
in
Teaching with Sakai Innovation Award: 
2009
Award Status: 
Entry
Contributing Institution: 
The University of Oxford

The Oxford International Programme in Evidence-Based Health Care was launched in October 2008. Its purpose was twofold: to help health care professionals develop the skills they need to be able to practice in an evidence-based way; to give clinicians the skills and understanding required to undertake useful and methodologically sound clinical research.

Further information about the course can be found online at http://cpd.conted.ox.ac.uk/ebhc/modules.asp

Course Information
Course Number/ID: 
000130
Course Length (number of weeks): 
8
Course Delivery Mode: 
Hybrid/Blended (some face-to-face and some online interactions)
Average Number of Enrolled Students: 
Between 10 and 30 students
Course Level: 
Graduate
Course Level: 
Vocational/Professional
Course Level: 
Other (please describe below)
Describe Other Course Level: 
Participants are expected to have several years experience in service or related area
Course Development & Delivery
Course Contributors: 

Course Director

Dr Amanda Burls

Module Leads

Dr Carl Heneghan

Dr Amanda Burls

Dr JoseEmparanza

Dr RafaelPerera

Dr Clare Bankhead

Dr Matthew Thompson

Professor Paul Glasziou

Sara Ryan

Invited Teachers

Sir Iain Chalmers

Professor Douglas Altman

Professor Jack Dowie

Dr Alex Sutton

Fawei Geng

Dr Helen Doll

Instructional designer

Fawei Geng

Course administrators

Caroline Elmslie

Lanson Moore

Rachel Parsons

Course Development: 

In health care, new knowledge is constantly being generated and the best way of diagnosing and treating patients is continually evolving. Therefore, health care professionals who want to provide the best care to their patients, cannot rely on what they were taught at medical school or learnt during their professional training - instead they need to practise life-long learning and have the skills and disposition to find, critically appraise and integrate up-to-date scientific evidence into their clinical decisions.

Traditionally medical education has been about giving people medical skills and knowledge and most doctors and health care professionals have not been trained in the skills required for evidence-based practice. The MSc in EBHC aims to address this deficit and enable participants to become innovative and adaptive learners. It teaches participants to recognise uncertainties and when there is a need to look for new medical knowledge and provides them with the skills to do this and integrate new research evidence into their decision making.

Designing the programme posed challenges:

  1. It was aimed at senior health professionals with considerable specialist knowledge and responsible positions and needed to recognize and build on participants' own expertise.
  2. Modern health care involves teamwork and we wanted participants to learn to work collaboratively and constructively together by incorporating this into the design.
  3. The course needed to be flexible, so it could be undertaken by people in full-time jobs, and be compatible with, for example, people being on-call for emergency cover, enabling students to do concentrated work when they had time and not get left behind when their jobs demanded their full attention.
  4. Participants come from all over the world and needed to be able to work together while geographically separated and in different time zones. The course needed to be sensitive to the different contexts (health care systems, needs, values and resources) in which people were working.
  5. The course needed to be relevant to participants from wide range of professions and specialties (e.g. doctors, nurses, pharmacologists, physiotherapists, researchers, managers, information specialists).
  6. We anticipated there would be considerable heterogeneity in background knowledge and skills (e.g. statistics) of participants and the course needed to be work for all.
  7. To provide the highest quality input the course we wanted the course to be delivered by a wide range of individuals in order to bring in the leaders at the forefront of development and research in this area.

The course team discussed options and decided a hybrid programme of online and face-to-face learning was the best method of addressing these challenges. Sakai-based WebLearn, being piloted within Oxford University, was chosen to support the course due its flexibility and ease of use.

The new programme is envisioned as part of a dynamic process building a Community of Practice to develop evidence-based health care and support course participants in the future. We therefore also constructed Sakai-based sites to provide an infrastructure for sustaining a Community of Practice of graduates and contributors to the programme in the longer term.

Course Delivery: 

The MSc requires satisfactory completion of 6 taught modules (2 core, 4 elective), each of which takes place over 8 weeks, and a research dissertation. The first week of each module is a preparatory week and aimed at encouraging people to interact online using Sakai, and ensure a minimum baseline understanding to provide a common starting point. For example in the Clinical Epidemiology and Decision Making module, two seminal texts on clinical epidemiology, written 20 years apart, were provided in Sakai, and students asked to discuss in a Forum whether they thought that the concept had changed during this period and, if so, how and why. The Forum tool was also used for participants to introduce themselves and start building a cohesive group. The second week is residential in Oxford. This intensive week covers most of the theory. The timetable, structure and materials are available on Sakai providing continuity with the online weeks. The face-to-face contact helps participants get to know and trust each other and facilitates supportive interactions during the following online activities and learning. The tremendous online activity observed, and positive feedback, suggests this structure helped the students bond well. Peer-supported learning arose spontaneously. For example, when a student used the Chat Room to ask about accessing a reference, another student emailed the reference. Others then explained how the he could overcome his access problems. We believe the active engagement in the course is due to three factors: deep interest in the subject, engaging teaching methods, the Sakai-based virtual learning environment.

Sakai helps keep momentum going and drives the self-directed, autonomous, learning culture. For example, a Wiki was provided for one exercise. In the following activity students decided to use the wiki to generate a group document identifying problems and potential solutions. The flexible infrastructure thereby helped develop team working and community building. In addition, students constructively challenge each other and deepen their understanding of the concepts discussed in class, thereby enhancing critical thinking skills.

The tutors' strategy is to use Sakai to encourage student-centred learning and allow the students' voice to predominate. However, tutors monitor activities and can intervene when necessary. For example, after an activity where students undertook searches on each others clinical questions and recorded and shared their process and findings, tutors identified searching skills as a learning need. They produced and uploaded to Sakai a screen-recording, with voice-over, demonstrating searching techniques on different databases. And, when discussions started exploring qualitative methodologies a qualitative researcher was able to join the site to offer support and signpost useful resources. Sakai's flexibility permits the MSc to be developed and improved continually based on tutor observation and feedback from students, making it dynamic and responsive to participants' needs. The Forum, Sakai polls and surveys also permitted immediate feedback facilitating real-time improvements in the course. The Site Info facility enabled tutors to identify people not participating and proactively support two participants who had personal health problems and family tragedies.

Course Self-Assessment
Communication & Collaboration Self-Assessment: 
Excellent
Communication & Collaboration Evidence: 

Although the Sakai website and course were designed to facilitate communication and collaboration, the extent to which this happened has been exceptional and far exceeded our expectations. Communication among the module participants begins in week one before they meet face-to-face. The online Forum allows course tutors to introduce themselves and their research interests to their students and students to say "hello" to each other and the tutors. All students were invited to do this in the first module and were successful before any training. In subsequent modules they did so spontaneously, e.g. one module, with just nine participants, generated 66 content-rich messages in the introductory week alone. The face-to-face week extensively uses interactive, problem-based, small group learning and this helped created trust and mutual respect amongst participants which facilitated communication throughout the subsequent online course.

The online, activity-based, collaborative learning design ensured that the communication flourished. In the six weeks following the residential weeks, messages flood in to discuss course-related topics, share new discoveries and give feedback. The Chat Room was also used by students to help each other with research topics, technical problems, and social activities.

Students are asked to reflect on their objectives at the start, during and at the end of each module to encourage them to take control of their own learning. A blog is used later in the course to facilitate this and allow private and shared reflections. Tutors are responsive to students' objectives and are able to adapt the course materials and VLE to students' needs. A wiki enabled students to work collaboratively to produce joint documents and the Forum to discuss topics taught in class. Participants also used the wiki as a dynamic forum for brainstorming, critiquing proposed technologies, and voting on potential solutions.

During the Oxford residential week, students have several invited opportunities to give verbal feedback about the course and there is an anonymous survey at the end about the quality, format, content and delivery and what can be improved. Actions were taken in real-time to respond feedback and observed needs. All modules have specific forums for feedback about the course organization, content and the website functionality and problems. Communication and collaborative activities were supported and enhanced by using many Sakai technologies, including: Wiki, Web Content, Videos, Discussion forum, Polls, Surveys, Resources, Drop box, Announcements, and Chat Room.

Students engaged in individual, group and team work. We had anticipated course leads needing to contribute to the discussion on at least a daily basis to ensure that people had responses to input and to stimulate and guide discussions but found that the forums rapidly acquired a dynamic of their own with students consistently responding positively and constructively to each other's contributions and providing help when colleagues met with problems. In this situation the tutors have kept a low profile to encourage active learning and not to interfere with the dynamic, only providing light guidance if the discussion required this.

Learning Material Self-Assessment: 
Excellent
Learning Material Evidence: 

The course was designed using ¨Learning Activity¨ design principles and a clear structure. The aims, learning objectives and expectations are given in the course and module handbook, Sakai, and at the start of every session. Each module has associated learning activities designed to build skills and knowledge in a natural progression. Each activity consists of objectives, pre-knowledge/skills required, resources to support the activity, roles played by participants and tutors, learning outcomes. Feedback on is given by tutors as required.

To help students understand the structure, learning activities, sequencing, and expectations, the Sakai sites:

  • Clearly explain: how to use the Virtual Learning Environment; the module structure; the module aims and learning outcomes; and the nature of the assessment.
  • Use the same basic structure for all modules and have a consistent design style across different learning activities within modules including: aims, objectives, tasks and readings.
  • Have an intuitive, easily navigable, ¨Resources Corner¨ with learning support resources which provides a structured single-access point for all compulsory reading, handouts, videos and supplementary reading materials and tools.

During the residential week, students are introduced to the design principle and course expectations. The first day starts with session on personal objectives and agreeing group objectives.

Core reading is identified in advance so that students can prepare in advance if that suits their work patterns. There is supplementary reading signposted that can be read if a participant wishes to go into depth in a particular area. Compulsory reading is clearly identified and the date by which it is required made clear. Students are also reminded shortly before the session in which it is important to have read a particular paper. We found students well prepared.

We used problem-based learning and participants work on clinical questions that have arisen from their day-to-day care of patients. This makes the learning more meaningful and ensures students are highly motivated on tasks. The degree to which this is the case can be seen from the fact that in module 1 some of the activities (non-assessed) actually resulted in publications in peer-reviewed journals and presentations at international conferences. "The style of teaching was largely responsible for how successful it was. It ensured the whole class participated and built on their existing knowledge. It also allowed us the opportunity to learn from each other."

Activities within modules are designed to build together and give the participants the skills required for the assignment. Assignments are designed to be formative not only summative and prepare participants for their research dissertation. This approach has resulted in high level of student motivation, interest and satisfaction for activities and assignments.

There are links to other online university training for example an online tutorial about plagiarism http://tallcourses.conted.ox.ac.uk/plagiarism/xhtml/introduction/Introductions.html and an course run by the Clinical Trials Support Unit, http://www.admin.ox.ac.uk/rso/clinical/gcp_online_training.shtml, successful completion of which provides a recognised certificate of Good Clinical Practice for researchers.

Learning Outcomes & Assessment Self-Assessment: 
Excellent
Learning Outcomes & Assessment Evidence: 

As the course is designed using the ‘Learning Activity¨ design principle, the learning outcomes are clearly defined in each learning activity. Activities link into the summative assessment for each module and, where possible, the summative assessment for each module links through to the dissertation, For example the assignment for the core module, Introduction to Research Methods, is the production of a research protocol. Most participants produce a protocol for their intended research dissertation and are thus able to get feedback and improve the quality of the methodology of their research projects.

The residential week give participants opportunities to confirm the course expectations and assessment details. Anonymous feedback was also provided in Sakai and in classroom. Peer feedback is also implemented in Sakai using Wiki, Forums and Chat Room. Face-to-face discussion and feedback were done during and at the end of residential week. The interaction between tutors and participants enabled participants to work as a team after leaving Oxford. In the course, many assessment tasks were based on a real clinical case, which challenged participants, and encourage active involvement in the course at the same time. The participants realize that undertaking the learning activities will contribute both their dissertation and professional practice in the future. Feedbackhas been overwhelmingly positive (all scores positive with modal score "excellent" and many free text comments such as: ""A fantastically taught and inspiring course!"; "Outstanding lectures, teaching, methodologies. Appreciate the interactive learner responsive sessions."

Course Look & Feel, Web Usability Self-Assessment: 
Excellent
Course Look & Feel, Web Usability Evidence: 

Each module is supported by a Sakai website throughout the distance learning and during the residential week. All sites have a consistent layout: toolbar on the left, content is in the middle, and latest developments on the right. Each module site has the same structure to provide a common "look and feel". The sites have been designed to be intuitive and easy to navigate, with commonly used links at the top of the menu and each week's activity in order of their chronological occurrence. Each activity has the same structure and icon symbols (Aim, Objectives, Tasks, Reading). Pages are kept clean and short (to minimise scrolling) with clear links to enable students to access further information if they want to. Links open into new windows to avoid the screen becoming cluttered and assist navigation. The purpose of the design is to make the interface unobtrusive and permit participants to concentrate on the content. The ability to create hidden content and schedule announcements enabled preparation of modules and additional content in advance and was a very useful tool for tutors and helped ensure the course was well organised, consistent, attractive and easy to use and that key information was presented in a coherent order. All references are linked to the URLs where they can be downloaded to save students time.

The first module started with only a few tools to help participants become comfortable with the online VLE and ensure that they did not feel overwhelmed by the technology initially. During the first face-to-face training students were introduced to the website and there was one-on-one support for those who lack confidence. As the participants gain confidence, new tools , such as a wiki to facilitate collaborative learning and critical thinking, blogs to encourage active reflective learning, and polls and surveys, were introduced. Despite initial concerns expressed by those less familiar with IT, feedback from students showed they loved working in the VLE:" I thoroughly enjoyed M3 and I believe all of us did. We learnt a lot. I am a believer in active learning by the student and I liked the fact that there were a lot of hands-on activities."

The sites use a wide range of media. Reading and other materials are given within each activity and, in addition, to make materials easy to find later, each site has a "resource corner" where handouts and essential and supplemental materials and reading are filed and signposted. Key talks from leaders in the field in the face-to-face weeks are videoed and uploaded so that students can review them should they need to and a digital library is gradually built up for future students and distance learning.

Learner Support Self-Assessment: 
Excellent
Learner Support Evidence: 

Each module runs 8 week and is very intensive for people working fulltime. Therefore the course team (Administrators, academics, tutors and IT support) consider every possible detail to support participants both online and in the office quickly and effectively. This was clearly appreciated by participants: "Everyone involved could not have been more helpful"; "The administrators... and all others behind the scenes deserve so much praise. They continually make this effortless for us and it is quite eye opening at times, the administration and support"; "I think had it not been for behind the scenes support, we would find it difficult to manage"; "We were treated very well and I feel like I have a new family. I like this. They are very prompt, deadlines, professional and do all to help - this is important."

Online: Apart from having instruction on how to use the online learning environment, during the week when student were at Oxford, an introduction to Sakai was arranged so that students can get familiarized with the online environment. The discussion forum feedback and problems are monitored and responded to at least once a day and students can also email the IT support team who respond during the 8 weeks of studying each module within less than 24 hours. "The computer support ... again was outstanding" In addition, all necessary and many supplemental learning resources are provided online. More resources, provided by tutors and participants, are added during the 8 weeks. The administrative staff also respond to administrative problems online and by email.

Offline: As the participants only stay in Oxford for one week, to ensure that they are not overwhelmed by the large amount of information they have to take in, the course team prepared comprehensive "Welcome pack" to provide necessary information about both academic and social activities during the week. One student wrote to say: "Again, I extend my appreciation to Rachel, Caroline and Lanson for always providing such amazing support. Thanks to all." We think a pleasant environment is essential promote social integration and a constructive learning environment, therefore there are free drinks and a welcome evening on the first day and superb food throughout with a formal dinner one evening. Indeed when students were invited to comment on how this week could be improved most replied "Nothing" and the only other response was "Nothing ...too much food...one could easily forget they are there to learn as the food is so delicious."

The online and hardcopy course and module handbooks contain extensive information about the online and/or campus environment and requirements for this particular course. A variety of resources and contact information is clearly presented. On the fly support material is developed throughout the semester when needed. Typical participant comments include: "The course was very good, stimulating, and provided me and us with lots of good material useful to our work."; "Thanks to all those involved in the preparation and organization. Well done and much appreciated!"

 

Teaching Innovation
Teaching Innovation: 

The MSc in EBHC started 1996 and used a VLE to provide handouts and information to students. EBHC is evolving and new resources are continually being developed, e.g the UK National Electronic Library for Health http://www.library.nhs.uk. Therefore In 2008 the programme was redesigned, using contemporary learning design principles, to capitalize on the revolution in information creation, distribution and access within health, increase flexibility and take advantage of the functionality offered by Sakai.

Innovative approaches to transform participants' learning, include:

  • Encouraging participants to identify and meet their own learning needs, e.g. forms for personal research skills analysis were developed enabling participants to map the methodological skills required by their research, assess their skill level, and identify where and how they can acquire training and support in these methodologies.
  • Real-time addition of materials and creation of videos by tutors to address knowledge and skills deficits revealed during online activities.
  • Weaving together different functionalities to support learning activities, e.g. students created a protocol for a joint research project on a wiki, discussed the direction of the project asynchronously and uploaded files using the Forum, while using the chat facility to have meetings when decisions needed to be made, leading to the production of an online research survey.
  • Orchestration of input and support from an extensive range of tutors and teachers with different skills in face-to-face and online teaching to provide a rich and stimulating environment. Tutors with statistical, information and qualitative research skills take a "horizontal" role across modules to provide specialist methodological support when required and ensure that learning in these areas builds in a coherent way. Leaders in the field, from within and outside Oxford, are invited and we have videoed these inspirational talks, making them available through Sakai, to build a useful resource for future students.

Student feedback: "What a pleasure to be taught by such knowledgeable, passionate folk."; "The course was very good, stimulating, and provided me and us with lots of good material useful to our work "Outstanding lectures, teaching, methodologies. Appreciate the interactive learner responsive sessions";" The course was excellent, length and material. It was of high calibre and the professors and speakers were very good. I like the ability to be interactive and partake."

An important innovation is that the new programme is conceived, not just as a course to which students come and leave more skilled, but as a dynamic process building a Community of Practice to support evidence-based health care as good patient care is essentially collaborative (from the synthesis of research evidence to the provision of care) and best practice requires lifelong learning. We, therefore, also built a Sakai infrastructure to support a sustainable Community of Practice of graduates and contributors to the programme. We hope this community will increase the numbers and skills of people supporting course participants and provide participants in other countries with face-to-face support when needed.