Category Archives: Uncategorized

Using Situational Judgment Testing

We explored a number of aspects of Situational Judgment Testing (SJT) in OpenLabyrinth over the past couple of years. This is a very useful assessment format and is widely adopted in selection of candidates in the UK.

Now we have finally pulled together our working notes, along with a few research questions that we would be interested in exploring with others. Check out the notes here: Using OpenLabyrinth for Situational Judgment Testing2

Contact us through these pages if you would like to explore this further.

Happy Milestones, Clinisnips

We just want say ‘Happy 3rd MilleniDay’, or should that be ‘3rd KiloDay’ to our sister project, Clinisnips. That is, it is 3001 days since the first video went live on the Clinisnips channel on April 16th, 2008.

In that time, there have been nearly 4.7M views of the channel, at a rate of about 100 views per hour, which has not dropped off at all since the channel was launched.

Google Analytics is a very powerful tool and we have pulled some impressive stats over that time period. We calculate that the Watch Time over that period is over 7.68 million minutes or more than 127,000 hours of CME!

Now, of course, as we recognized back in our article…

Topps, D., Helmer, J., & Ellaway, R. (2013). YouTube as a platform for publishing clinical skills training videos. Academic Medicine : Journal of the Association of American Medical Colleges, 88(2), 192-7. doi:10.1097/ACM.0b013e31827c5352

… not every viewer will be a healthcare professional. While we are happy that the reach of Clinisnips has been very broad, as demonstrated by the broad variety of Comments that we found in our qualitative analysis, we have not been able to track in detail who has been watching Clinisnips and what else they do around those times.

You can be sure, however, that Google has a very good idea of what its users do on all of its sites, services and channels. It is why they have grown to be the size they are today. This is big data, writ large. While they share little snippets of analytics with their contributors like us, they spend a lot of effort in tracking the activity metrics of all of us.

This is why we are becoming increasingly excited about what can be done with big data, and activity metrics (via xAPI etc) in the education research world. Imagine how much more effective we could make our educational materials, if we understood how they are used and what impact they have. We are late to the table, compared to commerce. It is well past time that we started looking at what our users do, not what they (or their teachers) say they do!

Turk Talk in action

Tomorrow, we will be testing out our Turk Talk function in OpenLabyrinth for the first time in a live teaching session. A number of nursing students at the University of Calgary will be putting it through its paces.

There have been some nice usability improvements since our early designs and it is now pretty easy to use. Michelle Cullen and her team at the School of Nursing have done a great job in debugging the cases. We are looking forwards to a fun session.

Testing this week has gone well and our facilitators even seem to have had fun! We hope the students do tomorrow as well.

WAVES: Widening Access to Virtual Educational Scenarios

The WAVES Project group had its first meeting in London over the past couple of days. More info will gradually be released on the project web site.

OpenLabyrinth will be extensively used in creating and supporting virtual scenarios for this project. And what the heck are virtual scenarios, I hear you ask? Scenarios in this context relate to the work of Ruth Colvin Clark in her book, Scenario-Based eLearning.

We have found this book to be very useful in our Scenario Based Learning Designs. Scenarios are basically groups of learning activities, put together so that you make best use of the resources available. We use the concept of Scenarios within OpenLabyrinth as a way to group together virtual patient cases (in a logically connected series if necessary), groups of learners, reports, counters, rules etc so that the Scenario Designer can take things way beyond a single simple case.

WAVES is coordinated by St George’s University, London, who are long time experts in OpenLabyrinth. This is a huge project involving health professional schools from many countries in Europe.

We are excited that the group is keen to explore the use of xAPI as a means of tracking activity metrics, and is also keen to work with Medbiquitous on forging common practices and Profiles around xAPI and Scenarios.

Medbiq xAPI workshop

At the annual Medbiquitous Conference in Baltimore, OpenLabyrinth provided the underpinnings for a workshop demonstrating the capabilities of the Experience API (xAPI).

Medbiq xAPI workshop team Using a simple Arduino computer, the team of Ellen Meiselman, Corey Albersworth, David Topps and Corey Wirun were able to track the stress levels of workshop participants as they played a very challenging series of OpenLabyrinth mini-cases. Sensors on the Arduino continuously measured heart rate and galvanic skin response which, even on these cheap ($30) kits, were easily sensitive enough to detect subtle changes in stress levels.

We intentionally set a very tight set of timers on the case series so that participants were increasingly pushed to make very rapid decisions. The data from the sensors were collected in our GrassBlade LRS, along with xAPI statements from our OpenLabyrinth cases.

Seeing such simple technology providing quite sophisticated tracking of learner stress levels prompted a lot of vigorous discussion in the workshop on how such activity metrics can be used in other ways.

We really appreciate the collaboration and help we received from the xAPI community in pulling this workshop together. We had naively thought that, since both Arduino and xAPI are simple to work with, this would be a nice quick effort. Corey A ran into a lot of tiny but time consuming quirks and put in many hours in getting this all to work smoothly.

We especially want to acknowledge the detailed help we received from Pankaj Agrawal (GrassBlade LRS) and Andrew Downes (Watershed LRS) for their patience and troubleshooting. For some parts of the project, we had some quite sophisticated statement pulls from GrassBlade to Watershed, due to the collaboration of these folks. It really showed us how much more you can achieve by blending the capabilities of these various devices and platforms,  using xAPI.

More xAPI stuff from OpenLabyrinth

Our work with activity metrics and the Experience API (xAPI) continues apace with the OpenLabyrinth platform. We have been able to integrate xAPI statements into our CURIOS video mashup tool.

Now when you insert a video mashup into one of our OpenLabyrinth cases, you can track how your users are using your videos, which bits they watch and replay again.

This will dovetail nicely with some of the xAPI features that we can now access with the H5P widgets. It will also allow us to track activities across a widening range of educational activities.

OpenLabyrinth v3.4 released

Delighted to announce that we released v3.4 of OpenLabyrinth today.

Lots and lots of changes in this one… maybe too many… we are considering putting out a v3.3.3 which had fewer changes.

Those who have been following this blog will be familiar with what we have been working on. We’ll put out a more detailed list of changes on the forum soon. The main things are as follows:

  • xAPI reporting to a LRS
  • H5P widget integration (https://h5p.org/)
  • Turk Talk for chat style small group communications
  • Improved LTI stability

For the latest release, server administrators can pull this from Github. For the rest of us, we always run the latest version of the software on our demo server so if you want to try these things out, contact us for a free trial account.

OpenLabyrinth at CCME

The Canadian Conference on Medical Education (CCME) starts tomorrow in Montreal, QC.

This annual gathering brings together medical educators from around the world, discussing a wide range of topics and research interests.

OpenLabyrinth features in several presentations, workshops and posters at this conference. If you are there, chat to us more about this educational research platform. Since OpenLabyrinth is free and open-source, we don’t have the funds to have a fancy exhibitor booth. Plus, we are not selling anything.

But we are always interested in talking to groups who are interested in educational research and who want to explore what can be done with activity metrics, branched pathways, embedded video or facilitated small group scenarios.

Turk Talk improvements in OpenLabyrinth

Today we released a new improved format for our powerful Turk Talk function in OpenLabyrinth. Many small tweaks have made this much more user friendly.

Check out the addendum to the User Guide which explains in better detail how to use this new functionality. Here is what the Turker now sees when guiding up to 8 learners simultaneously:

OLab Turker Chat panel

This is just a glimpse. The instruction notes show this much more clearly, but you can see 5 users waiting (pink columns) for the Turker to respond. The Turker can see where they are in the case, and who has been waiting longest.