We think the dust has settled. You can continue to use this URL to access our OLab3 demo server:

https://demo.openlabyrinth.ca/

We apologize because the transition was not nearly as smooth as we had hoped. We have copied all of the case information, media files, user identities etc from the old server to the new one.

We have repointed the https://demo.openlabyrinth.ca/ URL from the old server to the new one so that outside references to your cases should now work as before.

We have left in place the URL https://mvp.olab.ca/ to point to this same server so if any of you have altered your links and references, these should still work.

Phew. A lot of fuss for something that should have happened invisibly. Sorry that it did not but this resulted from a lack of resources at UCIT, who have many other things on their plates just now.

OLab3 Demo server moved

Sorry everyone for the inconvenience. We have been having issues with the server at https://demo.openlabyrinth.ca for a few weeks now.

We will need to switch to a more stable server – this will probably happen on Monday 5 DEC 2022. We will post the new server URL at that time.

In the meantime, what to do?

1. You can continue to use https://demo.openlabyrinth.ca until Monday.
2. The forgot-my-password link on that server will not function properly. (It will say that it will send you a link but that will not be sent.)
3. We will copy the data files from https://demo.openlabyrinth.ca to the new server on Monday. This means that…
   1. Case materials and edits on https://demo.openlabyrinth.ca should still be carried over
   2. User account details will remain the same
   3. Session data (who played what when) will be carried over.
4. The server at https://demo.openlabyrinth.ca/ will be running more slowly because of these internal problems.

We are continuing to work with the Alberta Chapter of CFPC on the creation of virtual scenarios for their Community Mentorship Network. (CMN)

Using both existing cases, and newly created scenarios, our OLab team is working with ACFP authors Cathy Scrimshaw and Leah Phillips on how to best adapt these scenarios for highly interactive learning.

As part of this we hope to engage new functions, such as TTalk and DFlow, in OLab 4.5 which allow for more natural language conversations.

For my heaviest users of OLab3 on this server at https://demo.openlabyrinth.ca/, I wanted to give you guys a heads-up about a few upcoming things.

This server is getting quite slow and bogged down at times. Yes, we all want to migrate to OLab4 asap… still getting a few hiccups on that… coming soon.

Sadly on this OLab3 server, some of the slow-downs seem to be because of memory leaks in the Kohana framework on which OLab3 is based. We also have some heavy class loads coming up so we are making a few wee changes so that this server does not slow down too much.

We hope also to give you access to some alternatives. Anyway, these are the important points: 

1. The demo.openlabyrinth.ca will be rebooted periodically to clear the memory leaks. The usual time for this will be 06:00 UTC. 
2. This will not be everyday. We should be able to set up a cron job so that it only happens one day per week, probably Sunday mornings.
3. Over the next couple of weeks, it may happen more often as we prep for some big groups of students.
4. Please contact me if there is a class or exam or whatever coming up that you want us to avoid.
5. We have been working on some alternate servers that you can also use. These will have largely the same code and functions. Initially the database of cases will be mostly the same between them but they do not share a database so cases developed on one will not automatically show up on another. Thank heavens for import/export and MVP.zip files.  

Our main demonstration server at https://demo.openlabyrinth.ca has now been up and running for a decade. And all through this time, we have been collecting a steady flow of activity metrics on how our materials are used. Every click, every response, even the thoughtful pauses, have all been recorded. So what does this tell us?

With over 25 million data points, we can sure say that this server is well used. Originally set up as a quick test bed to demonstrate what OpenLabyrinth v3 can do, it has been used by many for all sorts of purposes. We emphasize that this is a demo box with no guarantee of uptime but many authors and groups remain keen to use it. And it has been pretty reliable.

We just published a tech report on the various analytics that we have pulled from its databases. For those who are more technically inclined, feel free to peruse the report which delves into these numbers in more detail, along with some description of how we generated them.

Most of this data, we pulled directly from the SQL database that runs this server. But we have also used data pulled from our Learning Records Store, using xAPI statements.

This has given us a valuable source of orthogonal data to correlate with our SQL reports. This server has recorded nearly half a million user sessions and over 1.5 million clicks. There are  over 2000 maps and over 4000 registered users. And this is just for a single server.

We run 8 servers for PiHPES related services at UofC. Around the world, we are aware of many other installations of OpenLabyrinth but we do not track these. We know that users from around the world enjoy OpenLabyrinth. As noted a few years ago, we looked at the geolocations of some of them:

Now, let’s try to find those maps that we think have been played significantly i.e. 6 or more clicks. Here are the first 30 rows of MostPlayedCaseNamesView. The Cnt column shows how many clicks they have accumulated each. 

Each time a user clicks on a Link taking them to another Node, it is recorded, along with the state of Counters, the timestamp, user_id and whether this was part of a Course or Scenario. A single session can generate many thousands of rows in this table for cases like Medical Careers. But in crude terms it is an accurate representation of engagement or interaction with a case. The user cannot generate increased data or the appearance of activity by merely clicking randomly on the screen. Only valid links are recorded.

How much time is spent on a case? This is surprisingly difficult to answer across the board. While we can tell exactly how long each session was, there is such a wide variation, with lots of power law distributions, and it is not meaningful to use things like medium or maximum or standard deviation to describe them. We have lots of sessions where the user drops the case almost immediately — not the Droids they were looking for.

For a more complete look at how we analyzed this, check out our report at OLab3 case analytics tech report on Dataverse. We looked in more depth at cases which saw significant use, and used some approaches to filter out outlying data.

For an initial look at how much time might have been spent productively on the server, we estimate that comes to 10,500 hours of play time. This in itself is not inconsiderable and compares favorably with the activity metrics that we examined for our YouTube Clinisnips series: 100,000 hours: a decade on YouTube.

Which is the most popular case by amount of time spent on there? We found that to be Angus McWhindae, map 624, an interprofessional education case that was designed to be played in parallel by multiple small discussion groups. This case has accumulated more than 232 hours of learner interaction time. 

Let’s try the second one, map 24, Emil, one of our oldest palliative care cases and quite complex. This case has accumulated 151 hours for significant learner interaction. And Mildred ( a case on dizziness that we use in a large number of clinical teaching situations with individual learners), has 6200 sessions. This case has accumulated over 109 hours of interaction time. 

To give you an illustration of the power law curves that we see when looking at how long a user spends per session,

On the x-axis is duration in seconds; on the y-axis is the frequency count of sessions. 

For reference, 6000 seconds which we used for our cutoff, is 100 mins. This was the longest example of a credible, “truly played and engaged” session that we could find. This assessment of “engaged” was based on other metrics from that session, such as user_responses, consistent time spent per Node (no long interval while the user forgot about the case). 

Without filtering for very long or very short duration sessions, the curve is much more exaggerated but also less helpful.

What about ‘case completion’ metrics? As noted in the report, we found this quite difficult to assess. It does raise the higher issue in the minds of some educators as to whether the learner should be given credit for ‘completing’ the case. Indeed, this is foundational to SCORM and most other badge or credit-related systems that purport to assess ‘learning’.

If you have a resource that is essentially linear in style (such as most online course materials, which are just page-turners: a means to send information to the learner), then getting to the last page means that the learner ‘completed’ the lesson. Of course, this means very little educationally. 

If we accept the premise that the majority of our users are experienced learners and are discriminatory in the use of their time, they will not waste time on activities that do not contribute to their learning. There is very little frivolous play of these cases – that aspect is not like YouTube at all. So in relative terms for comparing cases with each other, or learners with each other, these simple metrics would appear to have value. Of course, this should be tested more thoroughly but at least this paradata gives us the tools to measure these factors. 

The purpose of this report was to illustrate, not just the extent to which OLab has been used, but also the wide variety of paradata that can be extracted from this usage. It does allow us to whittle out some weaker areas (of which there are many), and potentially learn from the stronger examples. 

It does provide a range of objective data that is orthogonal to our usual evaluation tools: happiness ratings and similar subjective surveys of learner opinions. The business world decided decades ago that customer satisfaction surveys tell you little but activity streams (who buys/does what and when) are a rich source of data that can be captured without threatening the woman-on-the-street with a clipboard. I think we are all a little tired of surveys.

We just created a tip for Teachers: Don’t Judge a Book by its Cover.

This is a short note about paying more attention to content than to making things pretty.

We are using a new format for OLab4 help files, using Gitbook. This note is built in that system. More on this shortly.

Thinking of using a chatbot to improve interactions with students?

Chatbots are getting better. Yes, some of the most powerful are now remarkably good. But what about the chatbot services available to the average educator?

Well, these are steadily improving and we have been testing them out for use with OLab. For more information on what we have found, check out this short article.

As you will see, they are proving to be very beneficial to large and medium-sized companies.But the resources needed to create a good scenario are still a bit beyond the average educator.

Spoiler Alert

Everyone hates spoilers. But sometimes they have a useful function. We recently came across a very neat way in which they had been used.

This reminded us of our Chats widgets in OLab3, a function that has not seen a lot of use. We created a map here that demonstrates how Chats can be used to selectively reveal spoilers and why you might want to consider them. Use ‘demo‘ to unlock the case.

Nice to have some good news in the current Covid outbreak: not a big item but it will help those who are trying to create content for online learning, which is much more needed now.

We have finally managed to fix an annoying bug which was preventing proper use of our CURIOS video mashup service. It is now fully functional again.

For those of you who may have missed it previously, here is more information on how to use the CURIOS video mashup tool.