Skeptical Science Online Course is a Stunning... 6% Success

So you guys might remember the Skeptical Science group put on an online course with the help of the University of Queensland for people to take titled “Making Sense of Climate Science Denial." If nothing else, you might remember the laughs we had over a nutjob there going off on widd rants because I disagreed with her, to the point she labeled the IPCC a right wing fringe group.

Or perhaps you'll remember how the course instructors didn't take issue with this, nor the user's repeated insults directed at me where she labeled me a sociopath and various other... offensive things. If you remember that, you'll likely remember how the course instructors instead threatened to ban me from their course because I dared to show people what was going on in their forum.

But whether your remember the course or not, I think we can all be interested in something I discovered today. The Skeptical Science group routinely promoted the number of people who enrolled in the course as a sign of how important the course was going to be. They made tons of remarks about 10,000+ people had already signed up, and things like that. They never really talked much about how many people actually completed the course though.

As it happens, I might know why. I recently found a report discussing things like the student activity for the course, and it says only 962 of the 16,861 enrolled students actually completed the course. That is, 94% of their students abandoned the course before it was finished.

Now I get online courses will necessarily have students who enroll but don't complete the course. That the number is 94% seems incredibly high though. Even amongst the 176 people who paid $100 for the course so they could get a special certification when they finished, only 105 completed the course. That is, a full 40% of the people who paid to take this course didn't even care enough about it to finish it.

I'd like to think if I charged $100 for a product, people purchasing it would actually take the steps to obtain that product. That wasn't the case here, for 40% of the people involved. Maybe they just wanted to donate some money to a cause they liked or something, but whatever the reason, they clearly didn't care that much about the course.

There are other interesting tidbits though. My favorite part of the report is this graphic:

2_25_MOOC_1

It shows nearly 4,000 people signed up for this course then never even looked at it. A full fifth of the students who enrolled in this course just didn't even think about it after they enrolled.

Another 9,000 people looked at the course once it started, then decided they weren't interested and didn't bother to follow it. That's over half the students who enrolled. Over half the students who enrolled in the course looked at it, decided they weren't interested and quit.

Then another ~3,000 students looked at some amount of material for half the course's chapters (that's the definition for "Only Explored") and decided they too weren't interested. They skipped out as well.

That left only 857 students who enrolled for free, and another 105 students who paid $100 for the course, who actually bothered to complete the course. I don't know how those numbers compare for other online courses (though seriously, who pays $100 for a certificate then doesn't bother to complete the course?), but to me, that's a resounding failure.

Here's another graphic which shows how uninteresting people found the course:

2_25_MOOC_2

You have to be mindful of how you interpret it given its logarithmic scale, but you can see thousands of peopple didn't even log into the course, and thousands more logged in all of once or twice. That shows people weren't all that interested.

Now, I'm not trying to say this course was a failure as far as online courses go. It may be common for only one in twenty or so students in online courses to actually complete the course. I don't know. What I do know is when one of the people behind the online course goes to the AGU with a presentationclaiming the course was successful and telling listeners how 78% of the enrollees accessed the course while 23% access at least half the course, he should have probably mentioned only 5.7% actually finished the course.

Because that's an interesting thing to be aware of. As impressive as 16,681 students may sound, it doesn't do anyone good to promote that number on its own. The most important number for any course is not how many people signed up for it. It's not even how many people bothered to attend a lecture or two.

The most important number for any course is how many people completed it. For the Skeptic Science course, that number was 962. I don't know how much time or money was put into the course, but personally, I don't think potentially reaching 962 was really worth it.

3 comments

  1. In most of academia the criteria for a courses success is the number of paying students. The revenue was $17,600.
    That would not have paid for the launch video, let alone
    - interviews with "75 scientific experts, including Sir David Attenborough, Katharine Hayhoe, Richard Alley, Michael Mann, and Naomi Oreskes."
    - Expenses for the committee of 13
    - Registering at the http://www.edx.org site
    - Funding for John Cook and others to produce the course
    Somebody is going to have made a huge loss. I hope it was not the Australian tax payers, as this would be a perversion of all what academia stands for.

  2. I'm pretty sure the University of Queensland footed a not-insignificant bill for this course. If I lived in Australia, I'd file an FOI regarding that. Especially since it'd be interesting to know if any of the participants in the interviews footed bills themselves or even got paid. If I had to bet, I'd say there were no payments save for costs, and it was all footed by the University of Queensland, but it'd be interesting to find out.

    Not living in Australia means I couldn't look into it very effectively though. Besides, I have a much more interesting topic to look at when it comes to Skeptical Science. I'm hoping to have a post up about it in a couple days, but it depends on how quickly I can work through something.

Leave a Reply

Your email address will not be published. Required fields are marked *