Tuesday, April 2, 2013

Completion

Number of students enrolled on the last day of Stat 2.1X: 52,661
Active in the last week: 10,609
Earned certificates: 8,181

If you define "completion" as "earned certificate," the completion rate is 15.5%.

For comparison, here are data from a blog by Katy Jordan. She has some beautifully presented data complete with sources; it's a pleasure to read what she writes. Her data points, provided in February 2013, consist of 27 MOOCs, mostly from Coursera. Here's a stem-and-leaf plot of the completion rates in her table.

Stat 2.1X students will know how to read this once I've told them that the first line reads 0.7%. For others, here is an expanded version of the first three lines; you can take it from there. The entries are percents.

0.7
1.7
2.3  2.3  2.6  2.7
and so on

COMPLETION RATES OF 27 MOOCS [source: February 2013 data summary by Katy Jordan]

0 | 7
1 | 7
2 | 3367
3 | 25
4 | 5678
5 | 24
6 | 056
7 | 036
8 |
9 |
10 | 118
11 |
12 | 56
13 | 8
14 |
15 |
16 |
17 |
18 |
19 | 2

The median is 5.4 by the conventional "half-way point of the data" definition. 5% is often quoted as a "typical" MOOC completion rate.

Coursera's Scala course is at 19.2. It was taught by Martin Odersky who designed Scala.

We're not at 19.2, but even so, 15.5% is exceptionally high on the scale of MOOC completion rates.

I've been thinking about what it means to "complete" a course like Stat 2.1X. "Earned certificate" is a measure that sticks with the usual conventions of exams, grades, and so on. But it's possible for a student to go through all the lectures and try the exercises without regard to due dates and grades, as long as the course materials are available. That's a form of completion too, and a perfectly reasonable one for students who want to learn the subject but don't need or want a certificate.

They're hard to keep track of. But I'm willing to bet they're a big group.

To them - indeed, to all those who made a determined effort in Stat 2.1X, especially those who got the certificate: well done, and thank you for exploring this new world with me.











23 comments:

  1. Dear Prof. Ani,
    although you already have many thank you messages, I wish to send one myself. I'll make it short. Just to let you and Prof. Philip know that I learned much more than seeing statistics as a friendly and useful tool. I learned a lot about teaching. Not the edX, which by itself is a wonderful well developed source of information, but your own skills to get the information through. I'm very inspired in becoming a better teacher by watching you.
    Thank you and see you in 2.2x.

    ReplyDelete
  2. Thank you,
    your way of teaching is amazing,
    it's been a good time.

    ReplyDelete
  3. Dear Prof. Ani,

    Unfortunately, the statistics can be misused, when they compare apples and oranges. While some courses maintain high standards and require 80% to receive a completion certificate, other courses are more generous and need only 35% to earn certificate. Therefore, comparison between different courses are misleading and wrong.

    ReplyDelete
  4. The analysis provided simply sets out the rates. There is no claim about why the rates are as they are. The data source is careful to point out that there are many different definitions of "completion."

    The only "apples to apples" comparison is this: students sign on to a course under conditions that they understand; some percent finish, according to those conditions; those are the percents being listed.

    I believe the most significant difference between the underlying conditions is not the cutoff score. Some courses have text answers that are peer graded; others involve computations graded by machine. Some are self-paced, while others have weekly deadlines that have to be met consistently. Some teach mathematics to people with no math background; others teach computer science to people with a background in that field. There's a lot more that's different than just cutoff scores.

    And those scores aren't directly comparable either. Comparing x% to y% seems like an apples to apples comparison as x and y are both numbers, but as Stat 2.1X tries to teach, before comparing numbers it's a good idea to see what has gone into them. Many courses in the list had the multiple choice format; that's different from the students having to enter numerical answers. A key factor that hardly gets discussed at all is the number of attempts allowed for each answer. Many courses allow several attempts; some allow just one or two. Instructors have to balance all these factors, along with those listed in the para above, to come up with cutoffs. The connection between x > y and "higher standards" isn't straightforward.

    In any case, the cutoff score is a minimum, not a maximum. If I ignore all the differences above, and apply the most stringent cutoffs that I can find among MOOCs including the 80% in your example, the new "completion rates" for Stat 2.1X are still in the top quartile.

    So I think it's time to focus on what motivates a student to persevere with studies when there is no penalty, in the conventional sense, for dropping out. I'll write about this some day in the blog. Stay tuned.

    ReplyDelete
    Replies
    1. "So I think it's time to focus on what motivates a student to persevere with studies when there is no penalty, in the conventional sense, for dropping out. I'll write about this some day in the blog. Stay tuned."

      As a student who has dropped of many online (and even some non-online courses), but that was so motivated by your course that I got a 95% overall score in it, let me say that I could not agree more. Congratulations.
      In great expectation for next week.

      Delete
  5. Dear Prof.A,
    It was a great class in stat2.1x. It was so interesting that I had no choice but to complete them. It almost matched any interesting novel... that you cant keep it down until its done! I am eagerly waiting for stat2.2x. I would be even more happy if more mathematical concepts can be added as pdfs,etc.

    ReplyDelete
  6. Prof A.

    You completely rock! Thank you for keeping up your enthusiasm for this form of learning and not letting anything bring you down. I am actually disappointed that so many people did -not- complete the course, particularly as the bar for completion was not overly demanding. However, I guess by MOCC standards, we had a fairly high completion rate and I am sure the everyone that participated to whatever extent learned something.

    Thank you again. I am half scared, half excited, but definitely looking forward to 2.2x. I am interested to see how many people will complete all three phases. The data analysis possibilities are endless. How fun!

    ReplyDelete
  7. Even the way you answer these questions is interesting. I understand and concur with each of your points. Many things to consider.

    Since you mentioned it, I greatly enjoyed having 2 opportunities to answer some of the questions correctly (those that deserved two chances). Sometimes, one is somewhat certain of the answer, but not certain enough to take a chance. It is a great time saver to allow you to make an attempt to answer as you see fit and if it is marked as incorrect, take the time to back up and try a different approach.... so thank you.

    I also -greatly- -greatly- enjoyed being able to see the correct answers on the quizes -immediately- following its completion. Immediately correction, reinforcement and/or use is the hallmark of adult education. That one thing made the 2.1x course so useful. Please continue that.

    Lastly, I searched online all I could and could not find what the "T" in the the Student's T-Test stands for, though I read all about William Sealy Gossett and his assumed pen name of "Student," which in itself was interesting.

    Does the "T" actually stand for "Tyche" the Greek Goddess of fortune and chance as you begin to talk about in your closing lecture? Thanks.

    ReplyDelete
  8. My guess: if it were possible to apply statistics here, I'm quite sure that course percentages would be in a beautiful linear relation with its Profs ;-)

    I agree with you in this finer observation:

    "Earned certificate" is a measure that sticks with the usual conventions of exams, grades, and so on. But it's possible for a student to go through all the lectures and try the exercises without regard to due dates and grades, as long as the course materials are available. That's a form of completion too, and a perfectly reasonable one for students who want to learn the subject but don't need or want a certificate.

    Impossible to keep track of this big minority (or anonymous majority, who knows).

    Best!

    ReplyDelete
  9. I propose that Prof Ani be cloned and sent all over the world to teach statistics- seriously!

    I have a fairly busy schedule but I was enjoying the classes so much that I had the course top on my priority list. I am impatiently waiting for Stat 2.2 and 2.3!

    ReplyDelete
  10. Hi prof A. And all and thanks once again for a great course!
    Let me make a guess .... I have a feeling that Stat2.2x will "beat" the 19.2 completion rate of Coursera's Scala course :-)
    See you all at Stat2.2x.

    Theodora

    ReplyDelete
    Replies
    1. I wish ... but I'm not counting on it. 2.2X is unusual in that it's harder for many students in the first half of the course than in the second half, so some students get discouraged early. I'll try to explain this at the start of the course. If students hang in there, they will find that in the second half they will be able to do some very interesting calculations rather easily.

      I'm looking forward to getting started. I enjoyed my 2.1X students and will be happy to be back working with them and their new classmates. Less than a week now ...

      Delete
  11. Dearest Prof A.
    It has been exceptional being your student, and for most of us, coming from developing countries like Pakistan, it is as real as it can get in terms of getting good quality education.

    I just love the way you play with data and keep everybody honest by not telling everything

    Prayers always

    Nasir

    ReplyDelete
  12. Dear Professor,

    I add my gratitude note to your long list. After the first lecture, I wrote in my facebook account that my professor has great style of talking. After the 2nd week lecture where got to see a video recording, I wrote on facebook that my professor is also extremely charming. At the end of the course, I am in daze as you are just amazing. If I had found you when I was in my 20s and learnt Statistics, I could have possibly given much more to the corporate world. However, I am not doing too badly at 44 and I will use every ounce of my learning from the course in the Big Data project I am involved in. I will come back for the further programs in Statistics that you have promised.

    Once again "Thank You" and pray to the Almighty for keeping people like you in the best of health and heart always.

    Regards,
    Partha
    http://www.parthamajumdar.org

    ReplyDelete
  13. Indeed it is surprising to see such a high drop out rate despite a "free course".I also noted that while some found the course simple others found it very tough.Motivation indeed is a very curious thing with some people learning for learnings sake while some do it jsut for a certificate.

    ReplyDelete
  14. Dear Prof. Anni,

    I would like to express my gratitude and appreciation for the times & efforts you have dedicated to bring Stat 2.1x class to us the way it was.

    Thank you so much indeed,

    Sincerely,
    Rabie

    ReplyDelete
  15. Dear Prof. A.,

    I took the course from my notebook here in Argentina and earned a certificate. But, much more important than that, I really got to understand all the subjects taught in the course. Thanks a lot for your generosity (which, I think, is one of the most important things for being a teacher)! I do think you are a great teacher with that particular talent for making everything more "understandable" (don't know if that word exists in English) than it would regularly be.
    This kind of courses are very important, not only for learning, but also because the certificates might be worthy for our CV's.
    See you over a week in Stat 2.2!
    Again, thanks a lot for taking the time and effort for making this kind of things possible.
    Best regardas!

    ReplyDelete
  16. Dear Prof A,
    Thanks again for offering this course. While the stats knowledge will help me in my daily work as a database engineer, what I really enjoyed about this class was the feeling of being in college again. It's been 22 years since I last felt the thrill of learning a new topic, the weekly rhythms of class, reading, and homework, and the crunch time preparing for finals. Alas, much of my college life was spent partying, procrastinating, and stressing about my poor academic results. I am able to redeem myself somewhat now, by achieving 100% in the first of the Stats 2.1 modules, and for that opportunity I thank you!!

    ReplyDelete
  17. Dear Prof. A,

    Thank you so much for a brilliant course and making such an effort.
    Unfortunately I wasn't able to take the final exam as I was traveling without internet access those days, but with a total score of 68% (without taking the exam) I thought I would still receive the certificate. Was I wrong in that assumption?
    I am looking forward to join Stat. 2.2

    Kind regards,
    Stella

    ReplyDelete
    Replies
    1. You weren't wrong at all. It should have been on the dashboard when you logged in. Please email EdX if you can't get at it. Best,

      Prof. A.

      Delete
  18. Ani-

    Thank you Thank you. I was greatly impressed with your skills to make this material accessible, learn-able and fun. I fit into the active but non-completion category as I was away from the "class room" for about two weeks. I met my goal of getting a refresher in basic statistical concepts. I really enjoyed the practice problems and the material in Philip's online text. You two make a great team for my learning style.

    Thank you again for helping me and several thousands of others dip our feet into the statistical ocean and feel the splash of the surf.

    Tom

    ReplyDelete
  19. Dear Prof. Ani,

    I worked trough the course stat 2.1x (here in The Netherlands) with a lot of joy doing the excersizes. Unfortunately I've signed up rather late, so I wasn't able to get a certificate. Will stat 2.1x enroll again (??), I do hope so very much.

    I wanted to complement you for making the subject so easy to understand. Especially for somebody who hase dyscalculia: that'll be me :) You gave me hope that I can complete statistics and maybe do a Master (Psychology) on the UNI here in The Netherlands when I've finished my curent study. So thank you for making me more confident when it comes to numbers, math and statistics, even with a handicap like dyscalculia.

    I'm looking forward to stat 2.2x!

    Best regards,

    ReplyDelete