Saturday, March 29, 2014
Content and the instructor
In the previous post I wrote about the smooth operation of a course as though that is somehow separate from the pedagogy. But while many aspects of "operation" are administrative or technical, the principal factor that makes a course go well for students is clarity of instruction.
The students' experience can't be smooth if they don't understand what's being taught. Their comments on the forum and on this blog show that they've noticed my effort to make the lectures clear; my thanks to all those who have posted.
The course content was created last year and is remarkably solid, given that I was putting much of it together in real time while the course was live, after a day's work and family responsibilities, often starting at 1:30 a.m. when the neighborhood and its dogs were less likely to contribute to the audio.
The lectures were the easy part. I've thought about this material and the related pedagogy for decades, I have a great man's example to follow, and I've worked with thousands of students at Berkeley. I knew what I wanted to say, and Beamer and Camtasia did the rest. There's just a handful of errata, and I have to work out the least resource-intensive way of fixing them: it's easy to redo the videos but that also entails redoing the transcripts. We'll get it done for the next run.
I did a few new things for this year, such as creating "no pause" pdf's of the lecture slides: some of last year's pdf's ran to 30 pages per segment because they grew line by line as the videos do.
And I took a hard look at the exercises.
Every time I see a request to "please post" more exercises, and especially when people ask for entire exercise sets if they miss a deadline, I know I should explain what's involved in creating each question. Here goes.
- First, I pick the pedagogical point of the question.
- Then I pick a setting that can be used to make that point; the data have to satisfy the assumptions I'm asking students to make; preferably the setting should come from a real study and not just be made up; and it should be fairly easily understood by students all over the world.
- Then I write the question, mindful that a large proportion of students don't use English as their daily language, and that all students need to be able to answer in a way that satisfies the auto-grader.
- Then I write the solution.
- Then I decide on the number of allowed attempts, the score, and the acceptable range if the answer is numeric.
- Then all of this is entered into the EdX auto-grading system. I'm grateful to my co-workers who have done the majority of this step.
- Then the exercise is tested. That is, we try it out, entering answers that students might plausibly enter, and trying to make the system fail.
- And then, once we're satisfied, we repeat all of the above with the next question.
Before the start of this year's class, I also went through every Spring 2013 forum conversation about the exercises, editing the exercises as needed to reduce students' doubts about what the statements mean. This editing has been associated with a noticeable change in the forum: there are fewer confused posts than there were last year, and the questions are more frequently about how to solve an exercise than about what the exercise means.
That's where the instructor comes in.
No matter how good or field-tested the course materials are, students will have questions, and they need someone with experience to answer them. Students do answer each other's questions on the forum, and that's one of the main reasons the MOOC world can survive. But as course material increases in complexity, it's harder for students to help others; even the most confident students ask to have their answers confirmed by the instructor. Few students, and no machine, can guess what a student meant to ask if what they did ask seems garbled, or make a connection with something broader than what the student asked.
Students have noticed my regular presence on the forum. I think my being there helps, and we're soon going to test whether others running the forum will help just as much. But the course will not work if nobody on the faculty is active on the forum. At best it will be the equivalent of handing students a textbook and leaving them to it, and more likely it will descend into the kind of nastiness that seems to be the bane of online communities.
Few students, and no machine, can calmly insist that students keep the discourse civil so that the focus remains on subject matter and not on someone's rant-du-jour.
A course needs an instructor whether or not it's online. Whether EdX, Berkeley, and I can come up with an arrangement by which I can teach both on campus and online remains to be seen.
Like clockwork
Will people know what that means, in a few years? "Like clockwork"? The "work" inside a digital timepiece isn't as visible as the delicately moving gears of an old mantel clock.
But there I go, showing my age again.
Smooth operation is one of my two main goals for any course, along with clear and effective pedagogy. In a large class – even 100 students, though that's tiny on the MOOC scale – malfunctioning systems or sloppy course organization can be as obstructive to students as poor instruction. I often teach large classes on campus, and am so persnickety about how they are run that teaching assistants wonder about the deep-seated neuroses I must surely have. Apologies to them, but my aim is that students should be able to focus on subject matter with minimal time spent on course logistics.
With 28,000 students, my persnicketiness comes in very handy. The current run of 2.1X has been so smooth that it's strange to recall the squawking that sometimes drowned out all else in the forum last year. I've become so much better at running MOOCs.
Based on experience from Spring 2013, 2.1X schedules and deadlines were simplified and displayed more prominently, expectations were made clearer and more explicit, work during exam weeks was lightened (no graded exercise sets), and basic instructions were repeated in every possible place. EdX has greatly improved its platform: it rarely crashes, things work as expected, and this year there is more flexibility and ease of operation for instructors and course engineers.
I believe we've now got the structure of 2.1X to a point where it is genuinely feasible to make the logistics work for any future run with barely more than the flip of a switch.
This kind of thing freaks out the people who are worrying about "faculty being replaced" by online courses. I'm not so worried about that. The next post will be about things the machine can't do.
But there I go, showing my age again.
Smooth operation is one of my two main goals for any course, along with clear and effective pedagogy. In a large class – even 100 students, though that's tiny on the MOOC scale – malfunctioning systems or sloppy course organization can be as obstructive to students as poor instruction. I often teach large classes on campus, and am so persnickety about how they are run that teaching assistants wonder about the deep-seated neuroses I must surely have. Apologies to them, but my aim is that students should be able to focus on subject matter with minimal time spent on course logistics.
With 28,000 students, my persnicketiness comes in very handy. The current run of 2.1X has been so smooth that it's strange to recall the squawking that sometimes drowned out all else in the forum last year. I've become so much better at running MOOCs.
Based on experience from Spring 2013, 2.1X schedules and deadlines were simplified and displayed more prominently, expectations were made clearer and more explicit, work during exam weeks was lightened (no graded exercise sets), and basic instructions were repeated in every possible place. EdX has greatly improved its platform: it rarely crashes, things work as expected, and this year there is more flexibility and ease of operation for instructors and course engineers.
I believe we've now got the structure of 2.1X to a point where it is genuinely feasible to make the logistics work for any future run with barely more than the flip of a switch.
This kind of thing freaks out the people who are worrying about "faculty being replaced" by online courses. I'm not so worried about that. The next post will be about things the machine can't do.
Friday, March 14, 2014
To Mathbabe
Dear Cathy,
Colleagues alerted me to the fact that you have two questions for me. I'm happy to answer. Thank you for your generous comments about my teaching.
I'll start by pointing out that outrageous, especially in italics, was not my choice of word; it's yours.
To be clear about what I was objecting to, it was people asking me to be an instructor and then telling me I shouldn't feel any particular obligation to instruct. When I'm an instructor, I instruct. Go figure.
As my only way to connect with students in the MOOC setting is through the forum, I made a conscious decision to be present on the forum every day, and people liked the course. That's association, not causation. And you're right that my contact with tens of thousands of people must necessarily be limited to those who read or post to the discussion forums or read my course updates.
So how much do I matter, you ask. Well, that question has occurred to others including me, as you can probably imagine, and my colleagues and I are studying it as statisticians should: with a randomized controlled experiment, as I said at MSRI. Hang in there and we'll actually have some data to talk about.
It's worth noting that MOOC students get the same number of hours of me in lecture as do my campus students, but they get more hours of me outside lecture. On campus, many faculty hold two or three office hours a week regardless of the size of their class, and connect only with those who show up. My half an hour to an hour a day for the MOOC is not only more hours per week, it's also archived: every one of the 28,500 students can listen in on all my conversations with other students. I can't help thinking that many of my on-campus students might consider this a better value proposition than what they get.
I'm actually not guarding against my MOOC being "replayed forever". I've been saying for a year that I'd like the course to be open all the time. It's an introduction to a much-maligned and poorly understood field, which is one of the main reasons I created it.
And in fact it's been replayed ever since it went online: it's archived and freely available to anyone who cares to register. They just have to go to "Past Courses" in EdX and there it is. They'll get the course materials for all 15 weeks. What they won't get is an instructor.
That's what I'm guarding against: the conflation of instruction and the provision of course materials. The two are not the same.
So I've asked simply to be called the author rather than instructor of my MOOC, unless I'm actually instructing. This was a novel thought, apparently, but it's been well received. One of the next iterations of Stat 2X will likely be used to develop some "best practices" for how to hand off a MOOC to an instructor who's not the author. I expect the MOOC world will have authors of course materials and instructors of courses as routinely as traditional courses do.
Which brings me to your other question, about how authors or creators of MOOCs can protect their rights. This is not so different from the same question about authors of textbooks, say. You choose your publisher and your contract carefully; if you don't, you might have trouble, as their lawyers are almost always bigger than your lawyers. But we've been playing this game for a long, long time. Nothing much is new here.
I don't actually have a contract with EdX. EdX has agreements with BerkeleyX, and BerkeleyX has agreements with ... etc. It's been hair-raising occasionally, as anything involving such agreements tends to be, but by and large people have been great. I think it helps that the Director of EdX is a faculty member and himself a great teacher who created a terrific MOOC.
Those were your two questions, and now you have my answers, imperfect as they are. The questions were thoughtful; thank you for asking.
In closing, I have to tell you about one moment of hilarity for me, reading your post; I don't know if you intended it. It's when you said people like me "loved MOOCs because MOOCs were working for them." How I wish I had known that when I was up till 3:30 a.m. for eight months, creating a course and video text book in real time, after my day job, for free. I'd have felt so much better.
MOOCs have sparked conversations about teaching, and will motivate faculty to re-assess how they teach, whether in traditional courses or OCs (I just made that up: MOOC - MO = OC, becoming more common on campus) or MOOCs. That is good for everyone in the profession. But at the moment they're mostly a labor of love, with many creative and adventurous people donating their time to explore this new world. That isn't sustainable. And once money starts getting into the act, I expect it's going to feel a lot like the old world ... I wouldn't start ordering wreaths for math research just yet.
Best,
Ani
Colleagues alerted me to the fact that you have two questions for me. I'm happy to answer. Thank you for your generous comments about my teaching.
I'll start by pointing out that outrageous, especially in italics, was not my choice of word; it's yours.
To be clear about what I was objecting to, it was people asking me to be an instructor and then telling me I shouldn't feel any particular obligation to instruct. When I'm an instructor, I instruct. Go figure.
As my only way to connect with students in the MOOC setting is through the forum, I made a conscious decision to be present on the forum every day, and people liked the course. That's association, not causation. And you're right that my contact with tens of thousands of people must necessarily be limited to those who read or post to the discussion forums or read my course updates.
So how much do I matter, you ask. Well, that question has occurred to others including me, as you can probably imagine, and my colleagues and I are studying it as statisticians should: with a randomized controlled experiment, as I said at MSRI. Hang in there and we'll actually have some data to talk about.
It's worth noting that MOOC students get the same number of hours of me in lecture as do my campus students, but they get more hours of me outside lecture. On campus, many faculty hold two or three office hours a week regardless of the size of their class, and connect only with those who show up. My half an hour to an hour a day for the MOOC is not only more hours per week, it's also archived: every one of the 28,500 students can listen in on all my conversations with other students. I can't help thinking that many of my on-campus students might consider this a better value proposition than what they get.
I'm actually not guarding against my MOOC being "replayed forever". I've been saying for a year that I'd like the course to be open all the time. It's an introduction to a much-maligned and poorly understood field, which is one of the main reasons I created it.
And in fact it's been replayed ever since it went online: it's archived and freely available to anyone who cares to register. They just have to go to "Past Courses" in EdX and there it is. They'll get the course materials for all 15 weeks. What they won't get is an instructor.
That's what I'm guarding against: the conflation of instruction and the provision of course materials. The two are not the same.
So I've asked simply to be called the author rather than instructor of my MOOC, unless I'm actually instructing. This was a novel thought, apparently, but it's been well received. One of the next iterations of Stat 2X will likely be used to develop some "best practices" for how to hand off a MOOC to an instructor who's not the author. I expect the MOOC world will have authors of course materials and instructors of courses as routinely as traditional courses do.
Which brings me to your other question, about how authors or creators of MOOCs can protect their rights. This is not so different from the same question about authors of textbooks, say. You choose your publisher and your contract carefully; if you don't, you might have trouble, as their lawyers are almost always bigger than your lawyers. But we've been playing this game for a long, long time. Nothing much is new here.
I don't actually have a contract with EdX. EdX has agreements with BerkeleyX, and BerkeleyX has agreements with ... etc. It's been hair-raising occasionally, as anything involving such agreements tends to be, but by and large people have been great. I think it helps that the Director of EdX is a faculty member and himself a great teacher who created a terrific MOOC.
Those were your two questions, and now you have my answers, imperfect as they are. The questions were thoughtful; thank you for asking.
In closing, I have to tell you about one moment of hilarity for me, reading your post; I don't know if you intended it. It's when you said people like me "loved MOOCs because MOOCs were working for them." How I wish I had known that when I was up till 3:30 a.m. for eight months, creating a course and video text book in real time, after my day job, for free. I'd have felt so much better.
MOOCs have sparked conversations about teaching, and will motivate faculty to re-assess how they teach, whether in traditional courses or OCs (I just made that up: MOOC - MO = OC, becoming more common on campus) or MOOCs. That is good for everyone in the profession. But at the moment they're mostly a labor of love, with many creative and adventurous people donating their time to explore this new world. That isn't sustainable. And once money starts getting into the act, I expect it's going to feel a lot like the old world ... I wouldn't start ordering wreaths for math research just yet.
Best,
Ani
Monday, March 10, 2014
"Teaching" a MOOC
It's Week 3 and the course has settled down. Students are posting about content more than they are about logistics. As before, their fellow students are volunteering help: I'm enjoying the contributions of kame, SusanArizona, motley, dennniswadams, weyedide, BaoAng, and others whom I'll name as the weeks pass.
Once a course is well established – and in the infant world of MOOCs, "well established" seems to mean "ran once and went reasonably well" – there are calls for it to be re-run. This happened with Stat 2X last August, as soon as the first run ended. "We'd like to run your course 3 times a year," someone on campus said blithely, clearly not expecting the loud squawk of alarm that emanated from me.
What was my problem, he wanted to know. What was the big deal? All the materials were there; we'd just get a GSI (= Teaching Assistant, at places other than Berkeley) to run the forum; I'd just have to glance over things, oh, once every couple of weeks max.
Really? That's what it means to "teach" this course?
Not to this teacher.
Teaching is communication between teacher and student. It's not simply plopping down a set of course materials in front of the students and then stepping back: no classroom teacher would just hand students a textbook and take off.
Yet in the online world, "the course" has become conflated with what I think of as "course materials." I'm resisting this every step of the way.
Though I'm a veteran (on the "ran once and went reasonably well" scale) of MOOCs, it continues to be hard for me to have little connection with the people I'm teaching. Yes, I've created a good set of materials. Yes, many students are able to learn just by using the materials in Courseware (they'd have been fine if we'd just mailed them a textbook, probably). But it's the forum that make the class a "class", and me a teacher.
That's why I run my eye over the forum whenever I get a chance. I don't use a GSI – the forum is my only chance to get to know my students, even if it's a small self-selected group of them. I answer questions that others haven't got to yet; I fix errors inadvertently posted by students in their haste to be helpful; and I try to throw in examples that have helped my students on campus. I tend not to answer those who clearly haven't watched the lectures, and I don't answer those who are rude. But there are very, very few of the latter. The Stat 2.1X discussion forum is a delightfully civil community.
It doesn't take much time. Years of teaching the course on campus have made me a great predictor of what the questions will be and what students will suggest in response. I have a mental bank of different ways of answering the same question; most likely at least one of those ways will make sense to the student, and if I'm lucky, I'll have picked the right way the first time around based on what I was able to guess about the student from how they wrote their question.
But it does take some time, and I certainly can't do it year-round. I do have a full time job doing other things. So I'm working on ways to hand Stat 2X off to other instructors.
Will it make a difference to students? I hope not much. The author of a text isn't the instructor of every class that uses the text.
I'm going to savor every minute of this pass through Stat 2X. I might not be teaching it again for a while.
Once a course is well established – and in the infant world of MOOCs, "well established" seems to mean "ran once and went reasonably well" – there are calls for it to be re-run. This happened with Stat 2X last August, as soon as the first run ended. "We'd like to run your course 3 times a year," someone on campus said blithely, clearly not expecting the loud squawk of alarm that emanated from me.
What was my problem, he wanted to know. What was the big deal? All the materials were there; we'd just get a GSI (= Teaching Assistant, at places other than Berkeley) to run the forum; I'd just have to glance over things, oh, once every couple of weeks max.
Really? That's what it means to "teach" this course?
Not to this teacher.
Teaching is communication between teacher and student. It's not simply plopping down a set of course materials in front of the students and then stepping back: no classroom teacher would just hand students a textbook and take off.
Yet in the online world, "the course" has become conflated with what I think of as "course materials." I'm resisting this every step of the way.
Though I'm a veteran (on the "ran once and went reasonably well" scale) of MOOCs, it continues to be hard for me to have little connection with the people I'm teaching. Yes, I've created a good set of materials. Yes, many students are able to learn just by using the materials in Courseware (they'd have been fine if we'd just mailed them a textbook, probably). But it's the forum that make the class a "class", and me a teacher.
That's why I run my eye over the forum whenever I get a chance. I don't use a GSI – the forum is my only chance to get to know my students, even if it's a small self-selected group of them. I answer questions that others haven't got to yet; I fix errors inadvertently posted by students in their haste to be helpful; and I try to throw in examples that have helped my students on campus. I tend not to answer those who clearly haven't watched the lectures, and I don't answer those who are rude. But there are very, very few of the latter. The Stat 2.1X discussion forum is a delightfully civil community.
It doesn't take much time. Years of teaching the course on campus have made me a great predictor of what the questions will be and what students will suggest in response. I have a mental bank of different ways of answering the same question; most likely at least one of those ways will make sense to the student, and if I'm lucky, I'll have picked the right way the first time around based on what I was able to guess about the student from how they wrote their question.
But it does take some time, and I certainly can't do it year-round. I do have a full time job doing other things. So I'm working on ways to hand Stat 2X off to other instructors.
Will it make a difference to students? I hope not much. The author of a text isn't the instructor of every class that uses the text.
I'm going to savor every minute of this pass through Stat 2X. I might not be teaching it again for a while.
Subscribe to:
Posts (Atom)