TOPcast Episode 16: Quality Episode Part 2

KELVIN		From the University of Central Florida’s Center for Distributed Learning, I’m Kelvin Thompson.	
			
TOM		And I’m Tom Cavanagh.	
			
KELVIN		And you’re listening to TOPcast: the Teaching Online Podcast. Hi Tom.	
			
TOM		Hey Kelvin. Here we are again.	
			
KELVIN		Here we are again.	
			
TOM		In the box. Recording.	
			
KELVIN		Will they let us out if we get it right this time??	
			
TOM		(laughter) Let’s hope.	
			
KELVIN		Yeah, there you go.	
			
TOM		At least throw some food under the door.	
			
KELVIN		Some peanuts. I don’t know. Something. Yes.	
			
TOM		So!	
			
KELVIN		Yes?	
			
TOM		Let’s waste no time.	
			
KELVIN		No time to waste.	
			
TOM		Let’s get right down to it.	
			
KELVIN		Down to it.	
			
TOM		What is in the thermos?	
			
KELVIN		Well, you gotta have your priorities, right?	
			
TOM		That’s right.	
			
KELVIN		No thermos today.	
			
TOM		Then what’s in the pockets?	
			
KELVIN		(laughter) Yes. I should get some pants like that, right? With a straw and some non-leak pockets because when you pour the coffee straight in there, it just is a mess, and people look at you funny wherever you go.	
			
TOM		I’m going to make no comment.	
			
KELVIN		(laughter) You were wondering before, but that’s my disclaimer. Now, well, let me give you this preamble. As you know, there are a lot of variables involved in an excellent cup of coffee, and hopefully, I usually bring an excellent cup of coffee.	
			
TOM		Indeed.	
			
KELVIN		Yeah, toward the end of the process is the roasting and the brewing, and I usually take care of the brewing part and get some roasting outsourced by a local roaster, but it’s kind of rare for the same person who roasted the coffee beans to be the same person who brews your coffee and pours it in your cup, but it does happen. Usually, when you’re in that kind of situation, we’re talking about some kind of attention to detail that produces really high quality results. Hard to figure out exactly where all the specifics are there, but when you got that all going on, good stuff usually happens. Now, today’s coffee is from that kind of place.	
			
TOM		That kind of place?	
			
KELVIN		That kind of place.	
			
TOM		So I have a cup in front of me.	
			
KELVIN		Yes, indeed. What’s it say on it?	
			
TOM		Um…	
			
KELVIN		“Tom’s cup”. (laughter)	
			
TOM		My hand was over the—(laughter) Lineage Coffee Roasting.	
			
KELVIN		That’s right! Lineage Coffee Roasting here in Orlando, where they have a bar—literally, a countertop—at an artisan market. And so this was roasted and brewed by the same folks. The particular coffee you’re drinking is a Colombia Santa Cruz. How do you like it?	
			
TOM		It’s good.	
			
KELVIN		Yeah. I like it, too. So, this roasting and brewing example seem to me at least analogous to the design plus teaching issue or maybe the design versus teaching issue within today’s topic of determining quality in online courses.	
			
TOM		Wow. What are the odds that you had thematically selected coffee?	
			
KELVIN		Oh, I keep something in my pocket just in case of emergency.	
			
TOM		(laughter) Yeah. No, it’s good. I like it, and I get how it relates to the conversation of quality. Now, this isn’t the first time that we’ve talked about quality.	
			
KELVIN		No, indeed. We did this with blended learning elsewhere in our podcast feed. We had blended learning, then we had blended learning part deux.	
			
TOM		Yes.	
			
KELVIN		Yeah. So I looked up some possible alternative titles for this episode.	
			
TOM		Did you?	
			
KELVIN		Yes.	
			
TOM		Do tell.	
			
KELVIN		So, being inspired from some of the most interestingly named sequels, we can call this episode Quality Too. T-O-O. Like Teen Wolf Too. Quality: The New Batch. From Gremlins 2: The New Batch. Quality: The Second One, which is inspired by Sharknado. (laughter) Or, that one might be the best, but my favorite so far is Honey, I Blew Up the Quality.	
			
TOM		Okay.	
			
KELVIN		For, like, Honey, I Blew Up the Kids.	
			
TOM		Oh, I got it.	
			
KELVIN		(laughter) So we’ll see which one—if any of those—make it into the—	
			
TOM		I am just old enough to get all of your references.	
			
KELVIN		What are the odds?	
			
TOM		Okay.	
			
KELVIN		Yes. So, yes. Not the first time. So in the earlier episode, we explored some of the broad issues on market—	
			
TOM		Magazine Street.	
			
KELVIN		Magazine Street.	
			
TOM		In New Orleans, we were at the OLC Innovate Conference.	
			
KELVIN		That’s right.	
			
TOM		And that was a bit of an impromptu podcast session, so we kind of—	
			
KELVIN		In our defense… (laughter)	
			
TOM		In our defense, yes. You know, little disclaimer: we had a bit of a free ranging conversation about quality.	
			
KELVIN		And that’s not always bad. The coffee was certainly good.	
			
TOM		It was. Probably better than the podcast. (laughter)	
			
KELVIN		If you haven’t listened to that one yet, you might want to give it a listen. I do think in thinking about that, I think we did tackle from a really high level some big issues in quality. So, give it a listen if you haven’t listened to it. But, we didn’t, though, get into the details of how we might operationalize quality markers for high quality online course design and high quality online teaching, so hey! How about we do that?	
			
TOM		Let’s do that. And, in fact, that might actually be more useful for people. And if you’ve listened to the podcast before, you may recall that we’ve discussed the concept of the iron triangle, and that’s quality, cost, and access, and you can’t necessarily impact any one without impacting the others, yet we feel that through creative application of innovative educational technology, you can potentially impact all three at the same time positively. And, we set that up as kind of a framework for what we would try to talk about this year occasionally—revisiting different elements of that iron triangle—and so it’s almost like you can’t talk about quality too much, you know?	
			
KELVIN		Right!	
			
TOM		It is often times, in online learning, the elephant in the room, where people talk about, “Yeah, you talk about convenience, and you talk about all this other stuff, but, you know, is it as good as…?” and all of that. When you talk about operationalizing the application of quality standards in an online course, I think that’s a way to answer that question, that elephant in the room of, “Well, here’s how we can—”	
			
KELVIN		(elephant noise) That was for the elephant.	
			
TOM		For the record, that was not an app.	
			
KELVIN		(laughter)	
			
TOM		That was Kelvin. It’s a way for us to say, “Look, this thing we’re doing is quality.” And neutralize objections and perhaps—	
			
KELVIN		And hopefully high quality.	
			
TOM		Yeah! Maybe in some cases—I am certainly an advocate of this—online learning can be higher quality than what happens in the classroom in many, many cases when it’s done right.	
			
KELVIN		Yeah. I think that’s right. You’ve alluded to rubrics or checklists, and I don’t know. As someone who once upon a time wrote a dissertation on a model to not have reductionist rubric-style evaluations of online course quality, I’ll tell you that as much as quality can’t be reduced to checklists or rubrics, people really want checklists and rubrics, and they really are helpful operationally. Just to move forward, you know. You can have these big conversations, but it’s like, well, what are you really concretely talking about? That’s where the rubber hits the road for some people.	
			
TOM		Sure, and people want to know—I think—how am I doing? Am I doing this right? Am I doing this well? How can I tell in comparing your course design against some sort of objective standard as a way to measure how you’re doing?	
			
KELVIN		Yeah, and I guess let’s give some shout outs. There are some really popular kinds of rubrics and tools that are in this range. What would you say is the number one recognized name in this area?	
			
TOM		I got to give my friend Deb props here. Quality Matters is the big gorilla and rightfully so. They do a fantastic job. I mean, they’ve got a rubric, but it’s also a process. People can subscribe to that process, and there are a number of new initiatives that Quality Matters is working on—including looking at MOOCs and some other things—but definitely that is always the first one off the tongue when people are talking about online course rubrics.	
			
KELVIN		Yes. Now, correct me if I’m wrong, money has to change hands to be a part of that. It’s a subscription—kind of a membership—kind of a thing. Can I share a pet peeve? It seems like lately I’ve been sharing a lot of pet peeves.	
			
TOM		What if I said no?	
			
KELVIN		I’d probably share it anyway. You know, so, might as well go with it. I’m amazed by the number of people who say things like, “Oh, well, we don’t use Quality Matters—we’re not a member—but we kind of use Quality Matters in our rubric here at our institution.” Now, what they really mean is they took those headings—those big categories—out of the rubric and are using that. They’re not really using the rubric, and it drives me a little bit batty.	
 			
TOM		The rubric has been out.	
			
KELVIN		Yes, at least a prior version, I think.	
			
TOM		Right.	
			
KELVIN		But they’re still careful to, you know—This is for like illustrative purposes. Don’t be thinking that you’re doing Quality Matters if you just kind of…you know? And that’s in fairness to them because that’s a robust process.	
			
TOM		And we have to be careful because we don’t want to get into—It’s a whole separate subject of faculty development models, and Quality Matters is certainly something that can be used in a faculty development process. So, for example, I’ve been asked by other institutions who are sort of asking us about how we do things here at UCF, “Well, how many courses do you approve for online learning every year?” Our model’s not like that. We’re a production model that’s large-scale, and it’s kind of decentralized with some centralized controls and support, but our courses do not go through some sort of “Every course must go through a Quality Matters approval process”, so that model doesn’t really fit here. But in some cases, it might.	
			
KELVIN		Yeah.	
			
TOM		And if you want that kind of vetting process, maybe you don’t have the kind of instructional design and production infrastructure that we have, then that’s a great way to ensure quality: to have every course reviewed by a peer committee against a standard.	
			
KELVIN		I’d like to say that we’re front-end heavy in that sense. We put all that investment in predesign and the design process, whereas a lot of institutions, they’ll use things kind of a rubric in an institutional way. I don’t know if I’d really call it the backend, but it’s down the road a little bit.	
			
TOM		Sure. It’s formative versus summative. And we’re more on the formative side, but we also have, you know, extensive buying and partnerships with the departments and the faculty themselves who all own the course.	
			
KELVIN		And not everybody has that.	
			
TOM		Correct. They’re all different models, so before we go any more down the cul-de-sac…	
			
KELVIN		No, agreed. Agreed. Would you agree with this? I certainly hear people saying this. I’m still trying to decide if I agree with this, but would you agree that in some respects, broadly speaking, [a] course quality rubric is a course quality rubric is a course quality rubric?	
			
TOM		Broadly speaking? Yeah. You know, the ones that I’ve seen that I really like—So Quality Matters is obviously one I like which I admire. I really liked the—it’s been a while since I’ve seen anything recently because we haven’t been working with Blackboard lately—but the old Blackboard Greenhouse rubric? I was a big fan of—I thought it did a really nice job. Chico State. I know that we’ve referred to some of their materials in the past.	
			
KELVIN		Cal State, Chico. Right.	
			
TOM		Right. Sorry.	
			
KELVIN		Usually it’s me.	
			
TOM		Cal State, Chico. And then there are others, as well.	
			
KELVIN		Yeah. SUNY’s come up with some stuff. OSCQR. Open SUNY. All kinds of good stuff. But maybe it would be helpful to look at some of the things that are typically addressed in those rubrics. And maybe just kind of riff on those a minute talk about what the nature of good quality is?	
			
TOM		Yeah, I think that’s worthwhile. Most of these rubrics tend to be…They tend to privilege design elements over delivery elements.	
			
KELVIN		Yup. Why is that?	
			
TOM		Well, it’s easier to observe. It’s the same thing as with a face-to-face class. Unless you’re sitting there watching somebody teach, you don’t know how well they’re delivering instruction.	
			
KELVIN		But interestingly, we don’t seem to privilege design face-to-face in the same way that we do online. And I think it’s weird, right? Because you could go observe face-to-face and that’s just a snapshot at a moment in time. You could do that online as well, though it’s a little more difficult, but there’s so much more material that one could review in advance. Face-to-face? I mean, you’ve got a syllabus or whatever, but it’s maybe the rare instructor who has, like, everything sort of in print. Kind of like doing an off-site review for your accrediting group.	
			
TOM		Show me evidence.	
			
KELVIN		That’s right. Having just recently gone through that.	
			
TOM		Well, and there’s also—A lot of faculty really enjoy the performance aspect of face-to-face instruction, and that’s much more improvisational than what you do online.	
			
KELVIN		My whole life’s an improvisation. You know?	
			
TOM		All right. So you want to lay out some of these categories?	
			
KELVIN		Sure. And here are—These are kind of what we’re doing, Tom, for everybody else’s benefit. We’re doing some work here in the state of Florida to try to get at maybe a state-wide set of principles, standards, markers, whatever, and then in that case we have decided to address both design and what some call delivery. Some people take issue with that term design and delivery. Teaching? How about we use that? There’s that term. So here are some…and I think while some specific details could—in different rubrics—could move from one heading to another, I think you get the gist of it when you kind of look at these headings. How about I just run them down and then we can kind of talk about anything we want to with those, right? So, course overview and content. I mean, kind of having clarity, right? Having organization and providing that kind of big picture and having meaningful content, that’s a thing. Interaction design. Designing for interaction, which is different than actually having the interaction. Course technology. Making that real apparent, clear for the students why they’re doing what they’re doing. Having help with it if they need it. Usability and accessibility. Is it—Can you find your way around in it? If you have any kind of need for accommodations, you know, does that cause an issue? Instructional design. We’ll call that all of the…your objectives aligned with your assessments aligned with your activities aligned with your content and you know, chunked and all that. And then on the teaching part—the delivery part—engaged teaching and effective course implementation. Those are some of the headings that our Florida group have used. The heading types change from rubric to rubric but that kind of stuff. [Do] you think that’s consistent with what you see elsewhere?	
			
TOM		Yeah, I think so. And I don’t think anybody would argue with those broad categories, especially people who’ve been doing this a while. The problem, I think, is in the details.	
			
KELVIN		Yeah.	
			
TOM		Especially when you have a committee—a state-wide committee—and of course, this cake is still baking. This isn’t done yet. But it has the danger of having people say, “Yeah, but you need to have this requirement. You need to have four interactions every course, or you need to have this proctored test every course,” or whatever it is. And if you don’t have that, then it can’t be quality. So, it’s like, well, some of that is opinion, and some of that is based upon the pedagogical style of the faculty member or the particular objectives of that course.	
			
KELVIN		Some nuances at that institution?	
			
TOM		Exactly! Culture at that institution, yeah, or context. And it’s hard sometimes for people to step back and have a rubric that’s more broadly applicable, and I’m not saying that we’re going to end up with that, but I think it has the danger of becoming too detailed.	
			
KELVIN		I agree with that. In fact, that’s been our conversation with our state-wide group, and something I’ve said a couple times—and some others have agreed—hopefully this is where we land. You said, it’s still a work in progress—still baking—but, what I’ve advocated for is, at a larger group like that, fewer things, more broad, and then your institutional, more detailed, content-specific stuff can find a home. It’s like little pigeons coming in to land in their own little nesting place. And I hope that’s where we end up. But you’re right. The process? I’ve been a part of several of these kinds of things and the process is always way nitty-gritty before it zooms out.	
			
TOM		I know. It’s the old kitchen sink conundrum.	
			
KELVIN		It really is. It’ll drive you batty before you’re done, for sure.	
			
TOM		But I think as a way to kind of broadly frame these things, I think it makes a lot of sense.	
			
KELVIN		Yeah, I think so. Now, as we’re recording this, we just recently had a little discussion/focus session with some faculty here at UCF, and we kind of posed some questions about quality. Now, we haven’t really—Actually, you and I haven’t really debriefed on this too much.	
			
TOM		Literally, like day before yesterday.	
			
KELVIN		What were your observations of this? Because to me, that conversation went a little bit different direction than some of these rubrics things typically go.	
			
TOM		Yeah, I thought it was a really good conversation, and unfortunately, I don’t think we had enough time to really explore this, and we had great participation. We filled the room with faculty who were interested in this question, and you know, these are all faculty who are experienced online instructors, so these people are really engaged. They know what they’re doing. And they have really informed opinions, so I found it a valuable discussion, but I made a few notes about just some broad themes. Now, they used an awful lot of specific examples to illustrate these themes, but I think they would all kind of agree with these themes. So the first one was that for quality, a course must have structure and consistency. And this includes both things like: try to have all your assignments due in a consistent way. Like on a certain day or a certain time or something, because if it’s random, it’s hard for students—particularly students who take courses online who might be working or whatever—to manage their time. As well as though, from a content perspective, structure from a scaffolding perspective, you know. Having content build upon each other to see you don’t feel like it’s just a random collection of content modules. There’s a design behind this, and they seem to really feel like the students valued a coherent structure for the course. Both mechanically as well as from a content standpoint. Speaking of mechanics—and I think this is obvious but it’s probably not a bad thing to say—the course must have clear navigation and usability. The very last thing you want is to have students enroll in an online class and not know what to do next.	
			
KELVIN		Yeah, right?	
			
TOM		I remember having conversations at other places where faculty had kind of a lot of autonomy to design the course the way they wanted. They could kind of change the location of buttons. They could change the interface colors and do all kinds of things.	
			
KELVIN		Could they make things blink?	
			
TOM		They could make things blink.	
			
KELVIN		(laughter)	
			
TOM		They could put red text on a purple background and all kinds of stuff, and there was some of that here, and there was some of that at my previous institution,, and I’ve heard stories from other colleagues from around the country, and you know, you don’t want to tie faculty’s hands and not let them express their knowledge and creativity, and you want to find a good balance, but if you have a course where a student is spending their intellectual energy trying to figure out what to click instead of learning the content you want, that’s not quality. So that’s navigation and usability, and usability also includes things like universal design. Instructor presence. This is more of your delivery/teaching. If students are emailing a faculty member, and it takes them a week to get back to them, students hate that. And that’s a big complaint.	
			
KELVIN		That’s not a normal turnaround time? (laughter)	
			
TOM		No, you know. You hear stories.	
			
KELVIN		You do.	
			
TOM		And so…	
			
KELVIN		They weren’t my students, were they?	
			
TOM		No. They’re not your students.	
			
KELVIN		Good.	
			
TOM		You would know. So, instructor presence. Being present for the students, and all of the people that were in the room the other day are the people that are getting back to students within like, you know, hours. They really cared. Being able to create a sense of community online. That was a theme that came up, and whether that’s facilitated by the instructor through something like a synchronous session of office hours or test review or whatever—bringing in a guest speaker—or it’s something that they enable students to do with themselves through some sort of a community-building mechanism within the course. Having a community online seems to be really important. And then the last theme I wrote down—and again, this was a brief conversation. I’m sure there would be more if we had more time—was not to waste the students’ time. No busy work. Students don’t mind spending time in a course if they feel like it’s relevant and it’s going to help them. That seems to be reinforced by a lot of the faculty. And I’ll just say from my own personal experience teaching many courses online, every single one of these I completely agree with. It’s been consistent with my own personal experience.	
			
KELVIN		Now, some of the things that I think faculty in that group said that I think is outside the—Because those things are all consistent with the kind of rubric-y kind of stuff that we’ve talked about. Is that a word? Rubric-y?	
			
TOM		Rubric-y.	
			
KELVIN		But some of the things that are out of the bound of the rubric-y kind of stuff are—Some of the faculty brought up class size is an issue.	
			
TOM		Sure.	
			
KELVIN		You know, design and class size matching. If you’ve got this kind of thing that was designed for a smaller class size, and you got a bigger class size and (groan). It doesn’t work.	
			
TOM		Yeah, designing your assessments. If you’ve got you know ten students versus a hundred students, it’s a very different kind of assessment strategy that you can explore.	
			
KELVIN		I was impressed, I think, by the number of faculty in the room who said various things that seemed to boil down to quality really kind of boils down to “Did the students learn stuff?” [It] was kind of really what they said. You know, I heard things like, “students achieving the learning outcomes” and things like that, but I thought, “Yeah, that’s probably right.” (laughter)	
			
TOM		We had a very spicy conversation—and it’d probably still be going if we could—about this concept of should online courses strive to meet minimum standards or excellence.	
			
KELVIN		Yeah. What do you think about that?	
			
TOM		Well, there was a lot of debate about, “What the heck do you mean by minimum standards?” and it was a little bit of, “How dare you.”	
			
KELVIN		I know.	
			
TOM		Which I was pleased, because the faculty in the room were like, “We’re about excellence, not minimum standards, so you better define what you mean by that.”	
			
KELVIN		And I’ll tell you what—	
			
TOM		It was a provocative question on purpose.	
			
KELVIN		I’m curious what you would say that that frames as, but I’ll tell you, looking down at a draft of a rubric right now, because it’s not a rubric. What we’ve got in front of—at least I’ve got in front of me, I think you’ve got it in front of you, too—I’ve got like a list of standards, and right now, there’s not a rubric that goes with that. There’s kind of like—I guess the implication is it’s binary, right? It’s either yes, this exists, or no, this doesn’t. And in that sense, I think that leads one down an expectation of minimum. Like okay. Check? Good. Not check? Bad?	
			
TOM		Well, yeah, you’re kind of making my point about the kitchen sink. The more granular you get and the more detailed sort of, “You must have this in your course,” if you don’t have it, does it mean the course is bad?	
			
KELVIN		Right.	
			
TOM		Not necessarily. It may be that that’s not appropriate for this kind of course. So, yeah, I fear that if you get below a certain threshold that it becomes much less useful.	
			
KELVIN		That’s right. Whereas a more robust rubric, you know, you’ve got kind of a high-end watermark, and you’ve got kind of lower-end, and maybe you even have an out for not applicable. I think that’s where a reviewer’s expertise really comes in. This is where my background in connoisseurship and so forth shows up, because so much of it is “Quality’s in the eye of the beholder,” and if you’re knowledgeable, then you can say some things like, “Oh! This was not even appropriate. You know, that’s not relevant for you.” And so we can go talk about this.	
			
TOM		But you know, I do think that there is—as much as the faculty legitimately push back on this concept of minimum standards versus excellence—there probably is something to minimum standards.	
			
KELVIN		I think so.	
			
TOM		Now, we would hope that all faculty would strive to exceed minimum standards, but as at least a starting point—especially for faculty who might be brand new to online teaching—it’s like, “Well, here’s what we kind of expect all courses to have: these elements.” And now let’s talk about, “How do we go beyond that?”	
			
KELVIN		So, with that in mind, you want to maybe say a word about what we’re kicking around here? This quick check idea?	
			
TOM		Sure. I’ll give credit to Bill Phillips, who we ran down in New Orleans about his chili.	
			
KELVIN		He’ll be looking for his commission from this episode. (laughter)	
			
TOM		Dr. Bill had, I thought, a terrific idea. He was calling it the Jiffy Lube, you know…all trademarks, yes, so we’re not calling it Jiffy Lube. But the quick check. The course quick check. The idea is that, you know, you take your online course, and you drive it into the shop here, and you put it up on the lift, and the instructional designer sits next to you.	
			
KELVIN		Ten-point inspection.	
			
TOM		Ten-point inspection thing, whatever [it] might be, and you go through the course, and say, “Well gosh! Yeah, looks like you got some pictures here without alt tags. So let’s mark that down, and looks like you’ve got some links that have gone bad over time.”	
			
KELVIN		Woops! Tires are worn!	
			
TOM		Yup, your tires are worn. Yeah, let me show you air filter, and so you can go through and—literally, literally a thirty-minute check of your course—and come out with a set of recommendations. 	
			
KELVIN		Which is far faster than any time I’ve done any kind of a more of elaborate course review. Even going fast, I think it’s taken me usually three hours to go through a fully online course.	
			
TOM		Yeah.	
			
KELVIN		So thirty minutes is pretty good.	
			
TOM		We’ve also got some pretty awesome technology tools that the team here has built like our UDOIT accessibility checker and some other things. That can really facilitate that kind of a speedy check-up.	
			
KELVIN		Yes.	
			
TOM		And so the faculty member would walk away with a report of, “Here’s some stuff that needs to be worked on. You can do it yourself, or we’ll be happy to work with you on it, or even in some cases, do things on your behalf. Just let us know.” And I think that’s something we can do side-by-side with faculty, or we can even proactively do on behalf of faculty, and then send them reports to try and help them strive for that kind of excellence, because, you know, faculty are busy, and they may not realize that a link has gone bad because they haven’t taught that course in a semester, and they haven’t gone to check it yet. We can help them out.	
			
KELVIN		Yeah, see to me, I think that’s the beauty of these kinds of tools. If you get dialogue generated, some self-evaluation, a little peer review, or here’s somebody professional who will come aside you and will say, “Hey, here’s some ideas. Let’s talk about them, or I can fix them for you.” That’s useful.	
			
TOM		Sure.	
			
KELVIN		That’s really useful. I think it’s maybe really helpful to make one caveat about blended learning. Much of what we’ve said here is applicable to blended as well. But, actual blended-specific rubrics are far and few between. You find them every once in a while. I’ll give it a shout out to—In the BlendKit course materials, we have a blended-specific self-evaluation/peer review form, and we’ll put that in the show notes. You can look for that, but bottom line is a lot of this stuff is the same, but at least you make allowance for the face-to-face part of the blended course, as well, if you’re going to do that. So what’s the bottom line, Tom?	
			
TOM		What is the bottom line?	
			
KELVIN		Do we have one?	
			
TOM		(laughter) We always have a bottom line. So, all right.	
			
KELVIN		The coffee’s out. That’s the bottom line.	
			
TOM		I can see my grounds. No. All right, so, you want me to take a crack at it here?	
			
KELVIN		Sure. Take a crack.	
			
TOM		So I think when we’re talking about quality, we need a common vocabulary so that we can discuss course design and high quality teaching. Rubrics can be very useful for self- or peer evaluation. In fact, that’s how we use it in our faculty development course. And perhaps [rubrics] can also assist in institutional quality assurance efforts, kind of like what we were talking about with our quick check. Now quick check might not necessarily be addressing, like, creative assessment strategies like, “Let’s do problem-based learning or something,” but at least will open the door and will address some of the minimums.	
			
KELVIN		That 80/20 rule kind of thing.	
			
TOM		Exactly.	
			
KELVIN		Short period of time, get most of the stuff, and go on from there. No, I think that’s good. Good for you.	
			
TOM		So, that’s our bottom line, I guess. 	
			
KELVIN		Okay!	
			
TOM		Would you agree?	
			
KELVIN		No, I think that’s excellent. I guess the only thing I would add to that—and I don’t know if you’d agree with this—be careful about too heavy a hand on the institutional stuff. Don’t want to do quality at teaching faculty. You want to have a collaborative collegial process as much as possible probably, but quality assurance is important.	
			
TOM		It is. All right, so, I know we said we were going to wrap it up, but let’s just keep going. This may be our longest podcast ever. There is a school of thought that quality is the last step of a particular design and development process, and yes, that’s true. You want a quality review or something, but, you know, if you’ve built in a culture of quality throughout, then it’s not about, “Okay, we design the course, now we have to check it for quality.” It’s something that’s just built into everything that you do. And maybe that’s the way to look at these rubrics. It informs everything you do on a formative basis.	
			
KELVIN		I think that’s excellent, Tom. That’s even better place to leave it than the first time you said that. So I guess we’ll leave it there, and for TOPcast, I’m Kelvin.	
			
TOM		And I’m Tom.	
			
KELVIN		See ya!