TOPcast Episode 33: Academic Integrity Online: Addressing Criticisms

TOM		From the University of Central Florida’s Center for Distributed Learning, I am Tom Cavanagh.	
			
KELVIN		And I’m Kelvin Thompson.	
			
TOM		And you are listening to TOPcast: the Teaching Online Podcast. Hi, Kelvin!	
			
KELVIN		Hey, Tom. I was just thinking.	
			
TOM		Sorry to interrupt your revelry here.	
			
KELVIN		I was just thinking I like podcasts and I like teaching online and I like TOPcast. 	
			
TOM		It’s like peanut butter and chocolate combining into delicious goodness. 	
			
KELVIN		Yeah, you said peanut butter and I went to jelly. Now I’m doing word association. I’m being psychoanalyzed. I’m thinking why did I go to peanut butter and jelly and not peanut butter and chocolate? 	
			
TOM		Maybe we are close to lunchtime. 	
			
KELVIN		That could be it. At any rate, I digress. 	
			
TOM		You don’t say.	
			
KELVIN		(laughter)	
			
TOM		Well, we may not have peanut butter and jelly sandwiches here…	
			
KELVIN		No, we don’t.	
			
TOM		But we do have beverages.	
			
KELVIN		We do have beverages. We have coffee, Tom.	
			
TOM		Coffee! What a surprise! I’m shocked to have coffee. 	
			
KELVIN		Well, you know, you’d think you’d pick up on this pattern by now. (laughter)	
			
TOM		I’m not bright but I am persistent. (laughter)	
			
KELVIN		So, you’re probably wondering what this is you’re drinking?	
			
TOM		I am wondering what this is I’m drinking. 	
			
KELVIN		It’s neither peanut butter, chocolate, nor jelly. It is a Guatemala San Isidro from Vessel Coffee Roasters via our colleague Ben Whitmore in the Spokane, Washington, area. 	
			
TOM		Hello, Ben! Thank you for the coffee!	
			
KELVIN		Yes, indeed. It's tasty, I think. I think you could say this is true. See if you follow this train of thought. The test of a high-end coffee roaster is in the cup. However, there are things that can get in the way of that namely like, me, you know, the person doing the coffee grinding and the coffee brewing but assuming that somebody like me doesn't screw it up, you can really taste the outcome of the coffee roasting process.	
			
TOM		Yeah, it’s good. But then you get a monkey like me across the table from you with some unsophisticated coffee palate who cannot appreciate all of those nuances and textures. All I know is it’s good. 	
			
KELVIN		Well it’s better than taking a swig and going, “Ugh!” (spit noise) “That’s horrible!”	
			
TOM		True dat. Yeah, it’s very good, and we appreciate Ben passing it along. 	
			
KELVIN		Thank you. And if you’re interested in Vessel Coffee Roasters, we’ll put a little thing in the Show Notes. You can check that out yourself. But I tried to make a little connection there for today’s topic, Tom. Do you see kind of a segue there somewhere?	
			
TOM		I’m not sure…	
			
KELVIN		Test is in the cup. You know, outcomes… 	
			
TOM		Oh, I see! Oof. Wow, it’s long way around, Kelvin.	
			
KELVIN		I’ve almost failed that test. I’m like, “Do you see a connection?” You’re like, “No, I don’t see anything. It’s opaque and I can’t make a connection between that.”	
			
TOM		I think I get it now. So, I can’t believe we’ve been podcasting for as long as we have and we have really not directly addressed this subject. 	
			
KELVIN		Yeah, it’s true. 	
			
TOM		It is one of the more common questions that I get and probably you, too. 	
			
KELVIN		Yeah, absolutely. 	
			
TOM		When I stand up in front of faculty, during faculty development, or I talk to members of the community or just anyone. When I went on the Tavis Smiley Show, it was one of the first questions I got asked. And the question is: don’t they cheat online?	
			
KELVIN		That’s right. 	
			
TOM		And I always have to take breath and think about who I’m talking to before I answer that question. 	
			
KELVIN		That’s right. How do you normally answer that question?	
			
TOM		Generally, the answer is no more than they do face-to-face. 	
			
KELVIN		(imitating drum noise) Ba-dum-bum. 	
			
TOM		Yeah, and it’s true. At least based on the research. And we’ve done quite a bit of it. I’ll give Chuck Dziuban andPatsy Moskal and some of the grad students that they’ve worked with credit for compiling some of these annotated bibliographies of the literature. And we’ve also looked at some data here. So, last year or so, I had asked our academic integrity office, our student conduct office, to give me a breakdown of the reported cases by modality, and there weren’t that many by percentage. There was fewer cases by percentage for online learning than we have percentage of online learning in general. 	
			
KELVIN		Right.	
			
TOM		If that makes sense. 	
			
KELVIN		Yeah, the proportions don’t match in the way that you would expect them to. All things being equal.	
			
TOM		To be fair, these are reported cases. So, it doesn’t necessarily mean that some shenanigans weren’t happening. It’s just they didn’t get reported. I found that encouraging. And we do some things to try to mitigate academic integrity issues beyond just… Well, maybe we should categorize it. There are technological blunt-force solutions, like catch ‘em kind of solutions. And then there are proactive assessment design kind of solutions. 	
			
KELVIN		The latter would be our preference generally speaking, and we can talk about why that is I suppose. 	
			
TOM		Do you want to start there? Do you want to talk a little bit about that?	
			
KELVIN		Yeah, why don’t we talk about that? So, man, it’s hard. I don’t do pithy, Tom. There is a school of thought that says when you go online, you try to replicate what you do in the classroom, and I’m not necessarily a fan of that mindset. 	
			
TOM		Yeah, me neither. 	
			
KELVIN		But under that umbrella mindset, there is sort of this let’s replicate how we deal with learning assessment which maybe the quippy response to that, the classification is “not well”. We deal with it not well. We have Scantron, multiple choice exams, or whatever…	
			
TOM		Or the analog for that online which is just multiple choice in the LMS.	
			
KELVIN		Exactly. And what we’ve seen historically in twenty plus years of online education is from the literature and I imagine you see this in your teaching practice—I see it in mine—that really when you try to bring about something that is higher level, that’s more involving of students, that has them actually producing something meaningful that’s really an outcome of how they can really apply what they are learning in the class. That’s much more powerful. That’s much more involving. You get to know them as people. You get to know them as people. It’s a much more human process. So, I would call that all and also to be quippy, there’s what we may refer to Higher Order Thinking Skills or HOTS, and Lower Order Thinking Skills or LOTS. And so, we’re better off kind of at the higher levels of Bloom’s Taxonomy of learning objectives, right?	
			
TOM		So, if I have more than an elevator ride to answer that question about you know, don’t they cheat online?	
			
KELVIN		Yeah, you hope I’m not on the elevator. (laughter)	
			
TOM		(laughter) Better be as John Hitt, our president said, “I hope it’s a tall building.” 	
			
KELVIN		(laughter) That’s right! That’s all I ever hope for. Don’t give me a short elevator ride. 	
			
TOM		Usually, I’ll try to go to the next thing, which is they don’t cheat anymore online than they do face-to-face. But what we try to do if we can—and it is part of our faculty development and the work that our instructional designers and others do with faculty—is to look at authentic assessments as opposed to the lower order Bloom’s and just knowledge recall, cognitive recall kinds of assessments. I refer to them as the P’s, so, you have like progressive papers, you know, various drafts. You have portfolios, projects…	
			
KELVIN		Oh, Tom! You’re very quippy! Look at you!	
			
TOM		Yeah, and there was one other too that is just escaping me that I had that was another P. 	
			
KELVIN		Progressive papers, portfolios, projects.	
			
TOM		And there was something else. I’ll think of it. There were four of them. 	
			
KELVIN		Huh. I like that. I have not heard you say that before. I’m going to steal that. 	
			
TOM		And the idea is to try to have the students produce that only they could. That you couldn’t look over at your roommate and have their answer be the same as theirs. Especially if you’re reviewing drafts of a paper, it’s really hard—unless you’re having somebody else write that paper for you over and over again—to cheat on that. If someone is doing that, that’s not a modality specific issue. You can do that in a face-to-face class. 	
			
KELVIN		And let’s just do this often draughted out thing as well, because it’s related. The whole thing of well, how do you know if the people doing the work online are the people who signed up for the class? Well, how do you know the person sitting in the physical classroom is the person who signed up for the class unless you are checking IDs at the door?	
			
TOM		Right.	
			
KELVIN		We don’t necessarily hold face-to-face to the same standard either. 	
			
TOM		No, we don’t, but there are some fairly straightforward things that you can do online to ensure identity including videos and IDs and other kinds of things that actually are easier to facilitate online than they are face-to-face. 	
			
KELVIN		Right. 	
			
TOM		If you had a lecture class of 100 people and you had to check IDs for every single one of them, there’s half your class spent trying to look at people’s identification. You can do the same thing online and everybody do it on their own and it wouldn’t take ten seconds to go through it.	
			
KELVIN		But speaking of that, right, that gets us into a little bit of proctoring. Proctoring is a big thing, right?	
			
TOM		It is. Boy, talk about a big industry. 	
			
KELVIN		Yeah, you know, taking nothing away from anybody, more power to you. Make the better mousetrap and so forth. I think that is part of the equation. I think you have to go there. 	
			
TOM		Yep. I think it needs to be a part of the portfolio of solutions that you offer but I don’t think in and of itself that it’s a magic bullet. 	
			
KELVIN		Because if that’s all your doing, if you’re just saying, “Okay, we’re going to take the same old learning assessment, the same old test, and we’re just going to come up with some efficient and affordable way to have a person or a robot or something look at Tom taking the test and figure it doesn’t look like he cheated, then we’ll go, okay, that’s fine.” We can do better than that. 	
			
TOM		I agree. Our colleague here Joel Hartman likes to describe it as whack-a-mole. 	
			
KELVIN		(laughter) Right.	
			
TOM		We can keep throwing various new and improved technologies at students, but students are awfully clever. And, you know, every time we whack one down, they’re going to pop up with some new thing somewhere else. We’ve all heard the crazy stories of the proctored exams where the camera accidently falls over or whatever and then you see all the answers pasted on the wall. 	
			
KELVIN		Pay no attention to the equations behind the curtain!	
			
TOM		Exactly. They were cheating, but you couldn’t see it because the camera wasn’t pointed at it until it was too late. Yeah, there’s all kinds of ways you can cheat. What I always tell faculty is, even if we’re using a proctoring solution, it’s not some sort of a card that allows you to suddenly do closed-book tests online. That’s still a bad practice. 	
			
KELVIN		Right. That’s exactly right. I remember talking to this associate dean once a few years back, and he was sort of in charge of kind of assessment at a large level for his college, and you could tell he was always on the lookout. I asked, “How are things going?” “Well, you know, found another method,” and I just said, “Hmmm, seems like they are coming up with those just as fast as you can detect them.” “Yes, I know!” But he was looking at is—I don’t —job security or what. He was on the hunt. I’m thinking, “Boy, I think I’d give up that hunt and go a different direction somehow.” 	
			
TOM		Yeah. I know there are a million different proctoring solutions and different companies and partners that you can go into business with to do live-proctoring or not live proctoring or whatever. We actually have a combination here on campus. We have one program that does use live-proctoring, but for the rest of the campus, we have built our own video proctoring solution that we’ve embedded into the LMS. It’s not live-proctored, but it does give faculty and administrators a picture of every test and where it’s used of the student taking the test and they have to hold up an ID and we compare it to the ID in the card office to ensure it’s their face. We are working on some programming to do some automatic facial recognition and some other kinds of stuff. 	
			
KELVIN		All for free. No charge to the student other than just having the hardware.	
			
TOM		But my point is that some faculty take that really seriously. I’ve heard of some faculty that even in a large class will spend the weekend watching every single video, and it’s a bit overkill for me.	
			
KELVIN		Get a life!	
			
TOM		Yeah, that is definitely the exception. Our faculty don’t generally do that.	
			
KELVIN		You’ll go nuts!	
			
TOM		You know, for us, it’s been a really useful tool when other factors have pointed to something suspicious. So, then we can go in and review, or they can or some combination. And then we triangulate that, right? Because I think actually online learning offers some affordances for prevention and enforcement that face-to-face learning just simply doesn’t offer. So, we’ve checked IP addresses. We’ve checked the time stamps of people taking different tests. We’ve checked all kinds of things to either tell faculty, “Yeah, something weird did happen here. You may want to investigate or report” or whatever. We don’t generally make judgements, but we’ll give all the facts that we find. Or in some cases, we’ve exonerated students. Yeah, this student is telling the truth. Their test crashed for whatever reason.	
			
KELVIN		Yeah and you know this, I sit on our academic integrity panel and hear cases related to academic misconduct and there, too, it’s a preponderance of evidence kind of thing. So, as an individual hearing officer or as part of a panel, you listen to everything, you look at all the evidence, and you make a determination of more likely than not because that’s the evidence standard. Something happened or something didn’t happen. So, all of that helps. If you’ve got documentation, that’s where the decision will be. You know, as we’re talking about this I keep thinking it’s kind of like an input versus output thing, right? The stuff we talked about at the beginning of our conversation: design, dealing with higher order thinking skills, designing activities that call upon students to do all those P’s that you talked about. That’s a much better place to be than dealing with outputs. To be chasing down the results of something that was not necessarily well thought through to start off with. Because it will be whack-a-mole. You’ll never catch everything. Gotta catch ‘em all! (laughter) Pokémon! That was your other P.	
			
TOM		(laughter) That must have been the fourth P. Pokémon. Of course! How could I have forgotten Pokémon?	
			
KELVIN		You cited Joel Hartman, I’ll cite Chuck Dziuban. Chuck has often quipped that, “Why do we expect technology to make up for a lack of ethics?”	
			
TOM		(laughter) Yeah. Drop the mic.	
			
KELVIN		It is funny how we paint ourselves into that corner. The ethical thing? That’s a bigger deal. We take that seriously here at UCF, and I’m sure other institutions do. Some institutions have honor code, we have our creed, and integrity is a big thing. That’s harder stuff, but it’s more important stuff. You got to do the technology stuff, but that’s not going to solve everything.	
			
TOM		No, it’s not. And even honor codes [won’t]. In years past, that has been seen as the primary solution because it creates a culture of integrity, but if you look at recent examples of schools that have really strong honor codes like the service academies, there have been cheating scandals at the service academies. And you don’t have a stronger honor code than you do at places like that. So, no place is immune from students trying to get an edge. 	
			
KELVIN		Yes, and that makes me think of something that we should probably note. Not to excuse any kind of cheating ever, but that particular example makes me think of—here’s a phrase that’s so important—high stakes. If the stakes are high—incredibly high perhaps—there is a fundamental structured incentive to cheat, one might say. Now, depending on one’s personal morals and ethics, right? It’s not an incentive to cheat if you’re like, “No, that’s not an option.” But if you’re like, “Eh”, and the stakes are higher, you know, the numbers don’t lie. I can tell you from going through many academic misconduct hearings where the stakes were super high, students will tend more to be tempted that direction. 	
			
TOM		It’s a cost-benefit in their mind. 	
			
KELVIN		It is! And it’s horrible. It’s sad. But so, when you are designing learning activities online— and learning activities, assignments, and assessments can all be the same thing or they can be different things—but as you’re designing those things to deemphasize the high stakes nature. Iterative assessment? I mean, that’s a very helpful approach, but it’s harder. It takes more thoughtful, deliberative design to get there.  	
			
TOM		Yeah, and maybe we can just spend a moment just looking forward. I totally agree. The days of a midterm and final and that’s your whole grade, I think, are behind us now, hopefully. You know, I’ve had conversations with faculty and department chairs about the use of adaptive learning as not just a really good pedagogical strategy—to meet students where they are, give them extra support if they need it, allow them to accelerate—but also as a strategy to mitigate collusion and cheating because it’s so personalized that everybody gets a different experience. You really can’t cheat. And even if you did work together, all you’re doing is helping each other learn the concepts, but you’re going to have a different answer set then the person sitting next to you. Which I think is a really interesting by-product/benefit of something like adaptive learning. We may have talked about this in the podcast in the past, but our math chair has talked about this concept of a course with no tests where through adaptive learning, the students are just continually assessed. As they go through the program, there’s these little micro-assessments. By the time the course is over, you don’t need to give them a test because you know what they know from everything that they’ve been assessed on all along. How do you cheat in something like that? It’s really hard. Especially with variablized questions where everybody gets a different number set. It’s almost impossible to cheat in something like that. 	
			
KELVIN		That’s right. Now, honestly and philosophically, you can do the same thing without technology, as well It’s just a lot harder. 	
			
TOM		It’s harder to scale. 	
			
KELVIN		That’s right. The philosophy is broadly applicable. But the technology—if you apply the philosophy to it—can get you to some really desirable places at a larger scale.	
			
TOM		Can I come back to something you said earlier?	
			
KELVIN		Please.	
			
TOM		This is often what I default to and I try to stop myself now. When we are asked the question, “Well, don’t students cheat online?” And the stats do show that they don’t cheat online any more than they do face-to-face. Maybe some of the kinds of cheating might be a little different. What Chuck and his team have talked about: panic cheating, planned cheating, and all those different categories. But the incidents of cheating are no great online than it is face-to-face. But when we say, “You’re holding online learning to a higher standard than you are face-to-face because you aren’t checking IDs at the door.” As many times as I have said that—and I have said that plenty—it’s never worked. 	
			
KELVIN		I know, right! Isn’t that funny?	
			
TOM		It’s never swayed anybody. I’ve never had anybody put their hand to their chin and say, “Oh, I’ve never thought about it like that.”	
			
KELVIN		I actually have, in fairness, had people go, “Huh,” but they are more thoughtful people than you talk to.	
			
TOM		(laughter) Yeah, than the cretins that I associate with. 	
			
KELVIN		(laughter) But continue on. 	
			
TOM		Well, I just have tried to—at least, if I mention it—to not dwell on it because I don’t think that ends the arguments. I don’t think it’s persuasive. So, I think it’s time to kind of move past that as a defense and to talk more proactively about the kinds of things we are doing that I think are better than the face-to-face environment. Some of these strategies we’ve already touched on.	
			
KELVIN		That’s a good point. When we talk about a different standard and a higher standard and better standard, it’s a good strategy, Tom. Lean into that. Embrace it. And say, “Yes, our standards are higher online than they are in face-to-face.”	
			
TOM		Yeah, and it kind of goes back to something we did talk about—this was a long time ago—about the no significant difference literature, where when people ask, “Is online learning as good as face-to-face?” which is kind of a derivative of this cheating question. And the answer that we have used in the past is based on the corpus of literature on no significant difference. Well, just telling people that it’s as good as face-to-face, I think, probably isn’t good enough anymore. I think we need to talk about how it could be better in some cases. Not to say that face-to-face is bad, right?	
			
KELVIN		Of course not.	
			
TOM		It’s great, but there are certain affordances that the online environment gives you that in certain situations and context that can make the experience more effective or better or whatever depending. And I think the same thing exists in the world of academic integrity. 	
			
KELVIN		I think that’s right. I have often said that when you go online, you make the formerly implicit more explicit. You make the formerly invisible more visible, and that’s a lot of what you’re talking about. You have at your disposal a lot of evidence and a lot of opportunities to make visible things that weren’t there before. So, you can enact that higher standard. Whether it’s data, analytics, visual records, whatever, it’s different than what you’ve had face-to-face. So, you might as well lean into it and embrace it. So that’s a good point.	
			
TOM		Yeah, people are not going to stop asking that question. And I think we need to be prepared with some answers depending upon who you are talking to. And you know what? We’ve got a forum for feedback on our website, and I would welcome people to share how they answer this question of academic integrity with us and with the community. Because I think collectively, it would be good for us to have some responses that are not feeding into sort of myths and misperceptions. 	
			
KELVIN		Yeah, so, Tom just gave you homework, folks. So, if you go to the TOPcast website. TOPcast.online.ucf.edu. Find this episode. Scroll down to the bottom. There’s a comments box there, and you can type away to your heart’s content on how you’ve address these issues. But our time and our coffee are growing thin.	
			
TOM		Yeah, I’ve seen the bottom of my cup. Alas. It was very good. Thank you, Ben. 	
			
KELVIN		Yes, indeed. Thanks very much for that. Very tasty coffee. So, as we get close to wrapping up, can I take a stab at where I think we are kind of landing this topic?	
			
TOM		Land the plane, Kelvin! Landing gear is down.	
			
KELVIN		Disagree if you like. And oh gosh, there’s so much more we could talk about here.	
			
TOM		This is more than a one episode podcast topic, that’s for sure. 	
			
KELVIN		We’ll throw some supporting resources on the Show Notes page and all, but we probably need to come back and revisit other things. We haven’t really talked about things like writing higher-ordered multiple choice exam items, and all manner of zoomed-in, on-the-ground ways of designing for higher-ordered thinking, and all the P’s you talked about, but that will be for another day. Here’s my stab here. Academic integrity online is more of a learning assessment and personal ethics issue than it is a technology issue. So, our approaches to solving those issues ought to match the challenges. That is, we ought to take them on as a personal ethics issue and as a learning assessment design issue rather than an implement technology thing. Would you agree?	
			
TOM		I would agree. I think it’s a multi-pronged solution. It’s not any single strategy that you employ is going to address this. Cool! Alright! So, maybe that’s a good place to wrap up. Until next time, for TOPcast, I’m Tom! 	
			
KELVIN		I’m Kelvin!	
			
TOM		See ya!