TOPcast Episode 34: Research and the Instructional Designer

TOM		From the University of Central Florida's Center for Distributed Learning, I'm Tom Cavanagh.	
			
KELVIN		And I'm Kelvin Thompson.	
			
TOM		And you are listening to TOPcast: the Teaching Online Podcast. Here we are again, Kelvin.	
			
KELVIN		Yeah, unless we had a time machine and then it would be here we are for the first time.	
			
TOM		Yeah. Well, we're working on that. We probably have some researchers over on campus exploring that. Perhaps the physics department.	
			
KELVIN		Ah. Theoretical.	
			
TOM		Yes.	
			
KELVIN		Theoretically.	
			
TOM		Some string theory perhaps?	
			
KELVIN		That's right. I think Sheldon Cooper's backed the string theory now on the Big Bang Theory.	
			
TOM		(laughter) Depending upon when you're listening to this.	
			
KELVIN		That's right. Or when you're traveling in time.	
			
TOM		Yeah, but at least in this dimension, you and I are sitting here in our favorite little room—	
			
KELVIN		(laughter)	
			
TOM		—talking to each other and the vast world of interested online learning professionals.	
			
KELVIN		Yes! You guys are vast.	
			
TOM		Vast. (laughter) That's no reflection on anyone's weight.	
			
KELVIN		(laughter) Way to go. You've offended everybody now.	
			
TOM		Sorry.	
			
KELVIN		Drink your coffee with nothing in it!	
			
TOM		(laughter) Maybe we should just jump to the coffee. Shall we?	
			
KELVIN		Alright. Always.	
			
TOM		What's in the Thermos today, Kelvin?	
			
KELVIN		Alright, so, first—you know you're in trouble when there's a preamble and a disclaimer. First, allow me to comment that I did not go to barista school. So, despite reading and watching online coffee-related content, I'm pretty much self-taught, you know? So, don't judge me. I bring this up because first, I'm still not sure if I got today's coffee brewed just right. So, that's not a reflection on the coffee. That's a reflection on me. And two, today's episode has something to do with gauging the expertise of instructional designers in a specific sub-domain: specifically, research. For instance, some IDs are experts in the area of research with formal credentials. Some are self-taught with varying levels of success. Others consider themselves novices when it comes to research. So, my possible brewing missteps notwithstanding, today's coffee comes to us via our colleague Dr. Wendy Athens of Utah Valley University from Jack Mormon Coffee Company in Salt Lake City. It's a dark roasted Mexico Cuarenteño. What do you think of the coffee?	
			
TOM		It's good! The only thing is something we've talked about is a temperature issue, but the coffee itself is good. Thank you, Wendy! That was sent to us after a visit that she and her team made here.	
			
KELVIN		Yeah! It was very kind of her. What do you think of the coffee connection?	
			
TOM		I think it's as good as any of your other coffee connections, Kelvin.	
			
KELVIN		Ouch! That's harsh! (laughter)	
			
TOM		(laughter) No! That could be awesome.	
			
KELVIN		It could be!	
			
TOM		There's a vast coffee connection. (laughter) No! I get it! Because we're talking about research and expertise and maybe stepping a little bit outside of your comfort zone to learn something new, as you have done with coffee. Well, you know, is there coffee barista school? Can you major in it? "I got my BA in pour over."	
			
KELVIN		Well, of course ASU's got a barista to BA, BA to barista thing.	
			
TOM		That's true.	
			
KELVIN		There's that.	
			
TOM		But they're not studying that.	
			
KELVIN		No, one would think not. Apparently, there are some. I did look that up, because I asked myself the same question. "Is there barista school? Do I want to sign up for it?" There are several programs. I think they focus on opening up your own boutique or coffee place, but I think there's some stuff that I saw that maybe even a Joe Schmoe like you or me could learn something.	
			
TOM		I heard a podcast one time that talked about somebody who started their own high-end coffee business. It was importing, I believe, beans from Yemen— he was from Yemen or his family was so he had some connections—but it was kind of self-taught, but there was a certain amount of apprenticeship, I believe, that was part of that, as well. Maybe that's the model, but that's cool! Thank you, Wendy, and thank you, Kelvin.	
			
KELVIN		You bet!	
			
TOM		I get it! So, there's a reason why we're talking about instructional designers and research, correct?	
			
KELVIN		Yes, there is.	
			
TOM		So, back in the fall, during the Online Learning Consortium Accelerate Conference, you had the opportunity to talk to our good friend Dr. Katie Linder from Oregon State University where she serves as Director of their Ecampus research unit. She's also the host of about 612 podcasts, 48 newsletters...	
			
KELVIN		(laughter) You know it does seem that way!	
			
TOM		Actually, she's the host of three podcasts that I know of.	
			
KELVIN		Yeah, maybe more brewing!	
			
TOM		Maybe. She's amazing. The one that's probably most relevant, and you and I have both appeared on together—and she has previously been on this podcast—is the Research in Action podcast. So, yeah, it's great to have her back because she's been on the show here before. She's always working on something interesting. We've both published and edited collections that she's put together, and I'm always interested in what she has to say. She was at OLC Accelerate talking specifically about a new project related to instructional designers and research, correct?	
			
KELVIN		That's right.	
			
TOM		So, do you want to do anything to sort of set up the interview?	
			
KELVIN		I guess the only thing I'll say is yes, this was right after she had released this study—which she talks about—about instructional designers and research. Our conversation goes into some related stuff that her research unit is doing, so I think they're doing really interesting work. So, maybe we'll cut to the interview and we'll come back and talk about anything that comes to mind after the interview. How about that?	
			
TOM		Sounds good. Here's Kelvin and Katie.	
			
		(musical transition)	
			
KELVIN		Hi Katie! Glad to have you back on TOPcast!	
			
KATIE		I am always glad to come on the show, Kelvin. Thanks for having me.	
			
KELVIN		Well, you are welcome. So, today, I'm excited to talk to you a little bit about a recent research study that came out of your office. Why don't you tell us a little bit about what it is before we talk about how it came to be?	
			
KATIE		Sure. So, we recently released a study on the research preparation and engagement of instructional designers, and we wanted to learn a little bit more about their formal education in terms of research and how they're currently engaging in research on teaching and learning.	
			
KELVIN		Now, you know—because I think we've touched on that area tangentially when you were on TOPcast before. We talked about the multiple roles of instructional designers. So, how did this particular study come about? What got stuck in your head about this?	
			
KATIE		Well, we have been doing a lot of presentations at different conferences like OLC Accelerate—which is where we're recording now—and we were doing presentations on research design, methodology around teaching and learning research, and we always have instructional designers in the audience. They had incredibly thoughtful and intelligent questions, but it was kind of clear from our interactions with them that they maybe didn't have the training that they needed to do this work, and they were always following up to ask, "Where can I learn more about this?" It just got us kind of curious because we were thinking—and we continue to think—that instructional designers are playing such an important role in that in-between space between students and faculty, and they have so much they can offer in terms of research. So, we really wanted to dig in and learn a little bit more about what is it that they want to be doing in terms of research and what are the kinds of things that they're hoping to grow into in their professional roles, but also how are they being prepared to do that if that is something they want to pursue?	
			
KELVIN		I'm barely containing myself asking you to hit some highlights from the study, but maybe it would be instructive for our listeners, too, to know a little bit about how you went about conducting this study. So, who were the instructional designers that you studied in this research?	
			
KATIE		Sure. So, we ended up doing about four weeks of recruitment, and we asked for a bunch of organizations to help us to recruit, because now there are kind of these instructional designer sub-groups with a bunch of places like UPCEA and ELI and OLC and all these different folks. So, we asked everyone to kind of help us recruit, and we had hundreds of people responding. We also did some personalized LinkedIn recruiting and things like that, and we ended up with a little over 300 that we included in the survey study. So, this was an online survey, about 60 questions, and we did both qualitative and quantitative analysis. So, we ended up with some really amazing, rich quotes from people as well in terms of how they're engaging in research.	
			
KELVIN		So, big picture highlights: what are the big takeaways from this?	
			
KATIE		Well, we will definitely link to the study in the Show Notes.	
			
KELVIN		Yes, absolutely.	
			
KATIE
			There are kind of five key takeaways we had from this study, but let me give you just some big picture things. One is definitely that instructional designers love to collaborate on research, and it's actually their preference to collaborate versus to do it independently, so that was something that was interesting that came out of our sample. We also found that there are a range of barriers that instructional designers are running into, and some of those are things like institutional barriers. They may not have access to the data they need to conduct the research. They might struggle to get through IRB if they're not seen as someone who can be a PI. There are also issues around credibility and value and who they thought was kind of thinking that they were more credible in terms of their own work when they were conducting this research. They definitely felt more credible in the eyes of the broader academic community and also in terms of the faculty that they were collaborating with, so that was really interesting. But, one of the key findings I think is just that instructional designers are definitely key players in this, and we should be thinking about them as possible researchers. Now, certainly not all of them want to be doing it, but a majority of the people in our study were interested.	
			
KELVIN		Yeah, I think those are all interesting points for sure. Just to comment aesthetically, I like how you laid out your site with the download for the report. That made that very accessible, and you know, I'm kind of a visual charts and graphs kind of a person, and you know, I don't really read books. I've said before I just read the back cover of books, right? So, give me a high-level infographic or chart, I'm happy. You've got that! And a lot of words as well that I was enticed to read the words, as well, so that's good! It's working for you.	
			
KATIE		That's good news for our marketing team that helps us with these website designs. For people who are hopefully not too intimidated, the study is about 80 pages, but that does include all the data tables and the instrument itself and there is a nice executive summary in the front if you just want to read a couple of pages.	
			
KELVIN		Which is very friendly for people like me. So, can I ask you about that, too, just at the risk of going down a trail for a second? That was a choice that you made to publish that on your site and make it so friendly, so accessible. So, why that route versus going through a traditional journal publishing venue?	
			
KATIE		That's a great question. This is actually the third research study that we've published through our site. We published a couple national studies last year, as well, and I think that often times what happens is we'll publish the initial report, which doesn't include everything. It includes our initial treatment of the data to get things out immediately so people can take a look, and with studies like this, I think that right now, instructional designers are a pretty hot topic. We're trying to kind of figure out what that role is as the field is professionalizing. People are very interested in it, and we had a lot of people who really wanted to know what the outcomes were. As your listeners may know, the publishing timeline for many peer review journals can be 18 to 24 months, and so often times we will put out the initial report and then follow up with some peer reviewed articles. That allows us to have a little bit more time to do a deeper dive with the data and to really kind of analyze it from a number of different angles that maybe we aren't doing for the report, the initial treatment of the data we put out. Also, we have several other publication projects in the pipeline that are longer term publications, as well, like edited collections, so we're kind of publishing out in a lot of different ways.	
			
KELVIN		Is part of your hope that your office will have a go-to site that will bring people in? To be thought of as, "Oooh, they have tasty research! I should go there first!"	
			
KATIE		Well, we always want to have tasty research, but I think that yes, we are hoping that the website has a number of different things that people can explore and look at. We have a bunch of stuff in our pipeline—some white papers that are going to be coming out in early 2018 on distance education research methodologies from some of our fellows that work within our fellows program and the faculty at Oregon State. That's really exciting. Also, I think it's just important for us to try to find a rich variety of ways to get this information out to people, and we found that research reports—especially in this field—are really accessible for people, and there's a nice history of doing these kinds of national surveys and then putting out a research report for people to digest.	
			
KELVIN		That's great. Now, as long as I've taken you down a rabbit trail, I'll just keep on digging deeper and we'll come back to the topic at hand. So, we were talking about this earlier. I know you recently also put together a database of research, so why don't you tell us about that, too, as long as we're talking about things on your site that might attract people to come visit?	
			
KATIE		Sure. So, this is something else we can definitely link to in the Show Notes, but a project that we've been working on since the summer is compiling all the research that compares different teaching modalities—online, face-to-face, and then hybrid and blended—so that people can dig in and really look at that modality-specific research and especially so that they can look at it from a disciplinary perspective. The reason that we wanted to do this is we get questions from our faculty all the time saying, "Well, what about the research in my discipline? How do we know that online learning works? How do we know that the outcomes are equivalent to face-to-face?" I think that there are definitely people in the field who are ready to move on from that question. (laughter)	
			
KELVIN		(laughter)	
			
KATIE		We were there, too. I mean, it was definitely something that we've had some meta studies done, and there's definitely a rich literature and we have 186 studies on the database that you can search through, but our faculty were still asking this question. We just felt if our faculty aren't ready to move on, we won't be ready to move on, either. We really wanted to provide this service for them, and so, this database is open for anyone to take a look at. I think the most important functionality of it—and it's the only one we know about that has this functionality—is you can search by discipline. So, if you want to know what are the efficacy studies that have happened in biology, you can look directly for that and find out what's going on in your field.	
			
KELVIN		That's great. I think that will be an interesting resource for folks who have those questions. I agree with you. I think the folks kind of corridor field, as you say, are ready to move on, but there are plenty of folks for whom this is still a new question. Discipline-specific folks. We have colleagues that we've talked about who have come into professional practice through teaching in a discipline and this is all new.	
			
KATIE		Absolutely.	
			
KELVIN		So those are honest questions. That's good. So, back to your study of instructional designers: did you find out anything else about instructional designers as you were conducting this study that was instructive? Demographically? Makeup wise? We talked when you were here before about the multiple hats, the multiple roles that instructional designers have. I know you were focused specifically on research preparation and how research intersects with their jobs, but anything else surface along the way?	
			
KATIE		One of the things that was really interesting is we found that a lot of the instructional designers in our sample had very recently come into working in higher education. They had quite a bit of experience working in other industries but had recently switched in the last maybe one to five years to coming into higher ed, which could explain why many of them have no training in academic research because they were trained to do instructional design maybe in a corporate setting. Some of them had worked in military settings or K-12 settings, and so, that I think is something interesting just to consider as we're starting to train instructional designers specifically for the culture of higher education, and there's a layer of research that's embedded in that.	
			
KELVIN		Yeah, that's right. That's a good observation. Is there a what next to this study? Is there something else you want to do to extend or follow-up on?	
			
KATIE		Well, one of the things that it raised for us—and this is something that we're hoping to look at more in early 2018—is who exactly is publishing in the field of distance education research, and so, we're starting to compile a new database that won't be public. It's just for kind of research analysis purposes, but we want to figure out who in maybe the last 10 years or so has been publishing in some of the select journals in distance education research, what are their position titles, and also what are some of the demographics so that we can get a sense of the diversity of who is publishing in our field, and I'm really curious to see if there have been trends that have changed over time. Maybe we started with more faculty publishing and we're starting to see more administrators or staff that are moving into publishing in those peer review journals. So, that's kind of one of the next steps that is leading off of this research with instructional designers, but also, we're making some changes as well at Oregon State in terms of how we're engaging our own instructional designers in research. We've created some space for them in our fellows program so they can collaborate with faculty researchers, and we've also been doing some dedicated research professional development around things like data visualization, so we're starting to look at our own practices to see how we can make sure that our instructional designers who want to engage in research can do that.	
			
KELVIN		Yeah, that's awesome. Sounds like things are going great guns at Oregon State, and you're always doing something interesting, and you're adding to what we know about our field itself, what we know about instructional designers and how we can be more effective and I know that a lot of our listeners do consider themselves instructional designers. So, as part of your resources that you're putting together on your site, if folks wanted to know more about how they might go about increasing their research competence, do you point them to some resources that would be helpful professional development-wise?	
			
KATIE		So, the first resource I would mention—and this is the main one that we produce in this area—is our podcast Research in Action, and so, this is the main place where we are talking with researchers in higher education about the topics and issues more general to higher education research, but we often point out different tips and strategies for how to engage in learning something new. We are going to be releasing a new mini-series specifically on quantitative literacy coming out on the show in the next few weeks, so that's very exciting, and that was an area in the study we found a lot of instructional designers were less confident when it came to things like running statistical analyses, cleaning data, that kind of thing. So, that is one resource, and definitely, I welcome people to reach out to us. I think there are so many different possibilities of how you can engage in professional development around research. It really depends on your preferences and whether you want to do like a massive open online course or you want to find a really good book that would be useful for you. The podcast would be a nice place to start.	
			
KELVIN		It is indeed, and I should have already given a nice plug to you for Research in Action. That's always a go-to source, especially for folks who are already listening to a podcast like this one. So, go right on over if you haven't subscribed and subscribe now in the same place that you found TOPcast. Well, Katie, as always, it was a great pleasure to have you on and hope to have you come back again before too long.	
			
KATIE		Thanks so much, Kelvin. This was really fun.	
			
		(musical transition)	
			
TOM		So, that was the conversation, Kelvin, you had with Katie. I found it really interesting. I always find listening to Katie interesting. I do listen to her podcasts. I think I mentioned to you recently I've been periodically binging on her Anatomy of a Book podcast.	
			
KELVIN		I've not made it there yet. I have good intentions.	
			
TOM		Yeah, well, we'll give a shout out to her three podcasts: the Research in Action, which we've already talked about; Anatomy of a Book; and the other one is You've Got This. They're all kind of slightly different. Back to the topic that you and she were talking about: I think it is interesting because she had said something about how she presents at these conferences and there's a lot of instructional designers in the audience and they'll come up to her afterwards and say, "How do I find out more about this? How do I do that kind of research?" I think it's so valuable to professionalizing the whole area of instructional design in online learning, and I was put in mind of something that you've been very involved with here: the TOPR. Too many TOP things.	
			
KELVIN		(laughter)	
			
TOM		It's the Teaching Online Pedagogical Repository, if you don't know. I encourage you to check it out. It's a really rich and deep resource.	
			
KELVIN		TOPR.online.ucf.edu	
			
TOM		Thank you. Hopefully that URL will remain in perpetuity.	
			
KELVIN		We sure hope so.	
			
TOM		We hope so. It's got all kinds of great resources, but one of the key elements in each of those entries about good practices in online instruction is that you link to some scholarly resources. This isn't just one faculty member's opinion of what's good. It's good because it's been proven in practice and it's been supported by actual scholarly literature and research. I think that's really important for instructional designers to understand. At the risk of sort of going on a bit of a soap box, I think that we're somewhat fortunate here in that many of our instructional designers kind of already think that way. We may have talked about this when Katie was on the show before, that our instructional designers are all technically in unit faculty. We have a unionized faculty, and they're considered part of the bargaining unit and are treated as faculty and have all the benefits that come with it with the exception of tenure. They're kind of like librarian faculty.	
			
KELVIN		There's a promotion plan that’s based on scholarship.	
			
TOM		Right, so yeah, scholarship is a component of that. Many of them—I think more than half of our instructional designers have terminal degrees so they've been sort of trained in research. There's an expectation. We've got the good fortune of having full-time researchers outside the faculty unit that can be kind of mentors to them very much like Katie serves in a similar role at Oregon State. We've certainly, I think, borne the fruit of that.	
			
KELVIN		Absolutely. I totally agree with that. I think, you know, additionally, instructional designers bring a very unique grounded—not in a grounded theory sense per se—perspective to inquiry, SOTL (scholarship of teaching and learning) and otherwise. So, it's not just an individual teaching faculty perspective or just an impartial researcher perspective, but that informed scholarly practitioner approach, which I think is real useful, as well.	
			
TOM		Yeah, and something else she talked about was pairing up the instructional designers with the faculty that they are supporting. She talked about, I believe, some fellow positions, and although we don't necessarily have something exactly like that, we do have an awful lot of our instructional designers partnering with the faculty they support to do scholarship of teaching and learning research.	
			
KELVIN		Which is awesome.	
			
TOM		It's great. They go to conferences together. They publish together. It helps them both. I think that's awesome.	
			
KELVIN		Yeah, in that sense, I hope we see more of that in the field, because I think it does make us all richer, for sure.	
			
TOM		So, thank you, Katie. That was fantastic. We'll probably have to make up another excuse to get her on the show.	
			
KELVIN		Yeah, right?	
			
TOM		Because she's got nothing but time.	
			
KELVIN		Apparently!	
			
TOM		Any other final comments? Something you wanted to highlight from your discussion with Katie?	
			
KELVIN		The only other thing I would say is I'm really intrigued by that secondary topic that we had: that disciplinary efficacy research database.	
			
TOM		Yeah.	
			
KELVIN		Really interesting. You know, that's not a direction we've gone a whole lot here, and she and I talked about it while the recorder was running and before and after a little bit. I've run across other places where we've been doing this here for a very long time. I don't think we've run into those questions as much, but when I talk to colleagues at other places and talking to Katie, it is a real question, and when you're zooming in in a new discipline. So, I think it serves a purpose.	
			
TOM		I think that's true, and even though I think that we've got pretty broad adoption here, we don't often get asked a question like, "Does this work?" Every once and a while we do, and it tends to be in the departments and the disciplines where we are not quite as engaged as we would like to be. If we had disciplinary research in these areas—discipline-based research in these areas—I could see it being enormously helpful.	
			
KELVIN		So, I think that's a real niche offering that she's ginned up there at Oregon State. We'll link to that in the Show Notes as well as her research study that we talked about. Take a look.	
			
TOM		Great. So, let me see if I can kind of put a bow on this.	
			
KELVIN		Put a bow on it!	
			
TOM		Given the inquiry-driven nature of our higher ed. settings, instructional designers tend to benefit when they're able to successfully make use of existing published research. We all benefit from research findings published by our instructional designer colleagues.	
			
KELVIN		Undoubtedly.	
			
TOM		So, research competence might even lead to more productive relationships between IDs and their faculty clients.	
			
KELVIN		We can only hope.	
			
TOM		Yeah, I mean, certainly we just mentioned one, which was the opportunity to do research together to benefit both their careers. So, awesome!	
			
KELVIN		Yes! I totally agree with that. That's awesome.	
			
TOM		(laughter)	
			
KELVIN		In the dwindling few seconds, Tom, it's been a while since we shamelessly quoted one of our listener testimonials from iTunes or what they're calling these days Apple Podcasts.	
			
TOM		They just can't leave that app alone.	
			
KELVIN		I know. So, maybe we'll do it today. So, today's testimonial comes from our colleague Dr. Jill Leafstedt of California State University Channel Islands, where Jill is Executive Director of Teaching and Learning Innovations and Senior Academic Technology Officer. Oh, and by the way, if you haven't listened to the HumanizED podcast that comes out of Jill's shop. So, Jill says, "Thank you for the opportunity to listen to TOPcast. Each episode is unique and engaging. You're addressing issues and topics relevant to all in higher education. The style of delivery is warm and enjoyable. I'll be sharing with my colleagues and plan to use future TOPcast episodes as jumping off points for faculty discussions and workshops." Isn't that awfully nice?	
			
TOM		Thank you, Jill!	
			
KELVIN		That was great.	
			
TOM		Shout out to Jill!	
			
KELVIN		Yeah! Those are good folks there: Mike, Michelle, Jill. All kinds of people. If you have something kind to say about TOPcast, please look us up on Apple Podcasts and leave a review.	
			
TOM		You, too, might get a shout out.	
			
KELVIN		You might! If you have something to say that warrants follow-up, feel free to record a voice memo and email it to TOPcast@ucf.edu so we can respond. We might even play that in a future episode!	
			
TOM		Wow!	
			
KELVIN		Wouldn't that be cool?	
			
TOM		That's awesome. Alright. So, thank you for the nice words, Jill, and thank you again, Wendy, for the coffee, and thank you, Katie for being on the show. Until next time, for TOPcast, I'm Tom.	
			
KELVIN		I'm Kelvin!	
			
TOM		See ya!