Show notes

Kathryn Baron (7:22): I’m wondering then if that disconnect is perhaps part of what leads to academic integrity problems in college and the frustration that you mentioned students often have?

Dr. Roy Swift (07:34): I do think K-12 is the foundational component to success in post-secondary education, in academia and Higher Ed. I believe it is crucial to individuals being able to make the right choices. There are several issues in regards to the system and preparing success in the post-secondary system. One is helping people understand how to learn to be able to identify resources, to build self-confidence in people. There is our need to move to more of a competency-based approach in Higher Ed, that’s transparent and can signal to the work world or the government or whoever that what the person not only knows, but what the person can do. The current transcript is not helpful in this regard. And the reluctance of faculty to move to competency outcomes versus general course descriptions is problematic.

I feel the issue is that the competency approach does put more pressure on the faculty to produce what they say they are producing because it is more transparent, and the assessment tools have to be more precise. The other disconnect is the lack of employability skills. The college is not teaching the behaviors that are expected in the workplace. Something as simple as coming to class on time, participating in class and being an active learner and working in teams often are forgotten.

Kathryn Baron (10:36): You mentioned a paper in an earlier conversation we had that you co-led on the integration of credentials, and I’m wondering if you can just tell us what were the primary takeaways from that and were you at all surprised by anything that you found when you were doing that work?

Dr. Roy Swift (10:54): Yes. Recently, I participated with the Higher Learning Commission, which is one of the national accreditors of universities and colleges. And because they are very interested in looking at the whole issue of credentials and how credentials may be integrated into a higher education system, industry credentials in this regard. The title of the paper was Institutional Accreditation at the Crossroads Drivers for Change, and it had four main themes. One was at the landscape and pressures on Higher Ed, employers and accrediting bodies are going to be increasingly to produce a product that is able to function at higher levels probably because of all the technology that is going on today will have to produce a very different kind of individual. Most people think technology will run people out of jobs, but it really looks like that what is going to happen is that it’s going to force and put pressure on producing people with higher level of knowledge in the ideas of robotics and artificial intelligence.

Dr. Roy Swift (13:36): ….there are over 8,000 industry certifications right now. And every week they develop more and more industry certifications, and it is one in which they can be complimentary.

But unless we understand, like I said at the beginning, the credentialing system and how they may interface and complement one another, we are going to develop competing systems. Which may not be the best way of thinking about these various because each credential tends to have a lot of strengths. And so, we should use the strengths of each credential to be able to see how they can be integrated. Our particular group, Workcred, is looking about the embedding of certifications into degree pathways. We think the two can complement one another because certification is about competency assessment.

Dr. Roy Swift (17:14): So, people who are trying to design Higher Education need to take a more systems thinking about what is the work world telling them? What is the government telling them? What do students desire? What’s the environment that we should be doing? And so, we take it from the, let’s just say the national system to the state systems, to the academic systems, to the subsystems of provost, deans, faculty, students, and understanding how those layers exist. Unfortunately, too often the K-12 system, which we talked about as being the foundation, is not producing individuals that have the psychological ego strength to face many of the issues that students are facing. And that threat, I think, does cause students to do things that may be unethical, such as cheating.

Dr. Roy Swift (26:12): Well, I think the first step is building more precise learning outcomes, competencies, whatever word that people feel comfortable in using, because I think that many times, I hear students talk about the unknown, oh, I don’t know what he’s going to ask. I don’t know what the expectations are. Competency gives more structure to the student as to what the expectations are in this regard. Competency-based assessment is really a more straightforward method of evaluating whether a skill has been achieved. And the students feel more secure when they know what it is that is expected of them, expected in the course, and expected on the assessment.

Kathryn Baron (30:50): ….what are your thoughts on what can be done in the training and workforce development industry to minimize cheating or even to, I don’t know, curb the impetus to cheat itself?

Dr. Roy Swift (31:04): Well, I do think it’s important to go back to transparency, relevance, and competency. I would say those are three main elements that has to be looked at. I would go back to my thoughts about competency-based education. When it’s about competency, and you can repeat the assessment until you’ve achieved the level of competency. There’s less reason to cheat. Let’s say, okay, I failed. I’ll go back and relearn. That’s a skill that’s taught in the military. I used to teach at the Academy of Health Sciences. It’s called something different now in the army. But one of the things that always happens in the military is that teach, test, reteach, retest. And generally, students are given several times to be able to achieve it because the military believes it’s about competency.

It might take them two or three times that one person can do it on the first time, but it doesn’t mean that the person who took three times isn’t just as good with that competency. So, I think we have to take that sort of an attitude in Higher Ed, instead of this, wow, we’re going to fail you, and that’s it. There’s no other chance in this ring. And it sets up a more feeling of freedom to fail. And don’t we tell people we learn by our failures?

On this episode of The Score, we look at cheating from a different angle than we have before. Our guest is Dave Tomar. From 2001 to 2010, Dave worked as the ultimate ghostwriter. He was a contract cheater. He wrote thousands of college essays, reports, and even master’s degree thesis. After a decade of putting words into other people’s work, Dave Tomar put the cheating life behind him. He’s since written two books about his experiences.

Dave Tomar (05:26): Well, I saw quickly that this type of service was popular with my classmates. But I had no idea how large the demand was, and when you start working for these companies, suddenly it’s not simply that you’re getting paid to write, it’s that you have more writing work than you can handle, which was a unique and exciting position for me to be in, honestly.

Dave Tomar (07:58): Yeah, it was a bit of a barter system as well on the college campuses. But no, the real difference was that while I was charging between $10 and $20 a page, both independently and while working for online companies, the online companies were charging twice that. I would get half of it, but that was the model for profitability. As an independent contractor, I would get half, they would get half, so I was essentially learning that I could have been charging twice as much on campus. However, it was worth splitting the proceeds because the work was so plentiful.

Kathryn Baron (09:19): About what did you earn a year?

Dave Tomar (09:21): I probably started when I went full-time earning just a little over $30,000, which so you know, was a raise from my legitimate job. By the end, bear in mind, inflation now applies, but this was 2010, I think I earned about $66,000 in my peak year.

Kathryn Baron (10:00): Do you have any sense of how many independent contractors like yourself there are working for these companies?

Dave Tomar (10:07): Certainly thousands. Every company that I’ve worked for has a different size pool. Some of them, you could tell was a couple of dozen, but others were sort of these broad online syndicates where when you get a sense of the surface level of this industry, there are big faces looking out to customers, but there maybe 20 of them affiliated with the same writing pool. The back door that I worked in for one company was a name that you would never see in public, but they pulled in assignments from a couple of dozen different outlets that are pretty well-known, and so that was a pool of hundreds. Now, when you get to the real essay mills, which are some of the lower-grade ones that might be operating overseas with even fewer rules, they could be working with stables of thousands.

Dave Tomar (11:11): The smaller companies would actually reach out to you with individual assignments. They’d say, “You interested in this one? You interested in this one?”, which is a bit of a clunky model, but I certainly have worked that way. The best companies that I have worked for use an automated system. You go onto a page like cheat.com and you order your assignment, and it automatically shows up on a board that I and hundreds of other writers have access to. As soon as it shows up, it tells me when it’s due, what it’s about, what the college level/graduate level is, and how much I’m going to get paid to do it, and you click the right button, and it goes into your box and you are responsible for it. From there, have it done by the deadline.

Dave Tomar (16:19): Now, this one’s really important, and I have to pull attention to the fact that when I read the typos and the grammatical errors in there, I don’t do so to mock this student, I do so to point out that this is a master’s-level student, and this is how their written communication appears. You can’t help but look at that email and say, “This person really lacks the academic qualifications to write the assignment that they’re outsourcing.” It’s an important point that I like to make a lot, which is that this desperation. This is not to dismiss the ethical implications of this conversation, but from a practical standpoint, this guy could not write this assignment, and that’s just a fact.

Dave Tomar (18:58): The rule is this, and this is an important thing to note about these paper-writing companies as well, revisions are important, repeat business is important, satisfied customers are important.

Dave Tomar (20:09): I worked with students through a full course, a full semester, three years of a program, you name it. If you’re working with a student on a thesis, or a dissertation, I know professors always say, “Well, how is that even possible? We’re constantly meeting, and they have to defend this and there’s feedback.” Well, it’s good-paying money because you are basically the student’s just a liaison between you and the professor at that point. Professor gives some feedback, the student brings it to you, and I say, “Okay, well, I got to work on my thesis a little.” That was how that process worked, so repeat business was important. Writer requests were very common. Not only that, but once you start buying assignments and submitting them in somebody’s voice, a savvier student knows not to raise red flags, so sticking with the same writer is usually a good idea.

Dave Tomar (21:24): Yeah. Well, it helps for students that go to school like the one that I did because Rutgers University was so large, and in so many contexts, so impersonal that it was maybe nobody’s looking. I witnessed it enough with my customers at Rutgers that it was a very, very easy thing to get away with when you’re dealing with graders and TAs and the professors teaching the course, but you never have once interacted with this person. That’s a very commonplace thing in a larger school. Now, I’m not saying that is the scenario always, but just as an example of how easy that might actually be to get away with.

Kathryn Baron (22:08): Well, this is a huge business, and I have to say, I was flabbergasted at how many of these companies exist. You list in one of the books, I think The Complete Guide to Contract Cheating, at the end, you have a list of about 470 cheating companies. You make a note that this is just a partial list. They operate like any other business. I looked up one company on the list at random and it made no attempt to obscure what it’s selling.

Kathryn Baron (22:59): Yeah, and I thought, “How do these companies avoid detection when the way they’re just so obvious and blatant with their advertising?”

Dave Tomar (23:12): Well, first of all, you can’t avoid detection because that’s poor marketing. It’s really, visibility is actually extremely important.

…Number two, and most importantly, this is the thing I do my best to impress upon educators at every single turn. It is very, very common and understandable to think of this as this sort of black market for papers.

Dave Tomar (24:16): Sure, it’s a shady business, but it’s not like drug dealing where these people are lurking in the shadows. It is an out-in-the-open business. It operates like an out-in-the-open business.

Dave Tomar (25:20): These are real companies, and they operate real companies and if we think of them as these shady black market/drug-dealing type of companies, then we undermine their danger. I paid taxes when I did this job, they paid taxes. It was very normalized, workaday sort of life with customer service, and everything else. While there are certainly shady companies out there, I think that’s probably true of every industry, those are not the ones that are going to survive in the long run. The companies that I worked for 20 years ago are still there and there is a reason.

Kathryn Baron (35:30): Did you ever hear back on what grades you earned?

Dave Tomar (35:35): No, not really. It’s funny. I know I read an email where the customer requested that they needed to have a certain grade. However, it was our official policy that we didn’t guarantee grades. As a matter of fact, to get back to the legal language, we made it very clear that these were study guides and that they were by no means meant to be submitted in a classroom, and so if you did that, then the consequences were really on you, and if you told me you didn’t like the grade you got, then you have violated the conditions of our agreement.

Kathryn Baron (36:13): Oh, gosh. Well, yeah, that’s kind of like Chegg saying, “This is just to help you understand how to answer the question.”

Dave Tomar (36:19): A hundred percent like. That’s exactly what it is.

Dave Tomar (37:40): As we led with, anytime anybody would ask what I did, I’d say, “Well,” very frankly, “I help students cheat for a living.” And people were just filled with questions about that. It took me a while to connect the dots that “Wow, people don’t realize this goes on.” It is very much out in the open. I was always very much out in the open. The companies are very readily Googleable. It was news to me to find out that people in education specifically were just not aware.

On this episode of The Score, we’re speaking with Dr. Sonny Ramaswamy, president of the Northwest Commission on Colleges and Universities, NWCCU. The Commission is one of seven institutional higher education accrediting organizations recognized by the US Department of Education. Their mission is to ensure that colleges and universities meet standard levels of quality set by the US Department of Education and each state.

 Dr. Sonny Ramaswamy (15:44):

But they’re going to have to vouch that that information is, the veracity of this person that is submitting that information. We want to make sure that it’s not just somebody that’s got some ax to grind or is ticked off with that particular thing. So, all of that including, if there’s any instances of cheating on campus, that is also submitted to us. And then if comes to our office, we go ahead and make that available to that group of evaluators as well. Remember, it’s not a one and done, we have a seven year cycle, we get these annual reports, we may get complaints or input coming any time of the year. When that comes, we actually follow up. We will investigate. My staff will go and investigate.

Dr. Sonny Ramaswamy (16:36):

At any time, you could have complaints coming to us, or anonymous letters come to us, or whatever else, I mean, particularly if it’s some issues related to integrity and cheating and things like that. We’re going to go ahead and follow up on those.

Dr. Sonny Ramaswamy (18:54):

When it comes to cheating, a lot of these things are dealt with locally. Now, if it’s just a one-off, let’s say if it was just one student, I felt sorry for that particular student, he or she got caught or whatever happened, and that they deal with it, there’s a grievance process. There’s all this efforts that really there’s no egregiousness on the part of the institution that they’re dealt with. Then they go on to do whatever it is based on the findings.

Dr. Sonny Ramaswamy (19:23):

But if it’s a huge number of students, like happened at some institutions that you and I know of, if that comes up, then it floats up to us at that level that, “Oh, wait a second. There’s something else going on.”

Dr. Sonny Ramaswamy (24:20):

Because, again, we can’t use hearsay and lack of evidence. You got to submit all of the evidence. If you’re accusing somebody of demonstrating zero or poor academic integrity, then you better have some very significant evidence for that. So, we ask you to go and submit that, it’ll come through our portal. And one of our staff members, we have assigned staff members to each institution, and they look at it. Then, we convene, including our general counsel and myself, we have a conversation. We look at all the evidence that’s been submitted.

Kathryn Baron (25:06):

So, as an accreditor, I think you have a unique perspective on the impact of cheating or on the integrity of a college or university. I’m wondering, when you’re looking at this part of the student success, the fundamental part of that, do you gather information, even if it’s not a giant big incident like Harvard and the hundred students who cheated, but just over the seven years, there’s a number of them that have been accrued. Do you gather that information to say, “Okay, we didn’t have this one giant situation, but we see here that,” I don’t know, “500 students have been sent to this academic integrity commission, because they were accused of cheating.”

Dr. Sonny Ramaswamy (26:02):

Again, I’ve been here as the president of the Northwest Commission now for a bit over four years, coming on almost five years. In that time, I’ve not seen that information. We do require institutions, that’s sort of happening on our 163 campuses. Now, we will… Not in the hundreds you’re talking about. Maybe the occasional one-off here and there.

Dr. Sonny Ramaswamy (27:20):

In terms of cheating, where there’s a wholesale cheating of tens and hundreds of them. I’ve not seen any evidence of that in the last four plus years that I’ve been here. In fact, if I were to go back and, of course, I’ve spoken with my colleagues, so what’s the situation going back eight, 10 years ago? And asking my colleagues that have been here for that length of time, and they have not seen it … has there been any prevalence of cheating on a particular campus? I’ve not come across any.

Kathryn Baron (28:16):

I guess, there are two ways of looking at those numbers. One are, when a student has been caught or is suspected of this and sent to through a formal process, or the student does it and nobody finds out. The reason I’m asking is because, on The Score, we’ve spoken to different researchers and their data is based on student surveys. In those, we’ve had 50% to 60% of students admitting that they cheated at least once. Most of this is probably not in any dataset, because it’s just them responding anonymously on a survey. I’m wondering then, you haven’t seen this as a big problem in terms of the official data, but do you think it’s as large as that? Do you have a sense of whether 50% to 60% of students have cheated at least once?

Dr. Sonny Ramaswamy (29:21):

Yeah. Well, I mean, first of all, you got to look at the nature of the question that is asked and how it’s framed and all that.

Dr. Sonny Ramaswamy (29:28):

If it’s an open-ended question, it’s anonymous and things like that, did you cheat? “Yeah. You know what, it was a closed-book exam, I opened the book or whatever. I’m fessing up.” Versus what happened at Harvard and other places. We’ve had at medical schools and dental schools and things like that, some very significant level of cheating. Also, you and I talked a little bit about this previously, some of those have not been borne out to be true anyways. Okay?

Dr. Sonny Ramaswamy (29:58):

So, the bottom line is, I’m not seeing those particular studies that you’re referring to. Is it possible? Anything is possible. I mean, we’re talking about human nature. I mean, given those studies that have been done by Stanford and other institutions, where they would go ahead and drop a dollar bill or a hundred dollar bill or whatever, you know how many of us don’t just pick it up and stick it in our pockets and walk away. I mean, how many of us are going to just actually take and say, “Hey, sir, did you lose this? I just found this.” Yeah, many of us do it. Of course, we were taught that, right from wrong, and things like that.

Dr. Sonny Ramaswamy (30:31):

This is like, I’m sure it’s happened to you. I go to a grocery store and I’m handed extra change or whatever. I give it back. That’s what I was taught to do. But some of us don’t and some of us do. Going back to this question of cheating, is the cheating a concerted effort to where a whole bunch of students hacked in to, let’s say, a database of questions or changed the grades? Which has all happened. I mean, we’ve seen, what do you call, instances of this. In fact, the Educational Testing Service, which does the tests of English and the foreign language or the GRE folks, that do GRE exams and all that.

Dr. Sonny Ramaswamy (31:59):

If 50% of the students are cheating, they’re doing little bitty things. But what you’re getting at is this, the death by a thousand cuts, if you have this sort of thing that’s going on, what does that say to student education? I mean, do we really want, as the joke goes, do we really want a person that cheated in a medical school to do surgery on your heart? I mean, these are the kinds of things that we need to consider as well. Then cheating is inappropriate or incorrect in any context, small or large, there is no such thing as a small cheat versus a large cheat, cheat is cheat. Yeah, that’s what I’d say.

Dr. Sonny Ramaswamy (33:04):

 That’s true not just of medical schools, and dental schools, and veterinary schools, and in the broadly writ health sciences, et cetera. But you see that sort of a post graduation credentialing taking place, i.e., you pass an exam, board exam, whether you’re an attorney, or a psychologist, or whatever, if you want to go onto practice or you apply that in practice, if you’ve cheated throughout your college career, you’re going to get busted, because you don’t have the knowledge base. That’s a built-in part of the checks and balances that we’ve got.

Dr. Sonny Ramaswamy (33:41):

But somebody that cheats on an English exam, copied, plagiarized or whatever else that happened. And we see that, you and I talk a little bit about this, too. We’ve got technology available today where we can see if the student has plagiarized or not. Also, we’ve got these mills, particularly in English speaking countries across the world, that will sell you an essay on whatever topic you want for 50 bucks or 10 bucks or whatever. You get all these kinds of things. Institutions, the faculty members are already tied up doing a lot of things and now they got to check whether this has been copied, or plagiarized, or somebody else wrote it and things like that. This is all part of this context that we’ve got.

Dr. Sonny Ramaswamy (35:06):

I mean, we all have our ethics statements and our institutions have, they post that on the wall of each room and the professor comes in and reads that out loud, saying, “You will adhere to these principles of being ethical and professional,” and things like that. By the time you come into college, as you know, it’s way too late, it should be happening at home. It should be happening… If you have a home, a lot of students don’t have homes, that, too. But it should be happening in K through 12. It should be happening in kindergarten. And first, and second, in elementary school, that’s where we learned these things about the good and the bad and the ugly that we’ve got of cheating or whatever else that we’ve got. I think, by the time you get to college, these things are already set.

On this episode of The Score, we’re speaking with Kylie Day and Sarah Thorneycroft, leaders in the field of design and implementation of online examinations. Kylie Day is the manager of exams and e-assessments at University of New England, in Australia, and Sarah Thorneycroft is the director of digital education at UNE.

 Kylie Day (03:58):
… we do have a central team and that’s been a feature at Australian universities for a long time. But what we’ve seen at other universities in Australia lately is that’s being distributed back out to academic areas. And I think I would say that’s a loss because I think it requires professional expertise to run what is probably the largest event a university will hold, high stress, high stakes, high numbers of people, really, really quite important.

And to pull that expertise in terms of how do I wrangle 10,000 people without making them cry, to be a little bit cynical, but that’s a skill. How do I communicate with people to achieve compliance with lots of different rules? How do I get people to actually do what they need to do so that everything coincides nicely for everyone and everyone has a good experience and how do I manage academic integrity issues well? I think distributing that out to academics who already have plenty to do it might not be their area of expertise, but to outsource that to them as well. I think you lose something there.

Kylie Day (07:43):
COVID helped us because we were at about 25% online exams before COVID, in the before times. And then we had a very rapid shift to 100% of all exams had to be held online with a 24-hour window in the online proctoring. So that really helped tear the bandaid off. And I think it helped people just take that step that they might not have been keen on doing. What we, my team put a lot of effort into was to make it really safe for them and massive amounts of support for students and for staff, so that nothing was too hard and that nothing went badly. And that’s why we put effort into being on call till 1:00 AM so that there were no stories from students about how they were just left at midnight with no one to help them. And I think that really helped. And when we did have people who wanted to be a bit innovative, we went out of our way to support that.

And so those then became the stories, the good examples that we could say, Hey, your colleague tried this and here are the metrics where we can see that student success increased. Students are happier. Students have more agency over all the demands on themselves. So they’re much more settled and more engaged. And just supporting that in a really safe way with a lot of support. The whole flexibility piece did take a lot of time for people to get their heads around. And I think that exams exist as a cultural archetype, that they’re hard, they’re tricky, they’re secret, they’re tough. You have to turn up or else, all this stuff that people have embedded in their brains about exams. Helping people realize that the way exams have been managed in the past is not necessarily the way exams should be managed and really calling into question every assumption that people have consciously or unconsciously about assessment and exams and flexibility and students. So it really has been a long change piece.

Sarah Thorneycroft (10:45):
Access too is key for students that don’t have to engage in geographical travel to get to locations. That can sometimes be a real barrier for our demographic. So being able to access online in your own home makes a real difference for a lot of students.

Kylie Day (11:02):
We had a student early on who actually rang crying tears of happiness and no one rings, right, to say what a wonderful exam they’ve just had, right? It’s a occupational hazard in our line of work that you only ever hear from people who have a bad time, but this student rang early on in the project when she really realized that the flexibility that she could have. She rang, crying tears of happiness to thank us to say that she had a spinal injury, which meant she was in chronic pain. Traveling was really hard and would make her really unwell with pain. And that she asked for a comfortable chair, but our idea of comfortable chair was not the same as her idea. And we couldn’t provide what she needed in the exam venue. When she realized that she could do, she had three exams in two days and she physically was not going to be able to do that at an exam center, which meant that she wasn’t going to finish her degree, which meant that she wasn’t going to be able to get that job that she had lined up, this dream job.

Once she realized that she could actually choose the timing of her three exams and sit one on a weekend, sitting in her lounge chair, which was much better for her and lay down if she needed to, she realized that she could access those exams. She could finish her degree. She was going to get that dream job that she’d lined up and that moved her to tears and probably moved us to tears a bit too when she rang to tell us that. So exams are an institutional barrier. Traditional exams are an institutional barrier to accessibility.

Kylie Day (13:13):
Certainly easier to get those metrics in an online assessment mode rather than paper. From my perspective, we do a survey after every exam period to say, how was it? Which bits were good, which bits were bad? Why did you like it? Why didn’t you like it? What impact did it have? And we also get various other pieces of feedback. And what we know is that students really appreciate being able to choose a time that suits them. They don’t like having to sit in an exam hall with 300 other people, sniffling and tapping and wobbling their desks. They don’t like having to travel, but I think Sarah can speak on the kind of metrics that you could get that would influence design.

Sarah Thorneycroft (13:58):
So in terms of designing our approach, getting metrics around when students choose to have their exams is really useful, because you can actually see the uptake of flexibility and understand when you make this available to students, how are they making use of it? And thus, to what extent you want to make sure you’re designing your assessments to maximize that capacity. And some of the other metrics, I know that some of the ones that we use a lot are around things like the test taker experience. So this isn’t necessarily about the design of assessment. A lot of the most effective actions you can take for assessment design are the things that don’t look like assessment design. Metrics around the test taker experience in terms of satisfaction, technical issues, academic integrity issues, the incidents of actual confirmed breaches and that kind of thing.

(section skip)

When you’re talking about an academic or you probably use the term professor, who’s talking to a student who had a bad experience in the exam, that’s really easy to understand as oh actually online exams are bad, but understanding that out of 10,000 exams, 85 to 90% of students are having a really positive experience

Sarah Thorneycroft (18:26):
The intangible costs are an important part of the conversation. In terms of dollars for instance, it’s reasonably more expensive than our learning management system, just as an example. But the key thing is because human individualization, human proctoring is a key part of our strategy. It’s not a platform cost it’s people, it’s people cost. So I think it’s important to contextualize that way is that it’s not a really expensive piece of technology. It’s actually a part of a whole ecosystem and it’s paying for the human experience.

On this episode of The Score, we’re speaking with Kylie Day and Sarah Thorneycroft, leaders in the field of design and implementation of online examinations. Kylie Day is the manager of exams and e-assessments at University of New England, in Australia, and Sarah Thorneycroft is the director of digital education at UNE.

Kylie Day (07:06):
… if we put our effort towards the student’s feelings and attitudes and decisions before the exam ever starts. So, in the same way as a community safety program or a community health program, you would do population-wide communications to talk about the risks involved, expected behavior, alternatives to risky behavior. In the same way that the highway patrol police are not expected to catch every single person who might speed, they have a presence and that serves a purpose to make it risky, to dissuade people from speeding.

Kylie Day (07:49):
But that’s not the only thing that one would do if you wanted to reduce say the road toll or the incidents of people breaking the road rules, you would expect to have a community safety program and narrative happening along with that. And when we catch people who might be cheating it’s not a good outcome for them, it’s not a good outcome for us as an institution.

Kylie Day (10:05):
… we see flexibility and easy flexibility as a key factor in letting students manage their own pressures in ways that allows them to succeed and not have to cheat to do that.

Sarah Thorneycroft (10:17):
That changes the cost benefit analysis.

Kylie Day (10:20):
So, we work with online exam proctoring service where our exams live in our learning management system, but we have highly skilled and trained supervisors who can… They have a view of the students’ screen. They can use software to lock down that student’s computer in ways that we ask them to, and they can also watch the student.

Kylie Day (12:01):
And that’s the first thing that our faculty said when we started having conversations about flexibility, flexibility is an F word, if I can be cheeky. Students will cheat, and so that’s when we talk about design. The assessment needs to be designed in the mode or in the context of the mode that it’s held. It should not be that we are just doing paper exams on a web page, it’s a whole second order change.

Kylie Day (12:31):
So, the design features might include using a question bank. So you would have just enough. I get a different question one to you. It’s still the same topic, same degree of difficulty. But if I say, “Hey, what did you put on question one?” That kind of collaboration will be disrupted because we get different question ones.

Sarah Thorneycroft (15:12):
This is where it’s really useful to help people make comparisons between the paper examination paradigm in which somebody is watching them, and often in more embodied ways of walking up and down and patrolling the physical room that people are located in. But we’ve also discovered, because online the proctor and student relationship is one to one, whereas in an exam hall it’s one to many. Yes, that proctor is watching because that’s the cultural condition for examinations that we’ve agreed on regardless of where they’re held.

Sarah Thorneycroft (15:49):
But the proctor can actually also provide support in situ, which can be both technical support or general encouragement. And we’ve had a lot of comments come through student evaluation that actually talk about how helpful and supportive the proctor was. So that’s one of the key reasons that we focus on human invigilation, not AI only invigilation, because of that personalized element and the ability to also provide benefits, not just stress and monitoring.

Kathryn Baron (22:57):
Do you have online practice exams to help students as well? I thought I had read that.

Kylie Day (23:05):
We do, and that’s one of our favorite things. We call it a try it out exam. And you have to book it, it’s supervised. You have to follow the rules, but it’s got questions like, Hey, did you know this is where you can see the countdown clock on your screen?

Kylie Day (23:24):
Or a question that suggests that you change the batteries in your wireless mouse or keyboard before your exam and do all your windows updates. It’s instructional around, how do I have a good time in my online exam? It has a thing on draw us a graph, which you can do, showing the correlation between the amount of caffeine that you consume compared to the amount of assignments you have due. So it’s intentionally lighthearted, but it allows a student to work out what buttons do I have to push? How does this thing work out? What does it feel like? What does it look like? What do I need to do in my own space to conform to exam conditions? And will my computer actually sustain the technical requirements and the bandwidth that I need?

Kylie Day (30:31):
What rings in my head a lot is the phrase demonstration beats explanation. So just starting with people who wanted to come and play really and making sure that went really well. Those people then become champions. You can publicize details and say, “You know what? We can talk all we like, but we tried it and this is what happened.” And having evidence to show people.

Kathryn Baron (32:44):
What are the concrete steps that these other universities can take?

Kylie Day (32:48):
One of the pieces of advice I give to people at other universities is that they should not consider it to be an IT project, nor should it be seen as a admin logistics project. That those pieces are really important, but the structure of the team I think is one of the reasons for our success in doing it.

Sarah Thorneycroft (33:10):
Yeah, I think I tend to frustrate my sector colleagues who hope that there might be a nice recipe of concrete steps and you just follow the steps and then it works, and it’s all good. And they come and talk to us and we are like, “Oh actually it’s a cultural change piece.”