Please contact us if you would like the full transcript.

On this episode of The Score, we’re speaking with Dr. Alexander Matros, a professor in the Darla Moore School of Business at the University of South Carolina, and Eren Bilen, Assistant Professor of Data Analytics at Dickinson College. Both are chess players and in their September 2020 study, online cheating, amid COVID-19, they examined the connection between cheating and online chess and the extent of online cheating in universities. The report describes how the International Chess Federation and the Internet Chess Club deal with cheating and suggests what universities can learn from that. Welcome to The Score.

Eren Bilen (05:08):

Yeah, the 2020 AP exams were the first time that these AP exams were given online because of, this was basically because of COVID. And so, what happened was this, so if you look at Google searches, and this is public information, you can just access this information, easily. What you see is this, so the 2020 AP exam for the math subject was given on May 12. This was in the afternoon Eastern time. So, we had 2:00 PM on May 12. And so, if you look at some of the keywords related to math concepts, such as derivative, integral, critical points, inflection point, things like that, you’ll see a spike, exactly 2:00 PM, and then following 3:00 PM, and so on, the spike basically disappears.

Eren Bilen (06:03):

And so, the next day, on May 13, it was the English literature subject. If you do a similar study, so you check, this time instead of checking math related keywords, you check literature related keywords. So, you can do imagery, literary techniques, diction, things like that. You get the spike, exactly at 2:00 PM on May 13. This is again the time of the test.

Eren Bilen (06:29):

And then last, you can even check physics, for example, this was the next day on May 14, but this time not 2:00 PM, it was 4:00 PM in the afternoon. And you get this spike on physics related keywords at exactly 4:00 PM on May 14. So, it looks like students basically do some Google searching in order to find the answers, was this helpful? Yes, no, we’re not sure, but at least students tried.

Kathryn Baron (06:57):

At least they tried to cheat. So, was this an unproctored online exam?

Eren Bilen (07:06):

That is correct. It was unproctored.

Eren Bilen (10:21):

Sure. Yeah. So, in the data, so we were quote on quote, “lucky,” in the sense that we had one special tool that enabled us to basically pinpoint what’s going on, what’s going on? The issue was this, so we looked at the time the students took to answer their questions. So we gave them basically a test with 20 questions. And these questions were not multiple choice. So, the students had to basically enter numbers using their keyboards. And what we saw was that some of the students had very strange timings.

Eren Bilen (11:02):

So, for example, on a question that you will expect a student to take on average, let’s say five minutes, the student gave an answer in seven seconds. You can say, “Okay, this is one occasion. The student just input a random number or something.” That was not the case. That was the correct answer. So, for example, the correct answer was let’s say 347. So, a student was able to pick that number 347 in less than 10 seconds. And this kept going and going. So next question. Similar. Third question, again, somethings similar. So, it kept on going for 20 questions.

Eren Bilen (11:40):

So, the overall time the student took to complete the exam was about 10 minutes.

Kathryn Baron (12:30):

But Eren, in seven seconds, how did they cheat, could they actually look something up online that quickly?

Eren Bilen (12:36):

So, you cannot do this in seven seconds. So, what we believe that students had was that they had the answers from other students who volunteered to take the test before they did, and they gave them the correct answers. And then you basically had a list in front of you with question names and then the correct answers. They basically looked at this test, the answer sheet, and it probably took them on average, 10 seconds to be able to figure out that was the question that they were seeing on the screen. And basically, they inputted the correct number using their keyboards. So, looks like this on average takes 10 seconds.

Kathryn Baron (15:51):

You earlier and mentioned fairness. And it does seem that this issue raises some huge ethical issues around fairness, because a student who works very hard to get good grades could very likely do worse in a class because that student didn’t cheat. And even though teachers and professors know from say homework assignments and classroom participation, which students are studying, what can they do when the test results don’t reflect that because of cheating?

Dr. Alexander Matros (16:21):

Yeah. I think in a sense, you ask very, very important questions. So, in a sense, during this pandemic during the whole year, so we had some expectations, we had some, you can call this social norm, so what we expect. So, let’s say people would come to a class and they would take a test and then you can rent them based on these results. And everything is from this point of view, more or less fair.

Dr. Alexander Matros (16:45):

Now, if you take a test at home, especially if it’s not proctored, so nobody knows who took this test. And then the situation now is such that we have another social norm when if you have these expectations if you have these beliefs that everybody else is teaching. So, this immediately puts you in situation when just, you might be the best student, but you feel that you have no chances to compete with this, as a students, unless you cheat as well.

Dr. Alexander Matros (17:15):

And we just move from what is called, maybe [inaudible 00:17:18] When you have these expectations, these are self-fulfilling expectations. And now if everybody cheats, everybody expects that. And then they play according to this morals.

Dr. Alexander Matros (20:06):

So, if you put a little bit of effort trying to check them, so maybe they would just abstain from this kind of behavior. And then this even simple monitoring can remove a lot, a lot of cheating. So, it would definitely not remove all cheating, but it would remove simple ones. So, for students like you describe, so who would actually prepare their rooms, you cannot eliminate that, but they put so much effort. So, if they would study instead, they would do so much better.

Dr. Alexander Matros (22:38):

But online, you have some clues, it’s never direct evidence. It’s only like indirect evidence. So, you can say, “okay, so the student took a test and finished this test in 5 minutes or 20 questions. It was multiple choice. And their answer is perfect.” But then is it possible? Yes, it’s possible. Because again, you can also win a lottery, so you just put the number and then you just like and you won. So, a student had a good day, so answer everything correctly. So, and then it’s possible. So, you cannot say this was impossible. So, student guess correctly, so perfect.

Kathryn Baron (24:39):

But do your colleagues feel that there is a lot of cheating going on in their classes or do they feel that their students, I’m just wondering is there a consensus that, “Yeah it’s going on,” or are they sort of in the dark about it?

Dr. Alexander Matros (24:54):

No, I think this is clearly a consensus that was cheating and what people will do. So they would try to find like some ways how deal with that.

Dr. Alexander Matros (26:14):

In my first 10 years, I had zero cases. And during pandemic yeah, I did report several cases

Eren Bilen (32:33):

Yeah. We have to move from a bad equilibrium room to a better one absolutely. I absolutely agree. In order to do that, we need to use some sort of proctoring. So, it could be in person proctoring, it could be live proctoring, but with the use of proctoring, we can basically move from those bad equilibria to the better ones. Because in a bad equilibrium basically, you give an option to student to cheat, but if you’re using proctoring, then hopefully 99% of the time, student won’t be able to cheat. So that’s the key takeaway that I want to point out.