How to cheat
Proctorio
Can AI Proctors Detect Online Exam Cheating? | Automated Online Exam Proctoring
Sharing buttons:
it's final season and many of you are
probably studying for your exams right
now
in fact i have to go back to studying
for mine once i finish filming this
video
but in an era of online learning our
exams may look a little
different instead of using tiny desks in
lecture halls or sitting in classrooms
or getting more take-home tests and
online exams and with that has come an
increased concern about cheating
after all there's no proctor or teacher
to watch you take the test
right well several companies have
developed ai-based solutions to those
concerns
from monitoring your behavior through
your webcam to tracking your keystrokes
these companies claim that they can tell
whether or not you're cheating during
exam using
automated remote proctoring but how true
is that and how does it even work
let's find out if you're new here i'm
jordan and welcome to the wonderful
world of artificial intelligence
consider subscribing if you're
interested in learning more and smash
that like button so that i know that you
enjoyed this video
also let me know in the comments if
you've taken an exam with online
proctoring ai based or
otherwise all right let's get into it so
online proctoring has actually been
around for way longer than i thought
when i set out to make this video around
10 years now and the original approach
was pretty straightforward you would log
into your exam using the online
proctoring software
and they would connect you via video to
a remote proctor who would watch you
take the test
ideally they would be familiar enough
with the exam to be able to answer any
questions you had about it but
just like an in-person proctor they
primarily served as a cheating deterrent
subsequent versions of this software
incorporated the ability to track you
as you took the test from monitoring
your video to monitoring your online
activity
are you googling the questions did you
just copy and paste something into your
essay all that activity would be visible
to the proctor who would use it to
determine whether or not you were
exhibiting cheating behaviors
from there it wasn't a huge leap to
automating the role of the proctor
after all the software already records
and monitors all of your activity
so the next step would be to train an
algorithm to detect whether or not an
activity would be considered cheating
other functionalities including facial
recognition and location tracking were
also added to make sure that you were in
fact the person who was supposed to be
taking the test
and some companies even offer audio
analysis to make sure that you're not
talking to someone else off screen who's
feeding you the answers
as well as a manual offline human review
to catch any errors that the algorithm
may have made
and now there are several companies that
offer these services predominantly to
universities they've been picking up
steam since well before coronavirus
forced everyone towards online learning
after all if you've ever been in a huge
lecture something like psych 101 at
cornell that has like a thousand people
in it
you know that it's functionally
impossible to determine whether
anyone is cheating because there's so
many students and so few teaching staff
and proctors
additionally exams are often offered in
the evening so
for commuter schools it may make more
sense to offer an online exam so that
students don't have to either stay late
on campus
or return to campus in the evening in
these scenarios automated online
proctoring is extremely useful it takes
the burden off of teaching staff to have
to keep track of all of the people
taking the test
can simultaneously monitor hundreds of
students
and allow students who don't necessarily
live on campus to take the exams from
home
but how does the software tell if you're
cheating well these are all proprietary
algorithms so i obviously can't pull
back the curtain
to show you but there are two
overarching strategies that i've seen
used first up is identity verification
or making sure that the person who's
supposed to be taking the test is
actually the person taking the test
this is often done at the beginning of
the exam usually by comparing your face
in the video feed to
a photo on record from your student id
but can also be done throughout the exam
second is activity monitoring this
includes anything from eye tracking to
making sure that you're focusing on the
screen
to disabling copy paste so that you
can't copy something from outside of the
exam
into it to analyzing your audio to make
sure that someone else isn't telling you
the answers
anything deemed by the algorithm to be a
suspicious behavior will trigger an
alert
either for the instructor for the
student or both the outcome of that
alert could range from anything from a
human review
of the alert itself to disciplinary
action on the part of the instructor
in short the software detects cheating
by being trained either through
simpler algorithms such as pattern
matching or through more complex
algorithms like deep learning algorithms
to track your identity and your activity
throughout the exam
and make a continuous prediction as to
whether or not you are cheating
now you might be wondering how an
algorithm decides whether an activity is
suspicious
the answer to this question honestly
deserves its own video as past work and
machine learning has shown that it's
very hard to make a training set that
doesn't have bias to it
and it can be very hard to extrapolate
the results of your model
to the real world as an example i'll
include a link in the description box to
a pre-print that tried to detect
deception using gate the authors faced
considerable criticism when this
preprint was originally uploaded to
archive and they've actually since
updated it to address some of the
limitations of their technology a bit
better
because the training data that they
generated was extremely contrived and
didn't seem to have a clear real world
parallel
and in fairness this is a challenge for
most of the work in this field because
determining the intent behind an action
is difficult for
humans let alone algorithms so the
software likely relies on a set of
training data that has been labeled
where some examples represent cheating
and some
do not it's hard to know more past that
now let's jump back to the question in
the title of this video
how well can automated online proctoring
identify cheating behaviors
obviously most of these companies as
well as the schools that use them claim
that they can identify cheating
behaviors better
than you can in normal in classroom
exams as well as in online exams i would
believe that claim to be true but would
also
question any accuracy statistics that
come with it because
it's unlikely that you can reliably
quantify how many people actually
cheated on a test more broadly i think
that online proctoring is something that
we're all going to have to get
comfortable with going forward
i don't think it's universally good or
bad after all there's a reason why we
normally have people proctoring exams
when we take them in person so it
doesn't surprise me and it does make
sense that you would have some
analogous system that can perform that
for online education
and automated online proctoring is
particularly useful right now when we
literally cannot
have proctors in the room with students
however it also has faults
students of color have complained that
the facial recognition system can only
identify them in specific lighting which
you may not have
access to wherever you're taking your
exam and these systems can require more
bandwidth than
students may have access to if you can
only take your online exam with the
library you might not be able to
download the necessary software to even
take the test
granted a lot of these things are also
challenges for online learning overall
there have also been some concerns about
data privacy after all the system is
recording while you take a test
and in the us we have a federal law
called ferpa which basically requires
that universities
maintain the privacy of your educational
records but i haven't found any claims
of data mismanagement or misuse so far
and it does seem like these companies
are doing their best to comply with
ferpa
now i'll say i personally haven't used
any of these systems in any of my online
exams so far
i have one final this semester and while
it is online as far as i know
it is not monitored in any way but in
short i think a lot of the work that
an ai proctor does in detecting online
cheating
isn't actually in the algorithm itself
it's in the psychology of having
someone or something watch you while you
take a test
and while this version of proctoring may
be different from what we're used to
it's likely not going away anytime soon
so you definitely shouldn't take it as
like a
challenge to fool the ai or anything
it's
not worth it if you want to learn more
about how artificial intelligence is
used in schools you can check out this
video up here
and if you'd like to subscribe to my
channel then you can click
right here to do that if you'd like to
learn about my phd life
you can follow me on twitter and
instagram thanks as always to all of my
patrons and i will see you guys next
friday
bye
