ChatGPT passes the Dutch exam (and fails in no time)

This is called “the ultimate test” Marc Ostendorp His experience. He asked ChatGPT to take the central VWO final exam in Dutch, and wrote about his findings on Neerlandistiek.nlan online magazine about Dutch language and literature.

The exam, which was administered to thousands of pre-college students earlier this week, consisted of a number of opinion pieces from newspapers and magazines, and students had to answer questions about the piece. Oostendorp, he writes, “entered the texts and then posed the questions to ChatGPT.” He then “always checked responses as closely as possible against the official response template”.

Not a terrible failure

According to the professor, it is not yet possible to determine exactly what ChatGPT would have obtained for a grade: no standard has yet been agreed, it is still too early for that.

But: ChatGPT got 33 points out of a total of 60 points. “She probably would have failed,” concludes Oostendorp, but just in time. “She didn’t get a terrible failing grade.”

What is this?

ChatGPT is based on an algorithm trained on large amounts of text, so it can learn how people talk and react. When you send a message, ChatGPT analyzes it and provides an appropriate response. This happens in real time, so it looks like you’re talking to a real person. It can be used for all sorts of purposes, like having conversations with friends, practicing a foreign language, or writing texts. It is also useful for companies looking for a way to communicate quickly and efficiently with their customers.

There are also extenuating circumstances: ChatGPT is not able to read PDF files, so Oostendorp had to convert the articles to another readable format. In doing so, he deleted the line numbers, “so the information was missing”.

Also, paragraphs are numbered in exam texts, and ChatGPT didn’t understand that either, says Oostendorp. “As a result, she probably missed crucial points,” he wrote. “She didn’t understand the structure of the exam.”

Thus, most of the errors ChatGPT makes relate to “exam skills”, not reading skills, on which students are tested and scored.

Oostendorp also says he’s not surprised that, had ChatGPT been trained with exams from previous years, he would have easily passed. “Anyway, there are quite a few answers that are incredibly good.”