The internet is littered with examples of ChatGPT passing certain difficult exams with flying colors. For example, many users cited ChatGPT managing to clear the US bar exam that gives lawyers the right to practice in the state of their residence. This has led many to assume that ChatGPT could actually replace lawyers in the real world, but in spite of the fact that this is the case the AI chatbot has ended up struggling with other types of exams, mainly those that involve high level math.
With all of that having been said and now out of the way, it is important to note that ChatGPT recently failed the CPA exam, also known as the Chartered Public Accountant exam. This is the exam that must be cleared in order to get registered as a Chartered Accountant in the US, and ChatGPT’s failure to answer all of the questions seems to suggest that it is not as advanced as people think it is.
Indeed, the chatbot’s performance was so bad that it tried to invent facts out of thin air to try to pass. This obviously wasn’t going to lead to any positive results, and it indicates that ChatGPT does not have what it takes to tackle exams that don’t involve linguistic analysis with all things having been considered and taken into account.
One thing that had a strong impact on how well ChatGPT performed was the type of question it got asked. The chatbot did far better with simple yes or no answer questions, getting 68.7% of them right. It also did well with multiple choice questions, obtaining a success rate of about 59.5%.
However, when it came to questions that required short form answers, ChatGPT was found lacking. It should be mentioned that this chatbot is still in a relatively early stage of its development. Students are still managing to outperform the chatbot in a wide array of areas, which is why it is unlikely that ChatGPT will serve any professional roles in the near future. Upcoming upgrades and advancements to the tech might allow it to get to the next level.
H/T: BYU / Study.
Read next: What’s the Best Tool for Detecting AI Generated Content?
With all of that having been said and now out of the way, it is important to note that ChatGPT recently failed the CPA exam, also known as the Chartered Public Accountant exam. This is the exam that must be cleared in order to get registered as a Chartered Accountant in the US, and ChatGPT’s failure to answer all of the questions seems to suggest that it is not as advanced as people think it is.
Indeed, the chatbot’s performance was so bad that it tried to invent facts out of thin air to try to pass. This obviously wasn’t going to lead to any positive results, and it indicates that ChatGPT does not have what it takes to tackle exams that don’t involve linguistic analysis with all things having been considered and taken into account.
One thing that had a strong impact on how well ChatGPT performed was the type of question it got asked. The chatbot did far better with simple yes or no answer questions, getting 68.7% of them right. It also did well with multiple choice questions, obtaining a success rate of about 59.5%.
However, when it came to questions that required short form answers, ChatGPT was found lacking. It should be mentioned that this chatbot is still in a relatively early stage of its development. Students are still managing to outperform the chatbot in a wide array of areas, which is why it is unlikely that ChatGPT will serve any professional roles in the near future. Upcoming upgrades and advancements to the tech might allow it to get to the next level.
H/T: BYU / Study.
Read next: What’s the Best Tool for Detecting AI Generated Content?
