Google’s AI Overviews might seem tempting at first but a new study is making users cautious of how you shouldn’t be blindly trusting the feature.
As per research experts, it failed to accurately respond to finance queries, 43% of the time. This means the majority of the time, it was giving false or misleading facts. So as you can imagine, with such a high inaccuracy rate, it might not be as reliable as the search engine giant claims.
The news comes to us thanks to a report by The College Investor which found significant errors in Google’s AI-generated replies and summaries. The theme in question here was finance-related. Out of 100 of the questions put forward in the search, 43% were wrong and misleading.
The study also looked at different aspects of finance including investments, loans, taxes, banking, and credit. Despite the diversity on display, the replies proved that 57% were right and 43% were wrong. 12% were completely misleading and 31% were missing critical elements in the response.
The researchers also noted how the search engine failed with those finance categories that are more pronounced and detailed than others. For instance, taxes, loans, and investments. Surprisingly, a lot of the data was outdated. This meant all basic queries were answered well but no additions of new trends or changes to policy were considered.
Anything complex, time-sensitive, or specific to a certain location was not handled too well by Google AI, the authors explained. However, when you ask for basic definitions, recent trends, or general inquiries, it does a great job.
So what does this mean? Should you rely on Google AI Overviews or not? The answer is ‘at your own risk’ because straightforward and basic will get you in the right direction. However, anything complex will not.
The founder of The College Investor says it’s alarming to see such results. A lot of people do put blind faith in Google as their companion when they’re not sure. What do you think?
Image: DIW-Aigen
Read next: Are Big AI Models Compliant Of EU Rules? The Answer In This Study Might Surprise You
As per research experts, it failed to accurately respond to finance queries, 43% of the time. This means the majority of the time, it was giving false or misleading facts. So as you can imagine, with such a high inaccuracy rate, it might not be as reliable as the search engine giant claims.
The news comes to us thanks to a report by The College Investor which found significant errors in Google’s AI-generated replies and summaries. The theme in question here was finance-related. Out of 100 of the questions put forward in the search, 43% were wrong and misleading.
The study also looked at different aspects of finance including investments, loans, taxes, banking, and credit. Despite the diversity on display, the replies proved that 57% were right and 43% were wrong. 12% were completely misleading and 31% were missing critical elements in the response.
The researchers also noted how the search engine failed with those finance categories that are more pronounced and detailed than others. For instance, taxes, loans, and investments. Surprisingly, a lot of the data was outdated. This meant all basic queries were answered well but no additions of new trends or changes to policy were considered.
Anything complex, time-sensitive, or specific to a certain location was not handled too well by Google AI, the authors explained. However, when you ask for basic definitions, recent trends, or general inquiries, it does a great job.
So what does this mean? Should you rely on Google AI Overviews or not? The answer is ‘at your own risk’ because straightforward and basic will get you in the right direction. However, anything complex will not.
The founder of The College Investor says it’s alarming to see such results. A lot of people do put blind faith in Google as their companion when they’re not sure. What do you think?
Read next: Are Big AI Models Compliant Of EU Rules? The Answer In This Study Might Surprise You
