A new study compiled by top researchers at Stanford University is challenging a very popular concept related to Large Language Models.
LLMs were usually considered to be great at taking in large documents and producing an accurate understanding of what exactly is taking place, churning out search results of a higher degree. But you’ll be amazed at how that might not be the case.
The study published recently claims that the exact opposite is actually true. Not only do such models fail to attain enough access to the document, but they’re also not great at making use of the right data provided to them when bombarded with long context windows. The latter is the term used for the text length processed and the response proved at any point in time. So it’s like thinking of it as a form of memory when conducting a chat with the AI-powered tool.
This study really did manage to attain a lot of attention across the board after it was published as so many developers started to experiment with it and took on the assumption that the bigger the context window, the better the accuracy and performance. And that would make it so much more useful across so many domains.
But now, the opposition is growing to how longer text documents aren’t great at enabling comprehension of a great degree and any queries put out by the user aren’t going to be given a response as one would hope.
With that being said, we’re still seeing so many top firms dealing with LLM produce more variations in models that are said to work great with large context windows. For instance, we’re talking 100k tokens for producing a summary of a long chat and putting out drafts of the sort.
When you go back to the study, it proves how so many facts being outlined about long context windows are getting super flawed. The chance to search and correctly analyze data is not up to the mark as per the researcher’s findings.
In reality, the models performed at their best when the right data arose at the start or toward the end but when the length was extra long, the performance seen at the middle of the document was poor for longer length documents. Meanwhile, the performance would just drop down when the content became longer.
As the study gains popularity and acceptance, many leading tech CEOs are using it while citing evidence about how you don’t need to stuff a whole document inside Windows to carry out a search.
So, what’s the solution as the problem seems to be a major one to deal with right now.
Experts feel the answer is a semantic search. There is no longer the need to stuff in documents. Instead, it’s a great idea to make use of vector databases that would stay viable for a long time. One common example is Pinecone who is providing such services to make the lives of users easier.
Pinecone is designed for the sole purpose of search. However, another point worth a mention is how this new study by Stanford is not claiming that putting documents of long length into the context window will prevent it from working.
The end result ultimately depends on the likes of the content in question that is being analyzed. As it is, LLMs aren’t too great at differentiating between several things at the same time that happen to be linked. However, they are great at noting down anything relevant, especially when the rest happens to be far from the topic of interest.
Read next: ChatGPT In The Spotlight: Where The Popular Yet Controversial App Stands Today May Surprise You
LLMs were usually considered to be great at taking in large documents and producing an accurate understanding of what exactly is taking place, churning out search results of a higher degree. But you’ll be amazed at how that might not be the case.
The study published recently claims that the exact opposite is actually true. Not only do such models fail to attain enough access to the document, but they’re also not great at making use of the right data provided to them when bombarded with long context windows. The latter is the term used for the text length processed and the response proved at any point in time. So it’s like thinking of it as a form of memory when conducting a chat with the AI-powered tool.
This study really did manage to attain a lot of attention across the board after it was published as so many developers started to experiment with it and took on the assumption that the bigger the context window, the better the accuracy and performance. And that would make it so much more useful across so many domains.
But now, the opposition is growing to how longer text documents aren’t great at enabling comprehension of a great degree and any queries put out by the user aren’t going to be given a response as one would hope.
With that being said, we’re still seeing so many top firms dealing with LLM produce more variations in models that are said to work great with large context windows. For instance, we’re talking 100k tokens for producing a summary of a long chat and putting out drafts of the sort.
When you go back to the study, it proves how so many facts being outlined about long context windows are getting super flawed. The chance to search and correctly analyze data is not up to the mark as per the researcher’s findings.
In reality, the models performed at their best when the right data arose at the start or toward the end but when the length was extra long, the performance seen at the middle of the document was poor for longer length documents. Meanwhile, the performance would just drop down when the content became longer.
As the study gains popularity and acceptance, many leading tech CEOs are using it while citing evidence about how you don’t need to stuff a whole document inside Windows to carry out a search.
So, what’s the solution as the problem seems to be a major one to deal with right now.
Experts feel the answer is a semantic search. There is no longer the need to stuff in documents. Instead, it’s a great idea to make use of vector databases that would stay viable for a long time. One common example is Pinecone who is providing such services to make the lives of users easier.
Pinecone is designed for the sole purpose of search. However, another point worth a mention is how this new study by Stanford is not claiming that putting documents of long length into the context window will prevent it from working.
The end result ultimately depends on the likes of the content in question that is being analyzed. As it is, LLMs aren’t too great at differentiating between several things at the same time that happen to be linked. However, they are great at noting down anything relevant, especially when the rest happens to be far from the topic of interest.
Read next: ChatGPT In The Spotlight: Where The Popular Yet Controversial App Stands Today May Surprise You
