Why LLMs are not replacing search

There has been a lot of talks recently about LLMs particularly ChatGPT and now Bard with many people starting to interact with this technology. There are few opinions that it’ll take over how we search, but I don’t feel traditional search is going anywhere, few of my reasons to believe so -

1. LLMs are impressive but not robust - When users first start interacting with these conversational models they are very impressed by the responses it generates, but slowly people can start to see that its not correct always and the results these models spit are not necessarily facts. Also users can make these conversational agents hallucinate things that don’t exist just by conversing with them. And once users start feeling they are getting wrong facts the trust in such models will decrease.

2. No concept of choice - These models spit out a single result, users don’t have any other choices. Many times users want to be able to look through multiple results to gain information about different aspects of a single search but a single essay answer enforces what the model believes on the user. While this can be useful in one off queries like a simple question where user just want a single answer its not useful for research purposes where people need to dig deeper into concepts. From the keynote of bing and from bards internal usages these essays will also link to internet sources as well but there is no utility for researchers who want to open multiple links anyways. Also Google does show results for quick questions on the top from actual websites which have human content so what the LLMs add to search is very little utility.

3. Internet is not only text - People don’t search things to read texts only, most of the fun in reading any information or article comes from multimedia and formatted text. Today people don’t search and expect a reading list for how to do things, instead they expect videos, images explaining how it’s done, bold heading highlighting important piece of information. Just imagine you want to search how to cook an omelette, would you find it better to watch a video or read an article with images or read bullet points on how to do it.

4. People don’t know what to search - Most of the time for humans its not easy to define what exactly we want so many searches just contain tokens, as users discover more information they get clarity on what exactly they are looking for and what more they want to look at, a single essay search doesn’t solve it.

5. Difficult to learn from new data - These models are not continuously learning, while there has been some talks that Bard will update over internet still it’ll be very hard to update such a big LLM every seconds or even every day. So people will naturally miss out on the new information.

6. Operating costs - ChatGPT is around 175B parameter model, which means if this model is to be loaded it requires 700 GB of memory. Obviously most companies will use a distilled version of these models but still the operating costs will be much more higher then a simple search, and when this scales to billions of searches every day it becomes much more huge. Though eng teams are always good at reducing these costs so it shouldn’t be a big issue and less used engines like bing may bet more money just to acquire more users.

I think in the end this tech will just become like Tesla FSD or some old projects like Google Stadia. Everybody talks about it but nobody trusts it. To be trustable these tech have to not be just 80% reliable but 99.999999% (6 9s).

I can see students copying from this tech but that is still a bad idea because it can spit out wrong information or mediocre results. So many people will be getting mediocre grades but even worst they won’t have any understanding of when they are getting fooled by.