ChatGPT's Forced "Auto" Internet Search Sucks: Give Users an Offline Mode
Auto internet search within ChatGPT degrades output quality
I’m not sure why nobody else has complained about this (that I’ve seen) so I’m bringing it up.
I absolutely hate the forced “auto” use of internet search (SearchGPT) in ChatGPT – there should be an option to use ChatGPT without “auto” internet search (i.e. 100% offline-mode).
I don’t even hate SearchGPT (it’s decent), just hate that OpenAI is forcing usage of SearchGPT within ChatGPT whenever it thinks internet searching is optimal (without any offline-mode option).
SearchGPT Rollout Timeline
I should note that I was in the beta testing phase of SearchGPT and stopped using it after a few days because I found it underwhelming relative to Perplexity Pro.
Announcement & beta testing: OpenAI announced the integration of web-browsing capabilities as part of its GPT-4 updates in March 2024.
Full SearchGPT rollout: After beta testing, SearchGPT rolled out widely in April 2024, specifically for ChatGPT Plus and Enterprise subscribers.
ChatGPT integration: As of October 31, 2024 – SearchGPT became fully integrated with ChatGPT, allowing real-time information access directly within conversations. ChatGPT began defaulting to “internet search” whenever it sensed users would be best served by an internet search (relative to no search).
I keep reading that “SearchGPT” is a game changer, amazing, etc. because it gives ChatGPT real-time, fully updated information.
In my experience, SearchGPT (ChatGPT + internet) is decent but the automatic usage of SearchGPT within ChatGPT is often times complete garbage.
This one simple alteration has turned me from a ChatGPT Enjoyer to a ChatGPT Hater in <1 month after integration.
Okay maybe not a full-blown hater, but I really dislike the fact that I no longer have a zero search (fully offline) mode.
If OpenAI simply added an icon to click that would allow users to disable the internet-search function (similar to what they already have for search with the globe icon), I’d be cool with ChatGPT again.
Forced “Auto” Internet Search Sucks in ChatGPT…
The only feature that made me start to hate ChatGPT was forced usage of internet (SearchGPT) in responses.
No zero internet search (offline) mode: There is currently no option to use ChatGPT without automatic internet search. Based on the specific prompt, ChatGPT is programmed to automatically use the internet. Yes you can include “don’t use the internet” or “no internet” in prompts – but adding this to every prompt is damn annoying. Sometimes ChatGPT defies this request.
Defaults to internet when offline responses would be superior: Many times SearchGPT automatically uses the internet when offline responses would be objectively better for users. This is for questions related to logic/rationale and internet is unnecessary. Using the internet for these types of queries degrades output quality (its citing online opinions instead of thinking things through on its own).
All OpenAI needs to do to make me (and probably others who hate this forced search) happy is provide a 100% zero-internet search mode option with the click of an icon in the query menu.
If they aren’t going to do this, they should provide both an offline and internet-based response side by side or something – giving users the option to pick the one they prefer (like they’ve been doing with new models).
Part of me thinks they’re forcing internet usage because they’re refining SearchGPT and want it to be used as much as possible to find errors, bugs, and get feedback, etc. – but I’m not sure if this is accurate.
If it isn’t accurate, I’m wondering why they give users no option to turn off “Search” or “Auto Search.”
Why not switch to Anthropic’s Claude? Because Claude Sucks (mostly due to its insanely low usage limits).
If Claude had the same usage limits as ChatGPT at the same price-point ($20/mo.) I would’ve already switched.
SearchGPT: Mediocre AI Search
Although SearchGPT is decent and will continue improving (such as to address many of these critiques), I think it has a long way to go before it catches up with Perplexity Pro.
Often fails to address the specific query: Many times I’ll ask something highly specific and SearchGPT generates a relatively highly generic response without digging for the specific answer. This has to do with the fact that it is unable to scan entire websites and use logic to process whether the info is likely accurate and directly addresses my specific query.
Repeats answers in subsequent queries: Ask something slightly different as a follow up query (following its initial internet-based answer) and it’ll often vomit the exact same reply as the previous internet-based answer even if it fails to address this subsequent query.
Cites junk sources on occasion: I’ve noticed that SearchGPT tends to cite mostly quality sources, but sometimes it cites complete junk. Occasionally I’ll ask for financials related to a certain company and it’ll cite PR Firm type sites (e.g. Business Wire). These can be accurate, but then if you ask for the latest info from a site like StockAnalysis or Yahoo Finance the data are completely different.
Scrapes information without analyzing quality: SearchGPT does not scan entire websites for the specific information you request. Instead it relies on RAG (retrieval-augmented generation) to process queries via Bing’s web index with a curated dataset (of allegedly validated and reliable sources).
Doesn’t use logic to evaluate quality of information: Assumes the authority is accurate no matter what – rather than thinking the results through before spitting them out.
No comparative analysis: SearchGPT doesn’t compare from multiple quality sources to determine whether one source might be inaccurate. It should likely be doing this to ensure accuracy (specifically regarding Stock metrics like P/E ratios, P/S ratios, etc.).
How to improve ChatGPT & SearchGPT…
It’s a really simple fix – all they need to do is give people the option to use offline-mode.
This would improve ChatGPT 1000x for me – currently I waste so much time saying “don’t use the internet” or “no internet.”
ChatGPT Improvements (Easy)
1. Offline-Mode as a Dedicated Option
Add a fully dedicated “Offline Mode” alongside the existing “Auto Mode” (where GPT decides) and “Search Mode” (which forces internet use).
This would ensure that users who prefer GPT’s logical reasoning and pre-trained knowledge don’t have to waste time with repetitive prompts to avoid internet usage.
Offline Mode should provide purely reasoning-based responses without deferring to internet-based content, ensuring faster and more concise answers.
2. Side-by-Side Internet & Offline Responses:
When in “Auto Mode” or “Search Mode,” offer both internet-based and offline-based responses side by side for the same query.
This would allow users to compare the reasoning-only answer with the web-enhanced answer and decide which one is more useful.
SearchGPT Improvements (How to Improve It)
1. Comparative Analysis for Source Accuracy
Enhance SearchGPT’s ability to compare data across multiple sources to identify inconsistencies, contradictions, or varying perspectives.
Presenting these side by side would help users see patterns or discrepancies, improving the accuracy of responses.
2. Use a Source Ranking Hierarchy
Employ a quality consensus algorithm to rank-order the sources SearchGPT pulls from.
This could factor in:
Recency: Ensuring the most recent and relevant information is prioritized.
Trust/Authority: Giving more weight to high-quality sources like academic publications, government sites (e.g., SEC.gov), or reputable industry reports.
Relevance: Ensuring the source directly addresses the query instead of providing tangential information.
3. Custom Source Ranking Options for Users
Allow users to rank-order specific sources they trust.
For example:
Someone researching financials might prefer SEC.gov filings over PR Newswire or Business Wire.
A scientist might prefer PubMed over general news outlets.
A general user might trust well-known sites like Wikipedia or BBC more than niche blogs.
Let users blacklist specific sources they find unreliable, ensuring SearchGPT avoids those in future responses.
4. Context-Aware Logic Layer
Introduce a logic layer that evaluates whether the fetched data makes sense before presenting it.
This could involve assessing whether the numbers or statements align with known patterns or GPT’s prior knowledge, reducing the risk of presenting flawed information.
5. Enhanced Aggregation of Information
Improve the aggregation process so that SearchGPT synthesizes data from multiple sources, combining key takeaways into a coherent and comprehensive response.
This would be especially useful for multifaceted queries.
6. Transparency in Source Selection
Add a feature that explains why certain sources were chosen over others, giving users greater confidence in the output.
For example, a response could include: “This data was selected due to recency, high authority, and relevance to your query.”
Perplexity Pro vs. SearchGPT (ChatGPT Search)…
In my experience (2024), Perplexity Pro currently demolishes ChatGPT search in: accuracy, quality, and directly addressing query specifics.
Perplexity Pro is still far from “perfect” and it occasionally makes mistakes.
On random occasions ChatGPT’s SearchGPT provides more accurate responses than Perplexity Pro (this is very uncommon though).
I’ve been using both now for a while (SearchGPT since launch & Perplexity Pro for ~1.5 years) – and Perplexity Pro continues to give higher-quality answers.
That said, I think SearchGPT remains ideal for the average Joe – as it comes with ChatGPT for $20/mo. and is still reasonably good for most queries.
The average Joe also doesn’t really care about where sources come from (even if it cites PR Newswire) – they just assume the citations are accurate and don’t want to waste time verifying source accuracy or doing any thinking).
Because of its different programming, Perplexity Pro combined with GPT-4o gives me far superior answers than SearchGPT via ChatGPT.
The best combo as of December 2024 is Perplexity Pro with Claude 3.5 Sonnet as the AI – but combination with 4o is better for more detailed replies.
I don’t think Perplexity Pro is worth the money for most people though… if you want the best-of-the-best, it’s worth it (this may change in the future).
What do you think of ChatGPT & SearchGPT?
Do you think ChatGPT automatically using SearchGPT at random helps you a lot? Do you actually like the auto-internet search feature?
Or are you like me such that you’ve become more of a ChatGPT hater when it searches the internet automatically – leaving you no option but to repeatedly tell it “no internet”?