The Rise of ChatGPT: Separating Hype from Reality (ML4Devs, Issue 19)
What ChatGPT can and can't do for you, and how to make it your friend rather than foe.
Originally published at ML4Devs.com. See all issues at https://www.ML4Devs.com/newsletter/
ChatGPT has captured everyone’s attention for good reasons. There are thousands of posts on ChatGPT on LinkedIn and Twitter. It is natural to feel overwhelmed. This issue provides an overview of ChatGPT, its limitations, and how to make the most of it.
Generative AI
Generative AI is a type of artificial intelligence that can generate new outputs. Just like humans can write essays and poetry, paint a picture, create music, and make a movie, Generative AI can generate text, images, audio, video, etc.
Deep Neural Networks, such as Generative Adversarial Networks (GANs), are trained on very large datasets to learn the underlying patterns in the data, and then use it to generate new outputs with similar patterns but still unique (i.e. not a blatant copy of existing work).
Everyone is excited about Generative AI because of its wide-ranging applications, such as content creation, art and music generation, and data augmentation. In fact, dozens of start-ups are working on these applications and have investors lined up. Here is the current landscape of Generative AI startups:
Large Language Models and ChatGPT
Large Language Models (LLMs) are Generative AI models trained on massive datasets of text from the internet, books, program source code, and other sources. They learn to predict the likelihood of a given sequence of words in a given context.
In the last couple of years, there has been a slew of LLMs coming out, such as BERT, XLNet, and RoBERTa.
GPT (Generative Pretrained Transformer) models are a series of large language models developed by OpenAI that have progressively demonstrated an unprecedented ability to generate human-like text. ChatGPT is the latest in that series, and GPT-4 is rumored to be around the corner.
ChatGPT has taken the world by storm due to its wide-ranging natural language abilities, but more importantly, its simple chatbot-like user interface that anyone can quickly try.
Here is a brief overview of ChatGPT by Yogesh Kulkarni:
Why Call it Hype?
It sounds great. If it is all true, then why call it hype?
The biggest issue with ChatGPT (and with any Generative AI models) is that it can confidently hallucinate sometimes, and it can make up “facts”. Andrew Ng tweeted an instance of ChatGPT being hilariously wrong as it confidently explains why an abacus is faster than a GPU. There is an archive of ChatGPT failures.
It is typical of a technology going through the hype cycle. According to Gartner, Generative AI is at the peak of the hype:
You can attribute that hype to the Dunning-Kruger effect. As people gain expertise and a deeper understanding, the ChatGPT hype will wean off, and people will use it at the right place and for the right reasons.
What ChatGPT Can’t Do?
ChatGPT can’t always accurately understand the context and make complex logical deductions as in documented ChatGPT failures. It can’t be trusted for precise facts and must be cross-checked. For example, it has sometimes produced a totally made-up summary of an actual research paper.
Since ChatGPT can confidently hallucinate and make up facts, it can’t be entirely trusted as a search engine. Traditional search engines use machine learning for information retrieval, on the other hand, ChatGPT is doing information generation.
Nonetheless, ChatGPT reportedly caused a Code Red at Google. Even though ChatGPT says that it can’t replace search engines, Google is paranoid that it may kill its search business. That fear is not entirely misplaced. What if you marry information retrieval and generative AI? Say, you first do a search to find 20 (or 100) top results, and then generate the answer only from those results. That will cause serious disruption to search ad revenue and the SEO industry.
What ChatGPT Can Do?
There are good reasons why ChatGPT set the record for the fastest-growing user base. It reached 100 million monthly active users in January 2023, just two months after the launch!
Microsoft is already using ChatGPT to power the intelligent recap feature in Microsoft Teams Premium. This is the fasted research-to-product story that I know of. While its chatbot-like interface is user-friendly, I won’t be surprised to see its fine-tuned and specialized integration with Microsoft Office, much like GitHub Copilot in Visual Studio.
Impact on Knowledge Workers
ChatGPT (and other Generative AI technologies) will certainly impact knowledge workers, including marketing professionals, copywriters, writers, graphic designers, photographers, artists, musicians, and programmers.
Because of its inabilities with complex logic and it can occasionally be untrustworthy, ChatGPT will require human judgment and oversight. It will not replace knowledge workers but will definitely change the way they work.
Despite its limitations, ChatGPT will become a necessary tool. I see it more as a friend than a foe. ChatGPT can be used to get a quick start and augment human decision-making, saving time and effort, and making us more efficient.
I expect ChatGPT to automate some of the writing tasks and provide a better starting point than a blank paper. For example, I first generated the title and subtitle of this article by prompting ChatGPT and then tweaked it to suit my idea and style.
Similar to how Google Search revolutionized information gathering and research for writing, ChatGPT can take it a notch further. Google Search has made us better and more efficient, and I believe ChatGPT will do the same. However, it’s important to understand its limitations and avoid its pitfalls.
PS: After I scheduled to send this newsletter issue, two news items show the intensity of the AI race:
Yesterday (Feb 6): Google announced Bard powered by LaMDA
Today (Feb 7): Microsoft announced Bing and Edge to be powered by ChatGPT and Prometheus
Update: Check out what Yann LeCun, father of CNN, Turing Award winner, and Chief AI Scientist at Meta think about LLMs in this tweet thread:
