When you think about the future of artificial intelligence (AI) technology, it’s likely that you don’t think of auto-completion. However, you probably should. In July 2020, OpenAI released a beta testing version of GPT-3, a new auto-completion program that could very likely define the next decade of AI programming.
In this blog post, I will introduce you to OpenAI’s GPT-3 model, and present the strengths, limitations, and potential for this new technology.
What is GPT-3?
Generative Pre-trained Transformer 3 (GPT-3) is AI technology developed by OpenAI, a company founded by Elon Musk and dedicated to AI investigation. It is the third model in OpenAI’s GPT series of autoregressive language tools.
GPT-3 is a language model that uses deep learning to create human-like text. Like other language processing systems, GPT-3 predicts the probability of the sequence of words based on the given text and automatically provides the most likely answer. It is similar to the auto completion you see when you type something in the Google search bar or in the messaging application on your phone.
How it Works
When GPT-3’s application programming interface (API) receives a small piece of text, it returns text based on the entry. The entry can be formulated as a phrase, a task, a question, or any kind of expression.
GPT-3’s auto-completion success is based on the amount of data from which it is able to gather statistical information. GPT-3 has access to data from a wide range of sources, including data sets, common crawl, books, news, and internet web pages. Where GPT-2, GPT-3’s predecessor, had 1.5 billion parameters to analyze, GPT-3 has 175 billion parameters. To put it in perspective, the entirety of the English Wikipedia only makes up about 0.6 percent of GPT-3’s dataset.