An Introduction to GPT-3

by

When you think about the future of artificial intelligence (AI) technology, it’s likely that you don’t think of auto-completion. However, you probably should. In July 2020, OpenAI released a beta testing version of GPT-3, a new auto-completion program that could very likely define the next decade of AI programming. 

In this blog post, I will introduce you to OpenAI’s GPT-3 model, and present the strengths, limitations, and potential for this new technology. 

What is GPT-3?

Generative Pre-trained Transformer 3 (GPT-3) is AI technology developed by OpenAI, a company founded by Elon Musk and dedicated to AI investigation. It is the third model in OpenAI’s GPT series of autoregressive language tools. 

GPT-3 is a language model that uses deep learning to create human-like text. Like other language processing systems, GPT-3 predicts the probability of the sequence of words based on the given text and automatically provides the most likely answer. It is similar to the auto completion you see when you type something in the Google search bar or in the messaging application on your phone. 

How it Works

When GPT-3’s application programming interface (API) receives a small piece of text, it returns text based on the entry. The entry can be formulated as a phrase, a task, a question, or any kind of expression. 

GPT-3’s auto-completion success is based on the amount of data from which it is able to gather statistical information. GPT-3 has access to data from a wide range of sources, including data sets, common crawl, books, news, and internet web pages. Where GPT-2, GPT-3’s predecessor, had 1.5 billion parameters to analyze, GPT-3 has 175 billion parameters. To put it in perspective, the entirety of the English Wikipedia only makes up about 0.6 percent of GPT-3’s dataset. 

Supervised vs. Unsupervised Machine Learning

There are two types of learning that can be accomplished with AI: supervised and unsupervised machine learning. The first, supervised machine learning, requires data to be well labeled. This means that any data the model reviews is tagged with the correct answer.

For example, you can tell the model that a photo of a cat is a cat. The algorithm will then create a specialized numerical identification for all cat photos you label. The more labels the model has, the more successful its processing potential becomes. You can also label items as “not a cat” to differentiate them from your cat labels. Once the data sets are complete, the model is then able to analyze unlabeled data and predict whether the new data is a cat or not. 

GPT-3 represents an unsupervised machine learning model. Unsupervised machine learning removes the need for labeled data, and instead allows the model to discover information on its own. With unsupervised machine learning you are able to perform more complex processing tasks, but your results can be more unpredictable. 

Utilization of GPT-3

Though GPT-3 is an auto completion tool, it can be utilized for a number of tasks, including: 

  • Maintaining a conversation. 
  • Taking a specific role in a conversation. For instance, you can ask to talk with a story character. It can talk about any kind of topic.
  • Translating text to any language.
  • Passing a sentence into a mathematical expression.
  • Generating news articles.
  • Designing/creating interface layouts. 
  • Creating pieces of code.
  • Translating code to different programming languages.
  • Summarizing text.

Now that it is in the beta testing phase, new uses for GPT-3 are constantly being discovered. 

Limitations of GPT-3

GPT-3 represents a significant step in the evolution of AI technology. However, despite its potential, there are still limitations to the GPT-3 model: 

  • The model is trained to calculate the probability of words based on the previous text, but it cannot understand the text it receives and generates.
  • The model cannot think independently, which limits usability. 
  • If the model generates or receives text that is meaningless, it will never know because it does not understand the context. However, it is able to say that it doesn’t understand. 

Additional Use-Cases for GPT-3

OpenAI has been quite reserved about the potential uses and use-cases of GPT-3. The proposed GPT-3 model use-cases include:

  • Summarizing and simplifying contracts to make them more understandable and easier to use.
  • Improving keyboard predictions on devices.
  • Writing first drafts of emails, letters, and other communications. 
  • Mass marketing across the web.
  • Creating stories and characters for video games to allow varied user experiences. 

As more users interact with GPT-3, it is likely that OpenAI will continue to update the proposed use-cases for the technology. 

GPT-3 is not an open source program; therefore, users must be granted permission to use it. To receive use rights for GPT-3, users must fill out a form on which they explain how they intend to use the model. Each request is evaluated individually by OpenAI before approval. OpenAI has also been closely investigating ways to mitigate the misuse of the GPT-3 API to avoid cases that cause social, physical or mental harm.

Though the full scope of GPT-3’s potential is not yet known, we can conclude that it has successfully ushered our world into a new phase of AI technology.  For more insight into what GPT-3 can do check out this op-ed article written by GPT-3. 

 

Leave a Reply

Your email address will not be published. Required fields are marked