WebGPT: Improving the factual accuracy of language models through web browsing

In this blog post, which is part of my blog post series on conversational AI and chatbots, I will review WebGPT proposed by OpenAI.

Introduction

WebGPT article discusses the challenge of long-form question-answering (LFQA) in natural language processing (NLP). LFQA systems have the potential to become a primary source of learning, as I use ChatGPT a lot these days to learn about different topics, but they currently perform below human standards. The focus of this work is on information retrieval and synthesis, and the article presents a solution that outsources document retrieval to the Microsoft Bing Web Search API and fine-tunes GPT-3 through unsupervised pre-training for the synthesis part. The article also emphasizes combining these components using more faithful training objectives and optimizing answer quality through human feedback, to achieve competitive performance with humans.

The main contributions of this work are as follows:

• Creating a text-based web-browsing environment that a fine-tuned language model can interact with. This allows for improving both retrieval and synthesis parts in an end-to-end fashion using general methods such as imitation learning and reinforcement learning.
• Generating answers with references: passages extracted by the model from web pages while browsing. This is crucial for allowing labelers to judge the factual accuracy of answers…