GPT3 has changed the level of language models and revolutionized AI by its capacity to learn with few examples, as GPT3 is a few-shot learner. However, it is not open-sourced, and access to OpenAI’s API is only available upon request. So EleutherAI is working on creating a similar model to GPT3, which is named GPT-Neo.
GPT-Neo is a transformer-based language model whose architecture is nearly the same architecture as the GPT3 model, and the results are also roughly equal to the lower versions of the GPT3 model. GPT-Neo is trained on the Pile Dataset. Same as GPT3, GPT-Neo is also a few-shot learner. And the good thing about GPT-Neo over GPT3 is it is an open-source model.