Introduction
Attempts at hand-crafting algorithms for understanding human-generated content have generally been unsuccessful. For example, it is difficult for a computer to “grasp” the semantic content of an image - e.g., a car, cat, coat, etc....… - purely by analyzing its low-level pixels. Color histograms and feature detectors worked to a certain extent, but they were rarely accurate for most applications.
In the past decade, the combination of big data and deep learning has fundamentally changed the way we approach computer vision, natural language processing, and other machine learning (ML) applications; tasks ranging from spam email detection to realistic text-to-video synthesis have seen incredible strides, with accuracy metrics on specific tasks reaching superhuman levels. A significant positive side effect of these improvements is an increase in the use of embedding vectors, i.e., model artifacts generated by taking an intermediate result within a deep neural network. OpenAI’s docs page gives an excellent overview: