Author: Roger Oriol

LLMs are very capable of performing a lot of feats that seem incredible to us, but they are bound by the borders of what Generative AI was originally meant to do: generate text based on the data it has been trained on.

They cannot access any services on the web to give us more accurate and recent answers, and instead, they are meant to answer using the training data that their creators were able to gather many years ago. They cannot access tools as simple as a calculator that would help to give mathematical answers instead of having to memorize the result of every mathematical operation. They are reliant on us to perform the actions they suggest to do in the real world, and we report back to them with the results.

LLMs do Natural Language Processing (NLP) to represent the meaning of the text as a vector. This representation of the words of the text is an embedding.

The Token Limit: The LLM Prompting Biggest Problem

Currently, one of the biggest problems with LLM prompting is the token limit. When GPT-3 was released, the limit for both the prompt and the output combined was 2,048 tokens. With GPT-3.5, this limit increased to 4,096 tokens. Now, GPT-4 comes in two variants. One with a limit of 8,192 tokens and another with a limit of 32,768 tokens, around 50 pages of text.

Language models have rapidly improved in recent years, with large language models (LLMs) such as GPT-3 and GPT-4 taking center stage. These models have become popular due to their ability to perform a great variety of tasks with incredible skill. Also, as the number of parameters of these models (in the billions!) has increased, these models have unpredictably gained new abilities.

In this article, we will explore LLMs, the tasks they can perform, their shortcomings, and various prompt engineering strategies.

Many enterprises choose a cloud environment to power the work of their data science team. If you chose the AWS SageMaker Studio, this article might interest you. Having both the data lake and the data scientist environment makes it easy to integrate them. You can choose what data any given data scientist is able to see. You might want a data scientist only to be able to use this data inside the SageMaker Studio environment. However, SageMaker Studio has a download button that lets data scientists download any data they have been working on. Once they have downloaded data to their computers, they are free to share it anywhere and with anyone.

Luckily, it is possible to disable this download button. Recently, it was only possible to disable the download button in SageMaker Notebooks. This article from Ujjwal Bhardwaj shows how to disable it in SageMaker Notebooks.

A feature store is a storage system for features. Features are properties of data calculated through an ETL process or feature pipeline. This pipeline takes raw data and calculates a property from it. This property - usually a numeric value - will be useful to a machine learning model. It is important to find adequate, correct, and quality features. The quality of those features is the most important contributor to a model's success. The model will use the features either to train itself or to make predictions. A feature store will help to organize and use those features.

At its core, a feature store is only a database. More specifically, there are usually two databases. There is an offline store equipped to store large sums of data, like an HBase or S3. There is also an online store equipped for fast data serving, like Cassandra. Features are organized in feature groups, which can be thought of as tables. Features that are used together are stored in the same feature group so access to them is faster and without joins. There are many ETL processes (think Spark) that write to the offline store. Data from the offline store replicates to the online store to keep them consistent. Data streams can also write both to the online and offline stores, for fast real-time data access.

In recent years, the web development landscape has discovered that, while super stylized websites and web applications with tons of very rich interactions made with Javascript might be appealing to users, the loading time of the page can be a hundred times more influential to the user's perspective of the experience. As a result, there has been a ton of effort to reduce the loading time metrics of websites. Many frameworks and patterns have (re-)emerged to give solutions to this problem, like Server-Side Rendering and Static Site Generation.

Enter Jarred Sumner. Jarred noticed that an action as simple as printing to the console was extremely slow in Node.js. Here's an example of the time it takes to execute a "Hello world" program in Node.js and Rust.