Author: Patrick McFadin

This blog will delve into the nuances of combining the prowess of DataStax Astra with the power of Ray and is a companion to this demo on GitHub. We’ll explore the step-by-step procedure, the pitfalls to avoid, and the advantages this dynamic duo brings to the table. Whether you’re a data engineer, a developer looking to optimize your workflows, or just a tech enthusiast curious about the latest in data solutions, this guide promises insights aplenty. Soon, you’ll be able to use Cassandra 5 in place of AstraDB in this demo — but for a quick start, AstraDB is a great way to get started with a vector-search-compliant Cassandra database!

Introduction

Vector search is a technology that works by turning data that we are interested in into numerical representations of locations in a coordinate system. A database that holds and operates on vectors is called a vector store. This functionality is coming to Cassandra 5.0, which will be released soon. To preview this functionality, we can make use of DataStax Astra. Similar items have their vector locations close to each other in this space. That way, we can take some items and find items similar to them. In this case, we have bits of text that are embedded. Embedding takes text into a machine-learning model that returns vectors that represent the data. You can almost think about embedding and translating data from real text into vectors. 

When creating real-time data platforms, data streaming is a low-latency, high-throughput method of moving data. Where batch processing methods necessarily introduce delays in order to gather a batch worth of data, stream processing methods act on steam events as they occur, with as little delay as possible. In this blog and associated repo, we will discuss how streaming data can be compatible with Cassandra, with Spark Structured Streaming as an intermediary. Cassandra is designed for high-volume interactions and, thus, a great resource for streaming workflows. For simplicity and speed, we are using DataStax’s AstraDB in this demo.

Introduction 

Streaming data is normally incompatible with standard SQL and NoSQL databases since they can consist of differently structured data with messages only differentiated by timestamp.  With advances in database technologies and continuous development, many databases have evolved to better accommodate streaming data use cases. Additionally, there are specialized databases, such as time-series databases and stream processing systems, that are designed explicitly for handling streaming data with high efficiency and low latency. 

The scenario that this blog will cover is a time-series forecasting of a specific stock price. The problem itself is very common and widely known. That being said, this is a technology demo and is in no way intended as market advice. The purpose of the demo is to show how Astra and TensorFlow can work together to do time-series forecasting. The first set is setting up your database.

Getting Test Data

Now that you have the database, credentials, and bundle file on your machine, let’s ensure you get the data needed to run this tutorial. This data is open source and available on many platforms. One of which is the Kaggle platform, where people have already done the work of gathering this data via APIs. 

Handwritten digit recognition is one of the classic tasks undertaken by students when learning the basics of Neural Networks and Computer Vision. The basic idea is to take a number of labeled images of handwritten digits and use those to train a neural network that is able to classify new unlabeled images. For this demo, we show how to use data stored in a large-scale database as our training data. We also explain how to use that same database as a basic model registry. This addition can enable model serving as well as potentially future retraining.

Introduction

MNIST is a set of datasets that share a particular format useful for educating students about neural networks while presenting them with diverse problems. The MNIST datasets for this demo are a collection of 28 by 28-pixel grayscale images as data and classifications 0-9 as potential labels. This demo works with the original MNIST handwritten digits dataset as well as the MNIST fashion dataset. 

ChatGPT plugins offer a way to extend the capabilities of OpenAI's ChatGPT by integrating custom functionalities directly into the conversational AI interface. These plugins enable users to interact with specialized features, transforming ChatGPT into a versatile tool for various tasks. Think of a ChatGPT plugin as a handy tool belt that equips OpenAI's ChatGPT with specialized superpowers. Just like adding a new gadget to your arsenal, a plugin empowers ChatGPT to perform specific tasks seamlessly within the conversation. 

In this blog, we'll dive into implementing the Cassandra to-do list ChatGPT plugin, which acts as a virtual personal assistant for managing your to-do list. It's like having a dedicated task organizer right beside you during your AI-powered conversations. With this plugin, you can effortlessly create, view, and delete tasks, bringing a new level of productivity and organization to your chat-based interactions with ChatGPT.

If you’re a frequent user of ChatGPT, you know the tendency it has to wander off into what is known as hallucinations. A great collection of statistically correct words that have no basis in reality. A few months ago, a prompt about using Apache Cassandra for large language models (LLMs) and LangChain resulted in a curious response. ChatGPT reported that not only was Cassandra a good tool choice when creating LLMs, but OpenAI used Cassandra with an MIT-licensed Python library they called CassIO. Into the rabbit hole we went, and through more prompting, ChatGPT described many details about how CassIO was used. It even included some sample code and a website. Subsequent research found no evidence of CassIO outside of ChatGPT responses, but the seed was sown. If this library didn’t exist, it needed to, and we started work on it shortly after.

Best hallucination ever.