Posted on

How to Migrate Your Data From Redshift to Snowflake

For decades, data warehousing solutions have been the backbone of enterprise reporting and business intelligence. But, in recent years, cloud-based data warehouses like Amazon Redshift and Snowflake have become extremely popular. So, why would someone want to migrate from one cloud-based data warehouse to another?

The answer is simple: More scale and flexibility. With Snowflake, users can quickly scale out data and compute resources independently by automatically adding nodes. Using the VARIANT data type, Snowflake also supports storing richer data such as objects, arrays, and JSON data. Debugging Redshift is not always straightforward as well, as Redshift users know. Sometimes it goes beyond feature differences that could trigger a desire to migrate. Maybe your team just knows how to work with Snowflake better than Redshift, or perhaps your organization wants to standardize on one particular technology.

Posted on

Data Replication for DBMS Using the Commit Log

Introduction

In this article, we will see how developers can break down information silos for their teams and business by replicating data across multiple systems. First, we will review why developers will replicate data and considerations for the cloud. Second, we will prepare for war with the replicators. Then we will examine the architecture of Postgres and MySQL and how their commit logs enable us to make exact copies of the data. Finally, we will connect Debezium to Postgres for a complete data replication solution.

Introduction to Data Replication

Data replication is the process of moving data between different database systems for various business use cases. In a typical SaaS (Software As A Service) application, data is stored in an operational database such as MySQL, PostgreSQL, Oracle, etc. There are other database systems such as data warehouses and search systems built for specialized use cases. Moving data between these systems is known as data replication.

Posted on

MySQL to Redshift: 4 Ways to Replicate Your Data

MySQL is the most popular open source cloud database in the world, and for good reason. It’s powerful, flexible, and extremely reliable. Tens of thousands of companies use MySQL to power their web-based applications and services every day.

But when it comes to data analytics, it’s a different story. MySQL is quickly bogged down by even the smallest analytical queries, putting your entire application at risk of crashing. As one FlyData customer said to us, “I have nightmares about our MySQL production database going down.”

Posted on

Distributed SQL Essentials

Distributed SQL databases combine the resilience and scalability of a NoSQL database with the full functionality of a relational database. In this Refcard, we explore the fundamentals of distributed SQL, including architecting for availability, handling schema design challenges, using JSON and columnar indexes, as well as assessing approaches to replication.
Posted on

Why Data Replication Should Not Be Done Using ESB-Based Integration Tools

This is one of the common questions we get when prospects come looking for data replication tools. It's more a question of Integration design patterns than of product implements.

Let's get started with what an ESB is - Enterprise Service Bus. This is an integration design pattern where messages are passed so that one or more Message Listeners can listen and consume the message - store and forward. These messages—like, say, emails—have a header (from and to), a payload (the message), and perhaps attachments. Based on the ESB, there might be some limitation on payload and attachments sizes.