Posted on

How to Choose a Stream Processor for Your Data

Data has become integral to most organizations. So it's no wonder that stream processing has become a critical part of big data stacks. This works wonders for consolidating and interpreting large amounts of data.

There are many end-to-end solutions available for streaming data pipelines in the cloud. Not to mention many terminologies to navigate the different stream processing tools to choose from.

Posted on

ETL and How it Changed Over Time

What Is ETL?

ETL is the abbreviation for Extract, Transformation, and Load. In simple terms, it is just copying data between two locations.[1]

  • Extract: The process of reading the data from different types of sources including databases.
  • Transform: Converting the extracted data to a particular format. Conversion also involves enriching the data using other data in the system.
  • Load: The process of writing the data to a target database, data warehouse, or another system.

ETL can be differentiated into 2 categories with regards to the infrastructure.