A Simple Introduction to Apache Camel

A "Stable, Consistent Architecture" is a myth. It can never be achieved. Over the years, each of us has come to accept this fact, and with a microservices architecture, it is a lot easier to live with this fact. But, that does pose a lot of other issues that we need to look at.

An important hurdle in this journey is that of application integration. A core tenet of the microservice architecture is that any interaction or integration between systems should be based on globally accepted standards rather than a mutually accepted protocol. If we build a system based on a protocol that is based on a simple mutual agreement between two developers, it can never work beyond those two developers. To enable a generic plug and play interface, the communication has to be based on globally accepted standards.

ETL and How it Changed Over Time

What Is ETL?

ETL is the abbreviation for Extract, Transformation, and Load. In simple terms, it is just copying data between two locations.[1]

  • Extract: The process of reading the data from different types of sources including databases.
  • Transform: Converting the extracted data to a particular format. Conversion also involves enriching the data using other data in the system.
  • Load: The process of writing the data to a target database, data warehouse, or another system.

ETL can be differentiated into 2 categories with regards to the infrastructure.