Introduction
File-based integration is one of the most popular and widely used approaches in data integration. Many software systems work with files — i.e., they either take in files as input, give their results as files, or do both — and working with multiple disparate systems requires connecting those systems together. Newer software systems also provide API-based access to information, and API-based integration is one of the approaches in connecting such systems. Although APIs provide advantages over traditional file-based systems, such as complex data structures and fine-grained security, many legacy software systems, and even modern ones, often use file-based approaches to export and import data.
Integration requires ensuring that two separate systems understand each other. And, although file structures and data types will likely be different in different software systems, we need an integration mechanism to map, transform, filter, and cleanse data in order for software systems to understand each other. Some of the most popular file formats used in general scenarios are CSV, EDI, JSON, and XML.