Hive Metastore supports various backend databases, among which MySQL is the most commonly used. However, in real-world scenarios, MySQL's shortcoming is obvious: as metadata grows in Hive, MySQL is limited by its standalone performance and can't deliver good performance. When individual MySQL databases form a cluster, the complexity drastically increases. In scenarios with huge amounts of metadata (for example, a single table has more than 10 million or even 100 million rows of data), MySQL is not a good choice.
We had this problem, and our migration story proves that TiDB, an open-source distributed Hybrid Transactional/Analytical Processing (HTAP) database, is a perfect solution in these scenarios.
