Posted on

Capacity and Compliance in Hybrid Cloud, Multi-Tenant Big Data Platforms

As organizations are realizing how Data-Driven insights can empower their strategic decisions and increase their ROI, the focus is on building Data Lakes and Data Warehouses where all the Big Data can be safely archived. Big data can then be used to empower various data engineering, data science, business analytics, and operational analytics initiatives to benefit the business by improving operational efficiency, reducing operating costs, and making better strategic business decisions. However, the exponential growth in the data that we humans consume and generate day to day makes it necessary to have a well-structured approach toward capacity governance in the Big Data Platform.

Introduction:

Capacity governance and scalability engineering are inter-related disciplines, as this requires a comprehensive understanding of our compute and storage capacity demands, infrastructure supply, and their inter-dynamics to develop an appropriate strategy for scalability in the big data platform. In addition to this, technical risk resolution and security compliance are equally important aspects of capacity governance.

Posted on

Externalizing Your Configurations With Vault for Scalable Deployments

Table of Contents:

  • Introduction
    1. The Solution
  • Setting Up Vault
    1. Creating API Admin Policy
    2. Creating Read-Only user policy
    3. Creating Token attached with API read-only policy
  • 1. Linux Shell Integration
  • 2. Java Integration
  • 3. Python Integration
  • 4. Nodejs Integration
  • 5. Ansible Integration
  • Conclusion

Introduction:

To implement automation for microservices or applications deployed to a large number of systems, it becomes essential to externalize the configurations to a secure, scalable, centralized configuration store. This is necessary to be able to deploy the application in multiple environments with environment-specific parameters without requiring human intervention and without requiring modification to the core application during automated deployments, scaling, and failover recoveries.

Besides the fact that manually managed configurations involve the risk of human error, they are also not scalable for 24x7 large-scale deployments, particularly when we are deploying several instances of microservices across various infrastructure platforms.