How to Deploy TiDB on Google Cloud Platform—Part 2

Overview

Welcome to the second of a two-part blog series on getting a simple TiDB cluster up and running on the Google Cloud Platform. The goal is to let you quickly set up TiDB and become familiar with its benefits. In Part 1, we created a GCP instance. Here in Part 2, we will install, configure, and run TiDB on that instance.

By the end of this article, your architecture may look similar to the following:

Google Cloud “GCP” Native NixOS Images Build

I am using NixOs as a Development environment that is running on Google Cloud VM.
NixOs official Web site does not provide an official GCP image. On the NixOs Wiki page, you can find an article how you can build your GCP NixOs Image that didn’t work well in the past but now seems all build image issue was resolved and now works well. NixOS Wiki page article also provides two GCP storage that you can find outdated GCP VM images that seem not actively maintained anymore.

  • nixos-images > gs://nixos-images
  • nixos-cloud-images > gs://nixos-images

In the below "HowTo," we will cover how you can build your images in GCP by using Google Cloud VM and store the images in your Google storage bucket.

Intro to Google Cloud VMware Engine: Establishing an SDDC in Google Cloud

This post is the first in a series on Google Cloud VMware Engine and Google Cloud Platform. This post walks through planning, prerequisites, and the process of deploying a Software-Defined Data Center (SDDC) using HCX in Google Cloud VMware Engine, as well as configuring a VPN gateway for initial access to the environment.

Before we dive into this, I want to set expectations for this blog series. My goal when working in the cloud is to create, modify and destroy resources programmatically. My tool of choice is Terraform, but I will also use CLI-based tools like gcloud. Occasionally I will inspect API calls directly and perform API calls with Python or cURL. I have found that learning a product's API is an excellent way to master it. Cloud consoles (GUIs) are adequate when getting started, but interfacing with the API, whether through Terraform or an SDK, is how these platforms are designed to work.

AWS vs. Google Cloud: Comparing the Giants

AWS and Google Cloud are two key rivals in the world of cloud computing and storage. While the first one is winning people’s hearts with its amazing flexibility and ample features, Google Cloud has managed to firm its feet in the industry with its superb backup services and cost-effectiveness. These two are performing such wonderfully at their respective fronts that it’s tough to pick one out of these two.

If you’re also at the crossroads of picking one, let’s help you decide. In this blog post of AWS VS Google Cloud 2021, we have covered few key differences between these co-workers cloud computing giants. Before we get into the details, let’s figure out the basics of these two.

Google Cloud Pub/Sub – Overview

Introduction

The Google Cloud Pub/Sub is a fully managed real-time messaging service and it helps our applications/services to send and receive messages independently between them. It helps us to build robust and scalable applications by integrating them asynchronously. It provides scalability, resilience and handles millions of messages simultaneously.

Why Google Cloud Pub/Sub?

Google Cloud Pub/Sub can be used for a lot of use cases. In general, If you want to process large amounts of data for analytics or do you want to simplify the event-driven microservices development?, then the Google Cloud Pub-Sub is the right choice.

How Are Zoom, Spotify, etc Slashing Their Cloud Costs by Millions?

In Q1 2021, Zoom reported that its gross margin widened to 73.9% from 69.4% in the previous quarter, primarily thanks to the optimization of public cloud resources. And Zoom is certainly not the only company that realized the value of optimizing the cloud infrastructure. As businesses migrate their workloads to the cloud and build cloud-native applications, they’re starting to realize that overprovisioning and cloud sprawl aren’t just urban legends. 

For startups, the cloud is an essential technology because of its unparalleled support for scalability. But the cloud may quickly turn into a struggle because of growing costs. Here's what a16z wrote in a recent analysis:

Introduction To Google Anthos

Introduction

Google has put over a decade’s worth of work into formulating the newly released Anthos. Anthos is the bold culmination and expansion of many admired container products, including Linux Containers, Kubernetes, GKE (Google Kubernetes Engine), and GKE On-Prem. It has been over a year since the general availability of Anthos was announced, and the platform marks Google’s official step into enterprise data center management.

Anthos is the first next-gen tech multi-cloud platform supporter designed by a mainstream cloud provider. The platform’s unique selling point lies in application deployment capability across multiple environments, whether on-premises data centers, Google cloud, other clouds, or even existing Kubernetes clusters.

7 Places to Host Your Jamstack Site

The Jamstack architecture is rapidly gaining popularity with web developers everywhere. Jam stands for “JavaScript, APIs, and Markup.” Jamstack sites are static sites built with HTML, JavaScript, and CSS; they’re text files, so they can be served up fast without requiring the overhead of traditional compiled or interpreted architectures.

Tim Berners-Lee created the first static website in 1991— in fact, it was the first website ever built. In the decades after, developers have moved to using interpreted and compiled code for websites and applications, requiring complicated configuration and additional overhead.

The Real Costs of Cloud Migration

As cloud computing rapidly proliferates enterprise IT and organizations migrate more of their traditional workloads and on-premise data in the cloud, the cloud will remain one of the fastest-growing segments of IT spend. 

According to Gartner, by 2024, more than 45% of IT spending on system infrastructure, infrastructure software, application software, and business process outsourcing will move from traditional IT infrastructures to the cloud. 

Enterprises are struggling to scale their infrastructure to the insatiable demands. They are adding up costs to hardware procurements, issue software updates, secure infrastructure, train staff, etc. On top of that, the ongoing pandemic is holding available human resources, limiting data center facilities, and shrinking hardware supply chains. Cloud computing brings much-needed relief.

No doubt, Flexera State of the Cloud report indicates 61 percent of organizations plan to focus on cloud migration this year. However, against what marketing pages of AWS and Microsoft Azure make you believe, cloud migration is more than moving VMs and databases in the cloud. You have to understand app dependencies, access technical feasibility, select the best instances, just to name a few. And then there are post migrations woes.

These challenges translate to added cost and not all of them are conspicuous until very later. To estimate the real cost of cloud migration for your business, you need to take a thoughtful, disciplined approach. The first step is to evaluate your current infrastructure.

The Real Cost of Current Infrastructure

If you don’t know how much your organization is paying for all those servers, software licenses, maintenance contracts, extended warranties, networking equipment, security contracts, then a visit to the accounting department will ease your day. 
Also, you can gather data on network bandwidth, storage, and database capacity, etc. In a typical SME setup with a five-year hardware upgrade cycle, the first-year cost can be $40000-$50000 considering these variables:

Considered variables

In addition to the first year's costs, there are recurring costs. You may not buy new servers every year but must maintain them. The same goes for all the pieces of hardware and software running your IT infrastructure and resources managing it. Your recurring cost should be anywhere between 10-20% of your first-year cost.
Recurring cost analysis
However, the real cost of your current IT implementation is more than direct, operating, and administrative costs. Did you know you lose money every time there is downtime? When a power system in British Airway’s data center failed, the airline lost roughly $68 million in fare refund alone, not to mention a 2.8% dip in their stock prices. If your on-premise setup has a 98% uptime, then you’re giving up 14 hours of productivity hours every month. Most cloud vendors guarantee a 99.9% availability and compensate for any additional downtime based on your contract. According to a Forbes report, for an hour of downtime, you could be losing thousands of dollars. However, for most SMBs, you could be losing anywhere from $3000 to $5000 per month in downtimes. 

So the cost of total ownership of your on-premise setup for three years is approx. $75000 to $85000.

The Real Cloud Infrastructure Cost

So the long story short: once upon a time, it was next to impossible to estimate cloud infrastructure cost up until the actual migration. There was little competition and cloud computing was all about AWS and little of Google Cloud and Microsoft Azure here and there. Now we have every major tech firm in the cloud business and the competition is at an all-time high. 

The pricing of major cloud vendors is still a little too complicated, but they now provide an approachable price estimation tool. Every major cloud vendor, along with AWS, offers some form of Total Cost of Ownership (TCO) Calculator, including Google Cloud and Microsoft Azure. 

Mostly, it is a simple tool that asks you a few basic hardware questions like your required bandwidth, RAM, processor core, storage, etc. If the tool feels a little arbitrary, then there is an advanced tool too that asks enough questions to cover your entire infrastructure, but the estimations are more accurate. 

So I ran estimations on TCO tool for AWS, GCE and Azure respectively for a typical SME setup consisting 32 vCPUs and 128 GB RAM.

Cloud Services comparison

A Practical Multi-Cloud Distributed SQL Strategy for 2021

In order to meet the needs of the growing demand of businesses, many global enterprises are choosing cloud infrastructure from multiple providers like AWS, Azure, Google, or private data centers. According to a Gartner survey of public cloud users, 81% of respondents are already using multiple clouds. Leveraging multiple clouds to support data infrastructure provides these benefits:

  • Reduced operating and infrastructure costs by avoiding vendor lock-in.
  • Improved application resilience and redundancy with geographically distributed data centers. (Cloud providers suffer from outages so putting all your workloads on one provider’s infrastructure increases the risk of an application becoming downtime.)
  • Improved customer experience and performance optimization by choosing a data center closest to end users that can serve the requested data with minimum latency.
  • Achieving data compliance, such as the EU’s GDPR which requires data to be held in particular geographical locations. (If your primary cloud does not support all the regions where your customers reside, you need to consider a multi-cloud strategy.)
  • Ability to expand into new markets by taking advantage of regional data centers.

Maximize Your Google Cloud Investment With Redis Enterprise

Google Cloud is used by many of the world’s top organizations, and provides an array of top-shelf solutions from developer tools to databases. Among these resources is Redis Enterprise Cloud, a robust in-memory data platform built by the same group that developed open source Redis.

Join Google Customer Engineer Matt Cowger and Redis Labs Senior Solution Architect Jake Angerman as they discuss how technology teams employ Redis Enterprise Cloud on Google Cloud to maximize the value of the cloud.

A Journey From MuleSoft to Serverless Integration

In the current highly volatile market trend the cost optimizationinfinite scalability, and agile development tools are very vital for any enterprise digital roadmap. The fast-growing trend of FaaS world with AWS Lambda, Google Cloud function, and Azure Fn has already received recognition of small, medium, and large enterprises and this indicates that the move from the iPaaS to FaaS is natural and unavoidable. This move opens a new world of Serverless Integration.

The key features of serverless integration are:

Using Containers and Google Cloud in Automation Testing

Using Containers for Automation Testing

With the advent of agile and continuous integration and deployment, continuous testing has become a requirement. There is no alternative to that. The emphasis on shorter release cycles to give the product an edge in the competitive market is crucial.
In the faster release cycles, maintaining the quality of delivery is a real challenge.

Case in Point

Let’s assume you build an accounts screen for a highly secured banking app. Now some of the users might be using Chrome to access the application, some might use Internet Explorer, some might use mobile apps, etc. Now in the mobile app also, some users may have an Android phone and some may have iOS. So while deploying the code on production, developers and testers have to make sure that the feature is compatible with all different OS, browsers, platforms.

AWS vs Azure vs Google Cloud

With the competition heating up in the public cloud service vendors, the addition of new features and a regular drop in the price will decide who the winner is. In this article, we intend to throw light on the mounting competition between AWS, Microsoft’s Azure, and GCP. AWS already has a head start of years over the others, but there is no denying the other two public cloud service providers are not far behind. It is important to compare them to understand which one suits your project the most.

A Detailed Comparison

Our comparison guide is a thorough take on all the three cloud service providers based on parameters like:

The 4 Best Jupyter Notebook Environments for Deep Learning

Notebooks are becoming the standard for prototyping and analysis for data scientists. Many cloud providers offer machine learning and deep learning services in the form of Jupyter notebooks. Other players have now begun to offer cloud-hosted Jupyter environments, with similar storage, compute and pricing structures. One of the main differences can be multi-language support and version control options that allow data scientists to share their work in one place.

The Increasing Popularity of Jupyter Notebook Environments

Jupyter notebook environments are now becoming the first destination in the journey to productizing your data science project. The notebook environment allows us to keep track of errors and maintain clean code. One of the best features although simple is that the notebook would stop compiling your code if it spots an error. Regular IDE’s do not stop compilation even if an error is detected and, depending on the amount of code, it can be a waste of time to go back and manually detect where the error is located. 

How to Grant User Privileges in Google Cloud MySQL

When you create a new user account in your Google Cloud MySQL database, it has the same privileges as a root user. It is, therefore, a good idea to limit the admin privileges of the new MySQL user with the REVOKE command and explicitly grant the required user privileges with the GRANT statement.

Google Cloud SQL User Privileges

You can use MySQL Workbench or Sequel Pro to connect to your Cloud SQL database with the root user. Make sure that your database has a public IP and your computer’s IP address is added as an authorized network in the Connections tab of your Database console.

SHOW GRANTS FOR db_user

If your MySQL user has root privileges, the statement will output the following:

GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, RELOAD, SHUTDOWN, PROCESS, REFERENCES, INDEX, ALTER, SHOW DATABASES, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, REPLICATION SLAVE, REPLICATION CLIENT, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, CREATE USER, EVENT, TRIGGER, CREATE TABLESPACE ON *.* TO 'db_user'@'%' WITH GRANT OPTION

As a first step, you can revoke all privileges from the user account. You can either specify individual permissions, separated by commas, but since the root user has many privileges, we can revoke them all and grant the required one in another statement.

REVOKE ALL PRIVILEGES, GRANT OPTION FROM db_user

Next, we tell the server to reload the privileges from the grant tables in the MySQL system schema.

FLUSH PRIVILEGES

Finally, grant the required privileges to the user. In our case, the user should only be able to read, insert, view and delete rows from all tables in a specific database.

GRANT SELECT, UPDATE, INSERT, DELETE ON db_name.* TO db_user

Execute the Flush Privileges statement again to apply the changes.

FLUSH PRIVILEGES

You may also run the SHOW GRANTS statement to verify that the correct privileges have been applied to the user.

SHOW GRANTS FOR db_user