Tips, Expertise, Articles and Advice from the Pro's for Your Website or Blog to Succeed
Snowflake is a cloud-based Data Warehouse platform that proudly boasts near zero management. This is part one of a series of articles that will unpack just what that means.
I was recently working for a major UK customer, where the system manager said, “Snowflake says they need Zero Management, but surely that’s just a marketing ploy.” Most Solution Architects use the term “Near Zero Management” which is probably more accurate. However, it got me thinking — is it true?
A few years ago, Hadoop was touted as the replacement for the data warehouse which is clearly nonsense. This article is intended to provide an objective summary of the features and drawbacks of Hadoop/HDFS as an analytics platform and compare these to the cloud-based Snowflake data warehouse.
First developed by Doug Cutting at Yahoo! and then made open source from 2012 onwards, Hadoop gained considerable traction as a possible replacement for analytic workloads (data warehouse applications), on expensive MPP appliances.
How do you tune the Snowflake data warehouse when there are no indexes, and few options available to tune the database itself?
Snowflake was designed for simplicity, with few performance tuning options. This article summarizes the top five best practices to maximize query performance.
Blockchain was a real technological breakthrough back in 2008. A new milestone in the world of money, security, e-commerce, and the Internet as a whole — it is literally everywhere now, and modern business utilizes it worldwide. The idea of recording the information without any ability to change it afterward was brilliant. Firstly, as you know, it was implemented in the first cryptocurrency, Bitcoin, but later, it came to banks, security providers, insurers, farmers, governments, etc.
Why were people so obsessed with the blockchain? Because this system guarantees that no one, no matter how powerful or rich this person is, can tamper with the information encoded within the public ledger. Yet, even the sun has dark spots. With time, people have realized that this system is not ideal, and there are problems to be solved. For example, the ledger gets heavier and slower with every new transaction, as the blockchain grows. Or, it cannot exist without miners, and thus — the fees are inevitable.
One of the greatest data management and data warehouse design challenges I faced was while working as a designer and DBA of a multi-terabyte Oracle project for a Tier 1 Investment bank. The project encompassed over a hundred designers, developers, and testers, all running in three parallel development streams, capped off with several System, and User Acceptance Test (UAT) projects in parallel. It was a nightmare to manage.
One of my responsibilities was to help design the procedures to manage fifteen multi-terabyte warehouse environments, ensuring everyone was running with the correct code version, database changes were correctly applied, and every platform loaded with the correct data.
This article describes the traditional database and machine architectures from Oracle, Microsoft, and IBM. It provides a high-level description of the main solutions, including shared memory, shared disk, and shared nothing. It describes how each option attempts to address the challenges of performance and scalability and describes the advantages and drawbacks of each.
It goes on to explain how a Data Warehouse startup has revolutionised database design with a solution designed for the cloud.