Posted on

2 Billion MySQL Records

Yesterday Gary Gray, a friend of mine sent me the following screenshot. He's got a database table with 2 billion records he intends to use Magic on, and he wanted to show it to me, thinking I'd probably be like, "That's cool." Gary works for Dialtone in South Africa, and is used to handling a "monster amount of data". For the record, neither Gary nor I would want to encourage anyone to handle 2 billion records in MySQL, but as you can obviously, see it is possible.

This of course is a legacy system. If it was started today, Gary would obviously use ScyllaDB, Cassandra, or something similar. Simply counting the records in the table above requires 5 minutes of execution in MySQL workbench. Obviously, such sizes are not for those faint at heart. Such a database also implies a lot of inserts.  This makes it impossible to add indexes, resulting in every select query you do towards it having some sort of where clause in it that implies a full table scan. However, it is possible.

Posted on

Using Render and Go for the First Time

My ability to remember things can drive my wife crazy. Earlier this week, she asked me two questions after I finished telling her a story that I had recalled from my long-term memory: 

  • How do you remember such things? 

Posted on

A Dynamic Task Scheduler for ASP.NET Core

There are many task schedulers for ASP.NET. Two of the more famous ones are Quartz.NET and Hangfire. Both of these are amazing tools, and I have personally used Hangfire myself with great success. But, they both suffer from a lack of flexibility and/or complexity in usage. 

Quartz.NET suffers from an overly complex API, based upon configuration files, in a difficult to understand format, giving you no syntax highlighting or AutoComplete when you declare your tasks. In addition, the amount of code required to actually create a simple task sometimes feels staggeringly large and unnecessarily complex. The fact that it was "ported" from Java shows as you try to learn its API.