key-value store | The Blog Pros

January 7, 2022

Debunking Four Metadata Myths

Data is being accumulated at an alarming rate due to the growth of unstructured data. Within that is metadata, which allows us to quickly find data files by identifying certain properties. It wasn’t until recently that developers had to pay closer attention due to the overwhelming effect it was having on our data engines. Metadata used to just be a thing that could be stored in memory behind the scenes without any worry, but its expansion presents the growing concern of where to store it and how to manage it.

Every database system, whether SQL or NoSQL, uses a data engine (aka storage engine), embedded or not, to manage how data is stored. Our ever-growing volume of data has outgrown our data engines’ infrastructures and with that, they are stretched to their limits and forced to make painful trade-offs necessary to support the scale and performance requirements of modern datasets.

December 31, 2021

How to Update Object Store Values Using Object Store V2 REST APIs: Mule4

Object Store in MuleSoft is used to store key-value pairs that we might need to preserve for future use. In this tutorial, we will learn how we can use Object Store V2 REST APIs to dynamically update the values present in our object store without making any changes to our code.

For a detailed explanation on Object Store and rest APIs please refer to official MuleSoft documentation.

November 9, 2020

Improving Apache Cassandra’s Front Door and Backpressure

We have improved Apache Cassandra's ability to handle high throughput workloads, while having enough safeguards in place to protect itself from potentially going out of memory. In order to better explain the change we have made, let us understand at a high level, on how an incoming request is processed by Cassandra before the fix, followed by what we changed, and the new relevant configuration knobs available.

How Inbound Requests Were Handled Before

Let us take the scenario of a client application sending requests to C* cluster. For the purpose of this blog, let us focus on one of the C* coordinator nodes.

May 24, 2019

Thoughts on Apache Ignite

Yahoo! JAPAN is one of the largest e-commerce platforms in Japan, and we are constantly working on improving users' shopping experience from attempts to understand their needs and behaviors and applying that knowledge to provide better shopping experience. We have a large number of services and APIs for that, and as the number of requests to them is growing (due to the release of new services, or an increase of calls from numerous microservices) we have to be able to scale accordingly and guarantee sub-second data access.

One of the services with a high demand for being able to process tens of thousands of requests per second and grow in near future is 'Recent purchases.' Querying the service for such purchases provides you the latest n items by particular categories in a specified depth. Originally, the data was stored in RDBMS and reached 500+ million records, and the old DB infrastructure couldn't cope with increasing number of requests and prevented the service adoption by other users. That now was the time to renovate the DB to meet higher demand from users.