Posted on

How Milvus Deletes Streaming Data in a Distributed Cluster

Featuring unified batch-and-stream processing and cloud-native architecture, Milvus 2.0 poses a greater challenge than its predecessor did during the development of the DELETE function. Thanks to its advanced storage-computation disaggregation design and the flexible publication/subscription mechanism, we are proud to announce that we made it happen. In Milvus 2.0, you can delete an entity in a given collection with its primary key so that the deleted entity will no longer be listed in the result of a search or a query.

Please note that the DELETE operation in Milvus refers to logical deletion, whereas physical data cleanup occurs during the Data Compaction. Logical deletion not only greatly boosts the search performance constrained by the I/O speed, but also facilitates data recovery. Logically deleted data can still be retrieved with the help of the Time Travel function.

Posted on

Edge Persistence Explained

Have you heard the term “edge persistence” floating around the webiverse? If so what does it mean to you? If your answer is “not sure” then this blog is for you! If you think you have an idea, let me know where you think I got it right and where I might have been off. Edge computing is the first aspect of this concept. Edge computing is the salt to cloud’s pepper. Blending edge computing with the cloud creates flexibility you would not be able to achieve with one or the other, with the added benefits of improved performance and reduced latency. So what do I mean when I say salt and pepper? They go great on dishes separately, but when combined can add the perfect finish to your cooking. Edge computing brings your computation and data storage closer to the location where you need it. It improves performance and reduces latency when running your application or technology. Now for the word persistence, I used a good old-fashioned dictionary. To be persistent is “existing for a long or longer than usual time or continuously.”

So edge persistence allows companies to globally distribute their applications, software, and technologies closer to the end-user location where it improves performance and reduce latency. When I say latency, what does that mean? Latency “is an expression of how much time it takes for a data packet to travel from one designated point to another”. It does so continuously for long periods of time.” I could end this blog here, but that wouldn’t be a very informative blog, so let’s go deeper.

Posted on

Distributed Letters Animation Layout

A while back I stumbled upon this great Dribbble shot by Seth Eckert. I love that letter effect and so I was thinking about a layout where I could incorporate this kind of animation.

After playing around a bit, I thought that a layout that allows for opening a panel would be interesting. So here it is; a little layout with a distributed letter animation on hover. When clicking, the letters will move to their respective place in the panel and the other elements will animate in.

I really hope you like this and that it comes in handy.

Thank you for checking by!

The post Distributed Letters Animation Layout appeared first on Codrops.

Posted on

SKP’s Java/Java EE Gotchas: Clustered Nodes Issue Using Apache Lucene 5.4.y

So, We just spent about 28+ days — using Stack Overflow and Hibernate Search Forums as also various other resources to 'FIX' the strangest of issues using Hibernate Search and Apache Lucene. The right help or pointer came from the 'Hibernate Search Committers/Creators' (Official Forums). Thanks a lot to 'sanne.grinovero'.

Issue Description: In a (non-replicated) clustered environment, if the Indexing by Lucene or Hibernate Search is done from the Same Database (Instance) using the Same Data the Results obtained are Different on Different Nodes. In other words, the Same Data from the Same Database Instance Indexed using the Same Code Yields a Different Result Set for the Same Search Term (Index Contents are Different on Different Machines).


Issue Resolution / Root Cause: The point to note is that, if you open a 'Hibernate Session' that you use for your 'Apache Lucene or Hibernate Search Indexing' and if a 'Runtime/Exception' occurs before you 'Close' the session, the results that are obtained across different machines, when queried using the same search term. Though this occurred after the 'console logs' reported that the Index is '100% Successfully Completed' — It has led to an inconsistent index across machine. My analysis is that 'session.close()' below may be leading to some 'flushing or committing to the indexing', which leads to the inconsistency when a runtime exception occurs — even after indexing is reported complete by the Hibernate/Hibernate Search on the 'Console Logs'.

Suggestion: Make sure that there are no Runtime Exceptions (or Errors) before Closing the Session. Even if you find that the Indexing is 100% Complete and there is an Exception right at the End — Please do not Ignore/Keep/Park it (Considering it a Minor or Unrelated Error) for a fix at the 'End of Dev Cycle' as it may Impact Results. This is true for any seemingly unrelated exception that may occur before calling 'session.close()'.

I am pasting the statement here from Hibernate Search Forums (Committer), which may be useful for all of you and helped us to finally resolve this error:  If a batch of documents failed, you might have a block of documents missing in one of the indexes (and this could happen several times). By default errors are logged, you might want to hook up an ErrorHandler to raise a more serious notification to your admins — if logs are ignored.



The culprit line of code, pointed out in red. [CULPRIT LINE OF CODE - WHERE RUNTIME EXCEPTION OCCURS - POINTED OUT USING ARROW]

Posted on

Make Your Existing Solution Tastier with Serverless Salt: NoOps

This article is the fourth in a series on serverless computing. I recommend starting at the beginning of the series, as I introduce concepts incrementally. The links to previous articles on DZone are included here:

In the previous articles, I demonstrated the benefits and challenges of serverless integration for business applications and illustrated them with an example based on the Bonita platform. As you know by now if you have been reading along, the main benefit of serverless is to abstract infrastructure with ephemeral containers using a FaaS service on the cloud.