Distributed Lucene: Full Text Searching in .NET for Scalability

As data has lately been coined as “the new currency,” Apache Lucene has gained traction as a popular full-text search engine, widely used in applications to incorporate flexible text search over huge amounts of textual data. Lucene uses inverted indexing, drastically cutting down the time to find documents related to a particular term.

However, it is a stand-alone solution that does not scale as your data grows — you need to rebuild entire Lucene indexes to search data which is an expensive and slow task, becoming a performance bottleneck. While a few Java and REST-based solutions now exist to cater to scalable full-text search, there is still a lack of a scalable full-text searching solution that may naturally fit in the .NET stack.