Introduction
This blog post shows how to use Nginx, Lua, Logstash, and Elasticsearch to log, store, and analyze HTTP request and response metadata. The metadata can either be generated by Nginx or by any upstream service that Nginx proxies to. For example, the metadata could be the request time, response size, or any information the upstream returns to log for purposes such as analytics.
First, a small example web service behind an Nginx server generates some data that we’d like to analyze via Elasticsearch. Nginx, with the help of Lua, parses and logs statistics returned by the example upstream service as it receives the response to send back to the client. This leads us to Logstash, which parses these logged statistics and forwards them to Elasticsearch for storage and analysis.