Tips, Expertise, Articles and Advice from the Pro's for Your Website or Blog to Succeed
"Service mesh" is an umbrella term for products that seek to solve the problems that microservices' architectures create. These challenges include security, network traffic control, and application telemetry. The resolution of these challenges can be achieved by decoupling your application at layer five of the network stack, which is one definition of what service meshes do.
Apache Flume is an efficient, distributed, reliable, and fault-tolerant data-ingestion tool. It facilitates the streaming of huge volumes of log files from various sources (like web servers) into the Hadoop Distributed File System (HDFS), distributed databases, such as HBase on HDFS, or even destinations like Elasticsearch at near-real time speeds. In addition to streaming log data, Flume can also stream event data generated from web sources like Twitter, Facebook, and Kafka Brokers.
Apache Flume was developed by Cloudera to provide a way to quickly and reliably stream large volumes of log files generated by web servers into Hadoop. There, applications can perform further analysis on data in a distributed environment. Initially, Apache Flume was developed to handle only log data. Later, it was equipped to handle event data as well.
Hive and Spark are two very popular and successful products for processing large-scale data sets. In other words, they do big data analytics. This article focuses on describing the history and various features of both products. A comparison of their capabilities will illustrate the various complex data processing problems these two products can address.
More on the subject:
Load balancing, traffic management, authentication and authorization, service discovery — these are just some of the interactions taking place between microservices. Collectively called a “service mesh,” these interconnections can become an operations headache when handling large‑scale, complex applications.
Istio seeks to reduce this complexity by providing engineers with an easy way to manage a service mesh. It does this by implementing a sidecar approach, running alongside each service (in Kubernetes, within each pod), and intercepting and managing network communication between the services. Istio can be used to more easily configure and manage load balancing, routing, security, and the other types of interactions making up the service mesh.
Logstash plays an extremely important role in any ELK-based data pipeline but is still considered as one of the main pain points in the stack. Like any piece of software, Logstash has a lot of nooks and crannies that need to be mastered to be able to log with confidence.
One super-important nook and cranny is the Logstash configuration file (not the software’s configuration file (/etc/logstash/logstash.yml) but the .conf file responsible for your data pipeline). How successful you are at running Logstash is directly determined from how well versed you are at working with this file and how skilled you are at debugging issues that may occur if misconfiguring it.
In a previous post, we introduced a new integration with Microsoft Azure that makes it easy to ship Azure logs and metrics into Logz.io using a ready-made deployment template. Once in Logz.io, this data can be analyzed using the advanced analytics tools Logz.io has to offer — you can query the data, create visualizations and dashboards, and create alerts to get notified when something out of the ordinary occurs.
In this article, we'll take a look at how to collect and analyze a specific type of log data Azure makes available — Azure Activity Logs.
Microsoft Azure has long proven it’s a force to be reckoned with in the world of cloud computing. Over the past year, Azure has made some significant steps in bridging the gap with AWS by offering new services and capabilities as well as competitive pricing.
A growing number of our users are Azure fans, and so we’re happy to introduce a new Logz.io integration for Azure as well as premade dashboards for monitoring different Azure resources!
In a previous article, we explained the importance of monitoring the performance of your servers. Keeping tabs on metrics such as CPU, memory, disk usage, uptime, network traffic, and swap usage will help you gauge the general health of your environment as well as provide the context you need to troubleshoot and solve production issues.
In the past, command line tools, such as top, htop, or nstat, might have been enough, but in today’s modern IT environments, a more centralized approach for monitoring must be implemented.
Kafka and the ELK Stack — usually these two are part of the same architectural solution, Kafka acting as a buffer in front of Logstash to ensure resiliency. This article explores a different combination — using the ELK Stack to collect and analyze Kafka logs.
More on the subject:
Let’s start with the happy ending — after a long search, we managed to identify a Netty memory leak in one of our log listeners and were able to troubleshoot and fix the issue on time before the service crashed.
More on the subject can be found below:
Gaining visibility into modern IT environments is a challenge that an increasing number of organizations are finding difficult to overcome.
Yes — the advent of cloud computing and virtually “unlimited” storage has made it much easier to solve some of the traditional challenges involved in gaining visibility. However, architecture has evolved into microservices, containers, and scheduling infrastructure. Software stacks and the hardware supporting them are becoming more complex, creating additional and different challenges. These changes have directly impacted the complexity of logging and it takes a very specific set of tools and strategies to be able to solve this visibility challenge.