Separate out analysis to preserve commit stage processing time
The entry point of a Continuous Delivery pipeline is its Commit Stage, and as such manages the compilation, unit testing, analysis, and packaging of source code whenever a change is committed to version control. As the commit stage is responsible for identifying defective code it represents a vital feedback loop for developers, and for that reason Dave Farley and Jez Humble recommend a commit stage that is “ideally less than five minutes and no more than ten” – if the build process is too slow or non-deterministic, the pace of development can soon grind to a halt.
Continuous Delivery and DevOps are interdependent, not equivalent
Since the publication of Dave Farley and Jez Humble’s seminal book on Continuous Delivery in 2010, its rise within the IT industry has been paralleled by the growth of the DevOps movement. While Continuous Delivery has an explicit goal of optimising for cycle time and an established set of principles and practices, DevOps is a more organic philosophy that is defined as “aligning development and operations roles and processes in the context of shared business objectives“, and gradually codifying into principles and practices. Continuous Delivery and DevOps possess a shared background in agile methods and Lean Thinking, and a shared desire to eliminate Waterscrumfall silos – but what is the nature of their relationship?
Building Continuous Delivery into an organisation requires radical change
While Continuous Delivery has a well-defined value proposition and a seminal bookon how to implement a deployment pipeline, there is a dearth of information on how to transform an organisation for Continuous Delivery. Despite its culture-focussed principles and an adoption process described by Jez Humble as ”organisational-architecture-process not tools-code-infrastructure“, many Continuous Delivery initiatives fail to emphasise an organisational model in which software is always releasable. This contravenes Lean Thinking and the Deming 95/5 Rule – that 95% of problems are attributable to system faults, while only 5% are due to special causes of variation. Building an automated deployment pipeline can eliminate the 5% of special causes of variation in our value stream (e.g. release failures), but it cannot address the remaining 95% of problems caused by our organisation structure (e.g. wait times between silos). From this we can infer that:
Technical debt is a fact of life for anyone working in software development: work that needs to be done to make the system cleaner and simpler and cheaper to run over the long term, but that the business doesn't know about or doesn't see as a priority. This is because technical debt is mostly hidden from the people that use the system: the system works ok, even if there are shortcuts in design that make the system harder for developers to understand and change than it should be; or code that’s hard to read or that has been copied too many times; maybe some bugs that the customers don’t know about and that the development team is betting they won’t have to fix; and the platform has fallen behind on patches.
It’s the same for most application security vulnerabilities. The system runs fine, customers can’t see anything wrong, but there’s something missing or not-quite-right under the hood, and bad things might happen if these problems aren't taken care of in time.
One of the big draws of the O'Reilly Software Architecture Conference was Adrian Cockcroft's talk, "Deliver Faster and Spend Less with Cloud Native Microservices." Cockcroft is an experienced speaker on the conference circuit and he's well-known as the architect who led Netflix into its new era of unprecedented scale and agility.
He now works for Battery Ventures, but he still draws primarily on his experiences at Netflix for his talks. He and his team were the ones behind the greatest success story for the latest trend in software architecture: microservices.
Containers weighing you down? Kubernetes can scale them.
To build a reliable, scalable containerized application, you need a place to run containers, scale them, update them, and provide them with networking and storage. Kubernetes is the most popular container orchestration system. It simplifies deploying, monitoring, and scaling application components, making it easy to develop flexible, reliable applications. This updated Refcard covers all things Kubernetes, exploring core concepts, important architecture considerations, and how to build your first containerized application.
The responsibility and accountability for security is rapidly shifting toward DevOps engineers, as they have greater visibility into the broader architecture of processes and systems used to deploy applications. Effective DevSecOps makes application deployments, operations, and service monitoring easier and more secure. In particular, DevOps engineers will be responsible for securing the Infrastructure as Code in which they build. In this Refcard, we explore IaC security, how it works, why it's important, and core practices for success.
Modern cloud environments are ever-changing, and so is the nature of cloud computing. The growing cloud assets accompany the attack surface expansion problem for organizations, which unveils the need for visibility of cloud resources. AWS Config addresses that exact demand. It can be challenging to understand resources within your infrastructure like:
- Seeing what resources you have
- Understanding your current configurations
- Knowledge of configuration changes and change histories
- Assessing if your resources are compliant with specific governances controls
- Having accurate and up-to-date audit information
Depending on the size of your AWS resources or deployment, overcoming these challenges and obtaining this information can become time-consuming and budget-intensive unless you use resource visibility and auditing tool like AWS Config.
I really like Maven, and I really like the declarative build style, but recently I finally came to understand why Gradle is better.
For small projects that produce a common library JAR, you can still use Maven, but real-life, complex software projects always contain a lot of support scripts for deployment, copying artifacts, and so on. For some of those tasks you can find Maven plug-ins, for most of them you can write Maven plugins, but in real life you have shell scripts to do the job.
From new and changing data protection laws to the widespread occurrence of data breaches, consumers are more aware than ever of how their data is being used and when their privacy is compromised. Companies must commit to an intentional, strategic approach for ensuring compliance across their infrastructure, throughout the software development and delivery lifecycles, and within the very fabric of their DevOps culture.
This Refcard covers the fundamental components of DevOps compliance as well as key steps for organizations to take in order to meet regulatory and security requirements, improve operational efficiency, and adapt as policies inevitably change.
The goal for a successful continuous delivery process is to minimize the time it takes for development teams to go from idea to usable software by practicing agile techniques and automating the entire software delivery system: build, deploy, test, release. This Refcard explains detailed patterns and anti-patterns for core areas of CD, including the delivery and deployment phases, rollbacks, pipeline observability and monitoring, documentation, as well as communication across teams and within the organization.
Kubernetes is everywhere. Transactional apps, video streaming services, and machine learning workloads are finding a home on this ever-growing platform. But what about databases? If you had asked me this question five years ago, the answer would have been a resounding “No!” — based on my experience in development and operations. In the following years, as more resources emerged for stateful applications, my answer would have changed to “Maybe,” but always with a qualifier: “It’s fine for development or test environments…” or “If the rest of your tooling is Kubernetes-based, and you have extensive experience…”
But how about today? Should you run a database on Kubernetes? With complex operations and the requirements of persistent, consistent data, let’s retrace the stages in the journey to my current answer: “In a cloud-native environment? Yes!”
Chaos Mesh is a cloud-native chaos engineering platform that orchestrates chaos experiments on Kubernetes environments. It allows you to test the resilience of your system by simulating problems such as network faults, file system faults, and Pod faults. After each chaos experiment, you can review the testing results by checking the logs.
But this approach is neither direct nor efficient. Therefore, I decided to develop a daily reporting system that would automatically analyze logs and generate reports. This way, it’s easy to examine the logs and identify the issues.
If you have anything to do with IT infrastructure, operations management, or DevOps, you've certainly come across the term "system hardening." While system hardening seems like a regular everyday activity in large IT teams with diverse infrastructure, the use and benefits cover a wide range of functions that secure your systems.
This blog explores what system hardening is, why it's needed, its benefits, and how Chef enables DevSecOps teams to harden heterogeneous systems with speed and ease.
Jenkins Pipeline is an automation solution that lets you create simple or complex (template) pipelines via the DSL used in each pipeline. Jenkins provides two ways of developing a pipeline — scripted and declarative. Traditionally, Jenkins jobs were created using Jenkins UI called freestyle jobs. In Jenkins 2.0, Jenkins introduced a new way to create jobs using the technique called pipeline as code. In the pipeline as code technique, jobs are created using a script file that contains the steps to be executed by the job. In Jenkins, that scripted file is called a Jenkinsfile. In this Jenkins tutorial, we will take a deep dive into Jenkins Declarative Pipeline with the help of Jenkins Declarative Pipeline examples.
Let’s get started with the basics.
