Continuous Integration Patterns and Anti-Patterns

Reap the full benefits of enhanced code quality, better testing practices, and early error detection with proper implementation of continuous integration processes. This Refcard explains detailed patterns and anti-patterns for core areas of CI, including version control, the build stage, pipeline monitoring, documentation, as well as communication and collaboration across teams and within the organization.

On Git and Cognitive Load

Any developer working in a modern engineering organization is likely working in git. Git, written by Linus Torvalds in 2005, has been since its creation, basically the ubiquitous tool for source control management. Git gained widespread adoption across engineering organizations as the successor to existing source control management such as Apache Subversion (SVN) and Concurrent Versions System (CVS), and this was mostly a byproduct of the timing.

Git predates cloud and improved network connectivity, and therefore the best solution to managing source control at scale was to decentralize and leverage local dev environments to power engineering organizations. Fast forward 17 years, this is no longer the case. Cloud is de facto, networking and internet connectivity is blazing fast, and this changes everything.

Simple code: Version control commits

Currently the most popular version control system is git and I'll be writing this based on git and it's functionalities and capabilities.

Git is often seen as a way to enable distributed programming i.e. multiple programmers can work on the same code repository quite easily without disturbing each others work (much). In addition to that just like other VCS's it's also a log of work but to my experience that part is often unfortunately neglected. What I will be focusing this time is the log part because I think it deserves more attention.

Why to create a meaningful log?

The git log should consist from small meaningful changesets where each commit addresses a single problem. By dividing the log to small commits it enables resilient way of working. Being resilient enables simple and fast procedures to rollbacks, reviews, tags, branching etc.

Lets say that a developer is implementing a REST API. The API needs a web layer that receives the HTTP requests, it probably has some sort of logic layer to do data transformations and validations and maybe some calculations and finally it has a data storage where the data is persisted. There are options how to record this work to the log. One option would be to implement the API and record a single commit or squash the commits before pushing the changes to remote so it would become a single commit. Another option would be to record commits every now and then while developing and finally push those commits as is to the remote repository. Yet another way would be to carefully pick what is recorded per commit in order to have a set of meaningful commits that each address a single issue.

Example of the first approach would be something like this:

A Programmer Learning List (for Beginners)

My friend has a son who's graduating high school soon. He's been learning some programming and is considering it for his career. He asked me a question I hear often: what should I learn next?

When I was first learning to code, I always assumed the answer to "what should I learn next" would be a new programming technique, a new language, a new library, or something along those lines. As I've progressed through my career, and especially as I've been on the other side of the interview desk, I've changed my tune significantly. My standard response to new programmers is that, in addition to honing their coding skills and learning new languages, they should cross-train in related fields (which I'll explain below).

How to Automate Project Versioning and Releases with Continuous Deployment

Having a semantically versioned software will help you easily maintain and communicate changes in your software. Doing this is not easy. Even after manually merging the PR, tagging the commit, and pushing the release, you still have to write release notes. There are a lot of different steps, and many are repetitive and take time.

Let’s look at how we can make a more efficient flow and completely automating our release process by plugin semantic versioning into a continuous deployment process.

Semantic versioning

A semantic version is a number that consists of three numbers separated by a period. For example, 1.4.10 is a semantic version. Each of the numbers has a specific meaning.

Major change

The first number is a Major change, meaning it has a breaking change.

Minor change

The second number is a Minor change, meaning it adds functionality.

Patch change

The third number is a Patch change, meaning it includes a bug fix.

It is easier to look at semantic versioning as Breaking . Feature . Fix. It is a more precise way of describing a version number that doesn’t leave any room for interpretation.

Commit format

To make sure that we are releasing the correct version — by correctly incrementing the semantic version number — we need to standardize our commit messages. By having a standardized format for commit messages, we can know when to increment which number and easily generate a release note. We are going to be using the Angular commit message convention, although we can change this later if you prefer something else.

It goes like this:

<header>
<optional body>
<optional footer>

Each commit message consists of a header, a body, and a footer.

The commit header

The header is mandatory. It has a special format that includes a type, an optional scope, and a subject.

The header’s type is a mandatory field that tells what impact the commit contents have on the next version. It has to be one of the following types:

  • feat: New feature
  • fix: Bug fix
  • docs: Change to the documentation
  • style: Changes that do not affect the meaning of the code (e.g. white-space, formatting, missing semi-colons, etc.)
  • refactor: Changes that neither fix a bug nor add a feature
  • perf: Change that improves performance
  • test: Add missing tests or corrections to existing ones
  • chore: Changes to the build process or auxiliary tools and libraries, such as generating documentation

The scope is a grouping property that specifies what subsystem the commit is related to, like an API, or the dashboard of an app, or user accounts, etc. If the commit modifies more than one subsystem, then we can use an asterisk (*) instead.

The header subject should hold a short description of what has been done. There are a few rules when writing one:

  • Use the imperative, present tense (e.g. “change” instead of “changed” or “changes”).
  • Lowercase the first letter on the first word.
  • Leave out a period (.) at the end.
  • Avoid writing subjects longer than 80 charactersThe commit body.

Just like the header subject, use the imperative, present tense for the body. It should include the motivation for the change and contrast this with previous behavior.

The footer should contain any information about breaking changes and is also the place to reference issues that this commit closes.

Breaking change information should start with BREAKING CHANGE: followed by a space or two new lines. The rest of the commit message goes here.

Enforcing a commit format

Working on a team is always a challenge when you have to standardize anything that everyone has to conform to. To make sure that everybody uses the same commit standard, we are going to use Commitizen.

Commitizen is a command-line tool that makes it easier to use a consistent commit format. Making a repo Commitizen-friendly means that anyone on the team can run git cz and get a detailed prompt for filling out a commit message.

Generating a release

Now that we know our commits follow a consistent standard, we can work on generating a release and release notes. For this, we will use a package called semantic-release. It is a well-maintained package with great support for multiple continuous integration (CI) platforms.

semantic-release is the key to our journey, as it will perform all the necessary steps to a release, including:

  1. Figuring out the last version you published
  2. Determining the type of release based on commits added since the last release
  3. Generating release notes for commits added since the last release
  4. Updating a package.json file and creating a Git tag that corresponds to the new release version
  5. Pushing the new release

Any CI will do. For this article we are using GitHub Action, because I love using a platform’s existing features before reaching for a third-party solution.

There are multiple ways to install semantic-release but we’ll use semantic-release-cli as it provides takes things step-by-step. Let’s run npx semantic-release-cli setup in the terminal, then fill out the interactive wizard.

Th script will do a couple of things:

  • It runs npm adduser with the NPM information provided to generate a .npmrc.
  • It creates a GitHub personal token.
  • It updates package.json.

After the CLI finishes, it wil add semantic-release to the package.json but it won’t actually install it. Run npm install to install it as well as other project dependencies.

The only thing left for us is to configure the CI via GitHub Actions. We need to manually add a workflow that will run semantic-release. Let’s create a release workflow in .github/workflows/release.yml.

name: Release
on:
  push:
    branches:
      - main
jobs:
  release:
    name: Release
    runs-on: ubuntu-18.04
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Setup Node.js
        uses: actions/setup-node@v1
        with:
          node-version: 12
      - name: Install dependencies
        run: npm ci
      - name: Release
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        # If you need an NPM release, you can add the NPM_TOKEN
        #   NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
        run: npm run release

Steffen Brewersdorff already does an excellent job covering CI with GitHub Actions, but let’s just briefly go over what’s happening here.

This will wait for the push on the main branch to happen, only then run the pipeline. Feel free to change this to work on one, two, or all branches.

on:
  push:
    branches:
      - main

Then, it pulls the repo with checkout and installs Node so that npm is available to install the project dependencies. A test step could go, if that’s something you prefer.

- name: Checkout
uses: actions/checkout@v2
- name: Setup Node.js
uses: actions/setup-node@v1
with:
    node-version: 12
- name: Install dependencies
run: npm ci
# You can add a test step here
# - name: Run Tests
# run: npm test

Finally, let semantic-release do all the magic:

- name: Release
run: npm run release

Push the changes and look at the actions:

The GitHub Actions screen for a project showing a file navigating on the left and the actions it includes on the right against a block background.

Now each time a commit is made (or merged) to a specified branch, the action will run and make a release, complete with release notes.

Showing the GitHub releases screen for a project with an example that shows a version 1.0.0 and 2.0.0, both with release notes outlining features and breaking changes.

Release party!

We have successfully created a CI/CD semantic release workflow! Not that painful, right? The setup is relatively simple and there are no downsides to having a semantic release workflow. It only makes tracking changes a lot easier.

semantic-release has a lot of plugins that can make an even more advanced automations. For example, there’s even a Slack release bot that can post to a project channel once the project has been successfully deployed. No need to head over to GitHub to find updates!


The post How to Automate Project Versioning and Releases with Continuous Deployment appeared first on CSS-Tricks.

You can support CSS-Tricks by being an MVP Supporter.

AWS Lambda Functions and Layers

Once upon a time, we used to have lambda functions with deployment packages reach 50MiB. We tended to test locally, zip all dependencies, and wait until it became available for testing after uploading the files. What makes it worse is when we have bad internet. Fortunately, now we have Layers!!

What are Lambda Layers?

In short, Lambda Layers is an approach that makes appending extra code to a Lambda function easier. It keeps the development package smaller and easy to maintain.

How to Create Your First GitHub Commit

Hello everybody. My name is Mar, and today I want to guide you through the most important step of your Tech career, your first interaction with Github.

I started off as a Computer Science student not long ago, in 2016. However, it took me a whole year to give this Github thing a try. I was hesitant and confused at first, but it did not turn out to be as complex as my peers made it sounds when they were talking about it. All it takes is some motivation! Now let’s dive right into it.

Gitting Started With Git

Let Octocat be your guide

In this edition of "Best of DZone," we take a look at all things Git, from basic commands, to theory behind its functionality, to development and deployment patterns. All tutorials and articles featured below are from our wonderful community members who continue to power DZone by sharing their knowledge of and passion for development with readers like you. 

Without further ado, let's git started!

How to Update PHP to the Recommended Version for WordPress

Using an outdated version of WordPress core, plugins or themes isn’t ideal. PHP is no different.

While WordPress technically works with PHP 5.6 and above, you’re sacrificing performance and compatibility, plus you’re open to security vulnerabilities by using older PHP versions. You should always aim to use the latest version of PHP.

In this post, you’ll learn everything you need to check what version of PHP you are running and how to update your PHP to the recommended version for WordPress.

What is PHP

PHP can do things like calculations, process conditional logic and pull information from the database so you can take a dynamic approach to build your webpages.

Unlike JavaScript, which modifies the HTML in the browser after the page has been constructed, PHP happens on the server. Since PHP happens before the HTML is rendered, you have more control over the final layout of the page.

If you look in your WordPress site’s files, you’ll see a lot of PHP files. These are the instructions that your server uses to build not only the webpages your visitor sees but also the WordPress admin backend. PHP is a major part of WordPress. PHP is a major part of the whole web, actually. It powers 70% of all websites.

WordPress PHP Version

You may recall that somewhat recently there was a big push to get everyone to upgrade WordPress to PHP 7. While it may have felt recent, that was a while ago. PHP has to be updated on a regular basis, just like WordPress core, themes, and plugins.

With PHP, each version receives support for two years after its initial release, then there is a period of another year where critical security updates are released as they’re needed. After this three year period, that version of PHP is no longer updated.

As of June 2019, when I’m writing this post, the current recommended PHP version for WordPress is 7.3 or greater.

The majority of quality hosting providers will require you to use the latest version of PHP in order to use their servers and will help you modify your site to ensure that the latest version of PHP will not break your site.

WPMU DEV Hosting, for instance, has your back and uses the latest PHP version automatically.

Pie chart depicting what PHP version WordPress sites use
Only 4.4% of sites are using the most up-to-date version of PHP

Less reputable cheap hosts will not take the initiative to encourage you to upgrade your PHP because of the additional cost of support time and resources to help you transition. Since they don’t want to break your site by updating to the latest version of PHP automatically, they’ll often leave your site to use an outdated version of PHP and hope you don’t notice.

This practice is quite common and most WordPress sites aren’t using the most recent version of PHP as a result. Take a look at these WordPress statistics:

  • 4.4% of WordPress sites are using PHP version 7.3
  • 18.4% of WordPress sites are using PHP version 7.2
  • 13.2% of WordPress sites are using PHP version 7.1
  • 16.9% of WordPress sites are using PHP version 7.0

PHP 7.0 made it’s debut on December 2015 and using our three-year rule, then we know it reached the end of its lifecycle in December 2018. This means 64% of WordPress sites are running a version of PHP that is no longer being actively supported and that’s problematic for several reasons.

Why you need to update PHP

Speed

For one, there are tremendous performance benefits that come with using the latest version. PHP 7.3 requires less time and resources to deliver the same number of requests to site visitors. It can handle as much as 3x as many requests per second as PHP 5.6.

According to this infographic, PHP 5.6 executes one WordPress request in just under 100M CPU instructions, while PHP 7 does the same using 25M CPU instructions, and PHP 7.3 uses even fewer. That’s a huge improvement that also goes a long way toward reducing memory usage on your host as well.

Since using the latest version of PHP takes less time to serve the same number of requests, you can give your visitor’s a much better user experience.

Ensuring Compatibility is Costly

Maintaining code takes effort, and that effort is multiplied when you need to ensure backward compatibility for additional versions.

No wonder people (probably overworked theme and plugin developers) at WordCamp US 2018 cheered when Matt Mullenweg proposed a plan to make PHP 7.0 the minimum required version by “as early as” December 2019.

Sometimes new versions of PHP introduce new features, such as error handling or new operators, as was the case with PHP 7. Other times, functions are deprecated or changed.

Rather than spending time parsing through the code to ensure that any changes to PHP do not affect the code, developers can spend their time adding new features or improving security instead.

Vulnerabilities

Just like plugins, themes and WordPress core get updates to fix security vulnerabilities, so does PHP. This is also why using a PHP version that is still getting updates is so important, as it protects you from vulnerabilities such as SQL injection, XSS and Dos attacks.

According to security vulnerability datasource CVE Details, there were 18 known vulnerabilities found in PHP in 2018. Compare this to the 43 vulnerabilities discovered in 2017 and the 107 found in 2016 and you can see why updating can help protect you from a lot known security exploits.

As a matter of fact, some hackers look to see what version of PHP you’re running in order to know what kind of attack would be effective.

How to Check PHP Version in WordPress

So how do you find out what version of PHP your site is using? Easy. You can see this information right within WordPress.

Go to Tools > Site Health

Site Health Menu
Site health was recently added in version 5.2

If your site is not using the latest version of PHP, then it will be shown as a recommended improvement.

If it is, then open the passed tests section to verify. You’ll see that is says PHP is up to date plus what version you’re running.

WordPress site health
Obviously if you don’t know what version you’re running yet, then you won’t know where to look exactly

How to Update PHP in WordPress

Once you check to see what version of PHP your hosting server is running, you’ll know if you need to upgrade to the latest version of PHP.

If you do, you’ll first need to see if your site will be compatible with the new version of PHP. You never want to test major changes like updating your PHP on your live site because if there are major incompatibilities, you’ll damage your site.

For this reason, you’ll need to create an alternate site to test your changes on first.

Step 1: Create a Copy of Your Site

You’ll first need to get a copy of your current site. You’ll need all of the files and a copy of the database. You can use SFTP for your files and export a copy of your database using MyPHP.

In some instances, you may be able to use your last backup, if your backups include both the files and the database.

Step 2: Create a Local or Staging Site

Once you have a copy of your site, you’ll need to set up a testing environment. You can either use a local environment or set up a staging environment if your host allows it.

See our post on how to set up XAMPP on your local computer if you need help setting up your local environment.

Once your testing environment is set up, you can add the files and database from the previous step to your testing environment, so you have an exact replica of your live site.

Then once you have a copy of your site in your testing environment set up, you’re ready to upgrade the PHP version of your testing site.

Check PHP Compatibility

Now comes the fun part, and by fun, I actually mean when the boring action happens.

You’ll need to go through your test site to make sure that nothing is broken. You’ll want to pay extra careful attention to the critical functionality of your site. Checkout processes, newsletter signups, web forms, navigation, and all of the other elements that contribute to the bottom line.

You want to catch these issues before they have a chance to wreak havoc on your live site and cause a bad user experience for your visitors.

Checking PHP Compatibility with Tools

There are some tools that can help you with this process.

PHP Compatibility Checker – This plugin is often recommended to help you scan your site for incompatibility issues. Unfortunately, it hasn’t been updated to be able to test PHP 7.3. It also is known to flag false positives, so be sure to double check incompatibilities manually.

PHP Compatibility Checker
PHP Compatibility Checker Dashboard

Error Log Monitor – This plugin will display a list of PHP errors in your WordPress dashboard which will help you diagnose the issues if you’re a developer. If you’re not a developer then it will help you relay the issue to support.

PHP Error Log
Error Log in WordPress Dashboard

WP_DEBUG – WP_DEBUG is a PHP constant (a permanent global variable) that can be used to trigger the “debug” mode throughout WordPress. To turn on debugging in WordPress, set the WP_DEBUG variable to true. You can find it in the wp-config.php file. Check out our other post on Debugging WordPress: How to Use WP_DEBUG.

Fix issues

If a plugin or your theme is causing issues, work out the issue on your test site. Find a suitable alternative that is compatible with the latest version of PHP. Once you’ve found a solution, be sure to also make the change to your live site.

Smush plugin showing compatible PHP versions
Check your active plugins to make sure they’re compatible with the latest version of PHP

When you’re looking for a suitable plugin in the WordPress repo, check to see what versions of PHP are supported.

How to Update PHP Version in WordPress

Once you’ve worked through all of the bugs and issues with compatibility, and have made the corresponding changes to your live site, then you can switch over your live site to the new version of PHP.

This process will vary from host to host, but will most likely involve changing a setting in your hosting cPanel. Ask your host for help on how to do this as some shared hosts will not let you access this setting.

Here’s a list with instructions to how to update the PHP version for 32 different hosts including popular hosts such as WP Engine, Pantheon, Kinsta, Pagely, Bluehost, Hostgator, GoDaddy and more.

If you never want to deal with having to update your PHP version, why not give WPMU DEV hosting a try and leave all the technical stuff to us.

Partnering with Your Host to Update PHP Version

Cheap shared hosting has convinced a lot of people that hosting websites is a simple and straightforward process, but the truth is, there is a lot that goes into maintaining your site files and database. Keeping all of your software up-to-date is one of the tasks that your host can and should help you with. If they’re unwilling to help you or hide your version to keep you in the dark, then that’s generally a sign that it’s time for you to switch hosts to something better.

At WPMU DEV, members get hosting for 3 free sites (even multisite) included with a monthly membership. Not only do we help you update your PHP version, we also backup your site daily, assist with caching configuration and monitor uptime. And if you ever need help fixing compatibility issues with your theme or plugins, our awesome support team is ready and willing to help. Sign up for a free 30-day membership to try everything out.

Graphical User Interfaces for Git

Git is command-line-driven software, but that doesn't mean you have to use the command line to make it work. There are lots of options! Some of the deepest programmer nerds I know prefer to use GUIs for Git (Graphic
User Interface, or you know, software you can see things and click stuff), and some near pure-designers I know prefer working with the command line for Git. Swear to Git.

Lemme round up what look like the major players for Git GUIs these days.


Tower

I've used Tower for ages and it's the one used the most. I'm not sure the exact release dates of all these, but I feel like Tower was an early player here. They've been around a long time and continuously improve, which I always respect.



Fork

It's free and actively developed, incredibly.



GitHub Desktop

This is a 2.0 of the original GitHub Desktop. I had some gripes with the 1.0 version in that its terminology was weird (to me) and seemed to vastly deviate from Git, which was more confusing than it was worth (again, to me). This version cleans most of that up. It's deeply integrated into GitHub so it makes GitHubb-y things (e.g. pull requests) feel like first-class citizens, but it will still happily work with any Git repo.



GitKraken

I'm pretty intrigued by this one. Upgrading (monthly cost) to get the in-app merge conflict tool seems worth it, but you also have to upgrade to access private repos. It seems highly feature rich, but I think my favorite part is the dark-with-rainbow-accent-colors theme.



Sourcetree

You might be compelled by Sourcetree if you're a big Bitbucket user because they are both Atlassian products. I know it works for any Git repo though. I imagine there is some smooth Bitbucket integration stuff with this, similar to the GitHub/GitHub Desktop connection.



Coda

You don't really think of Coda as a version control tool (it's more of a direct-to-FTP thing), and even though I'd argue the support for it is fairly half-baked, it does work! Seems likely the next evolution of Coda will address this.



VS Code

Having version control right in your IDE like this, to me, feels like kind of a tweener between GUI and CLI. There are a lot of features here, but it's not really a full-blown GUI to me, but you've got a terminal built in right there so it almost encourages that. A lot of Git usage is pretty basic pulling, committing, and pushing — so having this right within the app is kinda sweet.

(I imagine there are lots of other IDEs that offer version control features. PHPStorm, etc.)



The post Graphical User Interfaces for Git appeared first on CSS-Tricks.

Managing Infrastructure at Scale With Terraform

Eighty percent of software outages are due to improper changes — changes that occurred at the wrong time or were not properly executed.

To reduce these outages, you need to address the core problem: human error. The reality is that repetitive tasks, such as infrastructure configurations, should be designed by humans and executed by software. Having machines tackle the repetitive tasks reduces errors and leaves us to do what we do best — solve problems. This makes our systems more reliable and better designed.

Use Version Control for Everything

Version control is an important part of file management on every software development project I've ever worked on in the last two decades, regardless of the methodology. I've used Subversion and Git and several others in the process of building software.

I always find it a bit surprising that other industries don't use version control nearly as much as we do in the software industry. For example, I know several writers who know nothing about version control and they would benefit greatly from it. The same is true for several artists and graphics designers who I know. I find that version control it is extremely easy to work with and gives me some key benefits.

Delivering Quality at Speed With GitOps

At the inaugural online summit "Cloud Native Live" hosted by our friends at Twistlock, Weaveworks Customer Success Engineer, Brice Fernandes (@fractallambda) presented "Delivering Quality at Speed with GitOps".

Brice discussed how, by introducing and implementing GitOps best practices into your Kubernetes deployment pipelines, DevOps team can gain velocity without sacrificing quality.

Pushing Database Versioning to Its Limits

As most, current database systems still simply store current state or past states within one big relational table. We investigated what the performance drivers are and how to improve on the current state-of-the-art. We implemented an open-source storage system called Sirix(.io) from scratch, which stores small sized snapshots as well as supports sophisticated time-travel queries while competing with the efficiency of non-temporal database systems.

What Is a Temporal Database System?

It is a term used to describe that a system is capable of retrieving past states of your data. Typically, a temporal database stores both valid time, how long a fact is true in the real world, as well as transaction time when the data actually is committed to the database.