Today, we’ll briefly discuss why it’s important to write automated tests for any software project, and shed light on some of the common types of automated testing. We’ll build a to-do list app by following the Test-Driven Development (TDD) approach. I’ll show you how to write both unit and functional tests, and in the process, explain what code mocks are by mocking a few libraries. I’ll be using a combination of RTL and Jest — both of which come pre-installed in any new project created with Create-React-App (CRA).

To follow along, you need to know how to set up and navigate a new React project and how to work with the yarn package manager (or npm). Familiarities with Axios and React-Router are also required.

Why You Should Test Your Code

Before shipping your software to end-users, you first have to confirm that it is working as expected. In other words, the app should satisfy its project specifications.

Just as it is important to test our project as a whole before shipping it to end-users, it’s also essential to keep testing our code during the lifetime of a project. This is necessary for a number of reasons. We may make updates to our application or refactor some parts of our code. A third-party library may undergo a breaking change. Even the browser that is running our web application may undergo breaking changes. In some cases, something stops working for no apparent reason — things could go wrong unexpectedly. Thus, it is necessary to test our code regularly for the lifetime of a project.

Broadly speaking, there are manual and automated software tests. In a manual test, a real user performs some action on our application to verify that they work correctly. This kind of test is less reliable when repeated several times because it’s easy for the tester to miss some details between test runs.

In an automated test, however, a test script is executed by a machine. With a test script, we can be sure that whatever details we set in the script will remain unchanged on every test run.

This kind of test gives us the benefits of being predictable and fast, such that we can quickly find and fix bugs in our code.

Having seen the necessity of testing our code, the next logical question is, what sort of automated tests should we write for our code? Let’s quickly go over a few of them.

Types Of Automated Testing

There are many different types of automated software testing. Some of the most common ones are unit tests, integration tests, functional tests, end-to-end tests, acceptance tests, performance tests, and smoke tests.

1. Unit test
   In this kind of test, the goal is to verify that each unit of our application, considered in isolation, is working correctly. An example would be testing that a particular function returns an expected value, give some known inputs. We’ll see several examples in this article.
2. Smoke test
   This kind of test is done to check that the system is up and running. For example, in a React app, we could just render our main app component and call it a day. If it renders correctly we can be fairly certain that our app would render on the browser.
3. Integration test
   This sort of test is carried out to verify that two or more modules can work well together. For example, you might run a test to verify that your server and database are actually communicating correctly.
4. Functional test
   A functional test exists to verify that the system meets its functional specification. We’ll see an example later.
5. End-to-end test
   This kind of test involves testing the application the same way it would be used in the real world. You can use a tool like cypress for E2E tests.
6. Acceptance test
   This is usually done by the business owner to verify that the system meets specifications.
7. Performance test
   This sort of testing is carried out to see how the system performs under significant load. In frontend development, this is usually about how fast the app loads on the browser.

There’s more here if you’re interested.

Why Use React Testing Library?

When it comes to testing React applications, there are a few testing options available, of which the most common ones I know of are Enzyme and React Testing Library (RTL).

RTL is a subset of the @testing-library family of packages. Its philosophy is very simple. Your users don’t care whether you use redux or context for state management. They care less about the simplicity of hooks nor the distinction between class and functional components. They just want your app to work in a certain way. It is, therefore, no surprise that the testing library’s primary guiding principle is

“The more your tests resemble the way your software is used, the more confidence they can give you.”

So, whatever you do, have the end-user in mind and test your app just as they would use it.

Choosing RTL gives you a number of advantages. First, it’s much easier to get started with it. Every new React project bootstrapped with CRA comes with RTL and Jest configured. The React docs also recommend it as the testing library of choice. Lastly, the guiding principle makes a lot of sense — functionality over implementation details.

With that out of the way, let’s get started with building a to-do list app, following the TDD approach.

Project Setup

Open a terminal and copy and run the below command.

# start new react project and start the server
npx create-react-app start-rtl && cd start-rtl && yarn start

This should create a new React project and start the server on http://localhost:3000. With the project running, open a separate terminal, run yarn test and then press a. This runs all tests in the project in watch mode. Running the test in watch mode means that the test will automatically re-run when it detects a change in either the test file or the file that is being tested. On the test terminal, you should see something like the picture below:

You should see a lot of greens, which indicates that the test we’re running passed in flying colors.

As I mentioned earlier, CRA sets up RTL and Jest for every new React project. It also includes a sample test. This sample test is what we just executed.

When you run the yarn test command, react-scripts calls upon Jest to execute the test. Jest is a JavaScript testing framework that’s used in running tests. You won’t find it listed in package.json but you can do a search inside yarn.lock to find it. You can also see it in node_modules/.

Jest is incredible in the range of functionality that it provides. It provides tools for assertions, mocking, spying, etc. I strongly encourage you to take at least a quick tour of the documentation. There’s a lot to learn there that I cannot scratch in this short piece. We’ll be using Jest a lot in the coming sections.

Open package.json let’s see what we have there. The section of interest is dependencies.

  "dependencies": {
    "@testing-library/jest-dom": "^4.2.4",
    "@testing-library/react": "^9.3.2",
    "@testing-library/user-event": "^7.1.2",
    ...
  },

We have the following packages installed specifically for testing purpose:

1. @testing-library/jest-dom: provides custom DOM element matchers for Jest.
2. @testing-library/react: provides the APIs for testing React apps.
3. @testing-library/user-event: provides advanced simulation of browser interactions.

Open up App.test.js let’s take a look at its content.

import React from 'react';
import { render } from '@testing-library/react';
import App from './App';

test('renders learn react link', () => {
  const { getByText } = render();
  const linkElement = getByText(/learn react/i);
  expect(linkElement).toBeInTheDocument();
});

The render method of RTL renders the <App /> component and returns an object which is de-structured for the getByText query. This query finds elements in the DOM by their display text. Queries are the tools for finding elements in the DOM. The complete list of queries can be found here. All of the queries from the testing library are exported by RTL, in addition to the render, cleanup, and act methods. You can read more about these in the API section.

The text is matched with the regular expression /learn react/i. The i flag makes the regular expression case-insensitive. We expect to find the text Learn React in the document.

All of this mimics the behavior a user would experience in the browser when interacting with our app.

Let’s start making the changes required by our app. Open App.js and replace the content with the below code.

import React from "react";
import "./App.css";
function App() {
  return (
    <div className="App">
      <header className="App-header">
        <h2>Getting started with React testing library</h2>
      </header>
    </div>
  );
}
export default App;

If you still have the test running, you should see the test fail. Perhaps you can guess why that is the case, but we’ll return to it a bit later. Right now I want to refactor the test block.

Replace the test block in src/App.test.js with the code below:

# use describe, it pattern
describe("<App />", () => {
  it("Renders <App /> component correctly", () => {
    const { getByText } = render(<App />);
    expect(getByText(/Getting started with React testing library/i)).toBeInTheDocument();
  });
});

This refactor makes no material difference to how our test will run. I prefer the describe and it pattern as it allows me structure my test file into logical blocks of related tests. The test should re-run and this time it will pass. In case you haven’t guessed it, the fix for the failing test was to replace the learn react text with Getting started with React testing library.

In case you don’t have time to write your own styles you can just copy the one below into App.css.

.App {
  min-height: 100vh;
  text-align: center;
}
.App-header {
  height: 10vh;
  display: flex;
  background-color: #282c34;
  flex-direction: column;
  align-items: center;
  justify-content: center;
  font-size: calc(10px + 2vmin);
  color: white;
}
.App-body {
  width: 60%;
  margin: 20px auto;
}
ul {
  padding: 0;
  display: flex;
  list-style-type: decimal;
  flex-direction: column;
}
li {
  font-size: large;
  text-align: left;
  padding: 0.5rem 0;
}
li a {
  text-transform: capitalize;
  text-decoration: none;
}
.todo-title {
  text-transform: capitalize;
}
.completed {
  color: green;
}
.not-completed {
  color: red;
}

You should already see the page title move up after adding this CSS.

I consider this a good point for me to commit my changes and push to Github. The corresponding branch is 01-setup.

Let’s continue with our project setup. We know we’re going to need some navigation in our app so we need React-Router. We’ll also be making API calls with Axios. Let’s install both.

# install react-router-dom and axios
yarn add react-router-dom axios

Most React apps you’ll build will have to maintain state. There’s a lot of libraries available for managing state. But for this tutorial, I’ll be using React’s context API and the useContext hook. So let’s set up our app’s context.

Create a new file src/AppContext.js and enter the below content.

import React from "react";
export const AppContext = React.createContext({});

export const AppProvider = ({ children }) => {
  const reducer = (state, action) => {
    switch (action.type) {
      case "LOAD_TODOLIST":
        return { ...state, todoList: action.todoList };
      case "LOAD_SINGLE_TODO":
        return { ...state, activeToDoItem: action.todo };
      default:
        return state;
    }
  };
  const [appData, appDispatch] = React.useReducer(reducer, {
    todoList: [],
    activeToDoItem: { id: 0 },
  });
  return (
    <AppContext.Provider value={{ appData, appDispatch }}>
      {children}
    </AppContext.Provider>
  );
};

Here we create a new context with React.createContext({}), for which the initial value is an empty object. We then define an AppProvider component that accepts children component. It then wraps those children in AppContext.Provider, thus making the { appData, appDispatch } object available to all children anywhere in the render tree.

Our reducer function defines two action types.

1. LOAD_TODOLIST which is used to update the todoList array.
2. LOAD_SINGLE_TODO which is used to update activeToDoItem.

appData and appDispatch are both returned from the useReducer hook. appData gives us access to the values in the state while appDispatch gives us a function which we can use to update the app’s state.

Now open index.js, import the AppProvider component and wrap the <App /> component with <AppProvider />. Your final code should look like what I have below.

import { AppProvider } from "./AppContext";

ReactDOM.render(
  <React.StrictMode>
    <AppProvider>
      <App />
    </AppProvider>
  </React.StrictMode>,
  document.getElementById("root")
);

Wrapping <App /> inside <AppProvider /> makes AppContext available to every child component in our app.

Remember that with RTL, the aim is to test our app the same way a real user would interact with it. This implies that we also want our tests to interact with our app state. For that reason, we also need to make our <AppProvider /> available to our components during tests. Let’s see how to make that happen.

The render method provided by RTL is sufficient for simple components that don’t need to maintain state or use navigation. But most apps require at least one of both. For this reason, it provides a wrapper option. With this wrapper, we can wrap the UI rendered by the test renderer with any component we like, thus creating a custom render. Let’s create one for our tests.

Create a new file src/custom-render.js and paste the following code.

import React from "react";
import { render } from "@testing-library/react";
import { MemoryRouter } from "react-router-dom";

import { AppProvider } from "./AppContext";

const Wrapper = ({ children }) => {
  return (
    <AppProvider>
      <MemoryRouter>{children}</MemoryRouter>
    </AppProvider>
  );
};

const customRender = (ui, options) =>
  render(ui, { wrapper: Wrapper, ...options });

// re-export everything
export * from "@testing-library/react";

// override render method
export { customRender as render };

Here we define a <Wrapper /> component that accepts some children component. It then wraps those children inside <AppProvider /> and <MemoryRouter />. MemoryRouter is

A <Router> that keeps the history of your “URL” in memory (does not read or write to the address bar). Useful in tests and non-browser environments like React Native.

We then create our render function, providing it the Wrapper we just defined through its wrapper option. The effect of this is that any component we pass to the render function is rendered inside <Wrapper />, thus having access to navigation and our app’s state.

The next step is to export everything from @testing-library/react. Lastly, we export our custom render function as render, thus overriding the default render.

Note that even if you were using Redux for state management the same pattern still applies.

Let’s now make sure our new render function works. Import it into src/App.test.js and use it to render the <App /> component.

Open App.test.js and replace the import line. This

import { render } from '@testing-library/react';

should become

import { render } from './custom-render';

Does the test still pass? Good job.

There’s one small change I want to make before wrapping up this section. It gets tiring very quickly to have to write const { getByText } and other queries every time. So, I’m going to be using the screen object from the DOM testing library henceforth.

Import the screen object from our custom render file and replace the describe block with the code below.

import { render, screen } from "./custom-render";

describe("<App />", () => {
  it("Renders <App /> component correctly", () => {
    render(<App />);
    expect(
      screen.getByText(/Getting started with React testing library/i)
    ).toBeInTheDocument();
  });
});

We’re now accessing the getByText query from the screen object. Does your test still pass? I’m sure it does. Let’s continue.

If your tests don’t pass you may want to compare your code with mine. The corresponding branch at this point is 02-setup-store-and-render.

Testing And Building The To-Do List Index Page

In this section, we’ll pull to-do items from http://jsonplaceholder.typicode.com/. Our component specification is very simple. When a user visits our app homepage,

1. show a loading indicator that says Fetching todos while waiting for the response from the API;
2. display the title of 15 to-do items on the screen once the API call returns (the API call returns 200). Also, each item title should be a link that will lead to the to-do details page.

Following a test-driven approach, we’ll write our test before implementing the component logic. Before doing that we’ll need to have the component in question. So go ahead and create a file src/TodoList.js and enter the following content:

import React from "react";
import "./App.css";
export const TodoList = () => {
  return (
    <div>
    </div>
  );
};

Since we know the component specification we can test it in isolation before incorporating it into our main app. I believe it’s up to the developer at this point to decide how they want to handle this. One reason you might want to test a component in isolation is so that you don’t accidentally break any existing test and then having to fight fires in two locations. With that out of the way let’s now write the test.

Create a new file src/TodoList.test.js and enter the below code:

import React from "react";
import axios from "axios";
import { render, screen, waitForElementToBeRemoved } from "./custom-render";
import { TodoList } from "./TodoList";
import { todos } from "./makeTodos";

describe("<App />", () => {
  it("Renders <TodoList /> component", async () => {
    render(<TodoList />);
    await waitForElementToBeRemoved(() => screen.getByText(/Fetching todos/i));

    expect(axios.get).toHaveBeenCalledTimes(1);
    todos.slice(0, 15).forEach((td) => {
      expect(screen.getByText(td.title)).toBeInTheDocument();
    });
  });
});

Inside our test block, we render the <TodoList /> component and use the waitForElementToBeRemoved function to wait for the Fetching todos text to disappear from the screen. Once this happens we know that our API call has returned. We also check that an Axios get call was fired once. Finally, we check that each to-do title is displayed on the screen. Note that the it block receives an async function. This is necessary for us to be able to use await inside the function.

Each to-do item returned by the API has the following structure.

{
  id: 0,
  userId: 0,
  title: 'Some title',
  completed: true,
}

We want to return an array of these when we

import { todos } from "./makeTodos"

The only condition is that each id should be unique.

Create a new file src/makeTodos.js and enter the below content. This is the source of todos we’ll use in our tests.

const makeTodos = (n) => {
  // returns n number of todo items
  // default is 15
  const num = n || 15;
  const todos = [];
  for (let i = 0; i < num; i++) {
    todos.push({
      id: i,
      userId: i,
      title: `Todo item ${i}`,
      completed: [true, false][Math.floor(Math.random() * 2)],
    });
  }
  return todos;
};

export const todos = makeTodos(200);

This function simply generates a list of n to-do items. The completed line is set by randomly choosing between true and false.

Unit tests are supposed to be fast. They should run within a few seconds. Fail fast! This is one of the reasons why letting our tests make actual API calls is impractical. To avoid this we mock such unpredictable API calls. Mocking simply means replacing a function with a fake version, thus allowing us to customize the behavior. In our case, we want to mock the get method of Axios to return whatever we want it to. Jest already provides mocking functionality out of the box.

Let’s now mock Axios so it returns this list of to-dos when we make the API call in our test. Create a file src/__mocks__/axios.js and enter the below content:

import { todos } from "../makeTodos";

export default {
  get: jest.fn().mockImplementation((url) => {
    switch (url) {
      case "https://jsonplaceholder.typicode.com/todos":
        return Promise.resolve({ data: todos });
      default:
        throw new Error(`UNMATCHED URL: ${url}`);
    }
  }),
};

When the test starts, Jest automatically finds this mocks folder and instead of using the actual Axios from node_modules/ in our tests, it uses this one. At this point, we’re only mocking the get method using Jest’s mockImplementation method. Similarly, we can mock other Axios methods like post, patch, interceptors, defaults etc. Right now they’re all undefined and any attempt to access, axios.post for example, would result in an error.

Note that we can customize what to return based on the URL the Axios call receives. Also, Axios calls return a promise which resolves to the actual data we want, so we return a promise with the data we want.

At this point, we have one passing test and one failing test. Let’s implement the component logic.

Open src/TodoList.js let’s build out the implementation piece by piece. Start by replacing the code inside with this one below.

import React from "react";
import axios from "axios";
import { Link } from "react-router-dom";
import "./App.css";
import { AppContext } from "./AppContext";

export const TodoList = () => {
  const [loading, setLoading] = React.useState(true);
  const { appData, appDispatch } = React.useContext(AppContext);

  React.useEffect(() => {
    axios.get("https://jsonplaceholder.typicode.com/todos").then((resp) => {
      const { data } = resp;
      appDispatch({ type: "LOAD_TODOLIST", todoList: data });
      setLoading(false);
    });
  }, [appDispatch, setLoading]);

  return (
    <div>
      // next code block goes here
    </div>
  );
};

We import AppContext and de-structure appData and appDispatch from the return value of React.useContext. We then make the API call inside a useEffect block. Once the API call returns, we set the to-do list in state by firing the LOAD_TODOLIST action. Finally, we set the loading state to false to reveal our to-dos.

Now enter the final piece of code.

{loading ? (
  <p>Fetching todos</p>
) : (
  <ul>
    {appData.todoList.slice(0, 15).map((item) => {
      const { id, title } = item;
      return (
        <li key={id}>
          <Link to={`/item/${id}`} data-testid={id}>
            {title}
          </Link>
        </li>
      );
    })}
  </ul>
)}

We slice appData.todoList to get the first 15 items. We then map over those and render each one in a <Link /> tag so we can click on it and see the details. Note the data-testid attribute on each Link. This should be a unique ID that will aid us in finding individual DOM elements. In a case where we have similar text on the screen, we should never have the same ID for any two elements. We’ll see how to use this a bit later.

My tests now pass. Does yours pass? Great.

Let’s now incorporate this component into our render tree. Open up App.js let’s do that.

First things. Add some imports.

import { BrowserRouter, Route } from "react-router-dom";
import { TodoList } from "./TodoList";

We need BrowserRouter for navigation and Route for rendering each component in each navigation location.

Now add the below code after the <header /> element.

<div className="App-body">
  <BrowserRouter>
    <Route exact path="/" component={TodoList} />
  </BrowserRouter>
</div>

This is simply telling the browser to render the <TodoList /> component when we’re on the root location, /. Once this is done, our tests still pass but you should see some error messages on your console telling you about some act something. You should also see that the <TodoList /> component seems to be the culprit here.

Since we’re sure that our TodoList component by itself is okay, we have to look at the App component, inside of which is rendered the <TodoList /> component.

This warning may seem complex at first but it is telling us that something is happening in our component that we’re not accounting for in our test. The fix is to wait for the loading indicator to be removed from the screen before we proceed.

Open up App.test.js and update the code to look like so:

import React from "react";
import { render, screen, waitForElementToBeRemoved } from "./custom-render";
import App from "./App";
describe("<App />", () => {
  it("Renders <App /> component correctly", async () => {
    render(<App />);
    expect(
      screen.getByText(/Getting started with React testing library/i)
    ).toBeInTheDocument();
    await waitForElementToBeRemoved(() => screen.getByText(/Fetching todos/i));
  });
});

We’ve made two changes. First, we changed the function in the it block to an async function. This is a necessary step to allow us to use await in the function body. Secondly, we wait for the Fetching todos text to be removed from the screen. And voila!. The warning is gone. Phew! I strongly advise that you bookmark this post by Kent Dodds for more on this act warning. You’re gonna need it.

Now open the page in your browser and you should see the list of to-dos. You can click on an item if you like, but it won’t show you anything because our router doesn’t yet recognize that URL.

For comparison, the branch of my repo at this point is 03-todolist.

Let’s now add the to-do details page.

Testing And Building The Single To-Do Page

To display a single to-do item we’ll follow a similar approach. The component specification is simple. When a user navigates to a to-do page:

1. display a loading indicator that says Fetching todo item id where id represents the to-do’s id, while the API call to https://jsonplaceholder.typicode.com/todos/item_id runs.
2. When the API call returns, show the following information:
   • Todo item title
   • Added by: userId
   • This item has been completed if the to-do has been completed or
   • This item is yet to be completed if the to-do has not been completed.

Let’s start with the component. Create a file src/TodoItem.js and add the following content.

import React from "react";
import { useParams } from "react-router-dom";

import "./App.css";

export const TodoItem = () => {
  const { id } = useParams()
  return (
    <div className="single-todo-item">
    </div>
  );
};

The only thing new to us in this file is the const { id } = useParams() line. This is a hook from react-router-dom that lets us read URL parameters. This id is going to be used in fetching a to-do item from the API.

This situation is a bit different because we’re going to be reading the id from the location URL. We know that when a user clicks a to-do link, the id will show up in the URL which we can then grab using the useParams() hook. But here we’re testing the component in isolation which means that there’s nothing to click, even if we wanted to. To get around this we’ll have to mock react-router-dom, but only some parts of it. Yes. It’s possible to mock only what we need to. Let’s see how it’s done.

Create a new mock file src/__mocks__ /react-router-dom.js. Now paste in the following code:

module.exports = {
  ...jest.requireActual("react-router-dom"),
  useParams: jest.fn(),
};

By now you should have noticed that when mocking a module we have to use the exact module name as the mock file name.

Here, we use the module.exports syntax because react-router-dom has mostly named exports. (I haven’t come across any default export since I’ve been working with it. If there are any, kindly share with me in the comments). This is unlike Axios where everything is bundled as methods in one default export.

We first spread the actual react-router-dom, then replace the useParams hook with a Jest function. Since this function is a Jest function, we can modify it anytime we want. Keep in mind that we’re only mocking the part we need to because if we mock everything, we’ll lose the implementation of MemoryHistory which is used in our render function.

Let’s start testing!

Now create src/TodoItem.test.js and enter the below content:

import React from "react";
import axios from "axios";
import { render, screen, waitForElementToBeRemoved } from "./custom-render";
import { useParams, MemoryRouter } from "react-router-dom";
import { TodoItem } from "./TodoItem";

describe("<TodoItem />", () => {
  it("can tell mocked from unmocked functions", () => {
    expect(jest.isMockFunction(useParams)).toBe(true);
    expect(jest.isMockFunction(MemoryRouter)).toBe(false);
  });
});

Just like before, we have all our imports. The describe block then follows. Our first case is only there as a demonstration that we’re only mocking what we need to. Jest’s isMockFunction can tell whether a function is mocked or not. Both expectations pass, confirming the fact that we have a mock where we want it.

Add the below test case for when a to-do item has been completed.

  it("Renders <TodoItem /> correctly for a completed item", async () => {
    useParams.mockReturnValue({ id: 1 });
    render(<TodoItem />);

    await waitForElementToBeRemoved(() =>
      screen.getByText(/Fetching todo item 1/i)
    );

    expect(axios.get).toHaveBeenCalledTimes(1);
    expect(screen.getByText(/todo item 1/)).toBeInTheDocument();
    expect(screen.getByText(/Added by: 1/)).toBeInTheDocument();
    expect(
      screen.getByText(/This item has been completed/)
    ).toBeInTheDocument();
  });

The very first thing we do is to mock the return value of useParams. We want it to return an object with an id property, having a value of 1. When this is parsed in the component, we end up with the following URL https://jsonplaceholder.typicode.com/todos/1. Keep in mind that we have to add a case for this URL in our Axios mock or it will throw an error. We will do that in just a moment.

We now know for sure that calling useParams() will return the object { id: 1 } which makes this test case predictable.

As with previous tests, we wait for the loading indicator, Fetching todo item 1 to be removed from the screen before making our expectations. We expect to see the to-do title, the id of the user who added it, and a message indicating the status.

Open src/__mocks__/axios.js and add the following case to the switch block.

      case "https://jsonplaceholder.typicode.com/todos/1":
        return Promise.resolve({
          data: { id: 1, title: "todo item 1", userId: 1, completed: true },
        });

When this URL is matched, a promise with a completed to-do is returned. Of course, this test case fails since we’re yet to implement the component logic. Go ahead and add a test case for when the to-do item has not been completed.

  it("Renders <TodoItem /> correctly for an uncompleted item", async () => {
    useParams.mockReturnValue({ id: 2 });
    render(<TodoItem />);
    await waitForElementToBeRemoved(() =>
      screen.getByText(/Fetching todo item 2/i)
    );
    expect(axios.get).toHaveBeenCalledTimes(2);
    expect(screen.getByText(/todo item 2/)).toBeInTheDocument();
    expect(screen.getByText(/Added by: 2/)).toBeInTheDocument();
    expect(
      screen.getByText(/This item is yet to be completed/)
    ).toBeInTheDocument();
  });

This is the same as the previous case. The only difference is the ID of the to-do, the userId, and the completion status. When we enter the component, we’ll need to make an API call to the URL https://jsonplaceholder.typicode.com/todos/2. Go ahead and add a matching case statement to the switch block of our Axios mock.

case "https://jsonplaceholder.typicode.com/todos/2":
  return Promise.resolve({
    data: { id: 2, title: "todo item 2", userId: 2, completed: false },
  });

When the URL is matched, a promise with an uncompleted to-do is returned.

Both test cases are failing. Now let’s add the component implementation to make them pass.

Open src/TodoItem.js and update the code to the following:

import React from "react";
import axios from "axios";
import { useParams } from "react-router-dom";
import "./App.css";
import { AppContext } from "./AppContext";

export const TodoItem = () => {
  const { id } = useParams();
  const [loading, setLoading] = React.useState(true);
  const {
    appData: { activeToDoItem },
    appDispatch,
  } = React.useContext(AppContext);

  const { title, completed, userId } = activeToDoItem;
  React.useEffect(() => {
    axios
      .get(`https://jsonplaceholder.typicode.com/todos/${id}`)
      .then((resp) => {
        const { data } = resp;
        appDispatch({ type: "LOAD_SINGLE_TODO", todo: data });
        setLoading(false);
      });
  }, [id, appDispatch]);
  return (
    <div className="single-todo-item">
      // next code block goes here.
    </div>
  );
};

As with the <TodoList /> component, we import AppContext. We read activeTodoItem from it, then we read the to-do title, userId, and completion status. After that we make the API call inside a useEffect block. When the API call returns we set the to-do in state by firing the LOAD_SINGLE_TODO action. Finally, we set our loading state to false to reveal the to-do details.

Let’s add the final piece of code inside the return div:

{loading ? (
  <p>Fetching todo item {id}</p>
) : (
  <div>
    <h2 className="todo-title">{title}</h2>
    <h4>Added by: {userId}</h4>
    {completed ? (
      <p className="completed">This item has been completed</p>
    ) : (
      <p className="not-completed">This item is yet to be completed</p>
    )}
  </div>
)}

Once this is done all tests should now pass. Yay! We have another winner.

Our component tests now pass. But we still haven’t added it to our main app. Let’s do that.

Open src/App.js and add the import line:

import { TodoItem } from './TodoItem'

Add the TodoItem route above the TodoList route. Be sure to preserve the order shown below.

# preserve this order
<Route path="/item/:id" component={TodoItem} />
<Route exact path="/" component={TodoList} />

Open your project in your browser and click on a to-do. Does it take you to the to-do page? Of course, it does. Good job.

In case you’re having any problem, you can check out my code at this point from the 04-test-todo branch.

Phew! This has been a marathon. But bear with me. There’s one last point I’d like us to touch. Let’s quickly have a test case for when a user visits our app, and then proceed to click on a to-do link. This is a functional test to mimic how our app should work. In practice, this is all the testing we need to be done for this app. It ticks every box in our app specification.

Open App.test.js and add a new test case. The code is a bit long so we’ll add it in two steps.

import userEvent from "@testing-library/user-event";
import { todos } from "./makeTodos";

jest.mock("react-router-dom", () => ({
  ...jest.requireActual("react-router-dom"),
}));

describe("<App />"
  ...
  // previous test case
  ...

  it("Renders todos, and I can click to view a todo item", async () => {
    render(<App />);
    await waitForElementToBeRemoved(() => screen.getByText(/Fetching todos/i));
    todos.slice(0, 15).forEach((td) => {
      expect(screen.getByText(td.title)).toBeInTheDocument();
    });
    // click on a todo item and test the result
    const { id, title, completed, userId } = todos[0];
    axios.get.mockImplementationOnce(() =>
      Promise.resolve({
        data: { id, title, userId, completed },
      })
    );
    userEvent.click(screen.getByTestId(String(id)));
    await waitForElementToBeRemoved(() =>
      screen.getByText(`Fetching todo item ${String(id)}`)
    );

    // next code block goes here
  });
});

We have two imports of which userEvent is new. According to the docs,

“user-event is a companion library for the React Testing Library that provides a more advanced simulation of browser interactions than the built-in fireEvent method.”

Yes. There is a fireEvent method for simulating user events. But userEvent is what you want to be using henceforth.

Before we start the testing process, we need to restore the original useParams hooks. This is necessary since we want to test actual behavior, so we should mock as little as possible. Jest provides us with requireActual method which returns the original react-router-dom module.

Note that we must do this before we enter the describe block, otherwise, Jest would ignore it. It states in the documentation that requireActual:

“...returns the actual module instead of a mock, bypassing all checks on whether the module should receive a mock implementation or not.”

Once this is done, Jest bypasses every other check and ignores the mocked version of the react-router-dom.

As usual, we render the <App /> component and wait for the Fetching todos loading indicator to disappear from the screen. We then check for the presence of the first 15 to-do items on the page.

Once we’re satisfied with that, we grab the first item in our to-do list. To prevent any chance of a URL collision with our global Axios mock, we override the global mock with Jest’s mockImplementationOnce. This mocked value is valid for one call to the Axios get method. We then grab a link by its data-testid attribute and fire a user click event on that link. Then we wait for the loading indicator for the single to-do page to disappear from the screen.

Now finish the test by adding the below expectations in the position indicated.

expect(screen.getByText(title)).toBeInTheDocument();
expect(screen.getByText(`Added by: ${userId}`)).toBeInTheDocument();
switch (completed) {
  case true:
    expect(
      screen.getByText(/This item has been completed/)
    ).toBeInTheDocument();
    break;
  case false:
    expect(
      screen.getByText(/This item is yet to be completed/)
    ).toBeInTheDocument();
    break;
  default:
    throw new Error("No match");
    }
  

We expect to see the to-do title and the user who added it. Finally, since we can’t be sure about the to-do status, we create a switch block to handle both cases. If a match is not found we throw an error.

You should have 6 passing tests and a functional app at this point. In case you’re having trouble, the corresponding branch in my repo is 05-test-user-action.

Conclusion

Phew! That was some marathon. If you made it to this point, congratulations. You now have almost all you need to write tests for your React apps. I strongly advise that you read CRA’s testing docs and RTL’s documentation. Overall both are relatively short and direct.

I strongly encourage you to start writing tests for your React apps, no matter how small. Even if it’s just smoke tests to make sure your components render. You can incrementally add more test cases over time.

Related Resources

• “Testing Overview,” React official website
• “Expect,” Jest API Reference
• “Custom Render,” React Testing Library
• “jest-dom,” Testing Library, GitHub
• “Guiding Principles,” Getting Started, Testing Library
• “React Testing Library,” Testing Library
• “Recommended Tools,” Testing Overview, React official website
• “Fix the “not wrapped in act(…)” warning,” Kent C. Dodds
• “<MemoryRouter>,” React Training
• “screen,” DOM Testing Library
• “user-event,” Ecosystem, Testing Library Docs
• “The Different Types Of Software Testing,” Sten Pittet, Atlassian

(ks, ra, yk, il)

We will take a Test-Driven Development (TDD) approach and the set up Continuous Integration (CI) job to automatically run our tests on Travis CI and AppVeyor, complete with code quality and coverage reporting. We will learn about controllers, models (with PostgreSQL), error handling, and asynchronous Express middleware. Finally, we’ll complete the CI/CD pipeline by configuring automatic deploy on Heroku.

It sounds like a lot, but this tutorial is aimed at beginners who are ready to try their hands on a back-end project with some level of complexity, and who may still be confused as to how all the pieces fit together in a real project.

It is robust without being overwhelming and is broken down into sections that you can complete in a reasonable length of time.

Getting Started

The first step is to create a new directory for the project and start a new node project. Node is required to continue with this tutorial. If you don’t have it installed, head over to the official website, download, and install it before continuing.

I will be using yarn as my package manager for this project. There are installation instructions for your specific operating system here. Feel free to use npm if you like.

Open your terminal, create a new directory, and start a Node.js project.

# create a new directory
mkdir express-api-template

# change to the newly-created directory
cd express-api-template

# initialize a new Node.js project
npm init

Answer the questions that follow to generate a package.json file. This file holds information about your project. Example of such information includes what dependencies it uses, the command to start the project, and so on.

You may now open the project folder in your editor of choice. I use visual studio code. It’s a free IDE with tons of plugins to make your life easier, and it’s available for all major platforms. You can download it from the official website.

Create the following files in the project folder:

• README.md
• .editorconfig

Here’s a description of what .editorconfig does from the EditorConfig website. (You probably don’t need it if you’re working solo, but it does no harm, so I’ll leave it here.)

“EditorConfig helps maintain consistent coding styles for multiple developers working on the same project across various editors and IDEs.”

Open .editorconfig and paste the following code:

root = true
[*]
indent_style = space
indent_size = 2
charset = utf-8
trim_trailing_whitespace = false
insert_final_newline = true

The [*] means that we want to apply the rules that come under it to every file in the project. We want an indent size of two spaces and UTF-8 character set. We also want to trim trailing white space and insert a final empty line in our file.

Open README.md and add the project name as a first-level element.

# Express API template

Let’s add version control right away.

# initialize the project folder as a git repository
git init

Create a .gitignore file and enter the following lines:

node_modules/
yarn-error.log
.env
.nyc_output
coverage
build/

These are all the files and folders we don’t want to track. We don’t have them in our project yet, but we’ll see them as we proceed.

At this point, you should have the following folder structure.

EXPRESS-API-TEMPLATE
├── .editorconfig
├── .gitignore
├── package.json
└── README.md

I consider this to be a good point to commit my changes and push them to GitHub.

Starting A New Express Project

Express is a Node.js framework for building web applications. According to the official website, it is a

Fast, unopinionated, minimalist web framework for Node.js.

There are other great web application frameworks for Node.js, but Express is very popular, with over 47k GitHub stars at the time of this writing.

In this article, we will not be having a lot of discussions about all the parts that make up Express. For that discussion, I recommend you check out Jamie’s series. The first part is here, and the second part is here.

Install Express and start a new Express project. It’s possible to manually set up an Express server from scratch but to make our life easier we’ll use the express-generator to set up the app skeleton.

# install the express generator globally
yarn global add express-generator

# install express
yarn add express

# generate the express project in the current folder
express -f

The -f flag forces Express to create the project in the current directory.

We’ll now perform some house-cleaning operations.

1. Delete the file index/users.js.
2. Delete the folders public/ and views/.
3. Rename the file bin/www to bin/www.js.
4. Uninstall jade with the command yarn remove jade.
5. Create a new folder named src/ and move the following inside it: 1. app.js file 2. bin/ folder 3. routes/ folder inside.
6. Open up package.json and update the start script to look like below.

"start": "node ./src/bin/www"

At this point, your project folder structure looks like below. You can see how VS Code highlights the file changes that have taken place.

EXPRESS-API-TEMPLATE
├── node_modules
├── src
|   ├── bin
│   │   ├── www.js
│   ├── routes
│   |   ├── index.js
│   └── app.js
├── .editorconfig
├── .gitignore
├── package.json
├── README.md
└── yarn.lock

Open src/app.js and replace the content with the below code.

var logger = require('morgan');
var express = require('express');
var cookieParser = require('cookie-parser');
var indexRouter = require('./routes/index');
var app = express();

app.use(logger('dev'));
app.use(express.json());
app.use(express.urlencoded({ extended: true }));
app.use(cookieParser());
app.use('/v1', indexRouter);

module.exports = app;

After requiring some libraries, we instruct Express to handle every request coming to /v1 with indexRouter.

Replace the content of routes/index.js with the below code:

var express = require('express');
var router = express.Router();
router.get('/', function(req, res, next) {
  return res.status(200).json({ message: 'Welcome to Express API template' });
});
module.exports = router;

We grab Express, create a router from it and serve the / route, which returns a status code of 200 and a JSON message.

Start the app with the below command:

# start the app
yarn start

If you’ve set up everything correctly you should only see $ node ./src/bin/www in your terminal.

Visit http://localhost:3000/v1 in your browser. You should see the following message:

{
  "message": "Welcome to Express API template"
}

This is a good point to commit our changes.

• The corresponding branch in my repo is 01-install-express.

Converting Our Code To ES6

The code generated by express-generator is in ES5, but in this article, we will be writing all our code in ES6 syntax. So, let’s convert our existing code to ES6.

Replace the content of routes/index.js with the below code:

import express from 'express';

const indexRouter = express.Router();

indexRouter.get('/', (req, res) =>
  res.status(200).json({ message: 'Welcome to Express API template' })
);

export default indexRouter;

It is the same code as we saw above, but with the import statement and an arrow function in the / route handler.

Replace the content of src/app.js with the below code:

import logger from 'morgan';
import express from 'express';
import cookieParser from 'cookie-parser';
import indexRouter from './routes/index';
const app = express();
app.use(logger('dev'));
app.use(express.json());
app.use(express.urlencoded({ extended: true }));
app.use(cookieParser());
app.use('/v1', indexRouter);

export default app;

Let’s now take a look at the content of src/bin/www.js. We will build it incrementally. Delete the content of src/bin/www.js and paste in the below code block.

#!/usr/bin/env node
/**
 * Module dependencies.
 */
import debug from 'debug';
import http from 'http';
import app from '../app';
/**
 * Normalize a port into a number, string, or false.
 */
const normalizePort = val => {
  const port = parseInt(val, 10);
  if (Number.isNaN(port)) {
    // named pipe
    return val;
  }
  if (port >= 0) {
    // port number
    return port;
  }
  return false;
};

/**
 * Get port from environment and store in Express.
 */
const port = normalizePort(process.env.PORT || '3000');
app.set('port', port);

/**
 * Create HTTP server.
 */
const server = http.createServer(app);

// next code block goes here

This code checks if a custom port is specified in the environment variables. If none is set the default port value of 3000 is set on the app instance, after being normalized to either a string or a number by normalizePort. The server is then created from the http module, with app as the callback function.

The #!/usr/bin/env node line is optional since we would specify node when we want to execute this file. But make sure it is on line 1 of src/bin/www.js file or remove it completely.

Let’s take a look at the error handling function. Copy and paste this code block after the line where the server is created.

/**
 * Event listener for HTTP server "error" event.
 */
const onError = error => {
  if (error.syscall !== 'listen') {
    throw error;
  }
  const bind = typeof port === 'string' ? `Pipe ${port}` : `Port ${port}`;
  // handle specific listen errors with friendly messages
  switch (error.code) {
    case 'EACCES':
      alert(`${bind} requires elevated privileges`);
      process.exit(1);
      break;
    case 'EADDRINUSE':
      alert(`${bind} is already in use`);
      process.exit(1);
      break;
    default:
      throw error;
  }
};

/**
 * Event listener for HTTP server "listening" event.
 */
const onListening = () => {
  const addr = server.address();
  const bind = typeof addr === 'string' ? `pipe ${addr}` : `port ${addr.port}`;
  debug(`Listening on ${bind}`);
};
/**
 * Listen on provided port, on all network interfaces.
 */
server.listen(port);
server.on('error', onError);
server.on('listening', onListening);

The onError function listens for errors in the http server and displays appropriate error messages. The onListening function simply outputs the port the server is listening on to the console. Finally, the server listens for incoming requests at the specified address and port.

At this point, all our existing code is in ES6 syntax. Stop your server (use Ctrl + C) and run yarn start. You’ll get an error SyntaxError: Invalid or unexpected token. This happens because Node (at the time of writing) doesn’t support some of the syntax we’ve used in our code.

We’ll now fix that in the following section.

Configuring Development Dependencies: babel, nodemon, eslint, And prettier

It’s time to set up most of the scripts we’re going to need at this phase of the project.

Install the required libraries with the below commands. You can just copy everything and paste it in your terminal. The comment lines will be skipped.

# install babel scripts
yarn add @babel/cli @babel/core @babel/plugin-transform-runtime @babel/preset-env @babel/register @babel/runtime @babel/node --dev

This installs all the listed babel scripts as development dependencies. Check your package.json file and you should see a devDependencies section. All the installed scripts will be listed there.

The babel scripts we’re using are explained below:

Create a file named .babelrc at the root of your project and add the following code:

{
  "presets": ["@babel/preset-env"],
  "plugins": ["@babel/transform-runtime"]
}

Let’s install nodemon

# install nodemon
yarn add nodemon --dev

nodemon is a library that monitors our project source code and automatically restarts our server whenever it observes any changes.

Create a file named nodemon.json at the root of your project and add the code below:

{
  "watch": [
    "package.json",
    "nodemon.json",
    ".eslintrc.json",
    ".babelrc",
    ".prettierrc",
    "src/"
  ],
  "verbose": true,
  "ignore": ["*.test.js", "*.spec.js"]
}

The watch key tells nodemon which files and folders to watch for changes. So, whenever any of these files changes, nodemon restarts the server. The ignore key tells it the files not to watch for changes.

Now update the scripts section of your package.json file to look like the following:

# build the content of the src folder
"prestart": "babel ./src --out-dir build"

# start server from the build folder
"start": "node ./build/bin/www"

# start server in development mode
"startdev": "nodemon --exec babel-node ./src/bin/www"

1. prestart scripts builds the content of the src/ folder and puts it in the build/ folder. When you issue the yarn start command, this script runs first before the start script.
2. start script now serves the content of the build/ folder instead of the src/ folder we were serving previously. This is the script you’ll use when serving the file in production. In fact, services like Heroku automatically run this script when you deploy.
3. yarn startdev is used to start the server during development. From now on we will be using this script as we develop the app. Notice that we’re now using babel-node to run the app instead of regular node. The --exec flag forces babel-node to serve the src/ folder. For the start script, we use node since the files in the build/ folder have been compiled to ES5.

Run yarn startdev and visit http://localhost:3000/v1. Your server should be up and running again.

The final step in this section is to configure ESLint and prettier. ESLint helps with enforcing syntax rules while prettier helps for formatting our code properly for readability.

Add both of them with the command below. You should run this on a separate terminal while observing the terminal where our server is running. You should see the server restarting. This is because we’re monitoring package.json file for changes.

# install elsint and prettier

yarn add eslint eslint-config-airbnb-base eslint-plugin-import prettier --dev

Now create the .eslintrc.json file in the project root and add the below code:

{
  "env": {
    "browser": true,
    "es6": true,
    "node": true,
    "mocha": true
  },
  "extends": ["airbnb-base"],
  "globals": {
    "Atomics": "readonly",
    "SharedArrayBuffer": "readonly"
  },
  "parserOptions": {
    "ecmaVersion": 2018,
    "sourceType": "module"
  },
  "rules": {
    "indent": ["warn", 2],
    "linebreak-style": ["error", "unix"],
    "quotes": ["error", "single"],
    "semi": ["error", "always"],
    "no-console": 1,
    "comma-dangle": [0],
    "arrow-parens": [0],
    "object-curly-spacing": ["warn", "always"],
    "array-bracket-spacing": ["warn", "always"],
    "import/prefer-default-export": [0]
  }
}

This file mostly defines some rules against which eslint will check our code. You can see that we’re extending the style rules used by Airbnb.

In the "rules" section, we define whether eslint should show a warning or an error when it encounters certain violations. For instance, it shows a warning message on our terminal for any indentation that does not use 2 spaces. A value of [0] turns off a rule, which means that we won’t get a warning or an error if we violate that rule.

Create a file named .prettierrc and add the code below:

{
  "trailingComma": "es5",
  "tabWidth": 2,
  "semi": true,
  "singleQuote": true
}

We’re setting a tab width of 2 and enforcing the use of single quotes throughout our application. Do check the prettier guide for more styling options.

Now add the following scripts to your package.json:

# add these one after the other

"lint": "./node_modules/.bin/eslint ./src"

"pretty": "prettier --write '**/*.{js,json}' '!node_modules/**'"

"postpretty": "yarn lint --fix"

Run yarn lint. You should see a number of errors and warnings in the console.

The pretty command prettifies our code. The postpretty command is run immediately after. It runs the lint command with the --fix flag appended. This flag tells ESLint to automatically fix common linting issues. In this way, I mostly run the yarn pretty command without bothering about the lint command.

Run yarn pretty. You should see that we have only two warnings about the presence of alert in the bin/www.js file.

Here’s what our project structure looks like at this point.

EXPRESS-API-TEMPLATE
├── build
├── node_modules
├── src
|   ├── bin
│   │   ├── www.js
│   ├── routes
│   |   ├── index.js
│   └── app.js
├── .babelrc
├── .editorconfig
├── .eslintrc.json
├── .gitignore
├── .prettierrc
├── nodemon.json
├── package.json
├── README.md
└── yarn.lock

You may find that you have an additional file, yarn-error.log in your project root. Add it to .gitignore file. Commit your changes.

• The corresponding branch at this point in my repo is 02-dev-dependencies.

Settings And Environment Variables In Our .env File

In nearly every project, you’ll need somewhere to store settings that will be used throughout your app e.g. an AWS secret key. We store such settings as environment variables. This keeps them away from prying eyes, and we can use them within our application as needed.

I like having a settings.js file with which I read all my environment variables. Then, I can refer to the settings file from anywhere within my app. You’re at liberty to name this file whatever you want, but there’s some kind of consensus about naming such files settings.js or config.js.

For our environment variables, we’ll keep them in a .env file and read them into our settings file from there.

Create the .env file at the root of your project and enter the below line:

TEST_ENV_VARIABLE="Environment variable is coming across"

To be able to read environment variables into our project, there’s a nice library, dotenv that reads our .env file and gives us access to the environment variables defined inside. Let’s install it.

# install dotenv
yarn add dotenv

Add the .env file to the list of files being watched by nodemon.

Now, create the settings.js file inside the src/ folder and add the below code:

import dotenv from 'dotenv';
dotenv.config();
export const testEnvironmentVariable = process.env.TEST_ENV_VARIABLE;

We import the dotenv package and call its config method. We then export the testEnvironmentVariable which we set in our .env file.

Open src/routes/index.js and replace the code with the one below.

import express from 'express';
import { testEnvironmentVariable } from '../settings';

const indexRouter = express.Router();

indexRouter.get('/', (req, res) => res.status(200).json({ message: testEnvironmentVariable }));

export default indexRouter;

The only change we’ve made here is that we import testEnvironmentVariable from our settings file and use is as the return message for a request from the / route.

Visit http://localhost:3000/v1 and you should see the message, as shown below.

{
  "message": "Environment variable is coming across."
}

And that’s it. From now on we can add as many environment variables as we want and we can export them from our settings.js file.

This is a good point to commit your code. Remember to prettify and lint your code.

• The corresponding branch on my repo is 03-env-variables.

Writing Our First Test

It’s time to incorporate testing into our app. One of the things that give the developer confidence in their code is tests. I’m sure you’ve seen countless articles on the web preaching Test-Driven Development (TDD). It cannot be emphasized enough that your code needs some measure of testing. TDD is very easy to follow when you’re working with Express.js.

In our tests, we will make calls to our API endpoints and check to see if what is returned is what we expect.

Install the required dependencies:

# install dependencies

yarn add mocha chai nyc sinon-chai supertest coveralls --dev

Each of these libraries has its own role to play in our tests.

Create a new test/ folder at the root of your project. Create two files inside this folder:

• test/setup.js
• test/index.test.js

Mocha will find the test/ folder automatically.

Open up test/setup.js and paste the below code. This is just a helper file that helps us organize all the imports we need in our test files.

import supertest from 'supertest';
import chai from 'chai';
import sinonChai from 'sinon-chai';
import app from '../src/app';

chai.use(sinonChai);
export const { expect } = chai;
export const server = supertest.agent(app);
export const BASE_URL = '/v1';

This is like a settings file, but for our tests. This way we don’t have to initialize everything inside each of our test files. So we import the necessary packages and export what we initialized — which we can then import in the files that need them.

Open up index.test.js and paste the following test code.

import { expect, server, BASE_URL } from './setup';

describe('Index page test', () => {
  it('gets base url', done => {
    server
      .get(`${BASE_URL}/`)
      .expect(200)
      .end((err, res) => {
        expect(res.status).to.equal(200);
        expect(res.body.message).to.equal(
          'Environment variable is coming across.'
        );
        done();
      });
  });
});

Here we make a request to get the base endpoint, which is / and assert that the res.body object has a message key with a value of Environment variable is coming across.

If you’re not familiar with the describe, it pattern, I encourage you to take a quick look at Mocha’s “Getting Started” doc.

Add the test command to the scripts section of package.json.

"test": "nyc --reporter=html --reporter=text --reporter=lcov mocha -r @babel/register"

This script executes our test with nyc and generates three kinds of coverage report: an HTML report, outputted to the coverage/ folder; a text report outputted to the terminal and an lcov report outputted to the .nyc_output/ folder.

Now run yarn test. You should see a text report in your terminal just like the one in the below photo.

Notice that two additional folders are generated:

• .nyc_output/
• coverage/

Look inside .gitignore and you’ll see that we’re already ignoring both. I encourage you to open up coverage/index.html in a browser and view the test report for each file.

This is a good point to commit your changes.

• The corresponding branch in my repo is 04-first-test.

Continuous Integration(CD) And Badges: Travis, Coveralls, Code Climate, AppVeyor

It’s now time to configure continuous integration and deployment (CI/CD) tools. We will configure common services such as travis-ci, coveralls, AppVeyor, and codeclimate and add badges to our README file.

Let’s get started.

Travis CI

Travis CI is a tool that runs our tests automatically each time we push a commit to GitHub (and recently, Bitbucket) and each time we create a pull request. This is mostly useful when making pull requests by showing us if the our new code has broken any of our tests.

1. Visit travis-ci.com or travis-ci.org and create an account if you don’t have one. You have to sign up with your GitHub account.
2. Hover over the dropdown arrow next to your profile picture and click on settings.
3. Under Repositories tab click Manage repositories on Github to be redirected to Github.
4. On the GitHub page, scroll down to Repository access and click the checkbox next to Only select repositories.
5. Click the Select repositories dropdown and find the express-api-template repo. Click it to add it to the list of repositories you want to add to travis-ci.
6. Click Approve and install and wait to be redirected back to travis-ci.
7. At the top of the repo page, close to the repo name, click on the build unknown icon. From the Status Image modal, select markdown from the format dropdown.
8. Copy the resulting code and paste it in your README.md file.
9. On the project page, click on More options > Settings. Under Environment Variables section, add the TEST_ENV_VARIABLE env variable. When entering its value, be sure to have it within double quotes like this "Environment variable is coming across."
10. Create .travis.yml file at the root of your project and paste in the below code (We’ll set the value of CC_TEST_REPORTER_ID in the Code Climate section).

language: node_js
env:
  global:
    - CC_TEST_REPORTER_ID=get-this-from-code-climate-repo-page
matrix:
  include:
  - node_js: '12'
cache:
  directories: [node_modules]
install:
  yarn
after_success: yarn coverage
before_script:
  - curl -L https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64 > ./cc-test-reporter
  - chmod +x ./cc-test-reporter
  - ./cc-test-reporter before-build
script:
  - yarn test
after_script:
  - ./cc-test-reporter after-build --exit-code $TRAVIS_TEST_RESUL

First, we tell Travis to run our test with Node.js, then set the CC_TEST_REPORTER_ID global environment variable (we’ll get to this in the Code Climate section). In the matrix section, we tell Travis to run our tests with Node.js v12. We also want to cache the node_modules/ directory so it doesn’t have to be regenerated every time.

We install our dependencies using the yarn command which is a shorthand for yarn install. The before_script and after_script commands are used to upload coverage results to codeclimate. We’ll configure codeclimate shortly. After yarn test runs successfully, we want to also run yarn coverage which will upload our coverage report to coveralls.io.

Coveralls

Coveralls uploads test coverage data for easy visualization. We can view the test coverage on our local machine from the coverage folder, but Coveralls makes it available outside our local machine.

1. Visit coveralls.io and either sign in or sign up with your Github account.
2. Hover over the left-hand side of the screen to reveal the navigation menu. Click on ADD REPOS.
3. Search for the express-api-template repo and turn on coverage using the toggle button on the left-hand side. If you can’t find it, click on SYNC REPOS on the upper right-hand corner and try again. Note that your repo has to be public, unless you have a PRO account.
4. Click details to go to the repo details page.
5. Create the .coveralls.yml file at the root of your project and enter the below code. To get the repo_token, click on the repo details. You will find it easily on that page. You could just do a browser search for repo_token.

repo_token: get-this-from-repo-settings-on-coveralls.io

This token maps your coverage data to a repo on Coveralls. Now, add the coverage command to the scripts section of your package.json file:

"coverage": "nyc report --reporter=text-lcov | coveralls"

This command uploads the coverage report in the .nyc_output folder to coveralls.io. Turn on your Internet connection and run:

yarn coverage

This should upload the existing coverage report to coveralls. Refresh the repo page on coveralls to see the full report.

On the details page, scroll down to find the BADGE YOUR REPO section. Click on the EMBED dropdown and copy the markdown code and paste it into your README file.

Code Climate

Code Climate is a tool that helps us measure code quality. It shows us maintenance metrics by checking our code against some defined patterns. It detects things such as unnecessary repetition and deeply nested for loops. It also collects test coverage data just like coveralls.io.

1. Visit codeclimate.com and click on ‘Sign up with GitHub’. Log in if you already have an account.
2. Once in your dashboard, click on Add a repository.
3. Find the express-api-template repo from the list and click on Add Repo.
4. Wait for the build to complete and redirect to the repo dashboard.
5. Under Codebase Summary, click on Test Coverage. Under the Test coverage menu, copy the TEST REPORTER ID and paste it in your .travis.yml as the value of CC_TEST_REPORTER_ID.
6. Still on the same page, on the left-hand navigation, under EXTRAS, click on Badges. Copy the maintainability and test coverage badges in markdown format and paste them into your README.md file.

It’s important to note that there are two ways of configuring maintainability checks. There are the default settings that are applied to every repo, but if you like, you could provide a .codeclimate.yml file at the root of your project. I’ll be using the default settings, which you can find under the Maintainability tab of the repo settings page. I encourage you to take a look at least. If you still want to configure your own settings, this guide will give you all the information you need.

AppVeyor

AppVeyor and Travis CI are both automated test runners. The main difference is that travis-ci runs tests in a Linux environment while AppVeyor runs tests in a Windows environment. This section is included to show how to get started with AppVeyor.

• Visit AppVeyor and log in or sign up.
• On the next page, click on NEW PROJECT.
• From the repo list, find the express-api-template repo. Hover over it and click ADD.
• Click on the Settings tab. Click on Environment on the left navigation. Add TEST_ENV_VARIABLE and its value. Click ‘Save’ at the bottom of the page.
• Create the appveyor.yml file at the root of your project and paste in the below code.

environment:
  matrix:
  - nodejs_version: "12"
install:
  - yarn
test_script:
  - yarn test
build: off

This code instructs AppVeyor to run our tests using Node.js v12. We then install our project dependencies with the yarn command. test_script specifies the command to run our test. The last line tells AppVeyor not to create a build folder.

Click on the Settings tab. On the left-hand navigation, click on badges. Copy the markdown code and paste it in your README.md file.

Commit your code and push to GitHub. If you have done everything as instructed all tests should pass and you should see your shiny new badges as shown below. Check again that you have set the environment variables on Travis and AppVeyor.

Now is a good time to commit our changes.

• The corresponding branch in my repo is 05-ci.

Adding A Controller

Currently, we’re handling the GET request to the root URL, /v1, inside the src/routes/index.js. This works as expected and there is nothing wrong with it. However, as your application grows, you want to keep things tidy. You want concerns to be separated — you want a clear separation between the code that handles the request and the code that generates the response that will be sent back to the client. To achieve this, we write controllers. Controllers are simply functions that handle requests coming through a particular URL.

To get started, create a controllers/ folder inside the src/ folder. Inside controllers create two files: index.js and home.js. We would export our functions from within index.js. You could name home.js anything you want, but typically you want to name controllers after what they control. For example, you might have a file usersController.js to hold every function related to users in your app.

Open src/controllers/home.js and enter the code below:

import { testEnvironmentVariable } from '../settings';

export const indexPage = (req, res) => res.status(200).json({ message: testEnvironmentVariable });

You will notice that we only moved the function that handles the request for the / route.

Open src/controllers/index.js and enter the below code.

// export everything from home.js
export * from './home';

We export everything from the home.js file. This allows us shorten our import statements to import { indexPage } from '../controllers';

Open src/routes/index.js and replace the code there with the one below:

import express from 'express';
import { indexPage } from '../controllers';
const indexRouter = express.Router();

indexRouter.get('/', indexPage);

export default indexRouter;

The only change here is that we’ve provided a function to handle the request to the / route.

You just successfully wrote your first controller. From here it’s a matter of adding more files and functions as needed.

Go ahead and play with the app by adding a few more routes and controllers. You could add a route and a controller for the about page. Remember to update your test, though.

Run yarn test to confirm that we’ve not broken anything. Does your test pass? That’s cool.

This is a good point to commit our changes.

• The corresponding branch in my repo is 06-controllers.

Connecting The PostgreSQL Database And Writing A Model

Our controller currently returns hard-coded text messages. In a real-world app, we often need to store and retrieve information from a database. In this section, we will connect our app to a PostgreSQL database.

We’re going to implement the storage and retrieval of simple text messages using a database. We have two options for setting a database: we could provision one from a cloud server, or we could set up our own locally.

I would recommend you provision a database from a cloud server. ElephantSQL has a free plan that gives 20MB of free storage which is sufficient for this tutorial. Visit the site and click on Get a managed database today. Create an account (if you don’t have one) and follow the instructions to create a free plan. Take note of the URL on the database details page. We’ll be needing it soon.

If you would rather set up a database locally, you should visit the PostgreSQL and PgAdmin sites for further instructions.

Once we have a database set up, we need to find a way to allow our Express app to communicate with our database. Node.js by default doesn’t support reading and writing to PostgreSQL database, so we’ll be using an excellent library, appropriately named, node-postgres.

node-postgres executes SQL queries in node and returns the result as an object, from which we can grab items from the rows key.

Let’s connect node-postgres to our application.

# install node-postgres
yarn add pg

Open settings.js and add the line below:

export const connectionString = process.env.CONNECTION_STRING;

Open your .env file and add the CONNECTION_STRING variable. This is the connection string we’ll be using to establish a connection to our database. The general form of the connection string is shown below.

CONNECTION_STRING="postgresql://dbuser:dbpassword@localhost:5432/dbname"

If you’re using elephantSQL you should copy the URL from the database details page.

Inside your /src folder, create a new folder called models/. Inside this folder, create two files:

• pool.js
• model.js

Open pools.js and paste the following code:

import { Pool } from 'pg';
import dotenv from 'dotenv';
import { connectionString } from '../settings';
dotenv.config();

export const pool = new Pool({ connectionString });

First, we import the Pool and dotenv from the pg and dotenv packages, and then import the settings we created for our postgres database before initializing dotenv. We establish a connection to our database with the Pool object. In node-postgres, every query is executed by a client. A Pool is a collection of clients for communicating with the database.

To create the connection, the pool constructor takes a config object. You can read more about all the possible configurations here. It also accepts a single connection string, which I will use here.

Open model.js and paste the following code:

import { pool } from './pool';

class Model {
  constructor(table) {
    this.pool = pool;
    this.table = table;
    this.pool.on('error', (err, client) => `Error, ${err}, on idle client${client}`);
  }

  async select(columns, clause) {
    let query = `SELECT ${columns} FROM ${this.table}`;
    if (clause) query += clause;
    return this.pool.query(query);
  }
}

export default Model;

We create a model class whose constructor accepts the database table we wish to operate on. We’ll be using a single pool for all our models.

We then create a select method which we will use to retrieve items from our database. This method accepts the columns we want to retrieve and a clause, such as a WHERE clause. It returns the result of the query, which is a Promise. Remember we said earlier that every query is executed by a client, but here we execute the query with pool. This is because, when we use pool.query, node-postgres executes the query using the first available idle client.

The query you write is entirely up to you, provided it is a valid SQL statement that can be executed by a Postgres engine.

The next step is to actually create an API endpoint to utilize our newly connected database. Before we do that, I’d like us to create some utility functions. The goal is for us to have a way to perform common database operations from the command line.

Create a folder, utils/ inside the src/ folder. Create three files inside this folder:

• queries.js
• queryFunctions.js
• runQuery.js

We’re going to create functions to create a table in our database, insert seed data in the table, and to delete the table.

Open up queries.js and paste the following code:

export const createMessageTable = `
DROP TABLE IF EXISTS messages;
CREATE TABLE IF NOT EXISTS messages (
  id SERIAL PRIMARY KEY,
  name VARCHAR DEFAULT '',
  message VARCHAR NOT NULL
  )
  `;

export const insertMessages = `
INSERT INTO messages(name, message)
VALUES ('chidimo', 'first message'),
      ('orji', 'second message')
`;

export const dropMessagesTable = 'DROP TABLE messages';

In this file, we define three SQL query strings. The first query deletes and recreates the messages table. The second query inserts two rows into the messages table. Feel free to add more items here. The last query drops/deletes the messages table.

Open queryFunctions.js and paste the following code:

import { pool } from '../models/pool';
import {
  insertMessages,
  dropMessagesTable,
  createMessageTable,
} from './queries';

export const executeQueryArray = async arr => new Promise(resolve => {
  const stop = arr.length;
  arr.forEach(async (q, index) => {
    await pool.query(q);
    if (index + 1 === stop) resolve();
  });
});

export const dropTables = () => executeQueryArray([ dropMessagesTable ]);
export const createTables = () => executeQueryArray([ createMessageTable ]);
export const insertIntoTables = () => executeQueryArray([ insertMessages ]);

Here, we create functions to execute the queries we defined earlier. Note that the executeQueryArray function executes an array of queries and waits for each one to complete inside the loop. (Don’t do such a thing in production code though). Then, we only resolve the promise once we have executed the last query in the list. The reason for using an array is that the number of such queries will grow as the number of tables in our database grows.

Open runQuery.js and paste the following code:

import { createTables, insertIntoTables } from './queryFunctions';

(async () => {
  await createTables();
  await insertIntoTables();
})();

This is where we execute the functions to create the table and insert the messages in the table. Let’s add a command in the scripts section of our package.json to execute this file.

"runQuery": "babel-node ./src/utils/runQuery"

Now run:

yarn runQuery

If you inspect your database, you will see that the messages table has been created and that the messages were inserted into the table.

If you’re using ElephantSQL, on the database details page, click on BROWSER from the left navigation menu. Select the messages table and click Execute. You should see the messages from the queries.js file.

Let’s create a controller and route to display the messages from our database.

Create a new controller file src/controllers/messages.js and paste the following code:

import Model from '../models/model';

const messagesModel = new Model('messages');
export const messagesPage = async (req, res) => {
  try {
    const data = await messagesModel.select('name, message');
    res.status(200).json({ messages: data.rows });
  } catch (err) {
    res.status(200).json({ messages: err.stack });
  }
};

We import our Model class and create a new instance of that model. This represents the messages table in our database. We then use the select method of the model to query our database. The data (name and message) we get is sent as JSON in the response.

We define the messagesPage controller as an async function. Since node-postgres queries return a promise, we await the result of that query. If we encounter an error during the query we catch it and display the stack to the user. You should decide how choose to handle the error.

Add the get messages endpoint to src/routes/index.js and update the import line.

# update the import line
import { indexPage, messagesPage } from '../controllers';

# add the get messages endpoint
indexRouter.get('/messages', messagesPage)

Visit http://localhost:3000/v1/messages and you should see the messages displayed as shown below.

Now, let’s update our test file. When doing TDD, you usually write your tests before implementing the code that makes the test pass. I’m taking the opposite approach here because we’re still working on setting up the database.

Create a new file, hooks.js in the test/ folder and enter the below code:

import {
  dropTables,
  createTables,
  insertIntoTables,
} from '../src/utils/queryFunctions';

before(async () => {
  await createTables();
  await insertIntoTables();
});

after(async () => {
  await dropTables();
});

When our test starts, Mocha finds this file and executes it before running any test file. It executes the before hook to create the database and insert some items into it. The test files then run after that. Once the test is finished, Mocha runs the after hook in which we drop the database. This ensures that each time we run our tests, we do so with clean and new records in our database.

Create a new test file test/messages.test.js and add the below code:

import { expect, server, BASE_URL } from './setup';
describe('Messages', () => {
  it('get messages page', done => {
    server
      .get(`${BASE_URL}/messages`)
      .expect(200)
      .end((err, res) => {
        expect(res.status).to.equal(200);
        expect(res.body.messages).to.be.instanceOf(Array);
        res.body.messages.forEach(m => {
          expect(m).to.have.property('name');
          expect(m).to.have.property('message');
        });
        done();
      });
  });
});

We assert that the result of the call to /messages is an array. For each message object, we assert that it has the name and message property.

The final step in this section is to update the CI files.

Add the following sections to the .travis.yml file:

services:
  - postgresql
addons:
  postgresql: "10"
  apt:
    packages:
    - postgresql-10
    - postgresql-client-10
before_install:
  - sudo cp /etc/postgresql/{9.6,10}/main/pg_hba.conf
  - sudo /etc/init.d/postgresql restart

This instructs Travis to spin up a PostgreSQL 10 database before running our tests.

Add the command to create the database as the first entry in the before_script section:

# add this as the first line in the before_script section

- psql -c 'create database testdb;' -U postgres

Create the CONNECTION_STRING environment variable on Travis, and use the below value:

CONNECTION_STRING="postgresql://postgres:postgres@localhost:5432/testdb"

Add the following sections to the .appveyor.yml file:

before_test:
  - SET PGUSER=postgres
  - SET PGPASSWORD=Password12!
  - PATH=C:\Program Files\PostgreSQL\10\bin\;%PATH%
  - createdb testdb
services:
  - postgresql101

Add the connection string environment variable to appveyor. Use the below line:

CONNECTION_STRING=postgresql://postgres:Password12!@localhost:5432/testdb

Now commit your changes and push to GitHub. Your tests should pass on both Travis CI and AppVeyor.

• The corresponding branch in my repo is 07-connect-postgres.

Note: I hope everything works fine on your end, but in case you should be having trouble for some reason, you can always check my code in the repo!

Now, let’s see how we can add a message to our database. For this step, we’ll need a way to send POST requests to our URL. I’ll be using Postman to send POST requests.

Let’s go the TDD route and update our test to reflect what we expect to achieve.

Open test/message.test.js and add the below test case:

it('posts messages', done => {
  const data = { name: 'some name', message: 'new message' };
  server
    .post(`${BASE_URL}/messages`)
    .send(data)
    .expect(200)
    .end((err, res) => {
      expect(res.status).to.equal(200);
      expect(res.body.messages).to.be.instanceOf(Array);
      res.body.messages.forEach(m => {
        expect(m).to.have.property('id');
        expect(m).to.have.property('name', data.name);
        expect(m).to.have.property('message', data.message);
      });
      done();
    });
});

This test makes a POST request to the /v1/messages endpoint and we expect an array to be returned. We also check for the id, name, and message properties on the array.

Run your tests to see that this case fails. Let’s now fix it.

To send post requests, we use the post method of the server. We also send the name and message we want to insert. We expect the response to be an array, with a property id and the other info that makes up the query. The id is proof that a record has been inserted into the database.

Open src/models/model.js and add the insert method:

async insertWithReturn(columns, values) {
  const query = `
        INSERT INTO ${this.table}(${columns})
        VALUES (${values})
        RETURNING id, ${columns}
    `;
  return this.pool.query(query);
}

This is the method that allows us to insert messages into the database. After inserting the item, it returns the id, name and message.

Open src/controllers/messages.js and add the below controller:

export const addMessage = async (req, res) => {
  const { name, message } = req.body;
  const columns = 'name, message';
  const values = `'${name}', '${message}'`;
  try {
    const data = await messagesModel.insertWithReturn(columns, values);
    res.status(200).json({ messages: data.rows });
  } catch (err) {
    res.status(200).json({ messages: err.stack });
  }
};

We destructure the request body to get the name and message. Then we use the values to form an SQL query string which we then execute with the insertWithReturn method of our model.

Add the below POST endpoint to /src/routes/index.js and update your import line.

import { indexPage, messagesPage, addMessage } from '../controllers';

indexRouter.post('/messages', addMessage);

Run your tests to see if they pass.

Open Postman and send a POST request to the messages endpoint. If you’ve just run your test, remember to run yarn query to recreate the messages table.

yarn query

Commit your changes and push to GitHub. Your tests should pass on both Travis and AppVeyor. Your test coverage will drop by a few points, but that’s okay.

• The corresponding branch on my repo is 08-post-to-db.

Middleware

Our discussion of Express won’t be complete without talking about middleware. The Express documentation describes a middlewares as:

“[...] functions that have access to the request object (req), the response object (res), and the next middleware function in the application’s request-response cycle. The next middleware function is commonly denoted by a variable named next.”

A middleware can perform any number of functions such as authentication, modifying the request body, and so on. See the Express documentation on using middleware.

We’re going to write a simple middleware that modifies the request body. Our middleware will append the word SAYS: to the incoming message before it is saved in the database.

Before we start, let’s modify our test to reflect what we want to achieve.

Open up test/messages.test.js and modify the last expect line in the posts message test case:

it('posts messages', done => {
   ...
  expect(m).to.have.property('message', `SAYS: ${data.message}`); # update this line
    ...
});

We’re asserting that the SAYS: string has been appended to the message. Run your tests to make sure this test case fails.

Now, let’s write the code to make the test pass.

Create a new middleware/ folder inside src/ folder. Create two files inside this folder:

• middleware.js
• index.js

Enter the below code in middleware.js:

export const modifyMessage = (req, res, next) => {
  req.body.message = `SAYS: ${req.body.message}`;
  next();
};

Here, we append the string SAYS: to the message in the request body. After doing that, we must call the next() function to pass execution to the next function in the request-response chain. Every middleware has to call the next function to pass execution to the next middleware in the request-response cycle.

Enter the below code in index.js:

# export everything from the middleware file

export * from './middleware';

This exports the middleware we have in the /middleware.js file. For now, we only have the modifyMessage middleware.

Open src/routes/index.js and add the middleware to the post message request-response chain.

import { modifyMessage } from '../middleware';

indexRouter.post('/messages', modifyMessage, addMessage);

We can see that the modifyMessage function comes before the addMessage function. We invoke the addMessage function by calling next in the modifyMessage middleware. As an experiment, comment out the next() line in the modifyMessage middle and watch the request hang.

Open Postman and create a new message. You should see the appended string.

This is a good point to commit our changes.

• The corresponding branch in my repo is 09-middleware.

Error Handling And Asynchronous Middleware

Errors are inevitable in any application. The task before the developer is how to deal with errors as gracefully as possible.

In Express:

“Error Handling refers to how Express catches and processes errors that occur both synchronously and asynchronously.

If we were only writing synchronous functions, we might not have to worry so much about error handling as Express already does an excellent job of handling those. According to the docs:

“Errors that occur in synchronous code inside route handlers and middleware require no extra work.”

But once we start writing asynchronous router handlers and middleware, then we have to do some error handling.

Our modifyMessage middleware is a synchronous function. If an error occurs in that function, Express will handle it just fine. Let’s see how we deal with errors in asynchronous middleware.

Let’s say, before creating a message, we want to get a picture from the Lorem Picsum API using this URL https://picsum.photos/id/0/info. This is an asynchronous operation that could either succeed or fail, and that presents a case for us to deal with.

Start by installing Axios.

# install axios
yarn add axios

Open src/middleware/middleware.js and add the below function:

export const performAsyncAction = async (req, res, next) => {
  try {
    await axios.get('https://picsum.photos/id/0/info');
    next();
  } catch (err) {
    next(err);
  }
};

In this async function, we await a call to an API (we don’t actually need the returned data) and afterward call the next function in the request chain. If the request fails, we catch the error and pass it on to next. Once Express sees this error, it skips all other middleware in the chain. If we didn’t call next(err), the request will hang. If we only called next() without err, the request will proceed as if nothing happened and the error will not be caught.

Import this function and add it to the middleware chain of the post messages route:

import { modifyMessage, performAsyncAction } from '../middleware';

indexRouter.post('/messages', modifyMessage, performAsyncAction, addMessage);

Open src/app.js and add the below code just before the export default app line.

app.use((err, req, res, next) => {
  res.status(400).json({ error: err.stack });
});

export default app;

This is our error handler. According to the Express error handling doc:

“[...] error-handling functions have four arguments instead of three: (err, req, res, next).”

Note that this error handler must come last, after every app.use() call. Once we encounter an error, we return the stack trace with a status code of 400. You could do whatever you like with the error. You might want to log it or send it somewhere.

This is a good place to commit your changes.

• The corresponding branch in my repo is 10-async-middleware.

Deploy To Heroku

1. To get started, go to https://www.heroku.com/ and either log in or register.
2. Download and install the Heroku CLI from here.
3. Open a terminal in the project folder to run the command.

# login to heroku on command line
heroku login

This will open a browser window and ask you to log into your Heroku account.

Log in to grant your terminal access to your Heroku account, and create a new heroku app by running:

#app name is up to you
heroku create app-name

This will create the app on Heroku and return two URLs.

# app production url and git url
https://app-name.herokuapp.com/ | https://git.heroku.com/app-name.git

Copy the URL on the right and run the below command. Note that this step is optional as you may find that Heroku has already added the remote URL.

# add heroku remote url
git remote add heroku https://git.heroku.com/my-shiny-new-app.git

Open a side terminal and run the command below. This shows you the app log in real-time as shown in the image.

# see process logs
heroku logs --tail

Run the following three commands to set the required environment variables:

heroku config:set TEST_ENV_VARIABLE="Environment variable is coming across."
heroku config:set CONNECTION_STRING=your-db-connection-string-here.
heroku config:set NPM_CONFIG_PRODUCTION=false

Remember in our scripts, we set:

"prestart": "babel ./src --out-dir build",
"start": "node ./build/bin/www",

To start the app, it needs to be compiled down to ES5 using babel in the prestart step because babel only exists in our development dependencies. We have to set NPM_CONFIG_PRODUCTION to false in order to be able to install those as well.

To confirm everything is set correctly, run the command below. You could also visit the settings tab on the app page and click on Reveal Config Vars.

# check configuration variables
heroku config

Now run git push heroku.

To open the app, run:

# open /v1 route
heroku open /v1

# open /v1/messages route
heroku open /v1/messages

If like me, you’re using the same PostgresSQL database for both development and production, you may find that each time you run your tests, the database is deleted. To recreate it, you could run either one of the following commands:

# run script locally
yarn runQuery

# run script with heroku
heroku run yarn runQuery

Continuous Deployment (CD) With Travis

Let’s now add Continuous Deployment (CD) to complete the CI/CD flow. We will be deploying from Travis after every successful test run.

The first step is to install Travis CI. (You can find the installation instructions over here.) After successfully installing the Travis CI, login by running the below command. (Note that this should be done in your project repository.)

# login to travis
travis login --pro

# use this if you’re using two factor authentication
travis login --pro --github-token enter-github-token-here

If your project is hosted on travis-ci.org, remove the --pro flag. To get a GitHub token, visit the developer settings page of your account and generate one. This only applies if your account is secured with 2FA.

Open your .travis.yml and add a deploy section:

deploy:
  provider: heroku
  app:
    master: app-name

Here, we specify that we want to deploy to Heroku. The app sub-section specifies that we want to deploy the master branch of our repo to the app-name app on Heroku. It’s possible to deploy different branches to different apps. You can read more about the available options here.

Run the below command to encrypt your Heroku API key and add it to the deploy section:

# encrypt heroku API key and add to .travis.yml
travis encrypt $(heroku auth:token) --add deploy.api_key --pro

This will add the below sub-section to the deploy section.

api_key:
  secure: very-long-encrypted-api-key-string

Now commit your changes and push to GitHub while monitoring your logs. You will see the build triggered as soon as the Travis test is done. In this way, if we have a failing test, the changes would never be deployed. Likewise, if the build failed, the whole test run would fail. This completes the CI/CD flow.

• The corresponding branch in my repo is 11-cd.

Conclusion

If you’ve made it this far, I say, “Thumbs up!” In this tutorial, we successfully set up a new Express project. We went ahead to configure development dependencies as well as Continuous Integration (CI). We then wrote asynchronous functions to handle requests to our API endpoints — completed with tests. We then looked briefly at error handling. Finally, we deployed our project to Heroku and configured Continuous Deployment.

You now have a template for your next back-end project. We’ve only done enough to get you started, but you should keep learning to keep going. Be sure to check out the Express.js docs as well. If you would rather use MongoDB instead of PostgreSQL, I have a template here that does exactly that. You can check it out for the setup. It has only a few points of difference.

Resources

• “Create Express API Backend With MongoDB ,” Orji Chidi Matthew, GitHub
• “A Short Guide To Connect Middleware,” Stephen Sugden
• “Express API template,” GitHub
• “AppVeyor vs Travis CI,” StackShare
• “The Heroku CLI,” Heroku Dev Center
• “Heroku Deployment,” Travis CI
• “Using middleware,” Express.js
• “Error Handling,” Express.js
• “Getting Started,” Mocha
• nyc (GitHub)
• ElephantSQL
• Postman
• Express
• Travis CI
• Code Climate
• PostgreSQL
• pgAdmin

(ks, yk, il)

If you have been looking for an alternative to pagination, infinite scroll is a good consideration. In this article, we’re going to explore some use cases for the Intersection Observer API in the context of a React functional component. The reader should possess a working knowledge of React functional components. Some familiarity with React hooks will be beneficial but not required, as we will be taking a look at a few.

Our goal is that at the end of this article, we will have implemented infinite scroll and image lazy loading using a native HTML API. We would also have learned a few more things about React Hooks. With that you can be able to implement infinite scroll and image lazy loading in your React application where necessary.

Let’s get started.

The Intersection Observer API

According to the MDN docs, “the Intersection Observer API provides a way to asynchronously observe changes in the intersection of a target element with an ancestor element or with a top-level document’s viewport”.

This API allows us to implement cool features such as infinite scroll and image lazy loading. The intersection observer is created by calling its constructor and passing it a callback and an options object. The callback is invoked whenever one element, called the target, intersects either the device viewport or a specified element, called the root. We can specify a custom root in the options argument or use the default value.

let observer = new IntersectionObserver(callback, options);

The API is straightforward to use. A typical example looks like this:

var intObserver = new IntersectionObserver(entries => {
    entries.forEach(entry => {
      console.log(entry)
      console.log(entry.isIntersecting) // returns true if the target intersects the root element
    })
  },
  {
    // default options
  }
);
let target = document.querySelector('#targetId');
intObserver.observe(target); // start observation

entries is a list of IntersectionObserverEntry objects. The IntersectionObserverEntry object describes an intersection change for one observed target element. Note that the callback should not handle any time-consuming task as it runs on the main thread.

The Intersection Observer API currently enjoys broad browser support, as shown on caniuse.

You can read more about the API in the links provided in the resources section.

Let us now look at how to make use of this API in a real React app. The final version of our app will be a page of pictures that scrolls infinitely and will have each image loaded lazily.

Making API Calls With The useEffect Hook

To get started, clone the starter project from this URL. It has minimal setup and a few styles defined. I’ve also added a link to Bootstrap’s CSS in the public/index.html file as I’ll be using its classes for styling.

Feel free to create a new project if you like. Make sure you have yarn package manager installed if you want to follow with the repo. You can find the installation instructions for your specific operating system here.

For this tutorial, we’re going to be grabbing pictures from a public API and displaying them on the page. We will be using the Lorem Picsum APIs.

For this tutorial, we’ll be using the endpoint, https://picsum.photos/v2/list?page=0&limit=10, which returns an array of picture objects. To get the next ten pictures, we change the value of page to 1, then 2, and so on.

We will now build the App component piece by piece.

Open up src/App.js and enter the following code.

import React, { useEffect, useReducer } from 'react';

import './index.css';

function App() {
  const imgReducer = (state, action) => {
    switch (action.type) {
      case 'STACK_IMAGES':
        return { ...state, images: state.images.concat(action.images) }
      case 'FETCHING_IMAGES':
        return { ...state, fetching: action.fetching }
      default:
        return state;
    }
  }
  const [imgData, imgDispatch] = useReducer(imgReducer,{ images:[], fetching: true})
  // next code block goes here
}

Firstly, we define a reducer function, imgReducer. This reducer handles two actions.

1. The STACK_IMAGES action concatenates the images array.
2. FETCHING_IMAGES action toggles the value of the fetching variable between true and false.

The next step is to wire up this reducer to a useReducer hook. Once that is done, we get back two things:

1. imgData, which contains two variables: images is the array of picture objects. fetching is a boolean which tells us if the API call is in progress or not.
2. imgDispatch, which is a function for updating the reducer object.

You can learn more about the useReducer hook in the React documentation.

The next part of the code is where we make the API call. Paste the following code below the previous code block in App.js.

// make API calls
useEffect(() => {
  imgDispatch({ type: 'FETCHING_IMAGES', fetching: true })
  fetch('https://picsum.photos/v2/list?page=0&limit=10')
    .then(data => data.json())
    .then(images => {
      imgDispatch({ type: 'STACK_IMAGES', images })
      imgDispatch({ type: 'FETCHING_IMAGES', fetching: false })
    })
    .catch(e => {
      // handle error
      imgDispatch({ type: 'FETCHING_IMAGES', fetching: false })
      return e
    })
}, [ imgDispatch ])

// next code block goes here

Inside the useEffect hook, we make a call to the API endpoint with fetch API. We then update the images array with the result of the API call by dispatching the STACK_IMAGES action. We also dispatch the FETCHING_IMAGES action once the API call completes.

The next block of code defines the return value of the function. Enter the following code after the useEffect hook.

return (
  <div className="">
    <nav className="navbar bg-light">
      <div className="container">
        <a className="navbar-brand" href="/#">
          <h2>Infinite scroll + image lazy loading</h2>
        </a>
      </div>
    </navv
    <div id='images' className="container">
      <div className="row">
        {imgData.images.map((image, index) => {
          const { author, download_url } = image
          return (
            <div key={index} className="card">
              <div className="card-body ">
                <img
                  alt={author}
                  className="card-img-top"
                  src={download_url}
                />
              </div>
              <div className="card-footer">
                <p className="card-text text-center text-capitalize text-primary">Shot by: {author}</p>
              </div>
            </div>
          )
        })}
      </div>
    </div>
  </div>
);

To display the images, we map over the images array in the imgData object.

Now start the app and view the page in the browser. You should see the images nicely displayed in a responsive grid.

The last bit is to export the App component.

export default App;

The corresponding branch at this point is 01-make-api-calls.

Let’s now extend this by displaying more pictures as the page scrolls.

Implementing Infinite Scroll

We aim to present more pictures as the page scrolls. From the URL of the API endpoint, https://picsum.photos/v2/list?page=0&limit=10, we know that to get a new set of photos, we only need to increment the value of page. We also need to do this when we have run out of pictures to show. For our purpose here, we’ll know we have run out of images when we hit the bottom of the page. It’s time to see how the Intersection Observer API helps us achieve that.

Open up src/App.js and create a new reducer, pageReducer, below imgReducer.

// App.js
const imgReducer = (state, action) => {
  ...
}
const pageReducer = (state, action) => {
  switch (action.type) {
    case 'ADVANCE_PAGE':
      return { ...state, page: state.page + 1 }
    default:
      return state;
  }
}
const [ pager, pagerDispatch ] = useReducer(pageReducer, { page: 0 })

We define only one action type. Each time the ADVANCE_PAGE action is triggered, the value of page is incremented by 1.

Update the URL in the fetch function to accept page numbers dynamically as shown below.

fetch(`https://picsum.photos/v2/list?page=${pager.page}&limit=10`)

Add pager.page to the dependency array alongside imgData. Doing this ensures that the API call will run whenever pager.page changes.

useEffect(() => {
...
}, [ imgDispatch, pager.page ])

After the useEffect hook for the API call, enter the below code. Update your import line as well.

// App.js
import React, { useEffect, useReducer, useCallback, useRef } from 'react';
useEffect(() => {
  ...
}, [ imgDispatch, pager.page ])

// implement infinite scrolling with intersection observer
let bottomBoundaryRef = useRef(null);
const scrollObserver = useCallback(
  node => {
    new IntersectionObserver(entries => {
      entries.forEach(en => {
        if (en.intersectionRatio > 0) {
          pagerDispatch({ type: 'ADVANCE_PAGE' });
        }
      });
    }).observe(node);
  },
  [pagerDispatch]
);
useEffect(() => {
  if (bottomBoundaryRef.current) {
    scrollObserver(bottomBoundaryRef.current);
  }
}, [scrollObserver, bottomBoundaryRef]);

We define a variable bottomBoundaryRef and set its value to useRef(null). useRef lets variables preserve their values across component renders, i.e. the current value of the variable persists when the containing component re-renders. The only way to change its value is by re-assigning the .current property on that variable.

In our case, bottomBoundaryRef.current starts with a value of null. As the page rendering cycle proceeds, we set its current property to be the node <div id='page-bottom-boundary'>.

We use the assignment statement ref={bottomBoundaryRef} to tell React to set bottomBoundaryRef.current to be the div where this assignment is declared.

Thus,

bottomBoundaryRef.current = null

at the end of the rendering cycle, becomes:

bottomBoundaryRef.current = <div id="page-bottom-boundary" style="border: 1px solid red;"></div>

We shall see where this assignment is done in a minute.

Next, we define a scrollObserver function, in which to set the observer. This function accepts a DOM node to observe. The main point to note here is that whenever we hit the intersection under observation, we dispatch the ADVANCE_PAGE action. The effect is to increment the value of pager.page by 1. Once this happens, the useEffect hook that has it as a dependency is re-run. This re-run, in turn, invokes the fetch call with the new page number.

The event procession looks like this.

Hit intersection under observation → call ADVANCE_PAGE action → increment value of pager.page by 1 → useEffect hook for fetch call runs → fetch call is run → returned images are concatenated to the images array.

We invoke scrollObserver in a useEffect hook so that the function will run only when any of the hook’s dependencies change. If we didn’t call the function inside a useEffect hook, the function would run on every page render.

Recall that bottomBoundaryRef.current refers to <div id="page-bottom-boundary" style="border: 1px solid red;"></div>. We check that its value is not null before passing it to scrollObserver. Otherwise, the IntersectionObserver constructor would return an error.

Because we used scrollObserver in a useEffect hook, we have to wrap it in a useCallback hook to prevent un-ending component re-renders. You can learn more about useCallback in the React docs.

Enter the below code after the <div id='images'> div.

// App.js
<div id='image'>
...
</div>
{imgData.fetching && (
  <div className="text-center bg-secondary m-auto p-3">
    <p className="m-0 text-white">Getting images</p>
  </div>
)}
<div id='page-bottom-boundary' style={{ border: '1px solid red' }} ref={bottomBoundaryRef}></div>

When the API call starts, we set fetching to true, and the text Getting images becomes visible. As soon as it finishes, we set fetching to false, and the text gets hidden. We could also trigger the API call before hitting the boundary exactly by setting a different threshold in the constructor options object. The red line at the end lets us see exactly when we hit the page boundary.

The corresponding branch at this point is 02-infinite-scroll.

We will now implement image lazy loading.

Implementing Image Lazy Loading

If you inspect the network tab as you scroll down, you’ll see that as soon as you hit the red line (the bottom boundary), the API call happens, and all the images start loading even when you haven’t gotten to viewing them. There are a variety of reasons why this might not be desirable behavior. We may want to save network calls until the user wants to see an image. In such a case, we could opt for loading the images lazily, i.e., we won’t load an image until it scrolls into view.

Open up src/App.js. Just below the infinite scrolling functions, enter the following code.

// App.js

// lazy loads images with intersection observer
// only swap out the image source if the new url exists
const imagesRef = useRef(null);
const imgObserver = useCallback(node => {
  const intObs = new IntersectionObserver(entries => {
    entries.forEach(en => {
      if (en.intersectionRatio > 0) {
        const currentImg = en.target;
        const newImgSrc = currentImg.dataset.src;
        // only swap out the image source if the new url exists
        if (!newImgSrc) {
          console.error('Image source is invalid');
        } else {
          currentImg.src = newImgSrc;
        }
        intObs.unobserve(node); // detach the observer when done
      }
    });
  })
  intObs.observe(node);
}, []);
useEffect(() => {
  imagesRef.current = document.querySelectorAll('.card-img-top');
  if (imagesRef.current) {
    imagesRef.current.forEach(img => imgObserver(img));
  }
}, [imgObserver, imagesRef, imgData.images]);

As with scrollObserver, we define a function, imgObserver, which accepts a node to observe. When the page hits an intersection, as determined by en.intersectionRatio > 0, we swap the image source on the element. Notice that we first check if the new image source exists before doing the swap. As with the scrollObserver function, we wrap imgObserver in a useCallback hook to prevent un-ending component re-render.

Also note that we stop observing an img element once we’re done with the substitution. We do this with the unobserve method.

In the following useEffect hook, we grab all the images with a class of .card-img-top on the page with document.querySelectorAll. Then we iterate over each image and set an observer on it.

Note that we added imgData.images as a dependency of the useEffect hook. When this changes it triggers the useEffect hook and in turn imgObserver get called with each <img className='card-img-top'> element.

Update the <img className='card-img-top'/> element as shown below.

<img
  alt={author}
  data-src={download_url}
  className="card-img-top"
  src={'https://picsum.photos/id/870/300/300?grayscale&blur=2'}
/>

We set a default source for every <img className='card-img-top'/> element and store the image we want to show on the data-src property. The default image usually has a small size so that we’re downloading as little as possible. When the <img/> element comes into view, the value on the data-src property replaces the default image.

In the picture below, we see the default lighthouse image still showing in some of the spaces.

The corresponding branch at this point is 03-lazy-loading.

Let’s now see how we can abstract all these functions so that they’re re-usable.

Abstracting Fetch, Infinite Scroll And Lazy Loading Into Custom Hooks

We have successfully implemented fetch, infinite scroll, and image lazy loading. We might have another component in our application that needs similar functionality. In that case, we could abstract and reuse these functions. All we have to do is move them inside a separate file and import them where we need them. We want to turn them into Custom Hooks.

The React documentation defines a Custom Hook as a JavaScript function whose name starts with "use" and that may call other hooks. In our case, we want to create three hooks, useFetch, useInfiniteScroll, useLazyLoading.

Create a file inside the src/ folder. Name it customHooks.js and paste the code below inside.

// customHooks.js

import { useEffect, useCallback, useRef } from 'react';
// make API calls and pass the returned data via dispatch
export const useFetch = (data, dispatch) => {
  useEffect(() => {
    dispatch({ type: 'FETCHING_IMAGES', fetching: true });
    fetch(`https://picsum.photos/v2/list?page=${data.page}&limit=10`)
      .then(data => data.json())
      .then(images => {
        dispatch({ type: 'STACK_IMAGES', images });
        dispatch({ type: 'FETCHING_IMAGES', fetching: false });
      })
      .catch(e => {
        dispatch({ type: 'FETCHING_IMAGES', fetching: false });
        return e;
      })
  }, [dispatch, data.page])
}

// next code block here

The useFetch hook accepts a dispatch function and a data object. The dispatch function passes the data from the API call to the App component, while the data object lets us update the API endpoint URL.

// infinite scrolling with intersection observer
export const useInfiniteScroll = (scrollRef, dispatch) => {
  const scrollObserver = useCallback(
    node => {
      new IntersectionObserver(entries => {
        entries.forEach(en => {
          if (en.intersectionRatio > 0) {
            dispatch({ type: 'ADVANCE_PAGE' });
          }
        });
      }).observe(node);
    },
    [dispatch]
  );
  useEffect(() => {
    if (scrollRef.current) {
      scrollObserver(scrollRef.current);
    }
  }, [scrollObserver, scrollRef]);
}

// next code block here

The useInfiniteScroll hook accepts a scrollRef and a dispatch function. The scrollRef helps us set up the observer, as already discussed in the section where we implemented it. The dispatch function gives a way to trigger an action that updates the page number in the API endpoint URL.

// lazy load images with intersection observer
export const useLazyLoading = (imgSelector, items) => {
  const imgObserver = useCallback(node => {
  const intObs = new IntersectionObserver(entries => {
    entries.forEach(en => {
      if (en.intersectionRatio > 0) {
        const currentImg = en.target;
        const newImgSrc = currentImg.dataset.src;
        // only swap out the image source if the new url exists
        if (!newImgSrc) {
          console.error('Image source is invalid');
        } else {
          currentImg.src = newImgSrc;
        }
        intObs.unobserve(node); // detach the observer when done
      }
    });
  })
  intObs.observe(node);
  }, []);
  const imagesRef = useRef(null);
  useEffect(() => {
    imagesRef.current = document.querySelectorAll(imgSelector);
    if (imagesRef.current) {
      imagesRef.current.forEach(img => imgObserver(img));
    }
  }, [imgObserver, imagesRef, imgSelector, items])
}

The useLazyLoading hook receives a selector and an array. The selector is used to find the images. Any change in the array triggers the useEffect hook that sets up the observer on each image.

We can see that it is the same functions we have in src/App.js that we have extracted to a new file. The good thing now is that we can pass arguments dynamically. Let’s now use these custom hooks in the App component.

Open src/App.js. Import the custom hooks and delete the functions we defined for fetching data, infinite scroll, and image lazy loading. Leave the reducers and the sections where we make use of useReducer. Paste in the below code.

// App.js

// import custom hooks
import { useFetch, useInfiniteScroll, useLazyLoading } from './customHooks'

  const imgReducer = (state, action) => { ... } // retain this
  const pageReducer = (state, action) => { ... } // retain this
  const [pager, pagerDispatch] = useReducer(pageReducer, { page: 0 }) // retain this
  const [imgData, imgDispatch] = useReducer(imgReducer,{ images:[], fetching: true }) // retain this

let bottomBoundaryRef = useRef(null);
useFetch(pager, imgDispatch);
useLazyLoading('.card-img-top', imgData.images)
useInfiniteScroll(bottomBoundaryRef, pagerDispatch);

// retain the return block
return (
  ...
)

We have already talked about bottomBoundaryRef in the section on infinite scroll. We pass the pager object and the imgDispatch function to useFetch. useLazyLoading accepts the class name .card-img-top. Note the . included in the class name. By doing this, we don’t need to specify it document.querySelectorAll. useInfiniteScroll accepts both a ref and the dispatch function for incrementing the value of page.

The corresponding branch at this point is 04-custom-hooks.

Conclusion

HTML is getting better at providing nice APIs for implementing cool features. In this post, we’ve seen how easy it is to use the intersection observer in a React functional component. In the process, we learned how to use some of React’s hooks and how to write our own hooks.

Resources

• “Infinite Scroll + Image Lazy Loading,” Orji Chidi Matthew, GitHub
• “Infinite Scrolling, Pagination Or “Load More” Buttons? Usability Findings In eCommerce,” Christian Holst, Smashing Magazine
• “Lorem Picsum,” David Marby & Nijiko Yonskai
• “IntersectionObserver’s Coming Into View,” Surma, Web Fundamentals
• Can I Use…IntersectionObserver
• “Intersection Observer API,” MDN web docs
• “Components And Props,” React
• “useCallback,” React
• “useReducer,” React

(ks, ra, yk, il)