API Publish/Subscribe Between Zato Services

One of the additions in the upcoming Zato 3.2 release is an extension to its publish/subscribe mechanism that lets services publish messages directly to other services. Let’s check how to use it and how it compares to other means of invoking one’s API services. 

How Does It Work?

In your Zato service, you can publish a message to any other services as below. Simply point self.pubsub.publish to the target service by the latter’s name and it will receive your message.

Practical Serverless: A Scalable OCR Solution in 10 Minutes

Introduction

In this article, we will show you how to create a serverless solution for implementing a scalable Optical Character Recognition (OCR) system. In a system like this, scalability is a requirement. At certain times, we can expect possible bursts of traffic into the system where we need to process all of these requests and communicate the result back to the user in a timely manner.  To cater to this, we need a system that scales dynamically. One possible solution is to model the required workers and deploy them in a Kubernetes environment to achieve our scaling requirements. This approach has been implemented and discussed in this article.

Here, we will implement the same solution using Azure Functions in Ballerina (referred to as Ballerinalang in the rest of the article) and show how it can be implemented with considerably fewer lines of code, which resulted in lesser complexity and better maintainability.

A Non-Blocking State Machine

Learn more about non-blocking state machines in this quick post!

In a previous article, I presented a simple state machine for Spring Boot projects. I mentioned that the framework is easy to customize for new requirements. In this article, I illustrate the customization of the framework for a situation where one or more processors need to be non-blocking.

Understanding Async Await

When writing code for the web, eventually you'll need to do some process that might take a few moments to complete. JavaScript can't really multitask, so we'll need a way to handle those long-running processes.

Async/Await is a way to handle this type of time-based sequencing. It’s especially great for when you need to make some sort of network request and then work with the resulting data. Let's dig in!

Promise? Promise.

Async/Await is a type of Promise. Promises in JavaScript are objects that can have multiple states (kind of like the real-life ones ☺️). Promises do this because sometimes what we ask for isn't available immediately, and we'll need to be able to detect what state it is in.

Consider someone asks you to promise to do something for them, like help them move. There is the initial state, where they have asked. But you haven't fulfilled your promise to them until you show up and help them move. If you cancel your plans, you rejected the promise.

Similarly, the three possible states for a promise in JavaScript are:

  • pending: when you first call a promise and it’s unknown what it will return.
  • fulfilled: meaning that the operation completed successfully
  • rejected: the operation failed

Here’s an example of a promise in these states:

Here is the fulfilled state. We store a promise called getSomeTacos, passing in the resolve and reject parameters. We tell the promise it is resolved, and that allows us to then console log two more times.

const getSomeTacos = new Promise((resolve, reject) => {
  console.log("Initial state: Excuse me can I have some tacos");

  resolve();
})
  .then(() => {
    console.log("Order some tacos");
  })
  .then(() => {
    console.log("Here are your tacos");
  })
  .catch(err => {
    console.error("Nope! No tacos for you.");
  });
> Initial state: Excuse me can I have some tacos
> Order some tacos
> Here are your tacos

See the Pen
Promise States
by Sarah Drasner (@sdras)
on CodePen.

If we choose the rejected state, we'll do the same function but reject it this time. Now what will be printed to the console is the Initial State and the catch error:

const getSomeTacos = new Promise((resolve, reject) => {
  console.log("Initial state: Excuse me can I have some tacos");

  reject();
})
  .then(() => {
    console.log("Order some tacos");
  })
  .then(() => {
    console.log("Here are your tacos");
  })
  .catch(err => {
    console.error("Nope! No tacos for you.");
  });
> Initial state: Excuse me can I have some tacos
> Nope! No tacos for you.

And when we select the pending state, we'll simply console.log what we stored, getSomeTacos. This will print out a pending state because that's the state the promise is in when we logged it!

console.log(getSomeTacos)
> Initial state: Excuse me can I have some 🌮s
> Promise {<pending>}
> Order some &#x1f32e;s
> Here are your &#x1f32e;s

What then?

But here’s a part that was confusing to me at first. To get a value out of a promise, you have to use .then() or something that returns the resolution of your promise. This makes sense if you think about it, because you need to capture what it will eventually be — rather than what it initially is — because it will be in that pending state initially. That's why we saw it print out Promise {<pending>} when we logged the promise above. Nothing had resolved yet at that point in the execution.

Async/Await is really syntactic sugar on top of those promises you just saw. Here's a small example of how I might use it along with a promise to schedule multiple executions.

async function tacos() {
  return await Promise.resolve("Now and then I get to eat delicious tacos!")
};

tacos().then(console.log)

Or a more in-depth example:

// this is the function we want to schedule. it's a promise.
const addOne = (x) => {
  return new Promise(resolve => {
    setTimeout(() => { 
      console.log(`I added one! Now it's ${x + 1}.`)
      resolve()
    }, 2000);
  })
}

// we will immediately log the first one, 
// then the addOne promise will run, taking 2 seconds
// then the final console.log will fire
async function addAsync() {
  console.log('I have 10')
  await addOne(10)
  console.log(`Now I'm done!`)
}

addAsync()
> I have 10
> I added one! Now it's 11.
> Now I'm done!

See the Pen
Async Example 1
by Sarah Drasner (@sdras)
on CodePen.

One thing (a)waits for another

One common use of Async/Await is to use it to chain multiple asynchronous calls. Here, we'll fetch some JSON that we'll use to pass into our next fetch call to figure out what type of thing we want to fetch from the second API. In our case, we want to access some programming jokes, but we first need to find out from a different API what type of quote we want.

The first JSON file looks like this- we want the type of quote to be random:

{
  "type": "random"
}

The second API will return something that looks like this, given that random query parameter we just got:

{
  "_id":"5a933f6f8e7b510004cba4c2",
  "en":"For all its power, the computer is a harsh taskmaster. Its programs must be correct, and what we wish to say must be said accurately in every detail.",
  "author":"Alan Perlis",
  "id":"5a933f6f8e7b510004cba4c2"
}

We call the async function then let it wait to go retrieve the first .json file before it fetches data from the API. Once that happens, we can do something with that response, like add it to our page.

async function getQuote() {
  // get the type of quote from one fetch call, everything else waits for this to finish
  let quoteTypeResponse = await fetch(`https://s3-us-west-2.amazonaws.com/s.cdpn.io/28963/quotes.json`)
  let quoteType = await quoteTypeResponse.json()

    // use what we got from the first call in the second call to an API, everything else waits for this to finish
  let quoteResponse = await fetch("https://programming-quotes-api.herokuapp.com/quotes/" + quoteType.type)
  let quote = await quoteResponse.json()

  // finish up
  console.log('done')
}

We can even simplify this using template literals and arrow functions:

async function getQuote() {
  // get the type of quote from one fetch call, everything else waits for this to finish
  let quoteType = await fetch(`quotes.json`).then(res => res.json())

    // use what we got from the first call in the second call to an API, everything else waits for this to finish
  let quote = await fetch(`programming-quotes.com/${quoteType.type}`).then(res => res.json())

  // finish up
  console.log('done')
}

getQuote()

Here is an animated explanation of this process.

See the Pen
Animated Description of Async Await
by Sarah Drasner (@sdras)
on CodePen.

Try, Catch, Finally

Eventually we’ll want to add error states to this process. We have handy try, catch, and finally blocks for this.

try {
  // I’ll try to execute some code for you
}
catch(error) {
  // I’ll handle any errors in that process
} 
finally {
  // I’ll fire either way
}

Let’s restructure the code above to use this syntax and catch any errors.

async function getQuote() {
  try {
    // get the type of quote from one fetch call, everything else waits for this to finish
    let quoteType = await fetch(`quotes.json`).then(res => res.json())

      // use what we got from the first call in the second call to an API, everything else waits for this to finish
    let quote = await fetch(`programming-quotes.com/${quoteType.type}`).then(res => res.json())

    // finish up
    console.log('done')
  }

  catch(error) {
    console.warn(`We have an error here: ${error}`)
  }
}

getQuote()

We didn’t use finally here because we don’t always need it. It is a block that will always fire whether it is successful or fails. Consider using finally any time you’re duplicating things in both try and catch. I usually use this for some cleanup. I wrote an article about this, if you’re curious to know more.

You might eventually want more sophisticated error handling, such as a way to cancel an async function. There is, unfortunately, no way to do this natively, but thankfully, Kyle Simpson created a library called CAF that can help.

Further Reading

It's common for explanations of Async/Await to begin with callbacks, then promises, and use those explanations to frame Async/Await. Since Async/Await is well-supported these days, we didn’t walk through all of these steps. It’s still pretty good background, especially if you need to maintain older codebases. Here are some of my favorite resources out there:

The post Understanding Async Await appeared first on CSS-Tricks.

Publish/Subscribe and Asynchronous API Integrations

Publish/Subscribe and Asynchronous API Integrations

This article introduces features built into Zato that let you take advantage of publish/subscribe topics and message queues in communication between Zato services, API clients, and backend systems.

Overview

Let's start by recalling the basic means through which services can invoke each other.

Introducing the Couchbase Scala SDK

Couchbase Scala SDK

I'm very happy to report that Couchbase now has a supported Scala SDK, allowing you to get and fetch documents, run N1QL queries, and perform analytics and full-text search lookups — all with native Scala.

In this article, I'm going to touch on the key features and design principles of the Scala SDK. Or if you want to get going right away, then check out the getting started guide here. The Scala SDK is available to download right now in a pre-release alpha form.

How to Split Up Synchronous and Asynchronous Parts of Your System in Java

A lot of developers say that it's very complicated to switch their applications over to asynchronous processing because they have a web app with naturally synchronous communication. In this post, I would like to introduce one way to do it using a few well-known libraries and tools to use while designing their systems. The example below is written in Java but I believe it's more about the basic principles and the same app can be re-written into any language.

Tools and libraries needed:

20 Examples of Using Java’s CompletableFuture

This post revisits Java 8’s CompletionStage API and specifically its implementation in the standard Java library CompletableFuture. The API is explained by examples that illustrate the various behaviors, where each example focuses on one or two specific behaviors.

Since the CompletableFuture class implements the CompletionStage interface, we first need to understand the contract of that interface. It represents a stage of a certain computation which can be done either synchronously or asynchronously. You can think of it as just a single unit of a pipeline of computations that ultimately generates a final result of interest. This means that several CompletionStages can be chained together so that one stage’s completion triggers the execution of another stage, which in turn triggers another, and so on.

Applying Bulkheads and Backpressure Using MicroProfile (Video)

I’ve recorded a video on how to implement bulkheads and backpressure using MicroProfile Fault Tolerance. The idea behind bulkheads is to split applications into several execution units that isolate functionality. In enterprise Java applications, this typically means defining multiple thread pools.

Applying backpressure to clients results in either adding information about the current pressure on the system to the client so that they will react to it, or to explicitly deny the request with a temporary error response.

Asynchronous RDBMS Access With Spring Data R2DBC

Not too long ago, a reactive variant of the JDBC driver was released, known as R2DBC. It allows data to be streamed asynchronously to any endpoints that have subscribed to it. Using a reactive driver like R2DBC together with Spring WebFlux allows you to write a full application that handles the receiving and sending of data asynchronously. In this post, we will focus on the database and on connecting to the database and then finally saving and retrieving data. To do this, we will be using Spring Data. As with all Spring Data modules, it provides us with out-of-the-box configuration, decreasing the amount of boilerplate code that we need to write to get our application setup. On top of that, it provides a layer upon the database driver that makes doing the simple tasks easier and the more difficult tasks a little less painful.

For the content of this post, I am making use of a Postgres database. At the time of writing only Postgres, H2 and Microsoft SQL Server have their own implementations of R2DBC drivers.