TCP With Node

Introduction

Everyone who has any knowledge of the internet is well-aware of the Transport Control Protocol (TCP). Robert Kahn and Vinton Cerf invented TCP in the 1970s for satellite communications. In their work building a satellite communications network, they realized the need for a hardware-independent mechanism to move data from one computer to another. All prior work – remember, there was no internet – used systems highly customized for specific hardware implementations. They thought that a more general mechanism would make it easier to add connectivity to new systems.

They built TCP as a general communication mechanism that could easily be adapted to different computer systems. They architected it as a connection-oriented protocol that managed connections between client devices (initiators) and servers (responders).

API Concerns

To gather insights on the current and future state of API management, we asked IT professionals from 18 companies to share their thoughts. We asked them, "Do you have any concerns regarding the current state of API management?" Here's what they told us:

Design-First

  • A lot of great tools have become available for building and deploying APIs. However, design-first has not become as widely adopted as one might hope. Many times, developers are asked to provide an API, and they do just that. The person making the request for an API may have a very narrow need, and the developer can easily throw something together to satisfy that need, but then later additional users access the API and it comes up short in some way. 
  • One issue that might arise is the slow down of the applications. In some cases like creating an account, it's not unusual to have to communicate to multiple APIs to notify the action. For instance, sending a request to the analytics API, another to the payment processor to add the new user, one to add this new user to the chat & support service, and finally, one to send the welcome email.

    If these requests are made when creating the account, this can sum up to initial requests that can quickly take a few seconds to complete and might cause issues down the road. (What if the first two requests were successful, but not the last one? How can we retry them? Which one to prioritize?). That is why having some asynchronous call to external APIs implemented whenever possible, with retry mechanisms and alerting system in place a great way to mitigate these issues. 
  • The primary concern is that the market is ripe for consolidation. Solutions that are only 5-10 years old already have a “legacy” architecture that can’t keep pace with new architectural patterns like microservices. Customers need to be careful they don’t build their API capabilities around a solution that will need to be ripped out and replaced in just a few years.

Version Management

  • Dealing with the API contracts and open-API specification that tells you what an API can do. Version management or version changes to the APIs is still a challenge. Ways of communicating grow exponentially not a great way to handle. Moving so fast with features consumers have to support multiple versions at once.

Lack of Best Practices

  • A lot of progress has been made in the last few years. The big cloud providers are doing more. There's still a big learning curve for new developers. There are a lot of APIs out there that have not used best practices. For the development of new systems, the integration of APIs is still a big concern. 
  • So much of API management is still externally focused when there’s tremendous value in internal use cases. Focus on building a program internally that focuses on your internal developers. Give internal developers the information they need to take advantage of the services and get out of the way. Help your developers get their job done. If you create that portal for your internal audience where everything is well-documented and you’re guiding them down the path to get stuff done, that has a long-term benefit for internal teams and creates an innovation strategy with less fear of failure.
  • Too much of customer work is still required to put a good API product together, the vendors’ layout might be complicated and unintuitive. Lots of terms and product categories are fairly new and non-standard. As publishers switch their focus from a need to publish and focus more on the developer experience of the consumer, things will improve.
  • The lack of well-documented standards, especially when it comes to API versioning.

Other

  • How do you strike the balance between flexibility and optimization?  The public cloud is a good place to get started with small amounts of data but with large amounts of data, it starts to get complicated. We’re working to solve the throughput model of current APIs. Work is still needed on that front. With a microservices architecture, it gets more complicated as one service depends on many others. Tools like Istio and LinkerD become more important for circuit breaks and timeouts otherwise it just becomes a complete mess.
  • API management is a somewhat mature area. Key functionality blocks are pretty clear – gateway, portal, analytics, monetization capability, the key-value today is how effectively these pieces integrate together to deploy and get value from the API infrastructure. 
  • The domain is striving: complementary tools, techniques, practices, actors appear regularly - while existing ones are still being improved. The days when it was unclear whether selling an API could be a good idea, are long gone.
  • I think that there is often too much of a focus on quickly "pushing" APIs out and getting them online, without a deliberate effort to create a solid API-first culture. Runtime considerations such as security are certainly important, but everything that occurs before an API is deployed and running in production is equally important and often overlooked. 
  • Current APIM tools are heavily focused more on the provider and the consumer is an afterthought. We want to empower the API developers to know how many requests have been made by consumers in a given timeframe. We also want consumers to be able to handle situations such as an outage, API changes or retirement, etc.

    There usually is not a clear method for developers to communicate this information to the API consumers in a timely manner and the latter is oftentimes left in the dark.  Exposing this information is becoming more vital in today’s API economy. In addition, the lack of testing of an API (Contract, Automated, etc.) is something that will also need to be addressed. Implementing proper testing will ensure less likelihood of changes in the API breaking integrations, reliability and an ability to test at scale. 
  • API management is not something new. The old-style gateway is out there. It’s hard to manage. The trend happening in the market you see next-generation API gateway much lighter and endpoint as a proxy, open-source to see what’s going on, familiar with the microservices era. Using K8s backup by default. People are misrepresenting what their platform will do – suggesting they can facilitate microservices while they’re still running legacy, monolithic applications. 
  • We’re about a decade into the microservices era, but there’s still a paucity of tooling and practices to support companies who are delivering microservice APIs directly to customers.  At a standards level, the only option the OAuth2 spec provides is Scopes, and these are woefully inadequate for microservices APIs.  In anything beyond a trivial deployment, you need to reason about the resources a caller can access.

    The lack of standards around this makes authorization and access control a problem without a common solution, so everyone is left to roll their own or code to a vendor’s design. Given the asymmetry of changing API interfaces (and your access model is an absolutely critical part of your interface), we’ll be stuck with these proprietary implementations for years to come. 
  • Many API cost estimators are available, and few people understand the long-term costs they are imposing on themselves and their customers through their APIs. A conservative estimate is $25K to build an API, and $15K for your customer to integrate to it. Plus 50% annually to maintain it. This means an app with 100 APIs being consumed by 100 of your consumers is generating about $15,000,000 of cost mostly to your customers.

Here's who shared their insights:

How to Secure APIs

To gather insights on the current and future state of API management, we asked IT professionals from 18 companies to share their thoughts. We asked them, "What techniques and tools are most effective for securing APIs?" Here's what they told us:

Authentication

  • We are frequently providing API access to known B2B partners. In these kinds of situations, the ability to limit API access to partner IP addresses only (whitelisting) is very powerful. You still need to authenticate and rate limit, but you have cut traffic to only known partners. This eliminates a great deal of the nefarious traffic that is seen on APIs open to the broader Internet such as brute force attempts to gain access and denial of service attacks.

    Even with IP whitelisting in place, having an API gateway in place is still best practice. This aids in authentication and ensures the backend is only receiving properly formed API calls. 
  • The most common is OAuth and OAuth2 for communicating and securing communications between APIs. Underneath is token-based and claims-based authentication where the APIs are passing digitally signed tokens back and forth to verify the token is representative of who is making the call. 
  • Authentication is unevenly implemented by providers. Even with OAuth, the way tokens are persisted by others can be a problem. How is the lifecycle of tokens managed? Do the tokens get refreshed? Within our own infrastructure, we use one-time tokens heavily scoped to the type of operation we’re trying to do. It comes down to secure token management and certification-based authentication. 
  • Always authenticate APIs before Authorizing — There are many ways to do API authentication, but multi-factor authentication is the commonly recommended approach. For APIs, it is pretty common to get access token using an external process e.g. through OAuth protocol. Authentication keys are most sensitive and must be kept secure; however, it is recommended that a management store be used to automate the whole process.

    That said, authentication alone is not enough to grant access to an API, there should be an authorization step to determine what resources have access to the API. There various ways to check for proper authorization include content-based access control (CBAC), role-based access control (RBAC) or policy-based access control (PBAC) — these methods ensure business data remains fully protected against unapproved access. 

Rate Limit

  • Securing the API environment involves every API touchpoint — authenticating and authorizing API clients (third‑party applications and developers or microservices), rate-limiting API calls to mitigate distributed denial-of-service (DDoS) attacks and protecting the backend applications that process the API calls.

    Some techniques and tools for securing APIs are:

    1) Using JSON Web Tokens (JWTs) in authenticating and authorizing API clients — these tokens include information about the client such as administrative privilege or expiration date. When the client presents a JWT with its API request, the API gateway validates the JWT and verifies that the claims in it match the access policy you have set for the resource the client is requesting.

    2) Defining and implementing access control policies that allow only certain kinds of clients to perform write operations or access sensitive data such as pricing.

    3) Defining Role-based access control that allows only certain users (such as developers within a specific org) to publish APIs that expose sensitive information such as pricing or inventory levels.

    4) Securing the APIs themselves by applying a rate-limiting policy that sets a threshold on the number of requests the API gateway accepts each second (or other time period) from a specified source, such as a client IP address.

    5) Protecting the backend applications using HTTPS — HTTPS protocol should be used traffic between API gateway and the backend systems that process the API requests. 
  • Circuit Breakers - Throttling and Quotas — A good practice is to enforce per-app data usage quota so that the backend won’t be impacted in case of DoS, DDoS attacks or to prevent improper use of the API by unauthorized users. Throttling and quotas per resource not only act as a circuit breaker, but they can also prevent a negative impact on the downstream systems during an unwanted surge in usage. A sophisticated API management platform with policies like quotas and throttling can offer this capability. 

Broader

  • Three key areas in our approach to API security:

    1) Have a prescriptive approach

    2) Think carefully about how application identities are connected to user identities

    3) Think about API security in its broadest sense beyond authentication to mitigate intrusion attempts.

    Prescriptive model: Customers move towards OAuth 2 and overlay with Open ID Connect. There are a lot of choices to be made with OAuth 2, Open ID restricts the choice and directs people on best practices. Ultimately, it’s better to implement a clearly understood standard. Look at a balance between security, usability, and scalability similar to the world of storage.

    Roles of application and user identities: There is a number of API management vendors try to do both. It just isn’t scalable. Identity management is so robust you cannot keep up with the pace of change and new standard. We integrate with external identity management solutions and have full out of the box integration or OAuth2 and Open ID use cases. We have open interfaces for customers to swap us out for their own implementation if they want.

    For broader security concerns: We take the approach of distributing implementation of security. By default, API management is focused on providing an API gateway. We take an approach that an API gateway should be focused on authentication and authorization of traffic. We recommend taking a multi-layered approach and include a web app firewall in a separate layer with Apache Mod Security. When deploying additional security to deploy to the network edge. The last is using heuristics or machine learning to monitor traffic behavior and determine if traffic is malicious.

Other

  • API management security is still one of the areas where people are struggling. We need to recognize that APIs are inherently insecure. We need to relearn what is security for APIs and understand the internet fragility that is being attacked. Cloud providers are more mature. AWS and Azure API management tools provide an additional security layer. Think about the security of the user name, password, keys, and know who is using your APIs. Think about the data now that APIs are exposing legacy systems have the same mindset for securing the data that is there. More APIs consume more user data — how and where do you store? 
  • APIs can be a security risk in two key ways. First, as dependencies are like a set of Russian dolls. Inside your dependencies are the dependencies’ dependencies… and inside them your dependencies, dependencies’, dependencies’ and so on and so on. From a functionality perspective, this enables each development organization to focus on the code for which they add the most value.

    However, from a security perspective, as the end consumer, you inherit the entire chain of vulnerabilities. Application security professions must go beyond basic software composition analysis, which matches known CVEs to version strings because they only go one layer deep. As an example, Jackson-databind is a popular open-source library that is leveraged in many 3rd party communications SDKs. 

    The “jacksploit” deserialization vulnerability, that has plagued many version of the library, was exploitable for some of the SDK users, but the vulnerable version of Jackson-databind was obfuscated and not identifiable by traditional SCA. Hence, security professionals must look for tools that evaluate how the source code works, rather than relying on looking up CVE databases.

    Second, since APIs often pass data externally, it is important to map critical data flows to ensure critical data isn’t inadvertently leaked into them. This means identifying which variables are critical and mapping their journey from source to sink and all the transformations on the way. That way unencrypted or unredacted critical data can be caught in development before leaky routes are pushed into production. 
  • You need to have good API management platform in place. It will enforce the rules and roles you put in place to control who has access to what data. You can use a gateway or platform with options or use built-in authentication and authorization tools. Being able to control access in a friendly, easy-to-configure way. The web interface is the best way to do and control who has access without requiring a deployment or digging into the server and update complex configuration or files. Control at the gateway level and then it's easy to configure or manage. 
  • API security is important and requires a comprehensive approach to prevent data leakage. Because of the way data can be shared via API, users can inadvertently leak data in just a few clicks This occurs when employees authorize OAuth API connectivity between sanctioned apps and unsanctioned, third-party apps, granting the latter permissions to read and even write data (e.g. Email, Google Drive, Box, Office 365, and SharePoint). This functionality provides great business value but can expose data when not properly secured.

    Structured data can be synced between apps via JSON, XML, or file-based APIs (to name a few). This structured data may contain sensitive information that should be protected with encryption or tokenization. In these scenarios, it’s critical to leverage a bring-your-own-key (BYOK) approach; this is because data is not truly secure if an encryption key is stored with the app that houses the encrypted data. Fortunately, leading multi-mode cloud access security brokers (CASBs) provide visibility and real-time protections for all of these use cases, ensuring data is secure on any app, any device, anywhere. 
  • When it comes to securing APIs, serving them via HTTPS is a basic yet absolute requirement. It’s relatively straightforward to setup: the tools and processes involved are widely available and well understood. Moving away from the common username/password couple is also a practice we would recommend. Usernames are an easy target for social engineering. Passwords, while uses are evolving, are still too often created manually — constraining their size or composition leads to frustrating experiences. We prefer to go for API keys instead.

    Our API keys have the form we devised. We can assign several API keys to the same customer. A single API key can be exchanged to several entities for a given customer. We can revoke API keys. All of this can be done independently from actual customer account management. 
  • Enforce Encryption all the way — A fundamental security requirement is to encrypt the data using the latest cryptographic protocols to ensure proper protection while the data is in transit. Weak protocol version and ciphers should always be disabled. TLS1.0 and 1.1. Encryption in transit is great, but not enough. It should be enforced all the way, including while the data is at rest. 
  • Audit and Logging — Auditing is usually an afterthought, whereas it should be considered as a feature during the API development cycle. The more logging the better, since the cost of having this capability far outweighs the cost of not having them when you need this capability the most (during the attack). Logging should be systematic and independent, and resistant to log injection attacks. Auditing should be used for detecting and proactively deterring attacks.
  • Alert and Monitoring — Alert and monitoring are key to see what is going on in your world to protect against bad behaviors. Logging and auditing should be used proactively in detecting and alerting you against threats and attack.
  • Protect HTTP Verbs and Cross-Site Request Forgery (CSRF) — It is common with APIs to allow multiple methods for a URL for different operations. For example, a GET request to read data, PUT to update data, POST to create data, and DELETE to delete data. It is important for the APIs to restrict the allowed HTTP verbs so that only the allowed methods will work, while the others will result in an exception along with a proper response code sent back to the client. It is important for PUT, POST, and DELETE methods to be protected from Cross-Site Request Forgery (CSRF). Typically one would use a token-based approach to enforce this capability.
  • There's an external way and an internal way. External SSL or TLS for encryption, make sure traffic is secure and encrypted. Make sure the client and server are both SSL. Make sure you are encrypting data. Know who the person is that’s calling the application – OAuth, OpenID connect. The last one is securely based on who you are. External is a web application firewall to scan the request and make sure nothing is suspicious. Internally see other stuff like after already past the gateway two microservices talking to each other with MTLS. Limit the audience of the job. Only give permission to go to a certain place. Identity management.
  • Because we’ve taken microservices to their logical extreme, with each team producing APIs on the public internet for direct consumption, we don’t have the benefit of a gateway as a chokepoint for access.  In our world, there’s no such thing as a “backend API.”  That makes securing our APIs an especially diffuse problem, but it also means no one in our company relies on a false sense of security built on assumptions that only the “right” callers can reach their APIs.

    As a result, every team is responsible for the security of their APIs, and we achieve this by following a few key principles:

    1) Every caller (human or programmatic) authenticates using the same standard (OAuth2 bearer tokens in the form of JWTs signed by our identity vendor, Auth0)

    2) Every API validates the identity and access rights of the caller for each action

    3) Every API presents the original caller’s credentials for subsequent requests it must make to our APIs

    In this way, every API knows exactly who initiated a call chain and can accurately reason about the access rights of the caller. These techniques have served us well as every developer has the same expectations and responsibilities for securing access. We haven’t found great tooling to automate security testing.  We have found that human ingenuity and vigilance are always necessary.

    We use crowdsourced testing services for an external perspective but also empower internal teams to red flag, up to the highest levels of the organization, any security issues they identify in anyone’s APIs.  The latter works particularly well because, again, we’re all responsible for the security of APIs exposed on the public internet, so every developer has a well-honed sense for good and bad practices.
  • For RESTful APIs — OAuth2 and JWT are pretty much de-facto standard. We use both. OAuth from customer to our cloud-platform and JWT internally for performance benefits. With the growing popularity of Webhooks, it is a must to sign your payloads when customers “sign-up” for API notifications.

    We use HMAC signatures so customers can confirm data is coming from us and hasn’t been altered. I’m surprised how frequently we encounter API providers who are still not doing that. We use more advanced measures on some occasions, but those are sufficient for most cases.

    You also have to have some sort of single API entry point before API calls hit your microservices or monolith applications. Normally this is an API gateway that will block any calls very fast if they haven’t been properly authenticated.
  • Most security issues result from too much complexity and too many APIs. Simplicity is key, with all data security being handled centrally so there is one place to keep it current. Protocols like GraphQL and SPARQL allow consumers to describe their own data needs through a single API, substantially minimizing the need for so many APIs to manage to begin with.
  • Restrict the list of users that are authorized to access the token. Token-based authorization is there for a reason. Second, there’s a need to track and control the use of resources allocated for a single user, which is absolutely essential to protect from DDoS attacks.

Here's who shared their insights: