Artur PetrovBackend EngineerTravis TurnerTech EditorWelcome, digital explorer, to the vast terrain of microservices. Whether you’re navigating these challenging lands on your own, or comfortably residing in the serenity of a monolithic architecture, an Http client is an essential companion for every developer. But it’s not just about having one; it’s about equipping yourself with the right (or I.D.E.A.L) one. In this guide, you’ll learn essential techniques to ensure your trusty HTTP client is configured to avoid potential pitfalls. Plus, you’ll get insights about the advantages of separating a client code layer from the application.Let’s be honest—the modern web application is a complex beast. The days where it was possible to craft everything by yourself have long since faded into the past. Current applications are mostly a collection of services communicating further with each other to accomplish a task.And within this world, the HTTP client is the most important tool in your arsenal.This guide is about understanding the best practices for any HTTP client and how to leverage them to your advantage. This guide is not limited to backend applications (although most of the examples are in Ruby), the same principles are applicable for any language and platform, even the frontend.Let’s discuss the format of this post (a guide to a guide, so you know we mean business, here.)First, the mandatory pathway will guide you through the essentials of stability, debugging, documentation, and clarity. This foundational knowledge will ensure you’re always prepared for what lies ahead. Although the mandatory pathway is mandatory, it’s worth noting that we have no real means to enforce this policy.Next, the opinionated route will show the art of constructing a robust HTTP client. You’ll dive into the nuances of the abstraction layer, understand the importance of domain modeling, and arm yourself with advanced testing techniques that are not just about detecting issues, but preempting them.After reading this guide, you’ll be equipped with a powerful and properly configured HTTP client that will serve you well in even the most challenging environments. (Even underwater levels.) And perhaps, most importantly, you’ll more fully understand the value of assigning acronyms to pretty much any organization, process, or concept.There’s also a full opinionated example of a client that you can use as a starting point for your own.Let’s get started on our I.D.E.A.L. HTTP client journey!HTTP communications are unstable by nature. The network is always unreliable, and services are unpredictable. When venturing into the unknown, the first thing you need is some sense of stability.Bad things happen so you need to be prepared for the worst.Timeouts are the most important aspect of stability; you need to define a limit to how long you’ll wait for a response. Don’t let your services just hang indefinitely.Timeouts should be set to a reasonably low duration—several seconds is often sufficient. If you’re using Ruby, the default timeouts (60 seconds) are usually too high for most requests. (You might wonder why these default timeouts are set so high. The reason is compatibility; high timeouts help avoid errors in older code.)By the way, in JavaScript, the default settings normally involve no timeouts. Go’s default HTTP client doesn’t have timeouts either. Sounds like a pretty common pitfall, huh?Now, wait a second, timeout! You may be asking—if high defaults are meant to prevent errors, shouldn’t that mean it’s beneficial for my application? The answer is no. Failing to set appropriate timeouts can result in numerous requests from a borked service hanging and blocking your entire application, instead of allowing it to degrade gracefully while missing some functionality.Moreover, slow responses negatively impact your customers. If the UI is blocked with long requests, you’re doing your customers a cruel injustice and robbing them the chance to retry.This is particularly important to consider because data packets can sometimes be lost during transmission due to bad network conditions. (And there won’t be a delivery person you can yell at to make you feel better about it.) Moreover, while users might tolerate a few seconds of delay, if they find themselves waiting for minutes, they’re likely to leave your application.And here’s the most important consideration: you might not even notice the presence of slow requests might go unnoticed. It’s much harder to identify sluggish performance in your application when there are no clear complaints coming in. And customers, for whatever reason, are often reluctant to clearly communicate issues.So go with the motto of Erlang (or Elixir if you’re feeling young)—fail fast and loud, at least you’ll know something’s wrong.First of all, you need to set timeouts for all external requests:Broadly speaking, timeouts are tremendously important for any resilient application so you should set them wherever possible—be it the database, cache, or within other services.For all other HTTP-clients check out Ankane’s ultimate timeout guide with rich insights on ensuring stability.As we already know, errors are inevitable in HTTP communications, but it’s how you deal with them that will really determine your resilience in harsh environments. The most important thing is always expecting and handling HTTP errors gracefully. If you don’t, they’ll bubble up and crash your application. And, in most cases, you don’t want this:But please don’t overdo it. Remember: excessive error handling can mask the real problems in your application.Also, keep this in mind about error reporting: you don’t want to flood your error reporting system with the same errors for some small network disruption. This is because you need to be able to group them by the error type, so rather than using generic StandardErrors, you should always use custom error classes.Of course, you do need to be able to track the errors in your application. So get with the times and find a proper error reporting system like Sentry, AppSignal, Errbit, etc.This is a quick little trick to craft a consistent error hierarchy:This will organize your errors by domain and help handle errors in a more concise way:Since networks are unreliable, and there’s no way to avoid this, you should attempt to retry request to simplify things when recovering from sneaky flapped errors:Retries are a powerful tool, but you need to be careful with them; they’re useful for some types of errors but not for all of them.Generally, you can safely retry idempotent requests (GET, HEAD, PUT, DELETE, OPTIONS, TRACE), but you should never retry non-idempotent requests (POST, PATCH).Retrying non-idempotent requests can result in numerous duplicate entries in the system. One way to guard against this is by using a unique, one-time ID for every request and checking for duplicates before retrying. However, not all upstream services support this method. Additionally, be cautious of misbehaving backends where supposedly idempotent endpoints may produce erroneous side effects.Connection timeout errors are generally safe to retry, but you need to be careful with read and write timeouts, as requests may still be completed after your client has disconnected.Also, be mindful of the potential for further service disruptions. If you retry too often and too frequently, you risk generating a large volume of duplicate requests, and this may hinder service recovery. At the very least, you should limit the number of retries and the time between them.For example, you may retry several times with exponential backoff:Exponential backoff is an effective strategy to avoid overloading the downstream service with retries. Distributing your retries randomly over time should suffice.In computer science, this issue is commonly known as the “thundering herd problem”. That’s a pretty cool name for a problem, but the result is not so cool. This problem occurs when multiple processes, all awaiting a specific event, are simultaneously triggered once that event takes place, leading them to compete for the same resource. You can use a generic exponential backoff algorithm for your purposes such as Retriable or client-specific plugin implementations. A more advanced solution to this problem is to implement a circuit breaker pattern to prevent overloading the service with retries.Retries and circuit breaker can also be implemented on the gateway level (of course, if you’re interested in investing in API gateways between your microservices like Istio or Kong).Using a circuit breaker, after several failed requests to a downstream service, the breaker “opens”, causing all new requests to fail instantly. After a set timeout, it enters a “half-open” state, allowing limited test requests. If these succeed, the breaker “closes”, indicating the service has recovered. If they fail, the breaker remains open, waiting for another timeout before retesting.For example, Circuitbox is a nice implementation of this pattern in Ruby. As it requires data storage, it’s a bit more complex, but it’s a good way to protect your application from cascading failures:And remember, repeated attempts without breaks or reconsideration might lead you further astray.As we all know, every adventurer faces challenges out in the world. Like for example, having a bunch of bugs in your sleeping bag. So at one time or another, you’ll be destined to debug some problems. (We made the bug thing work.)During your journeys, make it a habit to keep logs of external requests:One of the real challenges with logs is that you often don’t realize you needed them until you’re already faced with a problem that could’ve been diagnosed if they had been in place from the start.While it might seem like extra work up front, we strongly suggest implementing them early on, especially if your distributed workflow is complex and spans transactional boundaries across various services. Your future self will thank you. (Unless your future self has already been corrupted by evil forces, then, all bets are off. Still, take this advice.)With error reporting and logging established, let’s move on to external request metrics, an often overlooked aspect in monitoring. It’s essential to monitor the regular flow of client requests and measure normal response times. This enables you to identify issues early and adopt a proactive approach by understanding how requests perform over a sufficiently long period of time.All common APM tools (New Relic, Skylight, Datadog, and so on) provide a way to monitor external requests. If you’re not using any of them, you might try a self-hosted, battle-tested solution like Yabeda to monitor your application.Even though it may be more time consuming to implement, a custom solution can be beneficial since you can mix it with your internal business metrics and get a more holistic view of your system.Sometimes things can go wrong on our side, such as implementing a nasty retry loop. If you’re a good citizen, you’ll have to identify your HTTP client to others:It’s a responsible practice to identify your client with a custom user-agent. This will help identify your requests in logs and for tracking them in external services. Including a version number will also help to track the client’s version and to detect problems with older versions.You’ve already seen correlation IDs in contemporary applications; a famous request ID from the Rails world is a sort of correlation ID. This is a unique ID attached to every request that’s passed between services to track the request and its state.There are a lot of tracing tools providing a way to trace requests across multiple services; you may have heard about open-source Zipkin. Although setup is a bit complex, it may be a good investment in the long run with microservices.It’s also a nice idea to add a correlation ID to your HTTP client to track requests to external services in the logs and to correlate them within request and response pairs:A well-constructed HTTP client is akin to a comprehensive survival manual; it’s quick when first setting things up, and also easy to use in the long run.The configuration of the HTTP client is the most important part of the client’s presentation, and it should be clear and easy to change. The best way to achieve this is to use some sort of configuration DSL. We may be biased here, but we recommend Anyway Config:You may want to separate the configuration into several files. This is a good way to keep your configuration clean and directed to a single service. With this approach, you’ll always be able to tame a misbehaving service just by changing its configuration.The really great thing about Anyway Config is that you may override the configuration from the environment variables. And believe us, there will be a situation when the next big thing in the branch is still undeployed, and the team needs to increase the timeout of the external service to make it work. With Anyway Config, this is just a matter of setting an environment variable:One last minor thing to consider is the thread safety of the configuration. If you’re using an instance-based configuration approach, this isn’t a concern. However, if you opt for a global, preinitialized object for the HTTP client, you’ll need to ensure that the shared configuration isn’t altered between threads. This is a common pitfall, especially since there’s often a need to adjust the configuration for different endpoints. While Rails makes multithreading “mostly ignorable” in applications, it doesn’t eliminate the inherent risk of different code paths altering the same global object.By the way, the best configuration is no configuration. So, try to keep the defaults sane for end users.Much like a seasoned adventurer wouldn’t start a journey without a proper travel plan (good snacks are essential to keep up morale), you can’t afford to overlook the performance of an HTTP client over the long run.One of the most common performance pitfalls is overlooking memory exhaustion due to large file downloads or uploads.A standard mistake is to load an entire file into memory, which can lead to performance bottlenecks or even application crashes. Streaming large files instead of loading them into memory is nearly a required practice.You can use a ready-to-use solution like the Down gem, or if you prefer, you can build your own custom solution leveraging the streaming capabilities that most HTTP clients provide:This issue can be particularly relevant to consider, as otherwise, it may go unnoticed for years. It might not catch your attention until, let’s say, a customer decides to upload their 1GB kitten photo archive to your application. You definitely don’t want to be caught unprepared in this situation; that would not be very a-meow-sing!Moving on to performance optimization, the most important part of this process is meaningfulness. Don’t forget to fine-tune your HTTP client, at the very least, based on application-specific performance metrics. What works best will vary depending on the unique needs of each individual application. Remember that simple solutions rock because they have fewer moving parts, which means less potential for errors.There are many ways to improve performance that require collaboration with upstream services. You could leverage HTTP/2 for connection reuse or employ binary protocols to reduce payload sizes. This is outside the scope of this guide, but you should definitely consider these options.One popular technique to improve performance is to avoid the overhead of establishing a new connection for every request:Persistent connections eliminate the overhead of establishing a new connection for every request, which is especially useful if you’re making multiple requests to the same host, like in microservices architecture.The downside of persistent connections is that they accumulate state, which may lead to annoying problems, such as mysteriously half-broken connections or resource leaks; your monitoring system should be ready to detect these issues. Additionally, persistent connections aren’t always faster because of connection pooling dependance, so it’s advisable to benchmark your application to find the best approach.To add to the “less work, more performance” mantra, you can also consider HTTP caching. It’s a great way to reduce the number of requests to external services. This approach is useful for requests that are not changed often, but beware of the cache invalidation issues; this is believed to be one of the hardest problems to deal with, so be extra wary.Another perspective on performance concerns parallelism, which is an excellent way to boost your application’s performance. However, implementing this can introduce significant complexity into the code. You may find it easier to switch to a different programming paradigm, such as the one demonstrated in the next example using Typhoeus, or something more asynchronous like Async::HTTP:It’s not always possible to parallelize requests. For instance, if you’re using an external service with a strict rate limit, this performance optimization technique simply can’t be utilized.The last mandatory part of the HTTP client to consider is standardization. It doesn’t matter if we’re dealing with a fully-featured client library or just an one-method HTTP request, it’s important to have a standardized way to handle external requests in an application. This will help your teammates mitigate the persistent need to investigate quirks of the next “best HTTP client”.We do not want to recommend any particular tool here. Honestly, in a world where microservices in different languages are common, it’s just a bad idea to choose “the best”. As a matter of fact, it’s up to you to decide what’s best for your project. But, just be sure to choose one and stick to it to avoid integration spaghetti. (Also worth noting, spaghetti may not really be an appropriate snack for adventuring, either. Go for foods in bar form.)You may want to stop reading here and start your adventure with your properly configured HTTP client. The previous mandatory parts of this guide should be enough to get going. By following those recommendations, you’ll have a solid foundation for your minimal HTTP client.If you’re interested in different layers of abstractions, we recommend the in-depth book Layered Design for Ruby on Rails Applications.But for those willing to ride shotgun with us, we now present the opinionated part of this guide.As discussed earlier, thinking carefully about how you integrate between services is important; ideally, you want to standardize on a small number of types of integration. But the entire application consists of different layers of abstractions. So, it’s important to choose the right level of abstraction for your HTTP client, too.This part of the guide is opinionated because implementing a fully-featured HTTP client as a separate library comes with a trade off: time. (Just like taking the scenic route and carrying the goods to set up a beautiful picnic. It can be worth it, but there are considerations).Implementing a separate library can seem like the natural way of doing things since there are clear boundaries between the application and the external service, and it’s easier to reuse common HTTP clients while custom business logic is clearly separated. But if you’re working with a service that is not so popular, it could be difficult to justify a full-featured integration.However, consider at least a minimal library implementation. Many of the techniques covered in previous chapters are easier to implement if you have a full-featured HTTP client library. But mostly it’s all about communicating your intentions to the team after all. We’re introducing lots of artificial layers in our applications to make it more maintainable. So, why not do the same with the external services and encapsulate them in a separate library?For a minimal implementation loosely inspired by OpenAPI (Swagger) syntax, take a look at Evil::Client.For illustration purposes, we’ll use Faraday as a great example of a layered HTTP client library. It’s a good idea to use it as a reference point for your own HTTP client library. Faraday is like that multipurpose tool every adventurer wishes to have; its versatility can be credited to its modular design, and you can always swap out the components to suit your needs better:You can find a ready-to-use example with an opinionated Faraday configuration here.It’s possible to use Faraday as a communication adapter for various HTTP client libraries. These can be changed up based as your requirements dictate, ranging from the simple Net::HTTP to the more complex Async::HTTP::Faraday.For a full list of Faraday adapters, check out Faraday’s curated list.It’s also possible to use Faraday as a middleware stack. It’s similar to Rack middleware, but for external HTTP requests. There are ready-to-use plugins for a large variety of tasks: from easy things like JSON request and response parsing, to more advanced topics like automatic retries. There is even a directory of community plugins. It’s a great way to reuse existing code and avoid reinventing the wheel:But this doesn’t mean that you absolutely must use Faraday; you can choose any other library that suits your needs. The most important thing to take away is Faraday’s experience of constructing a layered HTTP client library. A client library does not have to be flat. You may separate the HTTP client into several reused layers: configuration, unified error handling and error hierarchy, retry, logging and monitoring facilities, API client, and domain models.Wait, what are the domain models?Strong preparation is the key to success, and real-world survivalists know this, too. Again, snacks are key here, but also it’s important to have a sound structure for your HTTP client in oder to make it easy to use and maintain.You can work with requests and responses from external services as with plain hashes, but this is error-prone and hard to maintain. This is the same as working with a database without any ORM. There’s a way to make them more reliable with typed domain models.The trick here is to use a well-defined structure to represent the data. It’s a good practice to use a separate class for it to isolate the entity and to make it more maintainable. This is also a nice place to define type and validation logic for the data.There are several different gems in the Ruby ecosystem to implement typed domain models. You may have already heard about the most popular ones like ActiveModel::Model, dry-struct, Hashie, BloodContracts, and so on. They all have their pros and cons, but the most important part is to choose one and stick to it.Do you feel adventurous? Try to experiment with Sorbet’s T::Struct.An example of a typed domain model might look as simple as this:Typed models make it easier to understand code by using named fields instead of plain hashes. They also establish a contract between the application and external services, helping to prevent subtle errors that might occur if something changes in an external service in production. While serving a similar purpose to documentation, typed models are generally more reliable and easier to maintain.The final section (which is by no means of less importance than the others) of our guide concerns testing.We also highly recommend using a contract testing approach to ensure that your HTTP client works as expected. You may utilize a complex solution like Pact, or just use VCR for these tests. Although there’s some opposition to VCR within the Ruby community because of the effort required to maintain cassette recordings, we believe it still isn’t a good idea to use mocks and stubs in HTTP client tests; it’s better to use VCR to record and replay real HTTP interactions, so you’re not caught off-guard when the actual scenario plays out.Enlighten yourself with a radically different approach to testing by flipping the testing process upside down:One really neat trick to minimize re-recording struggles is to include preparation and cleanup phases for external services directly within the test files. While it may sound unconventional, it’s still common practice to manually prepare test data in these tests—things like test users in the auth microservice, required data in third-party providers, and so on. Why not automate this process by incorporating it into the test itself (the phases run ONLY when recording cassette):With this approach, you’ll always can re-record the cassette without any additional efforts. Delete the cassette, run the test, and it’ll be re-recorded with the automatically prepared data. Simple as that.This straightforward, common-sense approach can save huge amount of time and effort while re-recording cassettes. It’s particularly useful if you’re working with a large number of external services, as is common in a microservices architecture. The approach also makes tests more readable.In the challenging world of services, the HTTP client is an indispensable tool. Just make sure to properly configure it to make it work—and to work well:Just as every adventurer respects their gear, it’s time for us to give some love to our trusty HTTP clients. Good luck on your journey, explorer!At Evil Martians, we transform growth-stage startups into unicorns, build developer tools, and create open source products. If you’re ready to engage warp drive, give us a shout!Get all the new posts delivered directly to your inbox. Unsubscribe anytime.We’d love to hear from you! We’re not really all that evil, and we love discussing potential projects, intriguing ideas, and new opportunities. Complete the form below or drop us a line at [email protected]. Alternatively, schedule a Calendly appointment with us right now!A product development consultancy that works with startups and established businesses, while also creating open source-based products and servicesUnited States+1 888 400 548577 Sands St.Brooklyn, New York11201Portugal+351 308 808 570Rua Alexandre Oneill, 38,Porto4400-008Japan+81 6 6225 12429F Edobori Center Building, 2-1-1 Edobori, Nishi‑ku,Osaka550-0002Get all the new posts delivered directly to your inbox. Unsubscribe anytime.Designed and developed by Evil Martians