Test, Never Trust: Dealing with external services when using Elixir and Phoenix
In his short story Thin Cities 3, author Italo Calvino describes a city reduced to its plumbing — a network of pipes, stripped of the streets, walls, and floors that would ordinarily conceal them. I like to picture this “thin city” when I’m testing software; diving beneath the superficial layers to probe the essential connections that keep information and experiences flowing.
A “thin city” pipe network
The pipe analogy is especially apt when a project requires connections with third-party services, such as payment processors, mapping and navigation apps, support software, and cloud storage solutions. These connections are often essential to a product’s core functionality, so if you’re committed to a test-driven development culture, it’s critical to take third-party services into account in your test plan.
But how does this actually work in practice? The specifics will vary, of course, but looking at others’ approaches is still useful. In this post, I’ll explain how we’ve addressed third-party service testing on one recent project.
Testing tools
We currently use two main tools for third-party service testing. The first of these is Sage, an Elixir implementation of the saga pattern. The Sage library allows you to easily validate each step of a complex process, and to specify rules for handling subsequent steps if one step fails.
For example, say you’re building a travel booking service. Using Sage, you could implement a rule that responds to failure of a flight booking by automatically cancelling and refunding other parts of the trip:
Sage makes the code for this process robust, clean, and readable; you define the saga step-by-step, specifying a key to store the value that results from each step, a function to execute the step, and a function to handle an error. This structure is very reliable — you can easily handle errors and see exactly where things went wrong.
The second tool we rely on is mocks. Elixir creator José Valim recommends this approach to test third party code. Mocks work like this: let’s say we have a “pipe” module that connects our app with the Stripe payment service API. We want to test the connection, but we don’t actually want to hit the API and make real payments. So, we make a mock pipe module that serves as a stand-in for the real pipe in our development and testing environments.
The mock is useful not only because it allows us to test complex modules without calling the real API, but also because it’s highly configurable, i.e., we can force it to send any result (including a bad result) to confirm that the error handling we set up with Sage is working as expected. For example, if the Stripe connection fails, we can verify that our error handling opened a support ticket (nobody wants things to fail, but when they do, we want to show the user that we care!).
We can also use the mocks as “fishtanks” to store and retrieve information we need to verify. We accomplish this using Elixir’s Agents — a type of module that stores its own state, and is later retrievable. So, instead of just trusting that we are sending the correct information to the third-party service, we can store the information that would be sent to the real API, and confirm in the test that it’s retained correctly in the pipeline.
Use of a “fishtank” mock in place of an API
The bottom line — flaky third-party services don’t have to kill your platform or ruin your user experience. Anticipate errors, and know that there are tools and processes to help you handle them in the best possible way.
