Blog

This post shows how to get started with component tests (a.k.a. integration tests) for your API with Mountebank test doubles.
Starting with basic mocks, the post then shows samples on how to test functionality which cannot be covered with end-to-end tests.

If you need more information on why to implement other tests than just unit and end-to-end tests for a single API, Martin Fowler’s article on Testing Strategies in a Microservice is a very good introduction.

For more information on improving mocks and best practices check my post Component Tests for Distributed Systems.

All source code and a working example is available in https://github.com/AngelaE/service-tests/.

System Under Test

Let’s imagine we have started work on a nice new and shiny Book API which can serve a book catalogue. It has a data store and gets sales numbers from a public API.

Our team already has implemented some end-to-end tests, but we are hitting the boundaries of useful and efficient end-to-end tests:

1. We cannot manipulate data in the book stats API to test certain scenarios (error responses, very big sales stats, …)
2. The e2e tests are not stable, because the Stats API occasionally is not 100% available. To improve this behaviour we add a retry-strategy with Polly. The e2e tests seem more stable, but we cannot test the exact retry behaviour. Unit tests do not help, because Polly and we only want to test the configuration.

Pre-Requisites

We need to make it as simple as possible to write tests, otherwise extending the test automation will be a burden on the team. The following strategies make it easier to keep up with an agile service:

1. Generate the API client. Here is a sample for autorest, but any client generator works.
2. Generate response models for all downstream APIs. This means the mocked responses are more reliable. There are likely better choices than autorest in this case, but in the sample repo I have used it because it was easy.
3. Deploy the API in the most simple way possible. For example use in-memory implementations for data stores, caches etc.

Basic Test Scenarios with Mountebank

A very simple end-to-end test could look as follows, see full code here.

const bookApi = new BookApiClient({baseUri: `http://localhost:5000`});
it('returns a book with stats if stats exist', async () => {
  const bookWithStats = await bookApi.books.get(1);
  expect(bookWithStats.copiesSold).to.equal(21453);
})

The test code will not change if we write the same test as component test, but we need to set up mock responses from the Stats API. This sample uses Mountebank to mock the Stats API, and the client package @anev/ts-mountebank to create the mocks.

Step 1: Create a Mountebank Instance

The instance points to a running mountebank and uses the admin port 2525 to create and delete mock APIs. By default the instance points to http://localhost, but any URL can be passed in.

const mb = new Mountebank();

Step 2: Create a Mock Service with Responses

Mock services are called ‘Imposter’ in Mountebank. Each ‘imposter’ listens on a port. When a request comes in, the imposter checks whether it can find a matching ‘Stub’. A stub contains one or more responses it will return, and under which conditions.

In the sample below, the imposter will return the copies sold for all GET requests to /Stats/1. In all other cases it returns a 404 response.

let imposter = new Imposter().withPort(5011)
.withStub(new Stub()
    .withPredicate(new EqualPredicate()
      .withMethod(HttpMethod.GET)
      .withPath(`/Stats/1`))
    .withResponse(new Response()
      .withStatusCode(200)
      .withJSONBody({ bookId: 1, copiesSold: 2405 }))
)
.withStub(new Stub().withResponse(new NotFoundResponse()));

The imposter creation above is very verbose and will not scale very well. By using the builder pattern and creating an imposter builder for the Stats API simplifies the setup and will prevent bugs in the test code, see the v2 mountebank test setup.

let imposter = new StatsImposterBuilder('book.getv2', config.getStatsApiPort())
  .withBookStats(1, {bookId: 1, copiesSold: 2405})
  .withBookStatsError(1, {status: 404, title: 'Not Found'})
  .create();

In more complex scenarios, one imposter may contain stubs for different methods and paths. When an API makes requests to different downstream APIs, the test setup needs to create multiple imposters on different ports.

Step 3: Post the Imposter to Mountebank

Posting the imposter to Mountebank will delete any existing imposter on the defined port

await mb.createImposter(imposter);

The imposter is now visible in the Mountebank UI, check the sample code for the full test file.

Note:
If an imposter cannot match a request with a stub, it will return an empty 200 response by default.

Advanced Test Scenarios with Mountebank

In the basic test scenarios above we are testing the service response, but have no insight about what is happening inside. In a lot of cases it will be possible to test the behaviour manually or with end-to-end tests. However, there are important behaviours which are impossible or at the very least difficult to test that way. Samples are:

1. Resilience: How is the API handling problems with downstream services like transient errors, long response times or timeouts.
2. Caching: Is the cache working? Does it time out as expected?

Even though the API is a ‘black box’ from the perspective of the tests, we can verify the above scenarios by setting up the responses in the mock service.

Sample 1: Resilience of Downstream API Requests

Stubs in mountebank can return different responses. To check whether the book API can handle transient errors and retries, we set up an imposter which returns a 500 response for the first call, and then a valid response. (We may also want to test that 4xx responses do not cause a retry.)

it('Book API retries getting stats on transient error and succeeds', async () => {
    let imposter = new StatsImposterBuilder(5011)
    .withBookStatsResponses(1, [
        new RequestError('Internal Server Error', 500), // first response is an error
        {bookId: 1, copiesSold: 555} // second response is successful with book stats
    ])
    .create();
    await mb.createImposter(imposter);
    
    const book = await bookApi.books.get(1);  // Act
    expect(book.copiesSold).to.equal(555);
})

The test action/verify part looks exactly as for the e2e or basic test scenarios. From the outside it is impossible to tell that the service had to retry the request, apart from the test taking longer:

Sample 2: Cache and Cache Timeout

The cache and cache timeout can be tested in a similar way. A sample test flow would be:

1. Create an imposter with a successful response
2. Query the API and check that the copiesSold property contains the expected value.
3. Create an imposter with a 4xx response.
4. Query the book API – the copiesSold property should still have a value.
5. Wait for the cache to timeout or cause an even which would invalidate the cache.
6. Check that book API now returns the book without the copiesSold property.

Just testing the cache, or testing cache invalidation through an event will be fast. To test the cache timeout, the timeout needs to be very small, a few seconds at the most. Unfortunately, even then timeout tests will be slow and it may be a good idea to separate them from the fast tests.

Limitations of External API Tests

External tests as in the samples above work well for testing the deployed service and interactions with downstream services. However, some test cases are difficult to implement since the API is a ‘black box’ for the tests. For the following scenarios in-process tests will likely be a better fit:

1. Testing different configurations of a service when the tests cannot change the configuration on the fly.
2. Logs (i.e. to make sure they contain no sensitive information. Checking logs in too much detail can make tests very brittle.)
3. Anything happening internally which otherwise cannot is difficult to test. For example that data is encrypted in a data store, that sensitive data is not stored in the database, and more.

It is well worth considering in-process component in addition to external API tests. Check out my posts around this topic: https://angela-evans.com/tag/in-process-tests/

Repository with the sample code: https://github.com/AngelaE/service-tests

For best practices around component / API tests check my post about Strategies for Successful Component Tests.