Load Testing Apollo Federation GraphQL APIs

In this post, we will discuss strategies for load testing federated Apollo Federation GraphQL APIs, special considerations, and how to test these APIs using Multiple.

What is Apollo Federation?

Apollo Federation allows developers to declaratively compose GraphQL APIs (the subgraphs) into a unified graph (the supergraph) through their router. This enables teams to own their individual services with independent GraphQL APIs, while still providing a unified GraphQL API to clients. The diagram above illustrates a simple event ticketing Apollo Federation API with 3 subgraphs and services.

Special Considerations for Load Testing Apollo Federation APIs

• Apollo Federation queries can request resources from many services, with dependencies between them. Load testing can help identify potential bottlenecks and points of failure. We recommend turning federated tracing on during your load tests to monitor individual subgraph performance.
• Responses in GraphQL can partially fail but still return a 200 status code. Your load tests should account for this by checking for errors on the response object.
• Building microservices necessitates the separation of concerns - disparate teams need to come together to coordinate on load testing. This can make load testing both technically and organizationally challenging. Later in the post, we’ll discuss some ways to address this.

How to Design Apollo Federation Load Tests

• Identify workflows, query patterns, and mutations that reflect real-world usage. You can find this information in Apollo Studio. Use these queries to design realistic and relevant test query traffic.
• Incorporate security and authorization mechanisms in your load test scenarios. Authentication and authorization can add significant overhead, especially if you’re using coprocessors.
• Most load test platforms run many concurrent bots (also called Virtual Users or VUs) that execute test scripts. Ensure that each virtual user has its own unique identity. Testing if a single user performs an action many times can have a dramatically different effect on your system than many users performing the same action at once.
• Prioritize writing tests at the service layer and encourage the separate teams to collaborate on methodology.
• Once the service layer tests are complete, write end-to-end tests based on usage patterns found in your Apollo Studio history. This step will require collaboration across teams because access patterns across service lines may not perform well together. Deep knowledge of all services involved in multi-graph queries is necessary to optimize performance and fix issues. We strongly recommend using a single platform where tests and results are easily shared.

Executing Apollo Federation Load Tests

1. Run the service layer tests manually in isolation to pinpoint weaknesses or bottlenecks within specific services.
2. Add the service load tests into each service’s CI/CD workflows. Load testing in CI/CD will enable your team to spot degradations early and prevent deployment of slow code.
3. Run the end-to-end tests manually to identify new breaking points introduced by cross-service communication. Keep an eye out for new performance bottlenecks.
4. Integrate the end-to-end tests into your CI/CD to continuously monitor system performance.

We recommend running smaller-scale load tests continuously in CI/CD to prevent slow code from shipping, and manually running large-scale tests in preparation for significant events (major releases, marketing promotions, etc). Large-scale tests may uncover new critical issues, previously unseen due to concurrency issues in complex systems.

Load Testing Apollo Federation with Multiple

What is Multiple?

Multiple is a fully-managed load testing platform that runs tests written with JavaScript, TypeScript, and NPM packages. When you use Multiple’s CLI, your test specs can live directly in your code base, and you can import existing JS/TS code, just like you can for unit tests. This makes tasks like auth, generating fake data, and replicating client logic simple. You can add load testing to your deployment process by running two simple commands in your existing CI/CD workflows.

Using Multiple to Test Apollo Federation APIs

‍A major advantage of using Multiple to test GraphQL APIs is that you can use the @apollo/client in your tests, and accurately simulate API client behavior. Other benefits of using Multiple to test Apollo Federation APIs include:

• Tests are written in JS/TS and your existing code can be reused in the tests
• Multiple integrates into your code repository and CI/CD, so it’s easy to track the performance of across commits
• Nearly any type of service can be load tested with Multiple
• Multiple lets teams share tests and results with each other

Code Example

Below is a sample Multiple test spec that exemplifies some of the concepts and suggestions we’ve discussed today.

/**
* Apollo Federation Load Test Example w/ Multiple
* A Multiple test spec is defined as a class with lifecycle functions, shown below.
* Our sample Apollo Federation supergraph sits in front of a event ticketing application.
* It features subgraphs for three microservices - `events`, `tickets`, and `users`.
* This test will simulate users viewing a random page, viewing a specific event page, and buying a specific ticket.
*/

import { ApolloClient, createHttpLink, InMemoryCache } from '@apollo/client';
import { setContext } from '@apollo/client/link/context';
import type { JsonValue, MxCtx, MxTestSpec } from '@multiple/mx-cli';
import _ from 'lodash';

import {
 GET_ALL_EVENT_IDS,
 GET_EVENT_DATA,
 BUY_TICKET,
} from './your-project/queries';

class ApolloFederationExample implements MxTestSpec {
 // optional with Multiple CLI
 npmDeps = {
   '@apollo/client': '3.9.5',
 };

 defaultRunOptions = {
   numVUs: 50,
   testDuration: 300000, // ms
   rampUpDuration: 30000, // ms
   minVULoopDuration: 1000, // ms
 };

 /**
  * `globalInit()` runs once at the beginning of the load test.
  * In this example, we use an Apollo client to fetch event ids for use throughout the test.
  */
 async globalInit(): Promise<JsonValue> {
   // Set up an HttpLink to your Apollo Gateway
   const httpLink = new createHttpLink({
     uri: process.env.GRAPHQL_ENDPOINT, // use an environment variable
   });

   // Initialize Apollo Client with the HttpLink
   const client = new ApolloClient({
     link: httpLink,
   });

   const res = await client.query({
     query: GET_ALL_EVENT_IDS,
   });

   return {
     eventIds: res.data.eventIds,
     specialEvent: res.data.eventIds[0],
   };
 }

 /**
  * `vuInit()` runs once per VU (Virtual User), after `globalInit()` and before `vuLoop()`.
  * Here, we are logging in to get a JWT, and then initializing an Apollo Client with an Authorization header containing the JWT.
  */
 async vuInit(ctx: MxCtx): Promise<unknown> {
   // Create and log in a user and get a JWT for authorization
   // We are using Multiple's built-in axios instance to make the request
   const res = await ctx.axios.post(process.env.YOUR_AUTH_ENDPOINT, {
     // With Multiple, each VU has a unique, incrementing id
     email: `user+${ctx.info.vuId}@multiple.dev`,
     password: process.env.TEST_USER_PASSWORD,
   });

   // Set the Authorization header
   const authLink = setContext((_, { headers }) => ({
     headers: {
       ...headers,
       authorization: `Bearer ${res.data.token}`,
     },
   }));

   // Set up an HttpLink to your Apollo Gateway
   const httpLink = new createHttpLink({
     uri: process.env.GRAPHQL_ENDPOINT,
   });

   // Each Virtual User(VU) initializes an Apollo Client with the HttpLink and cache
   const apiClient = new ApolloClient({
     link: authLink.concat(httpLink),
     cache: new InMemoryCache(),
   });

   // Pass the Apollo client to the vuLoop
   return {
     apiClient,
   };
 }

 /**
  * `vuLoop()` runs repeatedly, until the end of the test.
  * Here, we simulate users viewing a random page, viewing a specific event page, and buying a specific ticket.
  */
 async vuLoop(ctx: MxCtx) {
   // Get the Apollo client from vuInitData
   const { apiClient } = ctx.vuInitData;

   const pageViewStart = Date.now();
   const pageViewRes = await apiClient.query({
     query: GET_EVENT_DATA,
     variables: {
       // use lodash to get a random event id
       eventId: _.sample(ctx.globalInitData.eventIds),
     },
   });
   ctx.metric('View Random Page', Date.now() - pageViewStart, 'ms');

   const specialPageViewStart = Date.now();
   const specialPageViewRes = await apiClient.query({
     query: GET_EVENT_DATA,
     variables: {
       eventId: ctx.globalInitData.specialEvent,
     },
   });
   ctx.metric('View Special Page', Date.now() - specialPageViewStart, 'ms');

   const buyTicketStart = Date.now();
   const ticketRes = await apiClient.mutate({
     mutation: BUY_TICKET,
     variables: {
       eventId: ctx.globalInitData.specialEvent,
     },
   });
   ctx.metric('Buy Special Ticket', Date.now() - buyTicketStart, 'ms');

   // Check the response for errors
   const errors = [pageViewRes, specialPageViewRes, ticketRes]
     .map((res) => res.errors)
     .flat();

   if (errors.length) {
     // Multiple automatically tracks and logs any thrown errors
     throw new Error(errors);
   }
 }
}

How to Get Started with Multiple?

You can start using Multiple for free by signing up here. To learn about Multiple and see more code examples, view our documentation here.

TL;DR

Load testing an Apollo Federation API is no small feat. A federated architecture requires load test systems that are flexible enough to test a variety of query types, query structures, and load patterns to identify bottlenecks and failure points. It also requires coordination from teams from across the organization supported by a load test platform that encourages collaboration.

Make sure that whatever load testing tool you build or buy can replicate Apollo Client logic includes team & organization management, generates unique virtual users to perform actions, and supports arbitrary logic to simulate user behavior.

Multiple provides a highly configurable interface for setting the frequency and scale of your load tests. It is easy to use for developers yet remains flexible enough to simulate complex user behaviors in JavaScript or TypeScript–languages that your teams are already proficient with. The Multiple CLI integrates seamlessly into your code base and CI/CD. Multiple also comes out of the box with features for organizations like the ability to share tests, results, and trends across teams.

We hope this introduction to load testing Apollo Federation APIs has been helpful. Please reach out to us with questions, feedback, or to share your own experiences with load testing.

‍