Designing an API Composition Layer: Aggregating Microservices, Handling Partial Failures, and Optimizing Client-Server Communication

When you decompose a monolith into microservices, you move coordination costs from inside a single process into the network. The first version is usually fine: the client calls the user service, then the order service, then the product service, and renders a screen. Four round trips later, the page loads. Nobody notices in development. In production, with real latency and real failure rates, you start to notice. The dashboard that renders a user profile with recent orders and recommended products makes five sequential requests, each with its own timeout budget, and a 2% failure rate in any single service becomes a visible problem in aggregate.

The API composition pattern solves this by moving aggregation out of the client and into a server-side layer that can parallelize calls, handle partial failures gracefully, and present a single endpoint to the client. This article covers how to design that layer, what can go wrong at each decision point, and how it compares to the alternatives: BFF services and GraphQL federation.

The Problem: N+1 Client Calls and the Chattiness Trap

The naive approach to microservices from a client perspective is to call each service independently and assemble the result in the client. This works until it doesn’t, and the failure modes are predictable:

Latency compounds. Sequential calls add their latency. If the user service takes 40ms, the order service takes 80ms, and the product service takes 60ms, a sequential client pays 180ms in service time before it can render anything, plus network round trips.

Partial failures produce broken UI. If one service returns a 500, the client has to decide: show a broken page, show nothing, or show a degraded view with the data it has. Most clients are not built to handle this gracefully. The composition layer is the right place to make that decision, not the browser.

Chattiness over mobile connections. Each HTTP request carries overhead: DNS lookup amortizes away, but TLS handshake, headers, and connection management add up. A mobile client on a poor connection paying five round trips will show measurably worse performance than a client making one call.

Client complexity escapes control. When you need to add a new field from a new service to an existing screen, you change the client. This coupling between the client and the internal service topology means every service refactor has a client impact. The composition layer is an abstraction boundary that hides service topology from clients.

The Composition Layer

A composition service sits between the client and the downstream microservices. It exposes a stable endpoint per product use case (or per screen, depending on how fine-grained you go) and owns the logic for fetching, merging, and returning the aggregated response.

The key design decision up front: what does this service own? It should own the fan-out strategy, the partial failure policy, and the response shape for its consumers. It should not own business logic that lives in the downstream services. Composition logic is structural: fetch these things, merge them this way, return this shape. If you find yourself adding domain rules to the composition layer, push them back downstream.

Parallel Fan-Out with Promise.allSettled

The core of any composition service is the fan-out: calling multiple downstream services and waiting for all of them. In TypeScript, Promise.allSettled is the right primitive because it does not short-circuit on rejection. Promise.all will throw on the first rejection, which forces you to either swallow errors in individual calls or propagate a full failure when only one service failed.

interface UserProfile {
  id: string;
  name: string;
  email: string;
}

interface RecentOrders {
  items: Array<{ orderId: string; total: number; createdAt: string }>;
}

interface Recommendations {
  products: Array<{ productId: string; title: string; score: number }>;
}

interface DashboardResponse {
  user: UserProfile | null;
  orders: RecentOrders | null;
  recommendations: Recommendations | null;
  degraded: boolean;
  missing: string[];
}

async function fetchDashboard(userId: string): Promise<DashboardResponse> {
  const [userResult, ordersResult, recommendationsResult] =
    await Promise.allSettled([
      fetchUserProfile(userId),
      fetchRecentOrders(userId),
      fetchRecommendations(userId),
    ]);

  const missing: string[] = [];

  const user =
    userResult.status === "fulfilled" ? userResult.value : null;
  const orders =
    ordersResult.status === "fulfilled" ? ordersResult.value : null;
  const recommendations =
    recommendationsResult.status === "fulfilled"
      ? recommendationsResult.value
      : null;

  if (userResult.status === "rejected") missing.push("user");
  if (ordersResult.status === "rejected") missing.push("orders");
  if (recommendationsResult.status === "rejected")
    missing.push("recommendations");

  return {
    user,
    orders,
    recommendations,
    degraded: missing.length > 0,
    missing,
  };
}

The degraded flag and missing array in the response are not an accident. The client needs to know whether to show a loading skeleton (transient failure, retry shortly) or a degraded state indicator (service is down, present what you have). Hiding partial failures from clients leads to silent data gaps that are harder to debug than explicit degraded states.

Sequential Fan-Out for Data Dependencies

Not all calls can be parallelized. Some downstream services require data returned by an upstream call. The order service might need the user’s account tier from the profile service to return the right pricing. In that case, the fan-out is sequential for the dependent portion and parallel where independence allows.

async function fetchOrdersWithPricing(userId: string): Promise<DashboardResponse> {
  // User profile is a prerequisite for order pricing
  const userResult = await fetchUserProfile(userId).catch(() => null);

  if (!userResult) {
    // Critical dependency failed; cannot meaningfully compose
    return {
      user: null,
      orders: null,
      recommendations: null,
      degraded: true,
      missing: ["user", "orders", "recommendations"],
    };
  }

  // Orders require account tier from profile; recommendations are independent
  const [ordersResult, recommendationsResult] = await Promise.allSettled([
    fetchRecentOrders(userId, { accountTier: userResult.accountTier }),
    fetchRecommendations(userId),
  ]);

  return {
    user: userResult,
    orders:
      ordersResult.status === "fulfilled" ? ordersResult.value : null,
    recommendations:
      recommendationsResult.status === "fulfilled"
        ? recommendationsResult.value
        : null,
    degraded:
      ordersResult.status === "rejected" ||
      recommendationsResult.status === "rejected",
    missing: [
      ordersResult.status === "rejected" ? "orders" : null,
      recommendationsResult.status === "rejected" ? "recommendations" : null,
    ].filter(Boolean) as string[],
  };
}

The decision tree here is: identify the dependency graph first, parallelize the independent subtrees, accept sequential latency only where dependencies force it. The composition layer is where this graph lives; nowhere else should need to know about it.

Partial Failure Handling

Partial failures in a composition layer fall into two categories: non-critical and critical.

Non-critical failures are services whose absence degrades the experience but does not break it. Recommendations are a canonical example: a user can view their orders without them. The composition layer returns null for that section, sets the degraded flag, and the client renders a fallback. The correct fallback depends on the client: in a browser app, show a “Recommendations unavailable” placeholder. In an API consumer, skip the section. The composition layer should not make that rendering decision; it should surface the information needed to make it.

Critical failures are services without which the response is meaningless. If the user profile service is down and the entire response hinges on the user identity, returning a degraded response is misleading. The composition layer should return a proper error in this case, not a hollow shell.

The classification is a product decision, not a technical one. Engineers should get explicit sign-off on what counts as non-critical before implementing fallbacks. Treating a billing data service as non-critical in a payments context is an incident waiting to happen.

Timeouts at the Composition Layer

Each downstream call needs its own timeout. Without them, a slow downstream service holds a composition request open indefinitely. The composition layer’s timeout budget should be shorter than the client’s timeout, leaving headroom for the client to retry.

async function fetchWithTimeout<T>(
  promise: Promise<T>,
  timeoutMs: number,
  label: string
): Promise<T> {
  return Promise.race([
    promise,
    new Promise<T>((_, reject) =>
      setTimeout(() => reject(new Error(`Timeout: ${label} exceeded ${timeoutMs}ms`)), timeoutMs)
    ),
  ]);
}

async function fetchDashboardWithTimeouts(userId: string): Promise<DashboardResponse> {
  const [userResult, ordersResult, recommendationsResult] =
    await Promise.allSettled([
      fetchWithTimeout(fetchUserProfile(userId), 200, "user-profile"),
      fetchWithTimeout(fetchRecentOrders(userId), 400, "recent-orders"),
      fetchWithTimeout(fetchRecommendations(userId), 300, "recommendations"),
    ]);

  // ... same aggregation logic as before
}

The timeout values are intentional. The user profile is fast, structural data; a 200ms budget is generous. Order history may involve a database scan; 400ms is reasonable. Recommendations may involve ML inference; 300ms is a tight budget that should prompt conversation about whether recommendations belong in the critical path or should be loaded asynchronously by the client after the main response arrives.

Caching at the Composition Layer

The composition layer is a natural place to cache, but it needs to be done per-section, not per-response. Caching the entire composed response means the TTL is constrained by the most volatile section. If recommendations update every 30 seconds but user profile changes only on explicit edit, a composite TTL of 30 seconds is correct for recommendations but unnecessarily aggressive for profile data.

Section-level caching means each downstream fetch checks its own cache first:

async function fetchUserProfileCached(
  userId: string,
  cache: Map<string, { value: UserProfile; expiresAt: number }>
): Promise<UserProfile> {
  const key = `user:${userId}`;
  const cached = cache.get(key);

  if (cached && cached.expiresAt > Date.now()) {
    return cached.value;
  }

  const profile = await fetchUserProfile(userId);
  cache.set(key, { value: profile, expiresAt: Date.now() + 60_000 }); // 60s TTL

  return profile;
}

In production, this cache is Redis or Memcached, not an in-process Map. The in-process version leaks between requests in worker processes and becomes a consistency problem in a multi-instance deployment. The key design principle stands regardless of backend: TTL lives with the data domain, not at the response level.

Tradeoffs: Composition Layer vs BFF vs GraphQL Federation

Approach	Strengths	Weaknesses	When to use
Composition layer	Single deployment, explicit aggregation logic, easy to debug	Can become a monolith if poorly scoped; all clients share one shape	One or two client types with similar data needs
BFF (Backend for Frontend)	Client-specific shapes, independent deployments, clear ownership	N services to maintain; shared concerns (auth, tracing) need coordination	Multiple clients with meaningfully different data needs
GraphQL federation	Client drives the query shape; schema distributed across services	Operational complexity of a gateway + subgraph mesh; debugging distributed traces is harder	Large orgs with many teams, many clients, need for self-service schema evolution

The composition layer is the right starting point for most systems. It is a single service, easy to reason about, and handles the core aggregation problem directly. Its failure mode is becoming too broad: teams start adding every cross-service join to it and it becomes a distributed join layer with business logic scattered across method calls. Discipline on scope prevents this.

The BFF pattern becomes necessary when clients diverge significantly. A mobile client on a 3G connection needs compact, one-call responses with minimal nesting. A web dashboard may want rich nested objects. When these shapes are so different that maintaining one composition layer requires constant if-branching on client type, splitting into separate BFFs with separate deployment cadences is the right call. The coordination cost is real: rate limiting, authentication middleware, distributed tracing instrumentation, and common utilities all need to be consistent across BFFs without becoming a monolith in disguise.

GraphQL federation is worth the operational overhead when you have multiple teams, each owning a service, who need to independently evolve their schema contributions without a central coordination layer. It moves schema composition to build time (or schema registry time) rather than runtime. The cost is real: debugging a federated query that touches four subgraphs through a gateway requires good distributed tracing, and the operational model of maintaining subgraph schemas, a schema registry, and a gateway adds meaningful infrastructure complexity. Reaching for federation in a two-team system is over-engineering. It earns its keep at four-plus teams with independent deployment cadences.

Production Considerations

Observability per downstream call. The composition layer should emit a span per downstream service call, not just a single span for the composite request. When the dashboard is slow, you need to know whether it was the user service, the order service, or the recommendation service. Without per-call spans, you are debugging by guessing.

Circuit breaking for downstream dependencies. If the recommendations service has a 40% error rate, the composition layer should stop calling it and return null immediately rather than accumulating 300ms timeouts on every request. The circuit breaker pattern (open after N failures, half-open after a cooldown, closed after M successes) belongs at each downstream client inside the composition layer, not at the outer edge.

Retry policy mismatch. Retries inside the composition layer on a non-idempotent call can produce duplicate writes. GET calls are safe to retry; POST, PATCH, and DELETE are not unless the downstream service is idempotent. Be explicit about which calls are retried and which fail fast.

Schema versioning. The composition layer is an abstraction boundary. Its response schema is a contract with the client. Downstream services can change their schemas as long as the composition layer’s output contract holds. This means the composition layer needs versioned output schemas and the ability to transform between downstream response shapes and the client contract. Without this, a downstream rename breaks all clients even though the composition layer exists to prevent exactly that coupling.

Cost of composition at scale. At high request volume, the composition layer becomes a fan-out multiplier. If you have 10,000 requests per second and each makes four downstream calls, you are generating 40,000 downstream requests per second. Section-level caching reduces this, but it does not eliminate it for cache misses. Profile the downstream call volume under load before you hit production.

The Structure That Holds

The composition layer is not a silver bullet. It solves the N+1 call problem, isolates partial failures, and gives you one place to manage the client-service contract. What it does not do is eliminate the distributed systems problems underneath: services still fail, networks still partition, and data consistency across services is still your problem.

The pattern earns its keep when you have more than two or three downstream services contributing to a client response, when clients are outside your control (public API, mobile apps), or when you want to hide service topology changes from clients without coordinating a client deploy. Start with a single composition service, keep business logic out of it, instrument every downstream call, and resist the urge to add client-specific branching logic until you genuinely need a BFF split.

The composition layer is infrastructure. Treat it like infrastructure: boring, stable, observable, and well-understood by the team that operates it.