Observability for Small Engineering Teams: What to Monitor When You Can't Monitor Everything

Most observability writing is written by platform engineers at companies with dedicated SRE teams, $50k/month Datadog bills, and the time to instrument every service with distributed tracing. If you are two engineers trying to keep a product alive while also shipping features, that writing is noise.

This covers what actually matters when you have limited time and a startup budget: what to instrument, what to ignore, which tools are worth the money at small scale, and when more sophisticated observability becomes worth the investment.

The Problem With Generic Observability Advice

The standard observability taxonomy (metrics, logs, traces, the “three pillars”) is correct. But it implies all three are equally important at all scales, which they are not.

For a 2-person team running a SaaS product:

• Metrics give you system health trends. High value, low cost.
• Logs tell you what happened when something broke. High value, moderate cost.
• Traces show you how a request traveled through distributed services. High cost, and often overkill before you have multiple services generating meaningful latency variance.

The mistake is jumping straight to full tracing before you have solved logging. Traces are debug tools. If you do not have structured logs telling you that something went wrong, traces will not save you.

The Four Golden Signals, Adapted for Startups

Google’s four golden signals (latency, traffic, errors, saturation) are the right mental model. Here is what they mean practically for a small team:

Latency. Track p50, p95, and p99 for your API endpoints. P50 tells you what users typically experience. P95 and p99 tell you about your tail, which is where trust erosion happens. Do not aggregate into a single mean, that number is useless.

Traffic. Requests per second, and specifically the breakdown by endpoint and by status code family (2xx, 4xx, 5xx). Traffic context is what turns “errors spiked” into “errors spiked on /api/billing/upgrade after the deploy at 14:22.”

Errors. Two distinct things: server-side errors (5xx responses, unhandled exceptions, database query failures) and client-side errors (JS exceptions, React render errors, failed fetches). Track both. They usually indicate different problems.

Saturation. CPU and memory utilization for your compute. Database connection pool usage. For serverless functions, cold start rate and concurrency limits. You do not need deep infrastructure metrics, but you need to know when you are approaching ceilings.

For a startup, the high-value version of these four signals is roughly a dozen time series. That is manageable. Do not start with a hundred.

Structured Logging: The Foundation

Before dashboards and alerts, you need logs you can query. Unstructured logs (raw print statements) are searchable but not filterable. When an incident happens at 2am, you want to filter by userId, tenantId, requestId, and errorCode, not grep through a wall of text.

The pattern in TypeScript is simple. Create a logger that always emits JSON with a consistent set of base fields:

import { randomUUID } from "node:crypto";

type LogLevel = "debug" | "info" | "warn" | "error";

type LogContext = Record<string, string | number | boolean | null | undefined>;

type LogEntry = {
  timestamp: string;
  level: LogLevel;
  message: string;
  requestId?: string;
  tenantId?: string;
  userId?: string;
  service: string;
  [key: string]: unknown;
};

function createLogger(baseContext: Partial<LogEntry>) {
  function log(level: LogLevel, message: string, context: LogContext = {}) {
    const entry: LogEntry = {
      timestamp: new Date().toISOString(),
      level,
      message,
      service: process.env.SERVICE_NAME ?? "api",
      ...baseContext,
      ...context,
    };
    // In production this goes to stdout, collected by your log aggregator.
    // In dev, pretty-print it.
    if (process.env.NODE_ENV === "production") {
      process.stdout.write(JSON.stringify(entry) + "\n");
    } else {
      console.log(JSON.stringify(entry, null, 2));
    }
  }

  return {
    debug: (message: string, ctx?: LogContext) => log("debug", message, ctx),
    info: (message: string, ctx?: LogContext) => log("info", message, ctx),
    warn: (message: string, ctx?: LogContext) => log("warn", message, ctx),
    error: (message: string, ctx?: LogContext) => log("error", message, ctx),
    child: (childContext: Partial<LogEntry>) =>
      createLogger({ ...baseContext, ...childContext }),
  };
}

export const logger = createLogger({});

In your request handler, create a child logger scoped to the request:

import { logger } from "./logger";
import { randomUUID } from "node:crypto";

export function requestLoggingMiddleware(req: Request, res: Response, next: NextFunction) {
  const requestId = (req.headers["x-request-id"] as string) ?? randomUUID();
  const requestLogger = logger.child({
    requestId,
    tenantId: req.auth?.tenantId,
    userId: req.auth?.userId,
    method: req.method,
    path: req.path,
  });

  // Attach to request so handlers can use it
  req.log = requestLogger;

  const start = Date.now();
  res.on("finish", () => {
    requestLogger.info("request completed", {
      statusCode: res.statusCode,
      durationMs: Date.now() - start,
    });
  });

  next();
}

This gives you log lines like:

{
  "timestamp": "2026-03-08T14:23:01.412Z",
  "level": "info",
  "message": "request completed",
  "service": "api",
  "requestId": "9a3f12b8-...",
  "tenantId": "tenant_abc",
  "userId": "usr_789",
  "method": "POST",
  "path": "/api/subscriptions/upgrade",
  "statusCode": 500,
  "durationMs": 843
}

Now your log aggregator can filter for all failed requests by a specific tenant, all slow requests to a specific path, or all errors from a deploy window, in seconds.

Error Capture: Don’t Lose Exceptions

Structured logs handle the happy path and expected errors. Unhandled exceptions need separate capture. The reason is practical: unhandled exceptions in production often come with stack traces that include line numbers across minified code, async context that is hard to reconstruct from logs alone, and browser environment context (user agent, URL, JS version) that matters for debugging frontend issues.

For a small team, a tool like Highlight.io covers both frontend and backend error capture under a single SDK, with session replay, at a free tier that handles most early-stage volumes. It is not the only option, but it is the one with the best cost-to-signal ratio at startup scale.

Backend setup in a Node.js service:

import { H } from "@highlight-run/node";

H.init({
  projectID: process.env.HIGHLIGHT_PROJECT_ID!,
  serviceName: "api",
  serviceVersion: process.env.GIT_COMMIT_SHA,
});

// Wrap your error handler
app.use((err: Error, req: Request, res: Response, next: NextFunction) => {
  H.consumeError(err, req, res);
  req.log?.error("unhandled error", {
    errorName: err.name,
    errorMessage: err.message,
  });
  res.status(500).json({ error: "Internal server error" });
});

The key addition is serviceVersion. Tagging errors with the deploy SHA lets you immediately answer “did this error start with today’s deploy?”

Metrics: What to Emit and How

For most small-team APIs, you do not need custom metric infrastructure in the first year. Your log aggregator can derive metrics from structured logs. Latency and error rate come directly from your request log lines if they include durationMs and statusCode.

What you do need custom metrics for:

• Business-level counters: signups, subscription events, payment failures. These do not come from generic request logs.
• Queue depth, background job lag, cron job success/failure.
• External dependency health: did the payment provider call succeed or fail.

A minimal metrics pattern using a StatsD-compatible interface (works with both self-hosted Prometheus and Grafana Cloud):

import { createSocket } from "node:dgram";

type MetricType = "counter" | "gauge" | "histogram";

const client = createSocket("udp4");
const STATSD_HOST = process.env.STATSD_HOST ?? "127.0.0.1";
const STATSD_PORT = parseInt(process.env.STATSD_PORT ?? "8125");

function send(metric: string, value: number, type: MetricType, tags: Record<string, string> = {}) {
  const tagStr = Object.entries(tags)
    .map(([k, v]) => `${k}:${v}`)
    .join(",");
  const suffix = type === "counter" ? "c" : type === "gauge" ? "g" : "ms";
  const line = tagStr
    ? `${metric}:${value}|${suffix}|#${tagStr}`
    : `${metric}:${value}|${suffix}`;

  client.send(Buffer.from(line), STATSD_PORT, STATSD_HOST);
}

export const metrics = {
  increment: (name: string, tags?: Record<string, string>) => send(name, 1, "counter", tags),
  gauge: (name: string, value: number, tags?: Record<string, string>) => send(name, value, "gauge", tags),
  timing: (name: string, ms: number, tags?: Record<string, string>) => send(name, ms, "histogram", tags),
};

Usage:

// In your subscription upgrade handler
metrics.increment("subscription.upgrade.attempted", { plan: payload.plan });
try {
  await upgradeSubscription(payload);
  metrics.increment("subscription.upgrade.succeeded", { plan: payload.plan });
} catch (err) {
  metrics.increment("subscription.upgrade.failed", {
    plan: payload.plan,
    reason: err instanceof PaymentError ? "payment" : "unknown",
  });
  throw err;
}

These events, tagged with domain context, are what turns your dashboard from “things look fine” to “three billing upgrades failed in the last ten minutes, all on the pro plan.”

Tooling at Startup Budgets

The honest comparison for a small team:

Tool	What it covers	Free tier	Paid cost
Grafana Cloud	Metrics, dashboards, alerting	10k series, 50 GB logs/month	~$0-50/month at startup scale
Axiom	Structured log ingestion and query	500 GB/month compressed	$25/month for 1TB
Highlight.io	Frontend + backend errors, session replay	500 sessions/month	$50/month
Sentry	Error tracking	5k errors/month	$26/month
Better Uptime	Uptime checks, status page	10 monitors	$20/month

For a team under 10 engineers running a SaaS:

• Grafana Cloud free tier covers metrics and basic dashboards well past seed stage.
• Axiom’s query speed and log retention economics beat self-hosted ELK for teams without dedicated DevOps time.
• Highlight.io is worth the $50/month if you have a frontend, because session replay turns “user reported a bug” from an hour of debugging into five minutes.

The total cost for a solid observability stack at startup scale is $70-120/month, not $5,000/month. You do not need to “instrument everything” to have good visibility.

Alerting Without Fatigue

Alert fatigue is real and it degrades your incident response. A team that ignores alerts because they are noisy is worse off than a team with fewer, higher-quality alerts.

The rule: alert on symptoms, not causes.

An alert on “CPU > 80% for 5 minutes” is a cause. It fires constantly, often harmlessly, and does not tell you if anything user-visible is wrong. An alert on “5xx error rate > 2% for 3 minutes” is a symptom. It fires when users are affected.

Practical alerting thresholds for a small SaaS:

p99 latency > 3000ms for 5 minutes on any API endpoint   → page
5xx error rate > 2% of requests over 3 minutes           → page
Payment failure rate > 5% over 10 minutes                → page
Background job queue depth > 500 for 15 minutes          → page
Uptime check fails 3 consecutive times                   → page

p95 latency > 1500ms for 10 minutes                      → Slack notification
Error rate > 0.5% over 5 minutes                         → Slack notification
Database connection pool > 80% utilization               → Slack notification

The separation matters. Paging someone at 3am for a Slack-worthy alert breaks trust in your alert system fast.

One rule that almost every small team violates early: every alert must have a runbook. Even a one-paragraph runbook. “This alert fires when X. Check Y first. If Y looks normal, check Z. Escalate if Q.” Without a runbook, the response is Googling while panicked, which is slow and inconsistent.

When Tracing Is Worth It

Distributed tracing is not premature if you actually have distributed services. But “distributed” is doing a lot of work in that sentence.

If your architecture is a monolith or a handful of services and you are not spending more than a few minutes per incident trying to understand which service introduced latency, you do not need tracing yet.

Add tracing when:

• You have 5+ services and latency investigations routinely require querying multiple logs to reconstruct a request path.
• A single user action fans out to multiple downstream services and you need to understand which leg is slow.
• You are debugging intermittent latency spikes and logs do not give you enough resolution.

For TypeScript services, OpenTelemetry is the right long-term standard. It is vendor-neutral, which means you can switch backends (Jaeger, Tempo, Honeycomb, Grafana Tempo) without re-instrumenting.

A minimal OpenTelemetry setup with Grafana Tempo as the backend:

import { NodeSDK } from "@opentelemetry/sdk-node";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";
import { getNodeAutoInstrumentations } from "@opentelemetry/auto-instrumentations-node";
import { Resource } from "@opentelemetry/resources";
import { SEMRESATTRS_SERVICE_NAME, SEMRESATTRS_SERVICE_VERSION } from "@opentelemetry/semantic-conventions";

const sdk = new NodeSDK({
  resource: new Resource({
    [SEMRESATTRS_SERVICE_NAME]: process.env.SERVICE_NAME ?? "api",
    [SEMRESATTRS_SERVICE_VERSION]: process.env.GIT_COMMIT_SHA ?? "unknown",
  }),
  traceExporter: new OTLPTraceExporter({
    url: process.env.OTEL_EXPORTER_OTLP_ENDPOINT,
    headers: {
      Authorization: `Bearer ${process.env.GRAFANA_TEMPO_TOKEN}`,
    },
  }),
  instrumentations: [
    getNodeAutoInstrumentations({
      "@opentelemetry/instrumentation-fs": { enabled: false }, // too noisy
    }),
  ],
});

sdk.start();

Auto-instrumentation covers HTTP, database drivers, and most popular frameworks automatically. Start there before writing manual spans.

What to Skip (For Now)

Things frequently recommended that small teams should deprioritize until they have a forcing function:

Custom dashboards. Grafana dashboards take time to build and maintain. Start with the out-of-the-box dashboards that come with Grafana Cloud’s integrations. Build custom dashboards only after you know what questions you keep asking that the defaults do not answer.

Log-based anomaly detection. Machine learning on logs sounds useful. In practice, at startup log volumes, it produces too many false positives and the tuning time is not worth it. Use simple threshold alerts instead.

Profiling (continuous profiling). Tools like Pyroscope are powerful for CPU and memory analysis. Reach for them when you have a specific, confirmed performance problem that you cannot isolate through metrics and logs. Not before.

Full HTTP request/response body logging. It feels thorough and it creates compliance risk (PII in logs), enormous storage costs, and makes your log queries slow. Log request metadata, not payloads.

Production Considerations

A few things that matter operationally and are often skipped in early setups:

Log sampling at high volume. If you grow to handling tens of thousands of requests per minute, logging every request at debug level becomes expensive. Use sampling: log 100% of errors and warnings, 10-20% of successful requests, and always log the complete context for requests above a latency threshold.

function shouldSampleRequest(statusCode: number, durationMs: number): boolean {
  if (statusCode >= 400) return true; // always log errors
  if (durationMs > 2000) return true; // always log slow requests
  return Math.random() < 0.15; // sample 15% of healthy requests
}

Correlation IDs across async boundaries. When a request triggers a background job, the job logs need the original requestId to be useful. Pass it explicitly when enqueuing work, and log it in the worker.

Alert on your observability infrastructure. If your metrics pipeline goes down, you lose visibility without knowing you lost it. Set up a synthetic check: emit a known metric on a schedule, alert if it stops appearing. A dead monitoring stack is worse than no monitoring stack because it gives false confidence.

The Right Sequence

The order in which to build this out:

1. Structured logging with request context (requestId, tenantId, durationMs, statusCode). Takes a day.
2. Error capture with source maps and deploy tagging. Takes a day.
3. Uptime monitoring. Takes an hour.
4. Five or fewer business-critical custom metrics (signup, payment, job failure). Takes a day.
5. Three to five high-signal alerts (error rate, latency, payment failure). Takes half a day.
6. Dashboards for the above. Takes a day.
7. Tracing when you have distributed services and a specific latency investigation problem.

That sequence, done in order, gives you genuine production visibility within a week. It is not exciting, but it is what separates teams that find out about problems from users from teams that find out first.

The goal is not comprehensive observability. It is fast enough signal to act before users leave.