DevOps ·

Structured Logging in TypeScript: From Console.log to Production Observability

console.log fails at scale in ways you will not notice until you are debugging a production incident. This covers structured logging fundamentals, library tradeoffs, log pipeline architecture, OpenTelemetry integration, cost-effective storage options, and a complete Hono middleware implementation.

Structured Logging in TypeScript: From Console.log to Production Observability

console.log works fine until it does not. The failure mode is specific: everything looks functional in development, your logs ship to production, and then an incident happens. You grep through gigabytes of unstructured text, find relevant lines, and realize you cannot correlate them across requests. You cannot filter by user. You cannot reconstruct what happened in what order across async boundaries. The logs exist but they are not queryable.

This is not a tooling problem. It is a data shape problem. The fix is making logs structured by default, and building a pipeline that can collect, ship, store, and query them at production volumes.

This covers the full stack: why unstructured logs fail, what structured logging actually means in practice, how the major TypeScript libraries compare, how to build a log pipeline that scales without breaking the bank, how to tie logs to OpenTelemetry traces, and how to handle serverless and edge environments where the rules change.

Why console.log Fails at Scale

The issue is not performance, though that matters. The issue is that console.log outputs unstructured strings. When you write:

console.log(`User ${userId} failed to upgrade subscription: ${err.message}`);

You produce a line like:

User usr_abc123 failed to upgrade subscription: Card declined

That line is human-readable. It is not machine-queryable. To find all subscription failures across a one-hour window, you write a regex. To correlate that failure with the incoming request that triggered it, you rely on timestamps and hope the logs are in order. To find all failures for a specific user across multiple services, you grep with a user ID pattern that might match false positives.

Structured logs solve this by making every field explicit:

{
  "timestamp": "2026-03-25T14:22:01.332Z",
  "level": "error",
  "message": "subscription upgrade failed",
  "service": "billing-api",
  "requestId": "req_9a3f12b8",
  "userId": "usr_abc123",
  "tenantId": "tenant_xyz",
  "errorCode": "card_declined",
  "durationMs": 1204
}

Now you can filter, aggregate, join, and alert on any of those fields without regex. The log aggregator can index them. Your dashboards can query them as data, not text.

The other failure mode is log volume. At a few hundred requests per second, console.log with JSON.stringify on every call is measurably slower than a library that uses a pre-compiled serializer. Pino benchmarks at roughly 5-7x faster than naive JSON serialization in tight loops, which matters when logging is on the hot path of every request.

Structured Logging Fundamentals

Three concepts drive every useful logging setup: consistent field schemas, log levels with enforcement, and correlation IDs.

Field schemas. Every log entry should have a fixed set of base fields: timestamp, level, message, service name, and environment. Request-scoped logs add request ID, user ID, tenant ID, HTTP method, path, and response duration. Error logs add error name, message, and stack trace. Stick to these shapes and your queries will be consistent across all services.

Log levels. The standard is debug, info, warn, error, and optionally fatal. The key production behavior: debug should be off by default in production (set via LOG_LEVEL=info environment variable), and every level above debug should ship to your aggregator. The mistake is logging everything at info because it is the default, which erases signal-to-noise ratio. Log info for business-relevant events (request completed, job processed, payment succeeded). Log warn for degraded but recoverable states. Log error for failures that require attention.

Correlation IDs. A correlation ID (also called a request ID or trace ID) is a UUID generated at the edge of your system and propagated through every log line, every async operation, and every downstream service call. Without it, you cannot reconstruct the sequence of events for a single user action that touches multiple services or spawns background jobs.

The pattern: generate at the boundary, read from incoming headers first (x-request-id is a common convention), and pass forward explicitly.

Library Comparison: Pino, Winston, Consola

The three libraries that dominate TypeScript logging each make different tradeoffs.

LibrarySpeedEcosystemBundle sizeStructured by defaultBest for
pinoFastest (5-7x over winston)Strong, pino-pretty for dev~120 KBYesNode.js services, high-throughput APIs
winstonModerateMature, many transports~370 KBNo (configurable)Legacy Node.js apps, complex transport routing
consolaModerateVue/Nuxt ecosystem~50 KBPartialCLIs, Nuxt apps, dev-focused tools

Pino is the right choice for production Node.js APIs. It serializes to JSON using a fast, low-allocation internal serializer, supports child loggers with inherited context (critical for request-scoped logging), and has strong TypeScript types. Its async transport mode writes to stdout in a worker thread, keeping the main thread free. The development experience is handled by pino-pretty, which formats JSON output into human-readable form when NODE_ENV !== "production".

Winston is the right choice when you need routing logic at the transport level: some logs go to CloudWatch, errors go to a separate Slack webhook, and debug logs go to a file. Its transport system handles this cleanly. If you are maintaining a legacy Express app already using winston, the switching cost to pino rarely justifies itself.

Consola is not a production logging library in the traditional sense. It targets developer experience: nice terminal output, integration with Nuxt’s module system, minimal configuration. Use it in CLIs and dev tooling, not in services that need queryable production logs.

Building a Production Logger with Pino

A minimal but complete setup:

import pino from "pino";

const isDev = process.env.NODE_ENV !== "production";

export const logger = pino({
  level: process.env.LOG_LEVEL ?? "info",
  // In dev, use pino-pretty for readable output.
  // In production, emit raw JSON to stdout.
  transport: isDev
    ? { target: "pino-pretty", options: { colorize: true, ignore: "pid,hostname" } }
    : undefined,
  base: {
    service: process.env.SERVICE_NAME ?? "api",
    env: process.env.NODE_ENV ?? "development",
    version: process.env.GIT_COMMIT_SHA ?? "unknown",
  },
  // Redact sensitive fields before they reach the aggregator.
  redact: {
    paths: ["req.headers.authorization", "req.headers.cookie", "*.password", "*.token"],
    censor: "[REDACTED]",
  },
  timestamp: pino.stdTimeFunctions.isoTime,
});

Create request-scoped child loggers in your middleware:

import { randomUUID } from "node:crypto";
import { logger } from "./logger";

export function createRequestLogger(requestId: string, context: Record<string, unknown>) {
  return logger.child({ requestId, ...context });
}

The child() call creates a new logger that inherits all parent bindings and adds new ones. Every log line emitted from a child logger includes the parent fields automatically, without any manual spreading.

Hono Logging Middleware

Hono is a fast, edge-native framework with first-class TypeScript support. Here is a complete logging middleware implementation:

import { Hono } from "hono";
import { randomUUID } from "node:crypto";
import pino, { type Logger } from "pino";

// Extend Hono's context Variables type
type Variables = {
  logger: Logger;
  requestId: string;
};

const app = new Hono<{ Variables: Variables }>();

const baseLogger = pino({
  level: process.env.LOG_LEVEL ?? "info",
  transport:
    process.env.NODE_ENV !== "production"
      ? { target: "pino-pretty", options: { colorize: true } }
      : undefined,
  base: {
    service: process.env.SERVICE_NAME ?? "api",
    version: process.env.GIT_COMMIT_SHA ?? "unknown",
  },
  redact: ["req.headers.authorization", "*.password"],
  timestamp: pino.stdTimeFunctions.isoTime,
});

// Logging middleware
app.use("*", async (c, next) => {
  const requestId =
    c.req.header("x-request-id") ??
    c.req.header("cf-ray") ?? // Cloudflare Workers provides this
    randomUUID();

  const requestLogger = baseLogger.child({
    requestId,
    method: c.req.method,
    path: new URL(c.req.url).pathname,
    // Pull from JWT/session if available
    userId: c.get("userId"),
    tenantId: c.get("tenantId"),
  });

  c.set("logger", requestLogger);
  c.set("requestId", requestId);

  // Propagate the ID downstream for service-to-service calls
  c.header("x-request-id", requestId);

  const start = Date.now();

  requestLogger.info("request started");

  await next();

  const durationMs = Date.now() - start;
  const status = c.res.status;

  const logMethod = status >= 500 ? "error" : status >= 400 ? "warn" : "info";
  requestLogger[logMethod]("request completed", { status, durationMs });
});

// Usage in a route handler
app.post("/api/subscriptions/upgrade", async (c) => {
  const log = c.get("logger");
  const { plan } = await c.req.json<{ plan: string }>();

  log.info("subscription upgrade initiated", { plan });

  try {
    const result = await upgradeSubscription(c.get("userId")!, plan);
    log.info("subscription upgrade succeeded", { plan, newTier: result.tier });
    return c.json({ success: true, tier: result.tier });
  } catch (err) {
    log.error("subscription upgrade failed", {
      plan,
      errorName: err instanceof Error ? err.name : "UnknownError",
      errorMessage: err instanceof Error ? err.message : String(err),
    });
    return c.json({ error: "Upgrade failed" }, 500);
  }
});

export default app;

The cf-ray header fallback matters for Cloudflare Workers: every request gets a globally unique Ray ID from Cloudflare’s edge network, and using it as the correlation ID ties your application logs to Cloudflare’s own request logs.

Log Pipeline Architecture

A log pipeline has four stages: collect, ship, store, and query.

Collect. Your application writes JSON to stdout. Do not write to files from the application process. Let the container runtime, systemd, or the serverless platform handle stdout capture. This keeps your application stateless and decoupled from the log destination.

Ship. A log collector reads stdout, buffers it, and forwards it to your storage backend. For Kubernetes, Fluent Bit or Vector are the standard choices, running as a DaemonSet. For Docker Compose, the fluentd or awslogs log drivers handle it. For serverless functions (Lambda, Cloudflare Workers, Vercel), the platform captures stdout automatically and routes it to its native log system.

Store. Where logs land for querying. The choices vary significantly on cost, query power, and operational overhead:

BackendStrengthsWeaknessesCost model
AxiomFast SQL queries, excellent DX, generous free tierClosed-source SaaSPer GB ingested + stored
Grafana LokiOpen source, tight Grafana integration, cheap storageLogQL is less expressive than SQLStorage cost, compute for queries
CloudWatch LogsZero-config on AWS, integrated with LambdaExpensive at volume, slow queries, poor DXPer GB ingested + stored
Elastic/OpenSearchPowerful full-text searchHigh operational overhead, expensiveCompute + storage

For teams that do not want to operate infrastructure: Axiom is the best developer experience at startup log volumes. For teams already on AWS that want zero integration work: CloudWatch is fine up to roughly $500/month before the cost model becomes painful. For teams self-hosting on Kubernetes who want to minimize vendor lock-in: Loki with Grafana is the right choice, and storage on S3 or GCS is cheap.

Query. Your aggregator’s query interface. Axiom uses APL (a Kusto-like language). Loki uses LogQL. CloudWatch uses CloudWatch Insights SQL. The practical question is: can your team write ad-hoc queries during an incident without referring to documentation? Axiom wins here for teams not already fluent in LogQL.

Integrating Logs with OpenTelemetry Traces

Structured logs and distributed traces answer different questions. Logs tell you what happened. Traces tell you how long each step took and where in the service graph the time was spent. They become much more powerful when correlated: clicking a trace in your tracing backend should let you jump to the logs for that specific request, and vice versa.

The correlation mechanism is adding the OpenTelemetry trace ID and span ID to every log line emitted within a traced request.

import { trace, context } from "@opentelemetry/api";
import pino from "pino";

// Custom pino mixin that reads the active OTel span context
const logger = pino({
  mixin() {
    const span = trace.getActiveSpan();
    if (!span || !span.isRecording()) return {};

    const ctx = span.spanContext();
    return {
      traceId: ctx.traceId,
      spanId: ctx.spanId,
      // traceFlags: 01 means sampled
      traceSampled: (ctx.traceFlags & 1) === 1,
    };
  },
  level: process.env.LOG_LEVEL ?? "info",
  timestamp: pino.stdTimeFunctions.isoTime,
});

export { logger };

The mixin function runs on every log call and merges its return value into the log entry. When a request is being traced, every log line automatically includes traceId and spanId. Your log aggregator can render a link to the trace in your tracing backend (Grafana Tempo, Jaeger, Honeycomb) by constructing a URL from those IDs.

For log aggregators that support the OpenTelemetry log data model natively (Grafana Loki 2.9+, Axiom), you can also ship logs through the OpenTelemetry Collector using the @opentelemetry/exporter-logs-otlp-http exporter, which keeps trace correlation automatic.

Serverless and Edge Environments

The rules change in serverless and edge contexts:

Lambda. AWS Lambda captures stdout/stderr automatically and ships to CloudWatch Logs. There is no persistent process, no file system, and no long-lived log buffer. Write JSON to stdout synchronously and let Lambda handle the rest. Pino’s default sync mode works without changes. The critical difference from long-running servers: do not use pino’s async transport (pino({ transport: { target: 'pino/file' } })) in Lambda, because the worker thread may be killed before it flushes.

Cloudflare Workers. No Node.js process, no stdout in the traditional sense. console.log in a Worker emits to Cloudflare’s logs system, accessible via Wrangler’s tail command or forwarded via a logpush job to an external destination. For structured logging, serialize your log object to JSON and pass it to console.log:

function log(level: string, message: string, ctx: Record<string, unknown> = {}) {
  // Cloudflare Workers: console.log output goes to Workers Logs
  // Use JSON so logpush can forward structured entries
  console.log(
    JSON.stringify({
      timestamp: new Date().toISOString(),
      level,
      message,
      ...ctx,
    })
  );
}

For Cloudflare Workers, you can also use the Tail Workers API to intercept log events programmatically and forward them to any HTTP endpoint before the invocation ends. This is the right approach if you need logs in Axiom or Loki rather than Cloudflare’s native system.

Vercel Edge Functions. Similar to Cloudflare Workers: no Node.js runtime, synchronous logging via console.log, with Vercel’s log drain feature forwarding to external destinations.

The shared constraint across serverless environments: you cannot rely on background flush operations completing after the function returns. Every log write that matters must be synchronous within the request lifecycle.

Log Sampling at Volume

Full request logging at high throughput gets expensive. A request-per-second rate of 500 with a 200-byte average log line generates roughly 85 GB/day. At Axiom’s pricing that is $2.50/day in storage, which is manageable. At CloudWatch’s pricing it is around $40/day, which is not.

Before cutting volume, exhaust the cheaper options: compress aggressively (Loki and Axiom both compress on ingest), set short retention for verbose log levels, and route debug logs to a cheaper tier or drop them entirely in production.

When you do need to sample, sample selectively:

function shouldLog(level: string, statusCode: number, durationMs: number): boolean {
  // Always log errors and warnings
  if (level === "error" || level === "warn") return true;
  // Always log slow requests regardless of status
  if (durationMs > 2000) return true;
  // Always log non-2xx responses
  if (statusCode >= 400) return true;
  // Sample 20% of healthy fast requests
  return Math.random() < 0.2;
}

Apply this in your response-logging middleware, not deep in your application code. Sampling at the middleware layer keeps the decision centralized and ensures errors always get through.

Production Tradeoffs

A few things that are genuinely unclear and worth being honest about:

Pino vs. rolling your own. Pino adds a dependency and learning curve. For simple services, a thin JSON wrapper around process.stdout.write is sufficient and has zero dependencies. The pino value proposition is the serialization speed, the redaction system, the child logger API, and the transport ecosystem. If you are not using those features, a 40-line custom logger is often cleaner.

Centralized log storage cost. At scale, log costs grow faster than most teams expect. The effective mitigation is aggressive log level discipline: never log debug in production unless investigating a specific incident, and turn it back off after. A well-disciplined info level will reduce volume by 60-80% compared to teams that log everything.

Synchronous vs. asynchronous transport. Pino’s async transport keeps logging off the critical path by writing through a worker thread. This is faster under sustained load but means you can lose the last few log lines if the process crashes hard (SIGKILL, OOM kill). In practice this is acceptable for most services. If you are in a regulated environment where log completeness is a compliance requirement, use synchronous transport and accept the latency.

The Setup That Actually Works

The sequence that avoids wasted effort:

  1. Drop console.log. Replace with a pino logger writing JSON to stdout. Add the mixin for OTel context if you are already tracing. This takes a few hours.
  2. Establish a field schema: the eight or so fields every log line must have. Enforce it in the child logger factory, not ad hoc in each handler.
  3. Set up your aggregator. Axiom for teams that want it working today, Loki for teams that want to own the stack.
  4. Add the request logging middleware. Every inbound request gets a correlation ID, and every log line within that request carries it.
  5. Add log-based alerts for the things that matter: elevated error rates on specific paths, payment failures, job queue depth. These come for free once your logs are structured and queryable.
  6. Add trace correlation when you have multiple services and latency investigations are costing you time.

The common mistake is inverting this order by starting with distributed tracing before logging is solid. Traces are powerful, but they tell you where time was spent. Logs tell you what happened. You need both, and logs are the cheaper investment with the higher early-stage return.

The goal is not comprehensive coverage. It is fast, queryable signal when something breaks at 2am.

More in DevOps

How Terraform Works Internally: HCL Parsing, Dependency Graph Execution, and the Provider Protocol Behind Infrastructure as Code
DevOps ·

How Terraform Works Internally: HCL Parsing, Dependency Graph Execution, and the Provider Protocol Behind Infrastructure as Code

A deep dive into Terraform's internal architecture covering HCL parsing, the expression evaluation engine, DAG-based dependency resolution, the gRPC provider plugin protocol, state mechanics, and the plan/apply lifecycle.

How Docker Works Internally: Namespaces, Cgroups, Union Filesystems, and the OCI Runtime Behind Every Container
DevOps ·

How Docker Works Internally: Namespaces, Cgroups, Union Filesystems, and the OCI Runtime Behind Every Container

A deep dive into what happens when you run docker run: Linux namespaces for process isolation, cgroups v2 for resource enforcement, overlay2 union filesystem for layered images, the OCI runtime spec and how runc spawns containers, content-addressable storage, the containerd shim architecture, and networking with veth pairs.

How Kubernetes Works: Pods, Scheduling, and the Reconciliation Loop
DevOps ·

How Kubernetes Works: Pods, Scheduling, and the Reconciliation Loop

A deep dive into Kubernetes internals covering the control plane architecture, etcd watch mechanics, the scheduler's filter and score phases, controller manager reconciliation loops, kubelet pod assignment, the CRI and CNI plugin models, and production considerations for resource requests, pod disruption budgets, and cluster autoscaler behavior.

How Docker Works: Namespaces, Cgroups, Union Filesystems, and the Container Runtime from Build to Execution
DevOps ·

How Docker Works: Namespaces, Cgroups, Union Filesystems, and the Container Runtime from Build to Execution

A deep dive into Docker internals covering Linux namespace isolation, cgroup resource constraints, OverlayFS copy-on-write layer mechanics, Dockerfile build cache invalidation rules, the containerd and runc OCI runtime stack, and production considerations for image hardening and startup latency.