Designing a Recommendation Engine: Collaborative Filtering, Embeddings, and Real-Time Personalization

Every recommendation system eventually converges on the same set of hard problems: what do you do with a new user who has no history, how do you serve thousands of candidates in under 50ms, and how do you know whether your new model is actually better or just differently wrong?

The textbook answer covers matrix factorization and cosine similarity. This article covers what you actually need to build a production system: the architecture, the failure modes, and the operational decisions that determine whether recommendations ship value or ship noise.

Collaborative Filtering

Collaborative filtering is the observation that users who agreed in the past tend to agree again. You don’t need to understand the content of an item at all. You just need enough user-item interactions.

User-User Filtering

For a target user, find the K nearest neighbors in interaction space, then aggregate their item scores weighted by similarity.

type UserId = string;
type ItemId = string;
type InteractionMatrix = Map<UserId, Map<ItemId, number>>;

function cosineSimilarity(
  a: Map<ItemId, number>,
  b: Map<ItemId, number>
): number {
  let dot = 0;
  let normA = 0;
  let normB = 0;

  for (const [item, scoreA] of a) {
    const scoreB = b.get(item) ?? 0;
    dot += scoreA * scoreB;
    normA += scoreA * scoreA;
  }

  for (const [, scoreB] of b) {
    normB += scoreB * scoreB;
  }

  if (normA === 0 || normB === 0) return 0;
  return dot / (Math.sqrt(normA) * Math.sqrt(normB));
}

function userUserRecommend(
  targetUser: UserId,
  matrix: InteractionMatrix,
  k: number,
  topN: number
): ItemId[] {
  const targetVector = matrix.get(targetUser);
  if (!targetVector) return [];

  const targetSeen = new Set(targetVector.keys());

  // Compute similarity to all other users
  const similarities: Array<{ user: UserId; score: number }> = [];
  for (const [user, vector] of matrix) {
    if (user === targetUser) continue;
    similarities.push({ user, score: cosineSimilarity(targetVector, vector) });
  }

  similarities.sort((a, b) => b.score - a.score);
  const neighbors = similarities.slice(0, k);

  // Aggregate item scores from neighbors
  const itemScores = new Map<ItemId, number>();
  for (const { user, score: simScore } of neighbors) {
    const neighborVector = matrix.get(user)!;
    for (const [item, rating] of neighborVector) {
      if (targetSeen.has(item)) continue; // already seen
      itemScores.set(item, (itemScores.get(item) ?? 0) + simScore * rating);
    }
  }

  return [...itemScores.entries()]
    .sort((a, b) => b[1] - a[1])
    .slice(0, topN)
    .map(([item]) => item);
}

User-user breaks down when your user base is large. Computing pairwise similarities across millions of users is O(n^2) and the neighborhood becomes noisy as the corpus grows. The shift to item-item similarity addresses this.

Item-Item Filtering

Item similarity is more stable than user similarity: popular items accumulate many ratings and their neighborhood doesn’t shift as fast. You compute item similarity offline once, then serve recommendations at query time by looking up items the user has interacted with and retrieving their neighbors.

type ItemSimilarityIndex = Map<ItemId, Array<{ item: ItemId; score: number }>>;

function buildItemSimilarityIndex(
  matrix: InteractionMatrix,
  topK: number
): ItemSimilarityIndex {
  // Transpose: item -> { user -> rating }
  const itemVectors = new Map<ItemId, Map<UserId, number>>();
  for (const [user, items] of matrix) {
    for (const [item, rating] of items) {
      if (!itemVectors.has(item)) itemVectors.set(item, new Map());
      itemVectors.get(item)!.set(user, rating);
    }
  }

  const index: ItemSimilarityIndex = new Map();
  const itemList = [...itemVectors.keys()];

  for (const itemA of itemList) {
    const similarities: Array<{ item: ItemId; score: number }> = [];
    for (const itemB of itemList) {
      if (itemA === itemB) continue;
      const sim = cosineSimilarity(itemVectors.get(itemA)!, itemVectors.get(itemB)!);
      if (sim > 0) similarities.push({ item: itemB, score: sim });
    }
    similarities.sort((a, b) => b.score - a.score);
    index.set(itemA, similarities.slice(0, topK));
  }

  return index;
}

function itemItemRecommend(
  userId: UserId,
  matrix: InteractionMatrix,
  index: ItemSimilarityIndex,
  topN: number
): ItemId[] {
  const seen = matrix.get(userId);
  if (!seen) return [];

  const scores = new Map<ItemId, number>();
  for (const [seenItem, userRating] of seen) {
    const neighbors = index.get(seenItem) ?? [];
    for (const { item, score: simScore } of neighbors) {
      if (seen.has(item)) continue;
      scores.set(item, (scores.get(item) ?? 0) + simScore * userRating);
    }
  }

  return [...scores.entries()]
    .sort((a, b) => b[1] - a[1])
    .slice(0, topN)
    .map(([item]) => item);
}

Item-item scales much better because the item catalog grows slower than the user base. The similarity index is precomputed and served from a fast store. The online query path is just a few lookups plus a small aggregation.

Content-Based Filtering

Collaborative filtering needs interaction history. Content-based methods work from item features alone: categories, tags, descriptions, structured attributes. They compute a user profile as a weighted average of the features of items the user has interacted with, then score candidate items by feature overlap.

Content-based has a hard ceiling. It only surfaces items similar to what the user already consumed. It will never recommend the surprising thing the user didn’t know they wanted, which is where collaborative filtering earns its keep.

The practical use case for content-based is hybrid scoring: use it as a signal in a ranking model alongside collaborative scores, or as a fallback for new items that have no interaction history yet.

Embedding-Based Methods

Modern recommendation systems represent both users and items as dense vectors in a shared embedding space. Items close to a user vector are good candidates. This approach generalizes beyond explicit co-occurrence: two items can be similar because their content embeddings are close, even if no user has consumed both.

The two-stage architecture is standard:

1. Retrieval: Given a user vector, find the top-K nearest item vectors from a corpus of millions using approximate nearest neighbor (ANN) search. Latency target: under 20ms.
2. Ranking: Score the top-K candidates with a heavier model that incorporates context (time of day, device, session behavior). Latency target: under 30ms additional.

interface UserEmbedding {
  userId: UserId;
  vector: Float32Array;
  updatedAt: Date;
}

interface ItemEmbedding {
  itemId: ItemId;
  vector: Float32Array;
  metadata: Record<string, unknown>;
}

interface RetrievalResult {
  itemId: ItemId;
  score: number;
  metadata: Record<string, unknown>;
}

// This interface maps to a vector DB client (Pinecone, Weaviate, pgvector, etc.)
interface VectorIndex {
  query(vector: Float32Array, topK: number, filter?: Record<string, unknown>): Promise<RetrievalResult[]>;
  upsert(id: string, vector: Float32Array, metadata: Record<string, unknown>): Promise<void>;
}

async function retrieveCandidates(
  userEmbedding: UserEmbedding,
  index: VectorIndex,
  topK: number,
  filter?: Record<string, unknown>
): Promise<RetrievalResult[]> {
  return index.query(userEmbedding.vector, topK, filter);
}

interface RankingFeatures {
  userId: UserId;
  itemId: ItemId;
  retrievalScore: number;
  itemPopularityScore: number;
  userAffinityTags: string[];
  itemTags: string[];
  hourOfDay: number;
  deviceType: "mobile" | "desktop" | "tablet";
}

async function rankCandidates(
  candidates: RetrievalResult[],
  userId: UserId,
  context: { hourOfDay: number; deviceType: string },
  rankingModel: (features: RankingFeatures) => Promise<number>
): Promise<RetrievalResult[]> {
  const scored = await Promise.all(
    candidates.map(async (candidate) => {
      const features: RankingFeatures = {
        userId,
        itemId: candidate.itemId,
        retrievalScore: candidate.score,
        itemPopularityScore: (candidate.metadata.popularity as number) ?? 0,
        userAffinityTags: [], // fetched from user profile store
        itemTags: (candidate.metadata.tags as string[]) ?? [],
        hourOfDay: context.hourOfDay,
        deviceType: context.deviceType as RankingFeatures["deviceType"],
      };
      const score = await rankingModel(features);
      return { ...candidate, score };
    })
  );

  return scored.sort((a, b) => b.score - a.score);
}

Embedding quality depends entirely on training data. Matrix factorization (SVD, ALS) works for explicit rating data. Two-tower neural networks work well for implicit feedback (clicks, views, completions) at scale. If you have rich item content, sentence transformers on item descriptions give you a useful starting point for cold items.

The Cold Start Problem

Cold start is not a single problem. It’s three separate problems with different solutions.

New user, no history: You have no interaction data. Options, in order of increasing personalization quality:

• Serve popularity-ranked items filtered to the user’s declared context (geography, device, onboarding choices).
• Ask for explicit preferences during onboarding (genre, category, goal). Even 3-5 explicit signals let you bootstrap a content-based profile.
• Use contextual bandits: start with a prior (popularity), explore with a small epsilon, update fast as the first interactions arrive.

New item, no interactions: Collaborative filtering can’t score it. Content-based can. Use item content embeddings to find similar items in the existing catalog, then proxy the new item’s score through its nearest neighbors until it accumulates real interaction data.

Returning user, sparse history: Fewer than ~20 interactions means collaborative filtering is noisy. Use a blended score: weight content-based more heavily early, decay toward collaborative as history grows.

function blendScores(
  contentScore: number,
  collaborativeScore: number,
  interactionCount: number
): number {
  // Content-based weight decays from 1.0 to 0.1 as history grows
  const contentWeight = Math.max(0.1, 1 - interactionCount / 25);
  const collabWeight = 1 - contentWeight;
  return contentWeight * contentScore + collabWeight * collaborativeScore;
}

The specific threshold (25 interactions here) should be calibrated against your data. Plot ranking quality (NDCG or precision@K) against interaction count and find where collaborative filtering reliably beats the content-based baseline.

Real-Time vs. Batch Scoring

The right architecture depends on whether personalization signals change faster than your batch cadence.

Batch scoring: Precompute top-N recommendations for every user offline, store in a fast KV store, serve at sub-millisecond latency. Simple, cheap, scales horizontally. Staleness is the tradeoff: if a user just watched five videos in a row, their batch recommendations won’t reflect that session for hours.

Real-time scoring: Score candidates at query time using a live user state (session events, recent interactions). Captures in-session behavior. Requires low-latency feature retrieval, a fast inference path, and careful timeout handling.

In practice you run both. Batch handles the base personalization layer. Real-time handles the session context layer that re-ranks or filters the batch result.

interface SessionState {
  userId: UserId;
  recentItemIds: ItemId[]; // last N interactions in this session
  sessionStartedAt: Date;
}

interface RecommendationRequest {
  userId: UserId;
  session: SessionState;
  surface: "homepage" | "item-detail" | "search-results";
  limit: number;
}

async function serveRecommendations(
  req: RecommendationRequest,
  batchStore: { get: (userId: UserId) => Promise<ItemId[]> },
  sessionRanker: (items: ItemId[], session: SessionState) => Promise<ItemId[]>
): Promise<ItemId[]> {
  // Fetch batch candidates with a hard timeout
  const batchPromise = batchStore.get(req.userId);
  const timeoutPromise = new Promise<ItemId[]>((resolve) =>
    setTimeout(() => resolve([]), 50) // 50ms hard limit
  );

  const batchCandidates = await Promise.race([batchPromise, timeoutPromise]);

  if (batchCandidates.length === 0) {
    // Fallback: serve popularity-ranked items for this surface
    return getFallbackRecommendations(req.surface, req.limit);
  }

  // Re-rank with session context; filter already-seen items
  const seenSet = new Set(req.session.recentItemIds);
  const unseen = batchCandidates.filter((id) => !seenSet.has(id));

  const reranked = await sessionRanker(unseen, req.session);
  return reranked.slice(0, req.limit);
}

async function getFallbackRecommendations(
  surface: string,
  limit: number
): Promise<ItemId[]> {
  // Serve from a pre-cached popularity list, always available
  return [];
}

One timeout pattern worth enforcing: the batch store should never block the response path. If it takes more than 50ms, serve the fallback. A slow recommendation store should not make the page slow.

A/B Testing Recommendation Quality

Online metrics for recommendations are deceptive. Click-through rate (CTR) improves easily if you recommend popular items. That does not mean quality improved. The metrics that actually matter:

• Precision@K: Of the K items shown, what fraction did the user interact with?
• NDCG@K: Normalized discounted cumulative gain. Rewards surfacing the most relevant items higher in the list.
• Diversity: Are the K items diverse across categories, or did the algorithm pile everything into one cluster?
• Coverage: What fraction of the catalog gets recommended at all? Low coverage means the long tail never gets exposure.

Run experiments at the user level, not the request level. If the same user sees both treatment and control across requests, the control group is contaminated.

interface ExperimentAssignment {
  userId: UserId;
  experimentId: string;
  variant: "control" | "treatment";
  assignedAt: Date;
}

interface RecommendationEvent {
  userId: UserId;
  experimentId: string;
  variant: "control" | "treatment";
  itemId: ItemId;
  position: number; // 0-indexed rank
  surface: string;
  eventType: "impression" | "click" | "conversion";
  timestamp: Date;
}

function computeNDCG(
  rankedItems: ItemId[],
  relevantItems: Set<ItemId>,
  k: number
): number {
  const dcg = rankedItems
    .slice(0, k)
    .reduce((sum, item, idx) => {
      const relevance = relevantItems.has(item) ? 1 : 0;
      return sum + relevance / Math.log2(idx + 2); // log2(rank + 1)
    }, 0);

  // Ideal DCG: all relevant items at the top
  const idealCount = Math.min(relevantItems.size, k);
  const idcg = Array.from({ length: idealCount }, (_, i) =>
    1 / Math.log2(i + 2)
  ).reduce((a, b) => a + b, 0);

  return idcg === 0 ? 0 : dcg / idcg;
}

Offline evaluation (NDCG on held-out test data) predicts online performance imperfectly. Use offline metrics to eliminate bad candidates quickly, then A/B test the survivors. Don’t promote a model based on offline numbers alone.

Tradeoffs

Dimension	Collaborative Filtering	Content-Based	Embedding (Two-Tower)
Cold start (new user)	Fails without history	Works with profile	Needs user tower input
Cold start (new item)	Fails without interactions	Works immediately	Works if content available
Serendipity	High (cross-genre discovery)	Low (same-category bias)	Medium to high
Scalability	Item-item scales; user-user does not	Scales with catalog size	Requires ANN infrastructure
Explainability	”Users like you also liked…"	"Based on items you liked”	Black box by default
Online update latency	Batch rebuild	Fast (profile update)	Requires embedding refresh
Catalog sparsity	Degrades with sparse matrix	Not affected	Degrades if few interactions

Production Considerations

Embedding staleness: User embeddings generated from last week’s interactions won’t reflect this week’s behavior. Decide whether your model supports incremental updates or requires full retraining. Two-tower models can update the user tower without touching the item tower, which makes online embedding refresh tractable.

ANN index consistency: When you upsert new items into the vector index, the index rebuild or segment merge introduces a window where new items aren’t yet retrievable. Track index lag explicitly: timestamp the last full sync and alert when it exceeds your SLA.

Feature store latency: Real-time ranking needs user features (affinity tags, recency signals) at query time. A separate feature store with sub-10ms p99 read latency is worth the operational overhead. Fetching features from the user DB at ranking time is a reliability risk.

Diversity enforcement: Left unconstrained, ranking maximizes relevance per item independently. The result is a list of 10 nearly identical items. Apply a re-ranking step (maximal marginal relevance or determinantal point processes) to enforce category diversity. Even a simple heuristic helps: max 2 items per category in the top 10.

Feedback loop drift: If you only train on items you recommended, the model learns to recommend what it already recommended. Inject a small fraction of exploration items (epsilon-greedy or Thompson sampling) on every page load to capture counterfactual signal. Log exploration items distinctly so you can train on them without polluting your precision metrics.

Serving fallbacks: At every layer, have a fallback. Embedding query fails: fall back to item-item index. Item-item index unavailable: fall back to batch precomputed list. Batch list expired or missing: fall back to global popularity. The failure path should never surface an empty recommendations slot.

Where to Draw the Architecture Boundary

If you have fewer than 100k active users: item-item collaborative filtering with a batch rebuild cadence (hourly is fine) and a popularity fallback for cold start. Keep the stack simple. A Redis sorted set per user is adequate serving infrastructure at this scale.

If you have millions of users with diverse, rapidly-shifting tastes: the two-tower embedding approach with ANN retrieval and a real-time ranking layer is worth the operational cost. The infrastructure overhead (vector index, feature store, embedding pipeline, online scoring service) is only justified when simpler methods visibly underperform and you have the data volume to train good representations.

If you have a small catalog (under 10k items): skip ANN retrieval entirely. Score all items at ranking time. The compute is trivial and you eliminate the retrieval recall ceiling that ANN introduces.

The retrieval-then-rank pattern matters more than the specific algorithm at each stage. Precision in the retrieval step sets a ceiling on final quality: if the right item isn’t in the top-1000 candidates, no ranking model can surface it.

Recommendation quality is fundamentally a data problem before it’s an algorithm problem. Better interaction logging, better negative sampling, and better coverage of the long tail in your training set will outperform algorithm upgrades on the same data almost every time.