Designing a Recommendation Engine: Collaborative Filtering, Content-Based Ranking, and Real-Time Personalization at Scale

Recommendations look simple from the outside: show users things they might like. In practice, you are solving a moving-target optimization problem where the input space is enormous, feedback is sparse and noisy, latency requirements are tight, and the cost of a bad recommendation is invisible until it accumulates into churn.

Netflix estimates that over 80% of content streamed is driven by recommendations. Spotify’s Discover Weekly had a 40% save rate in its first launch week. These systems did not get there by running a SQL query against a ratings table. They run multi-stage pipelines that generate thousands of candidates, score them with learned models, re-rank for diversity and business rules, and serve results in under 100ms.

This guide walks you through how to build that kind of system from the ground up.

The Core Problem

The naive approach is user-item matrix factorization: build a giant matrix of every user-item interaction, factorize it, and find similar users or items. That works at 10,000 users and 5,000 items. It stops working at 50 million users and 20 million items because the matrix is too sparse to factorize meaningfully, too large to fit in memory, and too slow to recompute in real time.

Production recommendation systems decompose the problem into two distinct phases:

1. Candidate generation: Given a user, retrieve a small set (100-1000) of plausibly relevant items from a catalog of millions. Speed is the constraint here.
2. Ranking: Given the candidate set, score each item with a richer model using more features. Quality is the constraint here.

This two-stage architecture is the backbone of systems at YouTube, LinkedIn, and Pinterest. Get the boundary between these two stages wrong and you either have terrible recall (relevant items never make it to ranking) or you are running an expensive model over too many candidates.

Candidate Generation

There are three core strategies for generating candidates. Most production systems use all three in parallel.

Collaborative Filtering

Collaborative filtering exploits the structure of user behavior: users who behaved similarly in the past will likely want similar things in the future. The modern approach is not matrix factorization directly but learned embeddings.

You train a two-tower model: one tower encodes users, one encodes items. The training signal is implicit feedback (clicks, watches, purchases) cast as a classification or contrastive learning problem. After training, you have user and item embeddings in a shared latent space.

interface EmbeddingModel {
  getUserEmbedding(userId: string): Promise<Float32Array>;
  getItemEmbedding(itemId: string): Promise<Float32Array>;
}

interface CandidateGenerator {
  generate(userId: string, limit: number): Promise<string[]>;
}

class CollaborativeFilteringGenerator implements CandidateGenerator {
  constructor(
    private readonly embeddingModel: EmbeddingModel,
    private readonly annIndex: ANNIndex,
  ) {}

  async generate(userId: string, limit: number): Promise<string[]> {
    const userEmbedding = await this.embeddingModel.getUserEmbedding(userId);
    const neighbors = await this.annIndex.query(userEmbedding, limit * 2);

    // Filter out items the user has already interacted with
    const seen = await this.getSeenItems(userId);
    return neighbors
      .filter((itemId) => !seen.has(itemId))
      .slice(0, limit);
  }

  private async getSeenItems(userId: string): Promise<Set<string>> {
    // Fetch from interaction store (Redis or DynamoDB)
    return new Set();
  }
}

The index behind annIndex is an approximate nearest neighbor (ANN) structure. Exact nearest neighbor search over millions of vectors is O(n) per query. ANN trades a small recall penalty for O(log n) to O(1) query time. The two dominant approaches are:

• HNSW (Hierarchical Navigable Small World): Graph-based, high recall, high memory. Used in Qdrant, Weaviate, pgvector.
• IVF (Inverted File Index) + PQ (Product Quantization): Cluster-based, lower memory, slightly lower recall. Used in FAISS.

For most production systems, HNSW at ef_search=128 gives 95%+ recall with sub-10ms query latency over 10 million vectors. That is your target.

Content-Based Filtering

Content-based filtering matches item attributes to user preference profiles. If a user consistently engages with long-form technical articles about distributed systems, you build a preference vector from those attributes and retrieve similar items.

This approach has an important advantage over collaborative filtering: it does not require other users’ behavior to generate candidates. A new item can be recommended immediately as long as its attributes are indexed.

interface ItemAttributes {
  itemId: string;
  category: string[];
  tags: string[];
  contentEmbedding: Float32Array; // From a text/image encoder
  duration?: number;
  difficulty?: string;
}

interface UserPreferenceProfile {
  userId: string;
  preferenceVector: Float32Array; // Aggregated from recent interactions
  categoryWeights: Record<string, number>;
  updatedAt: Date;
}

class ContentBasedGenerator implements CandidateGenerator {
  constructor(
    private readonly profileStore: ProfileStore,
    private readonly contentIndex: ANNIndex,
  ) {}

  async generate(userId: string, limit: number): Promise<string[]> {
    const profile = await this.profileStore.getProfile(userId);

    if (!profile) {
      return this.getFallbackCandidates(limit);
    }

    // Blend the preference vector toward recent interactions
    const queryVector = this.buildQueryVector(profile);
    const candidates = await this.contentIndex.query(queryVector, limit * 2);

    return this.applyAttributeFilters(candidates, profile).slice(0, limit);
  }

  private buildQueryVector(profile: UserPreferenceProfile): Float32Array {
    // Exponential recency weighting: recent interactions count more
    return profile.preferenceVector;
  }

  private applyAttributeFilters(
    candidates: string[],
    profile: UserPreferenceProfile,
  ): string[] {
    // Boost candidates from high-weight categories
    return candidates;
  }

  private getFallbackCandidates(limit: number): Promise<string[]> {
    // Return editorial or trending items for unknown users
    return Promise.resolve([]);
  }
}

Hybrid Retrieval

In practice you merge candidates from multiple sources before ranking. Each generator contributes a pool of candidates and you deduplicate them. A simple merge looks like this:

class HybridCandidateGenerator implements CandidateGenerator {
  private readonly generators: Array<{ generator: CandidateGenerator; weight: number }>;

  constructor(generators: Array<{ generator: CandidateGenerator; weight: number }>) {
    this.generators = generators;
  }

  async generate(userId: string, limit: number): Promise<string[]> {
    const perSourceLimit = Math.ceil(limit * 1.5);

    const results = await Promise.all(
      this.generators.map(({ generator }) =>
        generator.generate(userId, perSourceLimit).catch(() => [] as string[]),
      ),
    );

    // Merge with deduplication, preserving source diversity
    const seen = new Set<string>();
    const merged: string[] = [];

    // Round-robin across sources to maintain diversity
    const maxLen = Math.max(...results.map((r) => r.length));
    for (let i = 0; i < maxLen && merged.length < limit; i++) {
      for (const candidates of results) {
        if (i < candidates.length && !seen.has(candidates[i])) {
          seen.add(candidates[i]);
          merged.push(candidates[i]);
        }
      }
    }

    return merged.slice(0, limit);
  }
}

Ranking

Once you have 200-1000 candidates, you score them with a richer model. The ranking model has access to features that are too expensive to compute at retrieval time:

• User context: current session activity, time of day, device type
• Item features: freshness, popularity decay, completion rate
• Cross features: historical user-item affinity scores, category preferences
• Contextual bandits or reward signals from recent A/B experiments

A common architecture is a gradient-boosted tree (XGBoost, LightGBM) for tabular features combined with learned embeddings for the user-item pair. The output is a score you use to sort the candidate list.

interface RankingFeatures {
  userId: string;
  itemId: string;
  userItemAffinityScore: number;
  itemPopularityScore: number; // Decayed over time
  categoryMatchScore: number;
  recencyScore: number;
  sessionContextScore: number;
}

interface RankedCandidate {
  itemId: string;
  score: number;
  features: RankingFeatures;
}

class RankingService {
  constructor(private readonly scoringModel: ScoringModel) {}

  async rank(userId: string, candidates: string[]): Promise<RankedCandidate[]> {
    const features = await this.buildFeatures(userId, candidates);

    const scored = await Promise.all(
      features.map(async (f) => ({
        itemId: f.itemId,
        score: await this.scoringModel.predict(f),
        features: f,
      })),
    );

    return scored.sort((a, b) => b.score - a.score);
  }

  private async buildFeatures(
    userId: string,
    candidates: string[],
  ): Promise<RankingFeatures[]> {
    // Batch-fetch all feature data in parallel
    const [userFeatures, itemFeatures, crossFeatures] = await Promise.all([
      this.fetchUserFeatures(userId),
      this.fetchItemFeatures(candidates),
      this.fetchCrossFeatures(userId, candidates),
    ]);

    return candidates.map((itemId) => ({
      userId,
      itemId,
      userItemAffinityScore: crossFeatures[itemId]?.affinity ?? 0,
      itemPopularityScore: itemFeatures[itemId]?.popularity ?? 0,
      categoryMatchScore: this.computeCategoryMatch(userFeatures, itemFeatures[itemId]),
      recencyScore: this.computeRecency(itemFeatures[itemId]?.publishedAt),
      sessionContextScore: userFeatures.sessionScore ?? 0,
    }));
  }

  private computeRecency(publishedAt?: Date): number {
    if (!publishedAt) return 0;
    const ageHours = (Date.now() - publishedAt.getTime()) / 3_600_000;
    return Math.exp(-ageHours / 72); // Half-life of ~72 hours
  }

  private computeCategoryMatch(userFeatures: unknown, itemFeature: unknown): number {
    return 0;
  }

  private fetchUserFeatures(userId: string): Promise<Record<string, unknown>> {
    return Promise.resolve({});
  }

  private fetchItemFeatures(
    itemIds: string[],
  ): Promise<Record<string, Record<string, unknown>>> {
    return Promise.resolve({});
  }

  private fetchCrossFeatures(
    userId: string,
    itemIds: string[],
  ): Promise<Record<string, Record<string, unknown>>> {
    return Promise.resolve({});
  }
}

After ranking, apply a re-ranking pass for diversity and business rules: deduplication by author or category, exclusion of recently seen items, slot allocation for promoted content, and fairness constraints.

Real-Time Feature Computation

The gap between offline training and online serving is where most recommendation quality is lost. A model trained on yesterday’s interaction data does not know what the user did in the last five minutes.

The solution is a feature store with two tiers:

• Batch features (offline): Precomputed daily or hourly, stored in a fast key-value store (Redis, DynamoDB). User embeddings, historical preferences, item popularity scores.
• Streaming features (near-real-time): Computed from a Kafka event stream with a processing window of seconds to minutes. Session activity, trending items, fresh interaction counts.

User preference profiles should be updated incrementally using exponential moving averages, not recomputed from scratch on every event. This makes the update O(1) instead of O(history_length):

function updatePreferenceVector(
  current: Float32Array,
  newItemEmbedding: Float32Array,
  alpha: number = 0.1, // Learning rate
): Float32Array {
  const updated = new Float32Array(current.length);
  for (let i = 0; i < current.length; i++) {
    updated[i] = (1 - alpha) * current[i] + alpha * newItemEmbedding[i];
  }
  return updated;
}

Cold Start

Cold start is the hardest practical problem in recommendation. You have two variants:

New user cold start: You have no interaction history. Options in priority order:

1. Onboarding flow: ask the user to pick a few topics or items they like. Even 3-5 explicit signals break cold start.
2. Context signals: device type, referral source, location, time of day. A user who arrives from a Python tutorial link probably wants Python content.
3. Popularity fallback: show globally or contextually trending items. Not personalized, but safe and often converts well.

New item cold start: The item has no engagement history so it will not appear in collaborative filtering results. Content-based retrieval handles this because the item’s embedding is derived from its attributes, not its interaction history. Pair this with a freshness boost in ranking to give new items early exposure. Measure their true quality via exploration slots (5-10% of impressions) and update their rankings once you accumulate signal.

Tradeoffs

Approach	Recall Quality	Cold Start	Latency	Explainability	When to Use
Collaborative filtering (ANN)	High for active users	Poor (needs history)	Low (index lookup)	Low	Core retrieval for established users
Content-based	Good for niche items	Excellent	Low (index lookup)	High	New items, sparse-history users
Matrix factorization (batch)	Good	Poor	High (offline only)	Medium	Offline precomputation, not serving
Two-tower neural	Highest for active users	Poor	Low after training	Low	Large catalogs with rich interaction data
Popularity / trending	Low (no personalization)	Excellent	Very low	High	New users, fallback, freshness injection
Hybrid pipeline	Best overall	Good with fallback	Medium	Low	Any production system above 10K users

A/B Testing Recommendations

Standard A/B testing does not map cleanly onto recommendations because:

• Recommendations are shown repeatedly, so novelty effects decay
• Users self-select into engagement: a heavy user sees more recommendations than a light user
• The treatment and control groups can have correlated behavior through shared item popularity

Use these design choices to get cleaner signals:

• User-level splits, not session-level: All sessions for a given user go to the same variant.
• Holdout sets: Designate 5-10% of users as a holdout that never receives experimental algorithms. Use them to measure long-term baseline drift.
• Interleaved evaluation: For fast iteration, use interleaved tests where both algorithms contribute to a single result list and you track which positions users click. This is dramatically more statistically efficient than A/B for ranking problems.
• Metrics that matter: Optimize for downstream business metrics (subscription conversion, 30-day retention), not just CTR. High-CTR recommendations can be clickbait that hurts long-term retention.

Production Considerations

Serving latency budget: Break down your p99 budget. If you need results in 100ms, allocate roughly: 10ms for candidate retrieval (ANN queries in parallel), 30ms for feature fetching (batch from Redis), 40ms for ranking model inference, 20ms for re-ranking and response serialization. Cache pre-ranked results per user with a 5-minute TTL for read-heavy endpoints.

Embedding freshness: User embeddings trained offline will drift away from reality as users’ tastes change. Retrain embeddings at least daily. For high-activity users, apply real-time correction by adding the session’s interaction signals as a delta to their stored embedding before querying the ANN index.

Index staleness: New items will not appear in ANN results until they are added to the index. Add items to the index as they are published, not in a daily batch. Most ANN libraries support incremental inserts without a full rebuild, though periodic rebuilds (weekly) improve index quality.

Popularity bias: Collaborative filtering inherently amplifies popular items. If you never correct for this, your top recommendations will converge on the same 1% of the catalog. Use inverse frequency weighting in training and apply catalog-coverage monitoring in production: if the top 100 recommended items account for 60%+ of all impressions, you have a popularity collapse problem.

Feedback loops: When your model’s recommendations influence what users click, and those clicks become training data, the model learns to recommend what it already recommended. Break this loop with random exploration slots, counterfactual logging (record what you did not show), and off-policy correction in training.

Monitoring: Track CTR, skip rate (recommendation shown but user scrolled past), completion rate (started but not finished), and return rate (user came back within 24 hours). A model with high CTR and low completion rate is recommending misleading thumbnails or titles. A model with low CTR but high completion rate might be excellent but suffering from a presentation problem.

Closing

The architecture here, two-stage retrieval and ranking with a feature store bridging offline training and online serving, is the same pattern used at scale by the largest recommendation systems in the world. The details differ by domain, but the core tension is always the same: you want the richest possible model with the freshest possible features, constrained by a latency budget that does not care how much compute you throw at it. Design with that constraint in mind from the start and you will make better choices at every layer.