Building an AI-Powered Security Operations Pipeline: Threat Detection, Alert Triage, and Automated Response for Startup Security Teams

Most startup security teams are small. Two or three engineers, maybe a part-time CISO. They are responsible for monitoring infrastructure that generates thousands of events per hour across cloud providers, application services, and network boundaries. The commercial SIEM and SOAR platforms exist, but they cost six figures annually and require dedicated staff to tune and operate.

The math does not work. Industry data consistently shows that 90% or more of security alerts are false positives. A three-person team cannot manually triage 500 alerts per day. The result is alert fatigue, missed real threats, and engineers who burn out within a year.

This article walks through building a security operations pipeline that uses ML for anomaly scoring, LLMs for alert correlation and triage, and lightweight automation for response. The goal is not to replace a SOC analyst. It is to reduce the number of alerts that need human attention from hundreds to a handful.

Event Ingestion and Normalization

Security events come from everywhere: AWS CloudTrail, GCP Audit Logs, application authentication logs, VPC flow logs, WAF logs, endpoint agents. Each source has its own schema, timestamp format, and severity model. Before you can do anything useful, you need a common event format.

The normalization layer maps source-specific fields into a unified schema. This is unglamorous work, but it determines the quality of everything downstream.

interface NormalizedSecurityEvent {
  id: string;
  timestamp: Date;
  source: string;
  sourceType: "cloud_audit" | "application" | "network" | "endpoint";
  actor: {
    id: string;
    type: "user" | "service" | "ip_address";
    metadata: Record<string, string>;
  };
  action: string;
  resource: {
    type: string;
    id: string;
    region?: string;
  };
  outcome: "success" | "failure" | "unknown";
  severity: "info" | "low" | "medium" | "high" | "critical";
  rawEvent: Record<string, unknown>;
}

type SourceNormalizer = (raw: Record<string, unknown>) => NormalizedSecurityEvent;

const normalizers: Record<string, SourceNormalizer> = {
  cloudtrail: (raw) => ({
    id: crypto.randomUUID(),
    timestamp: new Date(raw.eventTime as string),
    source: "aws_cloudtrail",
    sourceType: "cloud_audit",
    actor: {
      id: (raw.userIdentity as any)?.arn ?? "unknown",
      type: "user",
      metadata: {
        accountId: (raw.userIdentity as any)?.accountId ?? "",
        sourceIp: (raw.sourceIPAddress as string) ?? "",
      },
    },
    action: raw.eventName as string,
    resource: {
      type: (raw.resources as any)?.[0]?.type ?? raw.eventSource as string,
      id: (raw.resources as any)?.[0]?.ARN ?? "",
      region: raw.awsRegion as string,
    },
    outcome: raw.errorCode ? "failure" : "success",
    severity: classifyCloudTrailSeverity(raw),
    rawEvent: raw,
  }),

  application_auth: (raw) => ({
    id: crypto.randomUUID(),
    timestamp: new Date(raw.ts as string),
    source: "app_auth",
    sourceType: "application",
    actor: {
      id: (raw.userId as string) ?? (raw.ip as string),
      type: raw.userId ? "user" : "ip_address",
      metadata: {
        ip: (raw.ip as string) ?? "",
        userAgent: (raw.userAgent as string) ?? "",
      },
    },
    action: raw.event as string,
    resource: {
      type: "auth_endpoint",
      id: raw.endpoint as string,
    },
    outcome: raw.success ? "success" : "failure",
    severity: raw.success ? "info" : "medium",
    rawEvent: raw,
  }),
};

function classifyCloudTrailSeverity(
  raw: Record<string, unknown>
): NormalizedSecurityEvent["severity"] {
  const action = raw.eventName as string;
  const destructive = [
    "DeleteBucket", "DeleteTrail", "StopLogging",
    "PutBucketPolicy", "CreateAccessKey", "AttachUserPolicy",
  ];
  if (destructive.includes(action)) return "high";
  if (raw.errorCode === "AccessDenied") return "medium";
  return "info";
}

async function ingestEvent(
  source: string,
  raw: Record<string, unknown>
): Promise<NormalizedSecurityEvent> {
  const normalizer = normalizers[source];
  if (!normalizer) throw new Error(`No normalizer for source: ${source}`);

  const event = normalizer(raw);
  const enriched = await enrichEvent(event);
  return enriched;
}

async function enrichEvent(
  event: NormalizedSecurityEvent
): Promise<NormalizedSecurityEvent> {
  const ip = event.actor.metadata.sourceIp ?? event.actor.metadata.ip;
  if (ip) {
    const geoData = await lookupGeoIP(ip);
    const threatIntel = await checkThreatFeed(ip);
    event.actor.metadata.country = geoData.country;
    event.actor.metadata.threatScore = String(threatIntel.score);
    event.actor.metadata.knownMalicious = String(threatIntel.listed);
  }
  return event;
}

Two things matter here. First, the rawEvent field preserves the original payload. You will need it when an analyst investigates. Second, enrichment happens at ingest time. Geo-IP lookups and threat intelligence checks are cheap when done once per event. They become expensive when done repeatedly during investigation.

ML-Based Threat Scoring

Not every event deserves the same attention. A successful login from a known IP at 2pm on a Tuesday is routine. The same login from a new country at 3am after five failed attempts is not. Threat scoring assigns a numeric risk value to each event based on how much it deviates from established patterns.

The approach combines two techniques: statistical baselines for known patterns (login times, source IPs, API call frequencies) and isolation forest scoring for detecting outliers across multiple dimensions simultaneously.

interface ThreatScore {
  eventId: string;
  score: number; // 0-100
  factors: ScoringFactor[];
  timestamp: Date;
}

interface ScoringFactor {
  name: string;
  weight: number;
  value: number;
  description: string;
}

interface ActorBaseline {
  actorId: string;
  typicalHours: number[]; // hours of day with activity
  knownIPs: Set<string>;
  avgDailyActions: number;
  stdDevDailyActions: number;
  commonActions: Map<string, number>; // action -> frequency
  lastUpdated: Date;
}

class ThreatScoringPipeline {
  private baselines: Map<string, ActorBaseline>;
  private isolationForest: IsolationForest;

  constructor(
    private readonly baselineStore: BaselineStore,
    private readonly config: {
      newActorScore: number;
      timeAnomalyWeight: number;
      ipAnomalyWeight: number;
      frequencyAnomalyWeight: number;
      threatIntelWeight: number;
    }
  ) {
    this.baselines = new Map();
    this.isolationForest = new IsolationForest({
      numTrees: 100,
      sampleSize: 256,
    });
  }

  async scoreEvent(
    event: NormalizedSecurityEvent
  ): Promise<ThreatScore> {
    const baseline = await this.getBaseline(event.actor.id);
    const factors: ScoringFactor[] = [];

    if (!baseline) {
      return {
        eventId: event.id,
        score: this.config.newActorScore,
        factors: [{
          name: "new_actor",
          weight: 1,
          value: this.config.newActorScore,
          description: "No baseline exists for this actor",
        }],
        timestamp: new Date(),
      };
    }

    // Time-of-day anomaly
    const hour = event.timestamp.getUTCHours();
    const timeAnomaly = baseline.typicalHours.includes(hour) ? 0 : 1;
    factors.push({
      name: "time_anomaly",
      weight: this.config.timeAnomalyWeight,
      value: timeAnomaly,
      description: timeAnomaly
        ? `Activity at unusual hour ${hour} UTC`
        : "Normal activity hour",
    });

    // Source IP anomaly
    const ip =
      event.actor.metadata.sourceIp ?? event.actor.metadata.ip ?? "";
    const ipAnomaly = ip && !baseline.knownIPs.has(ip) ? 1 : 0;
    factors.push({
      name: "ip_anomaly",
      weight: this.config.ipAnomalyWeight,
      value: ipAnomaly,
      description: ipAnomaly
        ? `New source IP: ${ip}`
        : "Known source IP",
    });

    // Threat intelligence
    const threatScore = Number(
      event.actor.metadata.threatScore ?? "0"
    );
    factors.push({
      name: "threat_intel",
      weight: this.config.threatIntelWeight,
      value: threatScore / 100,
      description: `Threat intel score: ${threatScore}`,
    });

    // Frequency anomaly using z-score
    const recentCount = await this.getRecentActionCount(
      event.actor.id,
      24
    );
    const zScore =
      baseline.stdDevDailyActions > 0
        ? (recentCount - baseline.avgDailyActions) /
          baseline.stdDevDailyActions
        : 0;
    const freqAnomaly = Math.min(Math.max(zScore / 3, 0), 1);
    factors.push({
      name: "frequency_anomaly",
      weight: this.config.frequencyAnomalyWeight,
      value: freqAnomaly,
      description: `Action count ${recentCount} vs baseline ${baseline.avgDailyActions.toFixed(1)} (z=${zScore.toFixed(2)})`,
    });

    // Composite score
    const weightedSum = factors.reduce(
      (sum, f) => sum + f.weight * f.value,
      0
    );
    const totalWeight = factors.reduce((sum, f) => sum + f.weight, 0);
    const score = Math.round((weightedSum / totalWeight) * 100);

    return { eventId: event.id, score, factors, timestamp: new Date() };
  }

  private async getBaseline(
    actorId: string
  ): Promise<ActorBaseline | null> {
    if (this.baselines.has(actorId))
      return this.baselines.get(actorId)!;
    const stored = await this.baselineStore.get(actorId);
    if (stored) this.baselines.set(actorId, stored);
    return stored;
  }

  private async getRecentActionCount(
    actorId: string,
    hours: number
  ): Promise<number> {
    return this.baselineStore.countActions(actorId, hours);
  }
}

The z-score approach for frequency anomalies works well because it adapts to each actor’s normal behavior. An automated service account making 10,000 API calls per day is normal. A developer account doing the same thing is a red flag. The isolation forest catches multi-dimensional outliers that individual checks miss, like a combination of slightly unusual time, slightly unusual IP, and slightly unusual action that individually would not trigger anything.

Baselines need regular updates. A weekly rebuild from the trailing 30 days of data works for most environments. Shorter windows catch behavioral drift faster but are more susceptible to noise.

LLM-Powered Alert Triage

Once events are scored, the high-scoring ones become alerts. But raw alerts are noisy. A brute-force attack generates hundreds of individual “failed login” events. A compromised credential triggers alerts across multiple services within minutes. Human analysts naturally group these. An LLM can do the same thing.

The triage layer takes batches of recent alerts, asks the LLM to identify clusters, and produces consolidated incident summaries.

interface TriageResult {
  incidents: IncidentSummary[];
  suppressedAlerts: string[];
  escalations: string[];
}

interface IncidentSummary {
  id: string;
  title: string;
  severity: "low" | "medium" | "high" | "critical";
  relatedAlertIds: string[];
  summary: string;
  suggestedActions: string[];
  confidence: number;
}

class LLMAlertTriage {
  constructor(
    private readonly llmClient: LLMClient,
    private readonly alertStore: AlertStore
  ) {}

  async triageAlertBatch(
    alerts: Array<ThreatScore & { event: NormalizedSecurityEvent }>
  ): Promise<TriageResult> {
    const alertSummaries = alerts.map((a) => ({
      id: a.eventId,
      score: a.score,
      actor: a.event.actor.id,
      action: a.event.action,
      resource: `${a.event.resource.type}/${a.event.resource.id}`,
      outcome: a.event.outcome,
      source: a.event.source,
      factors: a.factors
        .filter((f) => f.value > 0.3)
        .map((f) => f.description),
      timestamp: a.event.timestamp.toISOString(),
    }));

    const prompt = `You are a security operations analyst. Analyze these security alerts and:
1. Group related alerts into incidents (e.g., multiple failed logins from same actor, privilege escalation sequences)
2. For each incident group, provide a title, severity assessment, summary, and suggested response actions
3. Identify alerts that are likely false positives and can be suppressed
4. Flag any alerts that need immediate human escalation

Alerts:
${JSON.stringify(alertSummaries, null, 2)}

Respond in JSON matching this schema:
{
  "incidents": [{
    "title": string,
    "severity": string,
    "relatedAlertIds": string[],
    "summary": string,
    "suggestedActions": string[],
    "confidence": number
  }],
  "suppressedAlerts": string[],
  "escalations": string[]
}`;

    const response = await this.llmClient.complete({
      model: "gpt-4o",
      messages: [{ role: "user", content: prompt }],
      response_format: { type: "json_object" },
      temperature: 0.1,
    });

    const parsed = JSON.parse(response.content) as TriageResult;

    // Add generated IDs to incidents
    parsed.incidents = parsed.incidents.map((incident) => ({
      ...incident,
      id: crypto.randomUUID(),
    }));

    return parsed;
  }
}

A few notes on this approach. Temperature is set low (0.1) because you want consistent, deterministic triage decisions. The prompt deliberately avoids asking the LLM to make binary “malicious or not” decisions. Instead it groups, summarizes, and suggests. The human makes the final call.

Batching matters. Processing alerts one at a time removes the LLM’s ability to spot correlations. A five-minute window works well for most environments. High-severity alerts (score above 90) should bypass batching and go straight to escalation.

Automated Response Workflows

For high-confidence threats, waiting for human approval is too slow. A compromised credential being used to exfiltrate data needs an immediate response. The SOAR-lite pattern defines automated actions gated by confidence thresholds.

interface ResponseAction {
  type:
    | "block_ip"
    | "disable_account"
    | "isolate_host"
    | "revoke_sessions"
    | "notify";
  target: string;
  reason: string;
  reversible: boolean;
}

interface ResponsePolicy {
  condition: (
    incident: IncidentSummary,
    score: number
  ) => boolean;
  actions: (incident: IncidentSummary) => ResponseAction[];
  requiresApproval: boolean;
}

const responsePolicies: ResponsePolicy[] = [
  {
    // Auto-block known malicious IPs hitting auth endpoints
    condition: (incident, score) =>
      score >= 85 &&
      incident.title.toLowerCase().includes("brute force"),
    actions: (incident) => [
      {
        type: "block_ip",
        target: extractIP(incident),
        reason: "Automated: brute force detected",
        reversible: true,
      },
      {
        type: "notify",
        target: "security-channel",
        reason: incident.summary,
        reversible: false,
      },
    ],
    requiresApproval: false,
  },
  {
    // Disable compromised accounts but require approval
    condition: (incident, score) =>
      score >= 75 && incident.severity === "critical",
    actions: (incident) => [
      {
        type: "disable_account",
        target: extractActor(incident),
        reason: "Automated: suspected compromise",
        reversible: true,
      },
      {
        type: "revoke_sessions",
        target: extractActor(incident),
        reason: "Automated: session cleanup",
        reversible: false,
      },
    ],
    requiresApproval: true,
  },
];

class ResponseOrchestrator {
  constructor(
    private readonly executor: ActionExecutor,
    private readonly approvalQueue: ApprovalQueue,
    private readonly auditLog: AuditLogger
  ) {}

  async executeResponse(
    incident: IncidentSummary,
    score: number
  ): Promise<void> {
    const matchingPolicies = responsePolicies.filter((p) =>
      p.condition(incident, score)
    );

    for (const policy of matchingPolicies) {
      const actions = policy.actions(incident);

      if (policy.requiresApproval) {
        await this.approvalQueue.enqueue({
          incidentId: incident.id,
          actions,
          timeout: 15 * 60 * 1000, // 15 minute approval window
          fallbackAction: "escalate",
        });
        await this.auditLog.record({
          type: "response_pending_approval",
          incidentId: incident.id,
          actions: actions.map((a) => a.type),
        });
      } else {
        for (const action of actions) {
          await this.executor.execute(action);
          await this.auditLog.record({
            type: "response_executed",
            incidentId: incident.id,
            action: action.type,
            target: action.target,
          });
        }
      }
    }
  }
}

Every automated action must be reversible where possible and audited always. The reversible flag on actions is not decorative. When a false positive triggers an auto-block, you need a fast path to undo it. The audit log is non-negotiable. It is the only way to reconstruct what happened during a post-incident review.

Human-in-the-Loop Escalation

The approval workflow handles the middle ground: incidents that are serious enough to act on but not confident enough for full automation. The key design decision is what happens when no human responds within the timeout window.

There are three options: auto-execute (dangerous for account disabling), auto-suppress (dangerous for real threats), or escalate to a broader group. Escalation is the safest default. A 15-minute window before escalation gives the on-call engineer time to review without creating an unsafe gap.

Confidence thresholds should be tuned based on your environment’s false positive rate. Start conservative: high thresholds for automation, low thresholds for human review. Adjust downward as you build trust in the system. A reasonable starting point:

• Score 0-40: log and close automatically. Review a random sample weekly.
• Score 40-70: route to a low-priority review queue. Analyst reviews within 4 hours.
• Score 70-85: route to the on-call queue. Analyst reviews within 30 minutes.
• Score 85-100: trigger automated response policies. Notify on-call immediately.

The feedback loop matters here. When an analyst marks an alert as a false positive, that signal should feed back into the scoring pipeline. Without this, the system never improves. Store every analyst decision with the original event data and anomaly scores. Use this corpus to retrain baselines monthly and to evaluate whether your thresholds are calibrated correctly.

Tradeoffs Comparison

Approach	Alert Volume Reduction	Response Latency	False Positive Risk	Engineering Effort
Rule-based only	30-50%	Milliseconds	High (rigid rules)	Low
ML scoring + rules	60-75%	Under 1 second	Medium	Medium
ML + LLM triage	80-90%	30-60 seconds (LLM latency)	Low to medium	Medium to high
ML + LLM + auto-response	90%+	Seconds for auto, minutes for approval	Low (with tuning)	High
Commercial SOAR platform	85-95%	Varies	Low	Low engineering, high cost

The ML + LLM triage combination hits a reasonable balance for small teams. It gets you to 80-90% alert reduction with manageable engineering effort. Adding automated response on top requires more careful testing but handles the remaining cases where speed matters. The commercial SOAR path trades engineering effort for vendor cost and lock-in. For teams under five engineers, the build path often wins because it avoids the operational overhead of maintaining a complex vendor integration.

Production Considerations

Event ordering and deduplication. Distributed sources deliver events out of order and sometimes more than once. Use event IDs for deduplication and accept that your triage windows will have some jitter. Exactly-once processing is not worth the complexity for security events.

LLM reliability. The triage LLM will occasionally return malformed JSON or hallucinate alert IDs that do not exist. Validate all LLM output against the actual alert IDs in the batch. Implement retry with exponential backoff and fall back to rule-based grouping if the LLM is unavailable.

Baseline cold start. New environments have no behavioral baselines. Run the system in observation mode for two to four weeks before enabling automation. During this period, all alerts go to humans, and the system builds its baseline data.

Cost. LLM API calls on alert batches every five minutes add up. At roughly $0.01 per batch with 288 batches per day, that is about $90/month. Acceptable for most teams, but monitor it. Prompt size grows with alert volume, and you may need to truncate or sample within large batches.

Testing automated responses. Never test auto-block or auto-disable against production accounts without a circuit breaker. Implement a dry-run mode that logs what would have happened without executing. Run dry-run for at least two weeks before enabling live responses. Add a rate limiter on automated actions: if the system tries to block more than 50 IPs in an hour, halt automation and page the on-call engineer.

Regulatory and compliance. Some industries require human review of all security decisions. Check your compliance requirements before enabling fully automated response. The audit log helps demonstrate due diligence, but it may not satisfy all regulatory frameworks.

The pipeline described here will not replace experienced security analysts. What it does is give a small team leverage. Instead of drowning in hundreds of alerts per day and missing the few that matter, they review a manageable set of consolidated incidents with context, suggested actions, and confidence scores. That is a workload a three-person team can handle without burning out.