Hardening LLM Clustering Against Invalid IDs, Stale Assignments, and Incomplete Output

Treating Model-Generated IDs as Untrusted Input

I built a weekly clustering mutation that asks an LLM to group timeline events into story arcs, then saves those assignments to the database. Its first run against real data failed with a Prisma update error: one of the records targeted by an assignment did not exist. The model had returned a plausible-looking event ID that wasn't among the events I had loaded.

That failure made the boundary clear. The model could propose groups, but it couldn't decide which database records were eligible for modification. Valid JSON wasn't enough. A response could have the right structure and still contain invented references.

The original single-record update made this disruptive because it throws when no matching row exists. One invalid ID could stop the mutation before it processed the remaining valid assignments. The exception exposed the problem, but it was too brittle to serve as the only defense against model-generated input.

I separated two checks: whether a returned ID belonged to the loaded batch, and whether its row still existed when the application tried to write the assignment. The first limits the model to records supplied for this operation. The second accounts for database state changing after the load. Neither tells us whether the proposed grouping makes sense.

Filtering IDs and Tolerating Zero-Match Updates

I added a membership check using a set of IDs from the events already loaded into memory. Any returned ID outside that set is skipped before assignment writes. This is deliberately narrower than checking whether the ID exists somewhere in the database. A real event outside the selected batch shouldn't become eligible just because the model named it.

I also replaced single-record updates with updateMany. If no row matches, updateMany can affect zero rows without throwing. The membership filter rejects references outside the batch; the write operation tolerates a row no longer matching when persistence occurs. Other database errors can still throw.

This policy allows partial success. I chose to keep valid assignments rather than reject the entire response over one invalid ID. Rejecting the batch would make mismatches harder to overlook, but it would also discard usable work. For rerunnable, derived clustering state, partial progress was the chosen tradeoff. It doesn't mean every mutation that completes has done useful work.

I tightened the prompt to require exact matches to the supplied IDs as well. That instruction didn't eliminate invalid references, so it remains guidance, not enforcement. An output grammar could constrain response shape, but it wouldn't prove that an ID belongs to the batch.

These safeguards are implemented. The available evidence, however, does not include a separate post-change test showing both invalid-ID rejection and a loaded row disappearing before its assignment is written. Those checks remain distinct from the original failure that prompted the changes.

Resetting Weekly Assignments and Exposing the Failure Window

Regeneration revealed a separate problem. Events from an earlier clustering pass still held their previous arc assignments, so a query for unassigned events could return nothing. I observed a message saying there were no unassigned events even though the operation was supposed to regroup the week.

I changed the regeneration path to clear assignment foreign keys on events in the target week before clustering them again. That makes the full selected week available instead of treating the old groups as fixed and processing only leftovers. Clustering only unassigned events is still a reasonable option, but it is incremental assignment rather than full regeneration.

Clearing first creates a destructive failure window. If the model request fails, parsing yields nothing, or persistence stops afterward, the cleared events can remain unassigned. The available implementation details don't establish an atomic clear-and-replace operation, rollback behavior, or a lock coordinating concurrent runs. I can't describe regeneration as preserving the previous usable result until a replacement succeeds.

Concurrent regeneration is unresolved too. Two runs targeting the same week could clear or replace assignments while the other is working. Staging and validating proposed assignments before a short replacement transaction would reduce the early-reset risk. Serializing regeneration for the same week could address overlapping runs. Neither is a feature I implemented or verified here. Moving the clear later would also require a policy for events changed or deleted while the model was working.

Selecting Weekly Arcs Through Their Events

Clearing assignments exposed a read-side issue. Arc records from earlier groupings could remain in the database without attached events and still appear in the weekly view. Changing the assignment writes didn't remove those records or make readers ignore them.

I traced the weekly display behavior to a query that didn't require an arc to have events in the requested week. I changed it to include an arc only when at least one attached event falls within that week. The event relationship now determines whether an arc belongs in the view; the arc record's existence alone is not enough.

This matters even when an arc isn't empty overall. An arc with events elsewhere shouldn't appear in a particular week's results simply because it exists. The filter checks relevance to the requested week, not just whether the arc has any events.

Older data also produced empty-looking arcs through broader queries for arcs associated with timeline events, so I adjusted those reads too. Filtering reads was a narrower change than deleting historical arc records. It changed what the timeline selected without defining a cleanup policy for stored records.

Unused arc rows can therefore remain in the database, and other readers still need appropriate selection rules. The query changes are documented, but I don't have a separate post-change result showing every affected view with both legacy arcs and arcs emptied by regeneration.

Distinguishing Suspected Truncation From Other Parse Failures

I also observed a mutation that completed with zero assignments and left the UI unchanged. The existing logs didn't explain why. Unlike the invalid-ID failure, this path produced no usable parsed assignments for the membership filter or write operation to process.

The working diagnosis was that the response allowance was too small for the batch and the model's output had been truncated. But the available evidence includes neither a provider length-stop indicator nor a captured response visibly ending mid-JSON. Truncation is therefore suspected, not established. Malformed JSON or another response-handling failure could also explain the parsing failure.

I added logging for parse failures and zero-assignment outcomes. That distinguishes useful cases, but it doesn't prove why parsing failed. Diagnosing truncation would require evidence such as the provider's completion reason, token usage, or an incomplete returned structure. I haven't established that those details are captured.

Zero assignments can also mean several different things: the input batch was empty, a nonempty batch produced output that couldn't be parsed, every returned ID was rejected, or every write matched zero rows. These outcomes can look similar in the UI but call for different remedies. The added logging makes two of those points more visible; it doesn't establish complete outcome accounting or a successful rerun.

Sizing Input and Output Budgets Separately

I made two token-related changes: trimming event summaries before the model call and scaling maxTokens with batch size instead of using a fixed response ceiling. They address different limits and aren't equivalent protections against truncation.

Shorter summaries reduce input-token use. That can leave more room in a shared context window, but it doesn't directly reduce how many assignment IDs the response must contain. If every event still needs a returned assignment, the output must accommodate those identifiers and their surrounding structure. Trimming can also remove context that helps distinguish events. I haven't measured its effect on grouping quality.

Scaling the output allowance with batch size reflects expected growth in assignment output better than a fixed ceiling. Event count is still only a proxy, though. Response size also depends on identifier lengths, grouping structure, and any additional fields the model returns. The budget needs room for more than the IDs alone.

The available implementation description establishes proportional scaling, but not a defensible exact multiplier, fixed overhead, or cap. I can't derive a capacity bound from it. The model's output limit and total context limit still apply; requesting a larger allowance cannot override either.

I haven't established an explicit reject-or-split path for batches that won't fit. Smaller batches are an option, but related events split across calls may not be grouped together. Further summary trimming keeps the work in one call at the cost of context. Neither is demonstrated here as the implemented overflow policy.

Separating Implemented Safeguards From Verified Recovery

The implementation now includes a loaded-ID filter, assignment updates that tolerate zero matches, pre-clearing for regeneration, relationship-based arc selection, shorter input summaries, a batch-scaled response allowance, and additional failure logging. Each addresses a different part of the pipeline: eligible records, rows present at write time, regeneration scope, read relevance, or response handling.

I observed the original missing-record error, regeneration blocked by existing assignments, empty arcs appearing in read queries, and a zero-assignment run that the existing logs couldn't explain. Those observations support the changes. They don't establish that the revised pipeline completed successfully end to end.

I haven't presented a verified rerun showing a previously problematic batch parsing successfully, a completed regeneration restoring the week's assignments, or every affected read excluding empty arcs. I also haven't shown a controlled test in which a loaded row disappears before persistence. The expected behavior of the selected operations doesn't replace those checks.

Verification should test these failures separately. An out-of-batch ID should be rejected without losing valid assignments. A row missing at write time should not trigger the former missing-record exception. A failed regeneration should make clear what state remains after assignments are cleared. Read checks should cover both empty arcs and arcs whose events fall only outside the requested week. Response checks should distinguish incomplete output from other parse failures.

These are implemented safeguards with known recovery and verification gaps, not a demonstrated guarantee of resilient clustering at the observed batch sizes or larger ones.

Filtering hallucinated IDs and tolerating missing rows during clustering writes

interface TimelineEvent {
  id: string;
  summary: string;
}

interface ClusterAssignment {
  eventId: string;
  arcId: string;
}

interface UpdateManyResult {
  count: number;
}

interface EventStore {
  updateMany(args: {
    where: { id: string };
    data: { arcId: string };
  }): Promise<UpdateManyResult>;
}

async function callClusterModel(
  events: TimelineEvent[],
  maxTokens: number
): Promise<ClusterAssignment[]> {
  // Illustrative stand-in for an LLM call that may hallucinate IDs
  // or return truncated/malformed JSON.
  throw new Error("not implemented");
}

async function clusterWeek(
  store: EventStore,
  events: TimelineEvent[]
): Promise<{ applied: number; rejected: number }> {
  const loadedIds = new Set(events.map((e) => e.id));

  // Scale output budget with batch size rather than a fixed ceiling.
  const maxTokens = 256 + events.length * 40;

  let assignments: ClusterAssignment[];
  try {
    assignments = await callClusterModel(events, maxTokens);
  } catch {
    // Parse failure or suspected truncation; treat as zero assignments.
    return { applied: 0, rejected: 0 };
  }

  let applied = 0;
  let rejected = 0;

  for (const assignment of assignments) {
    if (!loadedIds.has(assignment.eventId)) {
      // Reject IDs the model invented or that fall outside this batch.
      rejected++;
      continue;
    }

    // updateMany tolerates a row that vanished between load and write,
    // instead of throwing like a single-record update would.
    const result = await store.updateMany({
      where: { id: assignment.eventId },
      data: { arcId: assignment.arcId },
    });

    if (result.count > 0) {
      applied++;
    } else {
      rejected++;
    }
  }

  return { applied, rejected };
}

This maps to the article's core reliability pattern: the model's returned eventIds are treated as untrusted, so each assignment is checked against loadedIds (the set of IDs actually loaded for this batch) before any write is attempted, and persistence uses an updateMany-style call that reports a zero-match count instead of throwing when a row no longer exists. The maxTokens scaling with events.length illustrates the article's point about sizing the output budget proportionally to batch size rather than using a fixed ceiling, and the catch block stands in for distinguishing a zero-assignment outcome (suspected truncation or parse failure) from a batch that genuinely had no work. The TimelineEvent, ClusterAssignment, EventStore interface, and callClusterModel function are illustrative scaffolding to make the filtering and write logic coherent, not a real schema or API. A production system would need to log why assignments were rejected or empty (invalid ID vs. zero-match write vs. parse failure) and capture provider-level signals like a finish/stop reason or token usage to confirm truncation rather than inferring it, which this snippet omits.