Back to the blog

Decoupling Weekly GraphQL Processing with SQS and Bounded Output Polling

Move weekly processing from a blocking GraphQL mutation to an SQS-backed Lambda worker, then observe output with bounded client polling and a pre-enqueue fingerprint baseline.

Moving Weekly Processing Out of the Mutation

I moved weekly processing out of a blocking GraphQL mutation and into an SQS-backed Lambda worker. The work generates arcs and scripts from underlying data. The web application now calls an enqueue mutation, then makes separate GraphQL fetches to look for changed output instead of keeping the original mutation open until processing finishes.

This separates accepting a request, executing it, and observing its output. I updated the GraphQL types and resolvers, added enums, removed methods that no longer fit the asynchronous flow, and switched the client to the enqueue path. The client change matters: adding a worker would not remove the blocking path if the application still called the original processing operation.

A successful enqueue response now means the request was accepted for asynchronous processing. It does not mean the arcs and scripts are ready. The expensive work runs outside the original mutation's execution window, while the client shows a waiting state and checks for output changes.

Keeping the work inside the mutation would offer a simpler contract: the response could represent completed processing. With the queue, the request can end sooner, but accepted work may not yet have visible output. Enqueue failures and later execution failures also need separate handling. The benefit is that separation—not an established improvement in processing speed or a guarantee that retries are safe.

Connecting SQS to the Worker Lambda

I added an SQS queue, a dedicated Lambda worker, and build configuration to package the queue-processing functionality for deployment. The path is straightforward: the GraphQL mutation submits a processing request to SQS, and the worker consumes it to perform the weekly work.

The packaging changes are part of that path. A queue trigger cannot run functionality that is missing from the deployed worker artifact. The resolver needs to submit the request, and the separately deployed consumer needs to contain the code that executes it.

SQS supports this producer-to-consumer handoff without requiring a workflow with explicit stages and transitions. An orchestrator could address requirements such as coordinated steps or persisted workflow progress, but the available implementation description does not establish those requirements. Starting an unfinished background task inside the resolver would not provide the same separation: a Lambda environment can be frozen or terminated after its handler returns.

The worker still has an execution limit. Moving the work does not give an invocation unlimited time. Repeated enqueue actions or repeated message delivery can also cause repeated processing unless the consumer guards against it. The supplied record establishes neither an idempotency mechanism nor a dead-letter or redrive policy, so I cannot describe duplicate handling or failure recovery as implemented guarantees.

Taking the Baseline Before Enqueueing

I changed the client to capture a baseline before submitting the enqueue mutation. It compares later response fingerprints with that starting snapshot. Taking the baseline first avoids a race: if the worker updated the data before a post-enqueue baseline was captured, the new output could become the baseline rather than a detectable change.

At the component level, the path is clear. The client captures response data and submits a weekly processing request through GraphQL. SQS carries the request to the worker, which generates arcs and scripts that the client can later observe through GraphQL. The supplied material does not show the enqueue payload fields, the worker's scope-selection logic or write targets, or the client's query arguments and selected fields. I therefore cannot identify a concrete week identifier or trace individual fields from the request to its output.

That missing contract matters. A baseline records what the client previously saw; it does not identify a processing request. The producer's scope, the worker's output scope, and the client's query scope must agree. Otherwise, completed work could remain invisible to the polling query, or another request could change the same observed data.

Enqueue acceptance and output observation remain separate. If enqueueing fails, there is no confirmed handoff. After it succeeds, subsequent fetches observe data—not the internal execution state of that particular request.

Stopping When the Fingerprint Changes

I built the polling comparison around a fingerprint of the response data. The client compares each later fingerprint with the pre-enqueue baseline. It keeps checking while they match and stops when one differs, subject to the attempt limit. The stop signal is a change from the starting snapshot, not a completion response from the worker.

The supplied evidence does not show which arc or script fields contribute to the fingerprint, how they are serialized, whether a hash function is used, or what normalization rules apply. Without those details, I cannot reconstruct an evidence-backed before-and-after example. What is supported is the comparison rule: an equal fingerprint allows polling to continue; a different one ends it.

Changed output is not necessarily completed work. An intermediate write or an unrelated edit to included data could change the fingerprint before the requested processing finishes. Successful regeneration could also produce identical output and leave the fingerprint unchanged. Depending on the implementation, array order, omitted versus null values, and changing metadata may affect equality too.

A persisted job identifier and status record would answer a different question: what happened to this request? That approach requires lifecycle storage, worker status updates, and API support. Output polling instead observes the data the client already fetches. The record does not establish why it was chosen over explicit status tracking. Its tradeoff is a narrower integration with a weaker completion signal.

Bounding Polling to 120 Five-Second Checks

I configured polling at five-second intervals with a maximum of 120 attempts—a nominal observation window of about ten minutes. That is a limit on automatic checking, not a processing-time target, a worker deadline, or evidence that jobs finish within ten minutes.

The supplied description does not show the scheduler, when the first fetch occurs, or when the attempt counter advances. It also does not establish whether failed fetches consume attempts, whether requests can overlap, or how an active cycle is cancelled or replaced. The interval and cap are configured; their runtime enforcement has not been demonstrated here.

Those details affect the bound. If the next check is scheduled after a fetch settles, request duration adds to the elapsed observation period. A fixed timer can allow requests to overlap when responses are slow. Neither behavior can be attributed to this implementation from the available material. An attempt cap also bounds repeated failures only if the counting and error paths enforce it.

Navigation, component teardown, or another enqueue action can make an existing polling cycle obsolete. A late response from that cycle should not control a newer one. The record does not establish cancellation, cycle replacement, or stale-response protection, so the supported claim is the configured polling policy—not verified timing or concurrency behavior.

Describing the Waiting State Accurately

I updated the queue banners to say that the client would check every five seconds for updates. That describes what the interface is doing without implying visibility into worker progress: it periodically fetches data and looks for a difference from its baseline.

The attempt limit also limits what the interface can conclude. If polling reaches the cap without detecting a difference, automatic observation has ended without seeing changed output. That does not establish worker failure or justify automatically submitting the work again. Resubmission could cause duplicate processing, especially because no idempotency guarantee is established.

Polling has a transport cost as well. Unchanged responses still consume API requests, and a published update remains unseen until a later successful fetch. WebSockets or server-sent events could reduce repeated fetching, but they bring connection-lifecycle and event-delivery responsibilities. The processing side would also need to publish an appropriate event. A different transport alone would not correlate a request with its output or define when all output is complete.

The implementation combines queue-based execution with bounded output observation. The queue separates the work from the initiating request, the polling policy limits automatic checking, and the banner explains that checking to the user. Keeping those roles distinct avoids implying progress information the design does not provide.

What Still Needs Verification

This is an implementation walkthrough, not a verified deployment report. The supplied record supports the enqueue path, SQS worker, packaging changes, client integration, fingerprint comparison, polling settings, and banner updates. It contains no test results or runtime observations demonstrating them together. I have not independently verified deployed queue acceptance, worker execution, or either polling stop condition.

Verification should start with the contracts missing from this account: the enqueue message fields, the worker's scope selection and writes, the client's query scope, and the fingerprint's inputs and representation. Those details are needed for a meaningful before-and-after example. They should establish that the worker produces the requested week's output and that the client includes it in the comparison.

The handoff then needs runtime checks. The mutation should return after queue acceptance while processing is still pending, and an enqueue failure should be surfaced rather than presented as accepted work. Separate deployment checks should demonstrate that the queue invokes the packaged worker and that the worker publishes the expected output. A successful build proves neither.

Controlled client responses can test observation independently. Matching fingerprints should keep polling active until the configured bound; a differing fingerprint should stop it. Timer-controlled checks should establish first-fetch timing, the exact attempt count, scheduling behavior, failed-fetch handling, and what happens with slow or overlapping requests. Navigation, repeated enqueue actions, and late responses should test whether obsolete cycles remain active.

Integration checks should also cover identical regenerated output, partial publication, unrelated edits, and message redelivery. These cases test both the limits of fingerprint-based observation and the safety of repeated execution. Until that evidence is available, the supported outcome is an implemented asynchronous handoff with configured bounded polling—not demonstrated end-to-end completion tracking.

Baseline Capture and Bounded Fingerprint Polling


interface WeeklyOutput {
  arcs: string[];
  scripts: string[];
}

interface PollConfig {
  intervalMs: number;
  maxAttempts: number;
}

function computeFingerprint(data: WeeklyOutput): string {
  // Illustrative only: real implementation would define exact
  // field selection, serialization, and normalization rules.
  return JSON.stringify({ arcs: data.arcs, scripts: data.scripts });
}

async function fetchWeeklyOutput(): Promise<WeeklyOutput> {
  // Stands in for a GraphQL fetch of the client's query scope.
  throw new Error("not implemented");
}

async function enqueueWeeklyProcessing(): Promise<void> {
  // Stands in for the enqueue mutation that submits to SQS.
}

type PollOutcome = { changed: true; data: WeeklyOutput } | { changed: false };

async function pollUntilFingerprintChanges(
  baseline: string,
  config: PollConfig
): Promise<PollOutcome> {
  for (let attempt = 0; attempt < config.maxAttempts; attempt++) {
    await new Promise((resolve) => setTimeout(resolve, config.intervalMs));
    const current = await fetchWeeklyOutput();
    if (computeFingerprint(current) !== baseline) {
      return { changed: true, data: current };
    }
  }
  return { changed: false };
}

async function requestWeeklyProcessing(): Promise<PollOutcome> {
  const initial = await fetchWeeklyOutput();
  const baseline = computeFingerprint(initial);

  await enqueueWeeklyProcessing();

  return pollUntilFingerprintChanges(baseline, {
    intervalMs: 5000,
    maxAttempts: 120,
  });
}

This mirrors the article's central decision: capture a fingerprint baseline before calling the enqueue mutation, then poll at a fixed interval up to a bounded attempt count, stopping as soon as the fingerprint diverges rather than waiting on any worker-reported completion signal. The computeFingerprint, fetchWeeklyOutput, and enqueueWeeklyProcessing functions are illustrative stand-ins for the real GraphQL query shape, serialization rules, and SQS-backed mutation the article says are not fully specified in the source material. The 5000ms interval and 120 attempts reflect the article's stated polling policy rather than a verified processing-time guarantee. A key production concern the snippet omits is cancellation: it has no mechanism to abandon a stale polling cycle when the user navigates away or triggers a new enqueue, which the article explicitly flags as an unverified gap that could let an obsolete cycle's late response affect newer state.