Somewhere between a webhook firing and a row landing in your database, one logical event was executed twice — or seven times, or forty — and no layer in the stack had a reliable way to say that it had already seen it.
That gap is missing idempotency: the absence of a contract between sender and receiver that makes repeated delivery safe. It is one of the most underengineered fault lines in modern iPaaS workflows.
You can build an elegant Zapier orchestration with branching logic, error handling, and neatly named steps, and it can still collapse when an upstream system replays an event. The platform will not save you. Trigger-side deduplication is not the same thing as idempotent processing. It is a limited filter that may prevent one trigger from appearing twice under normal conditions. It does not guarantee that every downstream action will happen only once.
The Anatomy of a Duplicate Cascade: Why Triggers Aren’t Enough
Zapier’s polling triggers do have a deduplication mechanism, and it is worth understanding what that mechanism actually protects. On each polling cycle, the platform retrieves recent items from the source application and compares them with identifiers it has already observed. In the ordinary case, a new item is passed into the Zap once, while previously seen items are ignored.
That is useful for a straightforward workflow: a record is created, the trigger finds it, the Zap runs, and the source identifier is not offered again on the next polling cycle. For low-volume integrations where the source emits clean, one-time events and every downstream action succeeds, this guardrail may be enough.
It is not a transaction. It does not wrap the entire workflow in a once-only guarantee.
A trigger can remember that it saw an event without knowing whether the rest of the workflow successfully processed it.
The distinction matters because a trigger and an action have different failure boundaries. A trigger may correctly identify one new Airtable record and start one Zap run. The next step may call a CRM API, receive a transient error, time out after the CRM accepted the request, or be retried by the platform. The trigger has done its job. It has no dependable knowledge of whether the downstream write created a record, partially completed, or never reached the destination.
This is how duplicate records appear in a Zapier CRM sync even when the trigger itself behaved correctly:
1. A source record creates one trigger event.
2. The Zap sends a create request to the CRM.
3. The connection times out before the Zap receives a definitive response.
4. The CRM has already accepted the request and created the contact.
5. The Zap retries because the response looked like a failure.
6. The CRM treats the retry as a new create request.
The same sequence can occur when a platform retries after a rate-limit response or when an operator manually replays a failed run. The payload may be identical, but a conventional create endpoint usually interprets each request as a new instruction.
The cascade becomes harder to trace when the workflow includes several action steps. One step may create a contact, another may create a deal, and a third may add a task. If the first action succeeds but the second fails, restarting the entire path can create another contact before the workflow reaches the point of failure again. The automation appears to be recovering, but it is actually repeating work that already completed.
This is why the number of records in the destination can exceed the number of trigger events. The system at the top of the stack believes it processed one event. The system at the bottom has received multiple create requests.
The 429 Trap: How Rate Limiting Triggers Unintended Retries
Rate limiting is one of the most visible ways to expose a missing idempotency contract. APIs impose request limits, and an automation can reach them quickly when it processes a burst of records, fans out across multiple paths, or performs a lookup followed by a create for each item.
An HTTP 429 response generally means that the receiver is asking the client to slow down or try again later. The dangerous part is not the response itself. It is the assumption that retrying the same request is automatically safe.
A retry can be harmless when the operation is naturally idempotent. Repeating a request that sets a resource to a specified state may produce the same final state. Repeating a create operation is different. If the server assigns a new identifier every time and does not recognize a client-supplied idempotency key, each retry can create another resource.
The exact retry behavior depends on the platform, the connector, and the type of failed step. A workflow may retry the action, replay a run, or leave the run for manual replay. The implementation detail changes, but the architectural question remains the same: can the receiver distinguish a legitimate new request from a repeated attempt to process the same event?
If the answer is no, a rate limit becomes a duplicate amplifier.
The risk is especially clear in a lookup-then-create pattern:
- The workflow looks for a matching contact.
- The lookup finds nothing.
- The workflow sends a create request.
- The request is accepted, but the response is delayed or lost.
- The workflow retries the lookup or create path.
- The second attempt still does not have a durable record of the first attempt, or the lookup is based on a field that was not indexed consistently.
- A second contact is created.
The workflow may be logically correct from the perspective of each individual step. The failure is in the gap between those steps. A lookup is a snapshot, not a reservation. It says that no matching record was visible at that moment. It does not reserve the right to create one.
The same problem appears when two workflows target the same destination. Workflow A and Workflow B may both respond to related source events, or both may process the same record through different paths. They can independently perform the same lookup and both conclude that the destination record does not exist. If both issue a create request, the result is a duplicate even though neither workflow has technically malfunctioned.
A sound design assumes that every create-or-update action may be called more than once for the same logical event. Rate-limit handling should therefore be paired with one of the following protections:
- a destination API that accepts and enforces an idempotency key;
- an upsert operation based on a stable external identifier;
- a database constraint that makes the business key unique;
- an atomic claim or insert operation in a middleware service;
- a queue or worker that owns retries and records completion state durably.
A delay or backoff can reduce pressure on the API, but it does not create idempotency. It changes when the request is sent, not what the receiver does when the same request arrives twice.
Architecting Downstream Idempotency for Webhook Receivers
When you control the receiving end of a webhook integration, idempotency belongs there. The sending platform may deliver an event more than once because of a timeout, a network interruption, a retry policy, or an operator replaying a failed run. In an at-least-once delivery system, repeated delivery is an expected condition rather than an exceptional one.
The most direct pattern is event-level deduplication using an identifier that remains stable across retries. The sender may provide an event ID, delivery ID, or another request identifier. If it does not, the receiving system may need to derive a key from stable source data. That derived key should represent the logical event, not merely the time at which the receiver happened to receive it.
For example, a key might combine the source system, source record ID, and event type. If the same record can produce several meaningful events, the event type or source version may also be required. A timestamp can be part of a key, but it is a poor substitute for an event identifier when clock precision and retry behavior are unclear.
The receiver then needs a durable record of the key and a safe way to claim it. The important operation is not simply check, followed by write. Those are two separate operations and can race.
A safe receiver usually follows this sequence:
1. Extract or calculate the idempotency key.
2. Atomically insert a processing record with that key, or atomically claim an existing pending record.
3. If the key already represents completed work, return a successful response without repeating the side effect.
4. If another worker is processing the same key, follow an explicit in-progress policy rather than starting a second operation.
5. Perform the business action.
6. Persist the result and mark the key as completed.
7. Return a response that matches the actual durability guarantees of the receiver.
The atomic insert is the crucial part. A database table with a unique constraint on the idempotency key can reject a second insert safely. A key-value store may provide an atomic set-if-absent operation. A purpose-built middleware service can expose the same behavior through a small endpoint. The implementation can vary, but the receiver needs a real concurrency control, not two adjacent workflow steps that happen to run quickly.
An in-memory set is rarely sufficient beyond a simple, single-process experiment. It disappears on restart, does not coordinate multiple instances, and cannot protect a receiver running behind a load balancer. A persistent store is required when retries may arrive after the original process has ended or when multiple workers can handle the same webhook.
Choosing a response strategy
Acknowledging a webhook before its payload is durably recorded creates a different failure mode: lost events. If the receiver returns success and then crashes before storing the request, the sender may reasonably stop retrying. The event has not been duplicated; it has disappeared.
A more resilient receiver persists the incoming event before acknowledging it. That persistence can be a queue, an inbox table, or another durable staging layer. The receiver can then process the event asynchronously while retaining enough information to retry the business action.
The inbox and the business result should be treated as separate states. An event can be received but not processed, processing but not completed, or completed successfully. A single Boolean field such as processed = true often cannot represent enough of that lifecycle.
Useful state fields may include:
- the idempotency key;
- the source system and event type;
- the first-seen timestamp;
- the current processing status;
- the number of attempts;
- the last error;
- the downstream resource ID, when one exists;
- the completion timestamp.
This information is not operational decoration. It is what allows a team to distinguish a safe retry from a duplicate side effect.
Idempotency keys and business keys are not always the same
A webhook delivery ID identifies a delivery attempt or a logical event, depending on the sender. A customer email, order number, or invoice number identifies a business object. Those keys solve different problems.
If a source sends two legitimate events for the same order, suppressing both under the order number could discard a real update. If the source retries one event under a new delivery ID, deduplicating only by delivery ID may allow the same business action twice.
The receiver should define what must happen only once:
- the delivery of a particular event;
- the creation of a particular business object;
- the transition to a particular state;
- or the application of a particular version of an update.
In some systems, both an event-level key and a business-level uniqueness constraint are needed. The event key prevents repeated processing of the same delivery, while the business constraint prevents two different events from creating the same supposedly unique object.
State-Checking Patterns: Using Zapier Tables and Delay Steps to Lock Records
Within Zapier, a state check can reduce duplicates, but it should not be described as a lock unless the underlying operation is atomic. A Zapier Table lookup followed by a separate action is still vulnerable to concurrency:
1. Run A looks for the event key and finds nothing.
2. Run B performs the same lookup before Run A writes its marker.
3. Both runs conclude that they are first.
4. Both continue to the create action.
This is the same race condition that appears in a Make webhook race condition fix. Slowing down one step may change the timing, but it does not remove the possibility that both executions reach the same decision before either has committed its state.
Zapier Tables can still be useful as an operational ledger. A workflow can record source IDs, destination IDs, processing status, and error details. A later lookup can stop obvious replays when the marker is already present. This is valuable protection against repeated polling results, manual replays, and some classes of workflow mistakes.
It is not, by itself, a guaranteed atomic deduplication layer for concurrent executions.
What a Delay step can and cannot do
A Delay step may help when the problem is eventual consistency. If a destination system needs time before a newly created record becomes visible to a subsequent lookup, waiting can reduce false negatives. A delay can also smooth a burst and lower the chance of sending a large number of requests at once.
But a delay does not serialize concurrent Zap runs. It does not reserve an event key, and it does not guarantee that the first execution will write its marker before the second execution performs its lookup. Two runs can simply wait for the same interval and then continue together.
Use a delay as a best-effort timing adjustment, not as a correctness mechanism.
A delay can give a system more time; it cannot give two concurrent executions ownership of the same event.
A practical comparison looks like this:
| Pattern | What it helps with | What it does not guarantee | Appropriate use |
|---|---|---|---|
| Zapier Table lookup | Detects keys already recorded in the table | Does not make lookup and write atomic | Replay detection and workflow audit state |
| Delay before lookup | Gives eventual writes time to become visible; smooths bursts | Does not serialize runs or close a race window | Best-effort timing adjustment |
| Filter step | Rejects events with an unwanted status or shape | Does not identify repeated valid events | Preventing duplicates caused by irrelevant state transitions |
| Destination upsert | Finds or updates a record by a stable external key | Depends on the destination’s matching semantics | CRM and database synchronization |
| Idempotency-aware middleware | Atomically claims keys and owns retries | Requires a service or endpoint outside the basic Zap | High-value or high-volume workflows |
| Database uniqueness constraint | Prevents two rows with the same business key | Does not automatically reconcile failed side effects | Systems where the database is the authoritative writer |
The safest pattern is to move the atomic decision to a system that can make one. For a custom application, that may be a database insert with a unique constraint. For a webhook receiver, it may be an idempotency-key store. For a CRM, it may be an upsert keyed by an external ID.
If the destination offers none of these, a Zapier Table can provide useful visibility and partial protection, but the workflow should be honest about the remaining race. It should not promise that a lookup and a later marker write have serialized the process.
Sequence matters, but no sequence repairs a non-atomic design
A common suggestion is to write the processed marker before the create action. That avoids the specific failure where the create succeeds and the marker write fails. It introduces another failure: the marker can be written while the create action never happens. The next run sees the marker and skips the missing business action.
Writing the marker after the action has the opposite trade-off. A failed marker write can cause a successful action to be repeated.
Neither ordering is universally correct because the two operations are not part of one transaction. The right solution is not to pick a more persuasive sequence. It is to use a destination operation that combines the uniqueness decision with the write, or to place both steps behind a service that can manage their state and recovery.
Where that is not possible, record enough information to reconcile the ambiguous state. Store the source event key, the intended destination, the attempt status, and any response identifier returned by the destination. An operator should be able to determine whether the action completed before replaying it.
Beyond the ID Field: Implementing Robust Deduplication Logic
Relying on a default id field is reasonable only when the source guarantees that the field is stable, unique, and present in every relevant payload. Those conditions do not hold across every iPaaS integration.
A Google Sheet row number is a familiar example. It can change when rows are inserted or removed, and it does not necessarily describe the business object represented by the row. A row containing a customer, invoice, or task may need a stable external identifier in a dedicated column. If no such field exists, a composite key may be built from several values, but that choice must account for legitimate repeated content.
A composite key might include:
- the source system;
- the source record identifier;
- the event or action type;
- a source version or update marker;
- a business identifier when the action concerns a unique object.
The objective is not to create the longest possible string. It is to identify the unit of work that must be protected from repetition.
Avoid unstable fields
Keys built from raw payloads can break when the payload changes format without changing its meaning. Whitespace, field order, date formatting, optional fields, and localized values can all produce different hashes for what a human would consider the same event.
If a payload hash is necessary, normalize the input first. Use a defined set of fields, stable formatting, and explicit treatment of missing values. Do not include volatile metadata such as receipt time unless a new receipt is meant to represent a new event.
At the other extreme, a key built from too few fields can collapse legitimate events together. An email address may identify a person but not a particular order. A task title may be repeated intentionally. A customer ID may be stable while a series of status changes must each be processed.
Deduplication is therefore a domain decision, not just a technical one. The correct question is not which field looks unique. It is what the business considers to be the same operation.
Prefer upsert semantics where they exist
An upsert combines matching and writing into a destination operation defined by a stable external key. Instead of asking whether a record exists and then issuing a separate create request, the client sends the external key to an endpoint that can create the record if absent or update it if present.
This removes one important race window. Two concurrent requests may still arrive, but the destination’s uniqueness rule decides whether they refer to one resource. The implementation should still be checked carefully: some connectors label an operation as an upsert while internally performing a non-atomic search followed by create.
When an API supports idempotency keys, send the same key on every retry of the same logical operation. The key should remain stable across connection timeouts and retry attempts, while a genuinely new business event receives a new key. The receiver should retain the key for a period appropriate to its replay window and should define what happens when the same key arrives with a different payload.
That last point matters. Silently accepting a changed payload under an existing key can hide upstream bugs. A robust receiver may compare a request fingerprint and reject or flag a mismatch. The policy should be explicit.
Handle partial success across multi-step workflows
Idempotency does not turn a multi-step Zap into one database transaction. If a workflow creates a CRM contact, then a deal, then a task, each side effect needs its own recovery strategy.
A durable design may:
1. assign a stable workflow or event key;
2. create or upsert the contact using that key;
3. store the resulting contact ID;
4. create or upsert the deal using its own derived key;
5. store the deal ID;
6. create or upsert the task using a key tied to the intended task;
7. retry only the incomplete step when possible.
This is more reliable than replaying the entire workflow from the beginning. It also makes the system easier to inspect. A failed task creation should not require guessing whether the contact and deal already exist.
For complex flows, an external orchestration layer may be justified. It can hold a state machine for each event, apply backoff, record responses, and retry individual operations. The point is not to add infrastructure for its own sake. It is to put state where the workflow can access it after a timeout, restart, or manual replay.
A Practical Design for Preventing Duplicate Tasks in an iPaaS
The most useful implementation question is what happens after an ambiguous response. If the system cannot tell whether the previous request succeeded, the next step must be safe in both possible worlds.
For a task synchronization workflow, that could mean:
- derive a stable key from the source task ID and the intended destination project;
- send that key as an external ID or idempotency key;
- use an upsert or atomic insert at the destination;
- store the destination task ID after success;
- retry with the same key when the response is lost;
- treat a key conflict as evidence that the task already exists, then retrieve and reconcile it.
The design should also distinguish between a retry of the same task and a new task with similar content. A title and due date may be useful for a human, but they are usually insufficient as the sole deduplication key.
For a webhook receiver, the equivalent flow is an inbox record with an atomic uniqueness rule, followed by asynchronous processing. For a Zapier CRM sync, it may be a CRM external ID field and an upsert action. For a Make scenario, it may require a datastore or middleware endpoint that performs a conditional insert rather than a plain search followed by create.
The tools differ. The invariant does not: repeated delivery of one logical event must not produce repeated business effects.
The Cost of Leaving the Gap Open
Duplicates are not merely untidy data. They can trigger a second cascade of their own:
- duplicate contacts receive the same campaign;
- duplicate deals distort pipeline reports;
- duplicate invoices enter an accounting review;
- duplicate tasks create conflicting ownership;
- duplicate webhook events trigger downstream notifications;
- duplicate records become linked to different child objects, making cleanup harder.
Deletion is rarely enough. A duplicate may already have sent an email, consumed an entitlement, created a payment attempt, or launched another automation. Merging records also has consequences: activities, associations, ownership, and audit history may not be combined symmetrically.
That is why idempotency belongs in the design phase, before the first burst of traffic or the first rate-limit incident. The relevant test is not whether the happy path runs once. It is whether the workflow remains safe when the sender retries, the receiver times out, two runs overlap, and an operator replays an ambiguous failure.
Final Position
Missing idempotency in Zapier workflows is not a minor configuration defect. It is a missing boundary between event delivery and business state.
A trigger can deduplicate what it has seen. A delay can soften timing. A filter can reject irrelevant transitions. A table can record useful processing state. None of those, alone, guarantees that a concurrent or retried create operation is safe.
The durable controls are more specific: stable event keys, atomic claim or insert operations, destination-side uniqueness, idempotent upserts, and retry logic that repeats the same logical request rather than blindly replaying an entire workflow. Where the no-code platform cannot provide those guarantees, a small middleware or receiver service can provide the missing contract.
The goal is not to prevent every repeated delivery. In a distributed system, that is often unrealistic. The goal is to make repeated delivery boring: the same event arrives again, the system recognizes it, and the business state remains correct.