Most integration diagrams end at the arrow. System A sends a request. System B receives it. A neat line connects the two boxes, and the design appears complete. The difficult work begins when nobody can answer a more ordinary question: what happens if the arrow only half works? A timeout does not tell you that nothing happened. The receiving system may have committed the change and lost the response. A retry may be correct, or it may create a duplicate. A green HTTP response may confirm that a message entered a queue, not that the business outcome completed. This is where many technically functional integrations turn into operational debt. The happy path is automated. The uncertainty is handed to a person who was never named in the design. Model the handoff as a state transition Treat the integration as a business state transition, not merely an API call. For each handoff, write down five things: the authoritative record before the handoff; the event or command being sent; the evidence that proves the receiving system accepted responsibility; the evidence that proves the business outcome completed; and the owner of every state between those two confirmations. Suppose a website sends an approved order to an operations system. A 202 Accepted response may prove receipt. It does not prove that stock was allocated, the order was scheduled, or a human exception was resolved. Those are different states and should be visible as different states. approved -> submitted -> accepted -> processing -> completed | | v v rejected exception The names will differ by system. The important point is that submitted and completed are not collapsed into a single boolean called synced. Give every request a stable identity Retries are unavoidable. Duplicate effects are not. Generate an idempotency key or stable business-event identifier before the first attempt and reuse it for every retry of the same intended action. Store it with the local record. The receiver should either return the original result or reject conflicting reuse. POST /orders Idempotency-Key: order_8421_submit_v1 The key must identify the intention, not the network attempt. Creating a fresh key after each timeout defeats the protection. There is also a less obvious requirement: operators need to be able to search by that identifier across logs, queues and the receiving system. A technically idempotent request that cannot be traced still creates expensive investigation work. Separate transport success from business success An integration usually has at least three kinds of outcome: Outcome What it means Typical next action Transport failure No reliable acknowledgement was received Retry with the same identity Accepted but incomplete The receiver owns the work, but the business result is pending Observe, poll or await an event Business rejection The request was understood but cannot proceed Route to a named decision owner These outcomes should not share one generic catch block. A retry policy belongs to transport uncertainty. A business rejection usually needs different data, approval, or a corrected decision. Replaying it without changing anything merely produces more noise. Make reconciliation a first-class path Even carefully designed flows drift. Events arrive out of order. Credentials expire. An operator corrects a record directly in one system. A deployment interrupts a consumer at the wrong moment. Build reconciliation before production, not after the first unexplained discrepancy. A useful reconciliation job compares authoritative identifiers and states, then emits a small set of actionable differences: present in the source but missing from the receiver; accepted but not completed inside the expected window; completed remotely but not reflected locally; rejected without an assigned owner; and conflicting versions that cannot be resolved automatically. The output should be a work queue with ownership, timestamps and evidence. It should not be a spreadsheet someone has to remember to download. Design the manual path as carefully as the automated one Human intervention is not automatically a failure of automation. Unplanned human intervention is. For each exception, define: who is allowed to decide; what evidence they see; which actions are safe; whether an action can be reversed; how the decision is recorded; and how the workflow resumes. Avoid a generic retry button that repeats an unknown operation. Give the operator the current state, the last confirmed transition and the effect of the next action. If the system cannot explain what it knows, what it does not know and what will happen next, the operator is being asked to guess. Test ambiguity, not only errors Most integration tests cover clear success and clear failure. Production incidents often live between them. Add tests for: the receiver commits the change, then the response is lost; the same event is delivered twice; events arrive out of order; the sender restarts after dispatch but before recording acknowledgement; a downstream operation remains pending beyond its expected window; a user changes the source record while processing is underway; and reconciliation runs while a retry is in flight. The expected result is not always that the request succeeds. It may be that the system enters an explicit uncertain state, prevents a conflicting action and assigns the case to the right owner. A small review before shipping Before calling an integration complete, ask: Can we distinguish receipt from business completion? Can the same intention be retried without duplicating its effect? Can an operator trace one handoff across every component? Does every rejection, timeout and stale state have an owner? Can the systems reconcile without relying on memory? Does the manual path preserve an audit trail? If any answer is vague, the missing work is not polish. It is part of the system boundary. Lumox uses the same questions when assessing whether a connected system is ready for production. The broader enterprise software readiness checklist covers ownership, information boundaries, assurance, recovery and handover. The arrow on the diagram is the easy part. The real integration is the agreement about who owns the uncertainty around it.