Flow Design

Practical patterns for designing Flows: picking goals correctly, fail-fast vs best-effort Step roles, collection deadlines, for_each fan-out, sub-flow composition, and an anti-pattern table for the mistakes you'll otherwise make.

Start with Goals

Define the outcomes you actually need. The engine builds the minimal Plan to reach them. Don’t list intermediate Steps as Goals.

Goal: send_confirmation
Engine computes:
  send_confirmation needs: transaction_id, customer_email
  transaction_id produced by: process_payment
  process_payment needs: amount, customer_id (from init)
  customer_email from init
Plan: process_payment → send_confirmation

Keep Flows to 1–3 focused Goals. More Goals usually means the Flow is doing too much.

Goal Step Failure vs Non-Goal Step Failure

This distinction matters: Goal Step failure terminates the Flow. Non-Goal Step failure does not, but it may make downstream Steps unreachable.

Flow goals: [send_confirmation]

Plan (built backward from the Goal):
  validate_customer → process_payment → send_confirmation
                                      ↘ audit_log (not a Goal)

audit_log is in the Plan because some optional output flows into send_confirmation. It’s not a Goal itself.

If process_payment fails:
  → send_confirmation is unreachable → Flow fails (the Goal can't be satisfied)

If audit_log fails:
  → Flow continues; send_confirmation can run without it
    (because the input from audit_log is optional, with a default or deadline)

If validate_customer fails:
  → process_payment unreachable → send_confirmation unreachable → Flow fails

The pattern: declare your real outcome as the Goal, route side-effect work through optional inputs so a failed side effect doesn’t take down the whole Flow.

Attribute-Driven Connections

Steps connect automatically by matching Attribute names. A Step that outputs customer_id feeds any Step that requires customer_id. No explicit wiring.

Use required.mapping.name or optional.mapping.name to bridge name mismatches between the Flow state and a service’s expected parameter names.

Optional Inputs and Defaults

Use optional Attributes with defaults to keep Steps reusable across Flow contexts:

{
  "attributes": {
    "customer_id": { "role": "required", "type": "string" },
    "notify_user": {
      "role": "optional",
      "type": "boolean",
      "optional": { "default": "true" }
    }
  }
}

Input Collection

When multiple Steps can produce the same Attribute, the collect policy controls how the consumer behaves. See Attributes for the full table.

first is the default and prunes remaining providers once a value arrives. For fan-in patterns where you need all results, use all or some.

Collection Deadlines

Optional inputs can declare optional.deadline (milliseconds). The deadline starts when the Step’s required inputs are satisfied. At expiry, the Step proceeds with whatever is available rather than waiting indefinitely. If all upstream providers complete before the deadline without satisfying the input, the Step resolves immediately. There is nothing left to wait for.

{
  "preferences": {
    "role": "optional",
    "type": "object",
    "optional": {
      "default": "{}",
      "deadline": 2000
    }
  }
}

For Each

Mark an array input with for_each: true to expand it into parallel Work Items:

{
  "attributes": {
    "order_items": {
      "role": "required",
      "type": "array",
      "required": { "for_each": true }
    },
    "stock_reserved": { "role": "output" }
  },
  "work_config": { "parallelism": 5 }
}

Outputs are only available after all Work Items complete. Don’t use for_each if you need per-item results before the Step finishes.

Predicates

Use Predicates to skip Steps based on Flow state. The Predicate evaluates when the Step’s inputs are ready:

{
  "predicate": {
    "language": "lua",
    "script": "return customer_tier == \"premium\" and order_total > 1000"
  }
}

Keep Predicates simple, 1 or 2 conditions. Complex logic belongs in a dedicated decision Step. A Predicate that errors causes the Step to fail, not skip.

Sub-Flows for Composition

Package reusable logic as a Flow Step. The Sub-Flow gets its own Execution Plan compiled at registration time:

{
  "id": "authorize-user",
  "type": "flow",
  "flow": { "goals": ["check-permissions"] },
  "attributes": {
    "uid": {
      "role": "required",
      "required": { "mapping": { "name": "user_id" } }
    },
    "authorized": {
      "role": "output",
      "output": { "mapping": { "name": "is_authorized" } }
    }
  }
}

Sub-Flow composition must be acyclic. A Flow Step cannot include itself through its Goals.

Idempotency

The engine deduplicates webhook completions by receipt token. For sync Steps, retries are safe as long as your handler is idempotent. Use Argyll-Receipt-Token to detect and reject duplicate invocations in your service if needed.

Anti-Patterns