Work Items

Expand collected input values into parallel Work Items and aggregate their outputs.

What Are Work Items?

Marking an input for_each: true tells Argyll to create one Work Item per collected value accepted by the Step’s Predicate, or every collected value when no Predicate is configured. Each Work Item executes independently with its own inputs, status, retries, and receipt token.

for_each is only valid when the input’s Collection Policy is all or some, since those are the only policies that resolve to a list of candidate values. The input’s match script, if present, filters that list exactly as it does for any other input, required or optional. Candidates are evaluated and nonmatching ones are removed before Work Items are expanded. A Step Predicate, when present, then evaluates each complete Work Item candidate after all for_each inputs have been combined.

Basic Example

{
  "id": "process-item",
  "name": "Process Item",
  "type": "service",
  "http": {
    "invoke": {
      "endpoint": "https://api.example.com/items/process",
      "timeout": 5000
    }
  },
  "attributes": {
    "items": {
      "role": "required",
      "required": { "for_each": true, "collect": "all" }
    },
    "processed": { "role": "output", "type": "number" }
  }
}

Flow init:

{
  "items": [
    {"id": "item-1", "value": 10},
    {"id": "item-2", "value": 20}
  ]
}

Each entry in the init list is a separate candidate value for items, the same shape multiple providers would produce. With collect: all, the resolved input is the full list of collected candidates, and for_each creates one Work Item per candidate. That gives two Work Items here, one per element, each calling the handler once.

Multiple for_each Inputs

Multiple for_each Attributes produce every combination of their collected values (the Cartesian product):

{
  "attributes": {
    "regions": {
      "role": "required",
      "required": { "for_each": true, "collect": "all" }
    },
    "products": {
      "role": "required",
      "required": { "for_each": true, "collect": "all" }
    }
  }
}

With regions collecting ["US", "EU"] and products collecting ["A", "B"], the engine creates 4 Work Items: US×A, US×B, EU×A, EU×B.

Parallelism

Work Items execute sequentially when parallelism uses its default value of 1. Increase it through work_config to run several items concurrently:

{
  "work_config": {
    "parallelism": 5,
    "max_retries": 3,
    "init_backoff": 100,
    "max_backoff": 5000,
    "backoff_type": "exponential"
  }
}

Match parallelism to your downstream service’s capacity. For I/O-bound Steps 10–50 is typical. For CPU-bound Steps keep it low.

Output Aggregation

Step outputs are only available once all created Work Items succeed. When multiple Work Items succeed, each aggregated output is an array, where each element includes the for_each input values alongside the output:

Work item 1 (item-1): processed = 100
Work item 2 (item-2): processed = 200

Aggregated "processed" attribute:
[
  { "items": {"id": "item-1", "value": 10}, "processed": 100 },
  { "items": {"id": "item-2", "value": 20}, "processed": 200 }
]

With one Work Item, the Step uses that item’s outputs without array aggregation. No collected candidates completes the Step without work or outputs. For a required input without a match predicate, this happens when the Flow’s initial state declares the input with an empty array. A required match predicate instead closes its gate when no candidates match, skipping the Step. An optional input declared in the initial state can resolve to zero candidates even when its match predicate rejects every initial value. An unfulfilled optional collection falls back to its configured default, typically an empty array, which also completes without work. If the Step Predicate rejects every candidate, the Step is skipped without outputs.

Idempotency

Each Work Item carries a unique Argyll-Receipt-Token header. The engine ignores duplicate completions for the same token (returning 200 either way), so your handler can safely be retried.

Partial Failure

A permanent Work Item failure marks the Step failed and abandons its pending items. Work Items already in progress still report their results and may have produced external effects. Configure compensation to reverse those effects after partial success.

Interaction with Predicates

The Predicate is evaluated once per candidate Work Item, after for_each expansion and before any work is created. A false result excludes that combination. Rejecting every candidate skips the Step. Without for_each, the same rule applies to one candidate with the collected inputs. Accepted Work Items keep their inputs and are not checked again before dispatch, retries, or recovery. See Predicates for an example.