Patterns
beginnerorchestration

Looping Collection Processor

Iterate through a list of items one by one — applying business logic, calling APIs, and making decisions per item. The simpler cousin of Fan-Out/Fan-In for smaller collections.

Views19
BPMN 2.0
On this page

Visual Flow

Rendering diagram…

When to Use This Pattern

Use a looping processor when:

  • You have a small to medium collection (1–100 items) to process
  • Items must be processed sequentially (order matters, or each depends on the previous result)
  • The per-item processing is simple (a few steps, not heavy API calls)
  • You don't need massive parallelism — simplicity beats speed for this use case

For larger collections or items that can be processed independently, see Fan-Out / Fan-In or Scheduled Batch Processor.

How It Works

items = get_collection()      // [item1, item2, item3, ...]
results = []
errors = []

FOR EACH item IN items:
    try:
        result = process(item)
        results.append(result)
    catch error:
        errors.append({ item, error })
        IF error.is_fatal:
            BREAK                 // Stop on critical error
        ELSE:
            CONTINUE              // Skip and move to next

generate_summary(results, errors)

Implementation Guide

Step 1: Get the Collection

Sources for your item list:

SourceHow to Retrieve
SharePoint listQuery list items action
DatabaseSQL query
Excel/CSVParse file into rows
API responseCall web service, parse response array
Repeating sectionRead form collection data
Manual listComma-separated text field
Step 2: Loop and Process

For each item in the collection:

  1. Extract the item's data — pull fields relevant to this iteration
  2. Validate — check required fields, data types, ranges
  3. Transform — apply business rules, calculations, lookups
  4. Act — create a record, update a field, call an API, send a notification
  5. Track — log success/failure for the summary
Step 3: Handle Per-Item Failures
StrategyWhen to Use
Skip and continueNon-critical items; collect errors for review at the end
Stop on first errorItems are dependent on each other; processing out of order would cause data corruption
Retry and continueTransient errors (API timeout); try once more before skipping
Accumulate errors and stop at thresholdOK to skip a few, but if >10% fail, something is systemically wrong
Step 4: Avoid Infinite Loops

Common causes and safeguards:

RiskSafeguard
Loop counter not incrementingSet a max iteration cap (e.g., 500)
Exit condition never metAdd a timeout (e.g., 30 minutes total run time)
Item reappears in collectionTrack processed IDs and skip duplicates
Workflow triggers itselfAdd a "processed by workflow" flag to items
Step 5: Generate a Summary

After the loop completes:

MetricValue
Total items47
Processed successfully44
Skipped (validation)2
Failed1
Processing time3m 12s
Error detailsItem #23: "Vendor API returned 404 — vendor not found"

Tips & Best Practices

Tip

If your loop takes more than 5 minutes, consider whether you can parallelize with Fan-Out/Fan-In instead. The break-even point is usually around 20 items or when per-item processing exceeds 15 seconds.

  • Process in batches. If calling an external API, batch API calls (e.g., send 10 records per request) instead of making one call per item. This is faster and gentler on the API.
  • Preserve loop index. Track which item number you're on (index) in a variable. If the workflow fails, you know exactly which item caused the issue.
  • Update progress. For long loops, update a status field periodically ("Processing item 35 of 47") so users can monitor progress.
  • Test with edge cases. Empty collection (0 items), single item, and a collection with duplicate entries.

Related patterns