Looping Collection Processor
Iterate through a list of items one by one — applying business logic, calling APIs, and making decisions per item. The simpler cousin of Fan-Out/Fan-In for smaller collections.
On this page
Visual Flow
Rendering diagram…
When to Use This Pattern
Use a looping processor when:
- You have a small to medium collection (1–100 items) to process
- Items must be processed sequentially (order matters, or each depends on the previous result)
- The per-item processing is simple (a few steps, not heavy API calls)
- You don't need massive parallelism — simplicity beats speed for this use case
For larger collections or items that can be processed independently, see Fan-Out / Fan-In or Scheduled Batch Processor.
How It Works
items = get_collection() // [item1, item2, item3, ...]
results = []
errors = []
FOR EACH item IN items:
try:
result = process(item)
results.append(result)
catch error:
errors.append({ item, error })
IF error.is_fatal:
BREAK // Stop on critical error
ELSE:
CONTINUE // Skip and move to next
generate_summary(results, errors)
Implementation Guide
Step 1: Get the Collection
Sources for your item list:
| Source | How to Retrieve |
|---|---|
| SharePoint list | Query list items action |
| Database | SQL query |
| Excel/CSV | Parse file into rows |
| API response | Call web service, parse response array |
| Repeating section | Read form collection data |
| Manual list | Comma-separated text field |
Step 2: Loop and Process
For each item in the collection:
- Extract the item's data — pull fields relevant to this iteration
- Validate — check required fields, data types, ranges
- Transform — apply business rules, calculations, lookups
- Act — create a record, update a field, call an API, send a notification
- Track — log success/failure for the summary
Step 3: Handle Per-Item Failures
| Strategy | When to Use |
|---|---|
| Skip and continue | Non-critical items; collect errors for review at the end |
| Stop on first error | Items are dependent on each other; processing out of order would cause data corruption |
| Retry and continue | Transient errors (API timeout); try once more before skipping |
| Accumulate errors and stop at threshold | OK to skip a few, but if >10% fail, something is systemically wrong |
Step 4: Avoid Infinite Loops
Common causes and safeguards:
| Risk | Safeguard |
|---|---|
| Loop counter not incrementing | Set a max iteration cap (e.g., 500) |
| Exit condition never met | Add a timeout (e.g., 30 minutes total run time) |
| Item reappears in collection | Track processed IDs and skip duplicates |
| Workflow triggers itself | Add a "processed by workflow" flag to items |
Step 5: Generate a Summary
After the loop completes:
| Metric | Value |
|---|---|
| Total items | 47 |
| Processed successfully | 44 |
| Skipped (validation) | 2 |
| Failed | 1 |
| Processing time | 3m 12s |
| Error details | Item #23: "Vendor API returned 404 — vendor not found" |
Tips & Best Practices
If your loop takes more than 5 minutes, consider whether you can parallelize with Fan-Out/Fan-In instead. The break-even point is usually around 20 items or when per-item processing exceeds 15 seconds.
- Process in batches. If calling an external API, batch API calls (e.g., send 10 records per request) instead of making one call per item. This is faster and gentler on the API.
- Preserve loop index. Track which item number you're on (index) in a variable. If the workflow fails, you know exactly which item caused the issue.
- Update progress. For long loops, update a status field periodically ("Processing item 35 of 47") so users can monitor progress.
- Test with edge cases. Empty collection (0 items), single item, and a collection with duplicate entries.
Related patterns
Saga with Compensating Transactions
Coordinate a multi-step business transaction that spans several systems by pairing each step with a rollback action. If a later step fails, run the rollbacks in reverse to restore a consistent state.
State Machine Workflow
Model a business process as a set of defined states with explicit transitions between them. Unlike linear workflows, items can move forwards, backwards, and loop — matching how real business processes actually behave.
Fan-Out / Fan-In
Split a workload into parallel branches, process them simultaneously, then aggregate the results. Dramatically reduces processing time for batch operations.