Fan-Out / Fan-In
Split a workload into parallel branches, process them simultaneously, then aggregate the results. Dramatically reduces processing time for batch operations.
On this page
Visual Flow
Rendering diagram…
When to Use This Pattern
Use fan-out/fan-in when:
- You have a collection of items to process (invoices, applications, line items)
- Each item can be processed independently of the others
- Processing takes significant time and you want to parallelize it
- You need to aggregate results after all items are processed (totals, summary, merged output)
How It Works
Rendering diagram…
Fan-Out: Split the input into individual items and launch parallel processing for each.
Fan-In: Wait for all parallel branches to complete, then combine the results.
Implementation Guide
Step 1: Prepare the Input Collection
Get the list of items to process:
- Query a database or list
- Parse a CSV or Excel file
- Receive an array from an API webhook
Step 2: Fan Out — Launch Parallel Processing
For each item in the collection, spin up an independent processing branch. Two approaches:
Approach A: Loop with child workflows
- Iterate over the collection
- For each item, start a child workflow (fire-and-forget)
- Each child workflow processes the item and writes its result to a shared location
Approach B: Parallel branches (fixed count)
- Use parallel branches in the workflow designer
- Each branch handles a subset of items
- Best when the number of items is small and known
Step 3: Process Each Item
Each parallel branch does the same work:
- Validate the item
- Call external services (API lookups, data enrichment)
- Apply business logic
- Write the result (to a list, database, or variable)
Step 4: Fan In — Wait and Aggregate
Wait for all branches to complete, then:
- Count successes and failures
- Sum totals (amounts, quantities)
- Merge outputs into a consolidated document or report
- Flag exceptions for manual review
Step 5: Handle Partial Failures
Not all items will succeed. Design for this:
| Scenario | Strategy |
|---|---|
| 1 of 50 items fails | Log the failure, continue with the other 49 |
| Half the items fail | Complete what you can, send a summary of failures |
| All items fail | Abort and alert — likely a systemic issue |
Example: Invoice Batch Processing
Fan-Out:
- Query "Pending Invoices" from ERP → returns 50 invoices
- For each invoice, start a child workflow:
- Validate line items
- Match to purchase orders
- Check budget availability
- Create approval task
Fan-In: 3. Wait for all 50 child workflows to complete 4. Generate summary:
- 45 invoices approved ($125,000 total)
- 3 invoices flagged for review
- 2 invoices rejected (PO mismatch)
- Send summary email with links to each invoice
Tips & Best Practices
Be careful with concurrency limits. If you fan out to 100 parallel API calls but the target system only handles 10 concurrent requests, you'll trigger rate limits or timeouts. Use the Retry with Exponential Backoff pattern within each branch.
- Cap the parallelism. Don't launch 1,000 parallel branches. Process in batches of 10-20.
- Use a shared status tracker. Write each item's status to a SharePoint list or database table so you can monitor progress in real-time.
- Make each branch idempotent. If a branch fails and gets retried, it shouldn't create duplicate data.
- Time-box the fan-in. Don't wait indefinitely — set a maximum wait time and handle stragglers separately.
Related patterns
Saga with Compensating Transactions
Coordinate a multi-step business transaction that spans several systems by pairing each step with a rollback action. If a later step fails, run the rollbacks in reverse to restore a consistent state.
State Machine Workflow
Model a business process as a set of defined states with explicit transitions between them. Unlike linear workflows, items can move forwards, backwards, and loop — matching how real business processes actually behave.
Work Queue with Pull-Based Claim
Put tasks in a shared queue and let qualified agents claim work rather than pre-assigning. Keeps the busiest person busy, the best person on the hardest case, and nothing in the wrong inbox.