Patterns
advancedorchestration Featured

Fan-Out / Fan-In

Split a workload into parallel branches, process them simultaneously, then aggregate the results. Dramatically reduces processing time for batch operations.

Views11
BPMN 2.0
On this page

Visual Flow

Rendering diagram…

When to Use This Pattern

Use fan-out/fan-in when:

  • You have a collection of items to process (invoices, applications, line items)
  • Each item can be processed independently of the others
  • Processing takes significant time and you want to parallelize it
  • You need to aggregate results after all items are processed (totals, summary, merged output)

How It Works

Rendering diagram…

Fan-Out: Split the input into individual items and launch parallel processing for each.

Fan-In: Wait for all parallel branches to complete, then combine the results.

Implementation Guide

Step 1: Prepare the Input Collection

Get the list of items to process:

  • Query a database or list
  • Parse a CSV or Excel file
  • Receive an array from an API webhook
Step 2: Fan Out — Launch Parallel Processing

For each item in the collection, spin up an independent processing branch. Two approaches:

Approach A: Loop with child workflows

  1. Iterate over the collection
  2. For each item, start a child workflow (fire-and-forget)
  3. Each child workflow processes the item and writes its result to a shared location

Approach B: Parallel branches (fixed count)

  1. Use parallel branches in the workflow designer
  2. Each branch handles a subset of items
  3. Best when the number of items is small and known
Step 3: Process Each Item

Each parallel branch does the same work:

  1. Validate the item
  2. Call external services (API lookups, data enrichment)
  3. Apply business logic
  4. Write the result (to a list, database, or variable)
Step 4: Fan In — Wait and Aggregate

Wait for all branches to complete, then:

  • Count successes and failures
  • Sum totals (amounts, quantities)
  • Merge outputs into a consolidated document or report
  • Flag exceptions for manual review
Step 5: Handle Partial Failures

Not all items will succeed. Design for this:

ScenarioStrategy
1 of 50 items failsLog the failure, continue with the other 49
Half the items failComplete what you can, send a summary of failures
All items failAbort and alert — likely a systemic issue

Example: Invoice Batch Processing

Fan-Out:

  1. Query "Pending Invoices" from ERP → returns 50 invoices
  2. For each invoice, start a child workflow:
    • Validate line items
    • Match to purchase orders
    • Check budget availability
    • Create approval task

Fan-In: 3. Wait for all 50 child workflows to complete 4. Generate summary:

  • 45 invoices approved ($125,000 total)
  • 3 invoices flagged for review
  • 2 invoices rejected (PO mismatch)
  1. Send summary email with links to each invoice

Tips & Best Practices

Warning

Be careful with concurrency limits. If you fan out to 100 parallel API calls but the target system only handles 10 concurrent requests, you'll trigger rate limits or timeouts. Use the Retry with Exponential Backoff pattern within each branch.

  • Cap the parallelism. Don't launch 1,000 parallel branches. Process in batches of 10-20.
  • Use a shared status tracker. Write each item's status to a SharePoint list or database table so you can monitor progress in real-time.
  • Make each branch idempotent. If a branch fails and gets retried, it shouldn't create duplicate data.
  • Time-box the fan-in. Don't wait indefinitely — set a maximum wait time and handle stragglers separately.

Related patterns