Workflow synchronization

How workflow synchronization from Zuora works

Workflow synchronization is the heart of Zuora Workflow Manager. This process downloads workflows from the Zuora API and saves them to the local database for visualization and management.

How it works

Synchronization flow

Pagination

Synchronization automatically handles pagination:

• Page size: 50 workflows per page (configurable)
• Auto-pagination: Continues until last page
• Progress tracking: Monitors progress via Moox Jobs

Synchronization modes

1. Manual synchronization (UI)

From the Customers list:

1. Click the Sync Workflows icon in the Actions column
2. The job is queued immediately
3. Monitor progress in Jobs → Jobs Waiting

When to use:

• After creating a new customer
• When updated data is needed immediately
• To test Zuora connection

Method	Trigger	Monitoring
UI Button	Manual click on customer	Jobs → Jobs Waiting

2. Automatic synchronization (Scheduler)

The scheduler automatically executes synchronization:

// routes/console.php
Schedule::command('app:sync-workflows --all')
    ->hourly()  // Every hour (default)
    ->name('sync-customer-workflows');

Frequency configuration:

->hourly()              // Every hour
->everyThirtyMinutes()  // Every 30 minutes
->everyFiveMinutes()    // Every 5 minutes
->daily()               // Once a day
->dailyAt('02:00')      // Every day at 2:00

Start scheduler:

# Development
lando schedule

# Production (cron job)
* * * * * cd /path/to/app && php artisan schedule:run >> /dev/null 2>&1

3. CLI synchronization

Synchronize from terminal:

# Synchronize all customers (asynchronous)
lando artisan app:sync-workflows --all

# Synchronize a specific customer
lando artisan app:sync-workflows --customer="Acme Corp"

# Synchronize all customers (synchronous, no queue)
lando artisan app:sync-workflows --all --sync

When to use:

• During deployment
• For troubleshooting
• For immediate sync without queue

Option	Parameter	Description
All customers	--all	Synchronize all customers
Specific customer	--customer="Name"	Synchronize a specific customer by name
Synchronous mode	--sync	Run sync immediately without queue

Detailed process

Step 1: Credential validation

Before starting, the system validates:

// Verify that the customer has:
- zuora_client_id (not empty)
- zuora_client_secret (not empty)
- zuora_base_url (valid URL)

If credentials are not valid, the job is terminated without retry.

Step 2: OAuth 2.0 authentication

// ZuoraService->getAccessToken()
1. Check cache (key: zuora_access_token_{hash})
2. If not in cache:
   - POST /oauth/token
   - grant_type=client_credentials
   - Save token in cache (TTL: 1 hour)
3. Return access token

Step 3: Fetch workflows

// ZuoraService->listWorkflows()
1. GET /workflows?page=1&page_length=50
2. Normalize API response
3. Extract workflow data and pagination info
4. Repeat for each page until hasMore=false

Step 4: Download JSON export

For each workflow:

// ZuoraService->downloadWorkflow()
1. GET /workflows/{id}/export
2. Return complete workflow JSON
3. Includes tasks, triggers, schedule, etc.

Step 5: Database save

// WorkflowSyncService->syncWorkflowRecord()
1. firstOrNew(['zuora_id' => $id, 'customer_id' => $customerId])
2. Fill attributes:
   - name, description, state
   - created_on, updated_on
   - last_synced_at = now()
   - json_export = downloaded JSON
3. save()

Step 6: Task extraction

// Workflow->syncTasksFromJson()
1. Extract tasks from json_export['tasks']
2. For each task:
   - updateOrCreate(['workflow_id', 'task_id'])
   - Save task attributes
3. Delete tasks no longer present (cleanup)

Step 7: Cleanup obsolete workflows

// WorkflowSyncService->deleteStaleWorkflows()
1. Find local workflows no longer in Zuora
2. whereNotIn('zuora_id', $zuoraIds)
3. delete() (cascade deletes tasks too)

Error handling

Retry logic

Jobs have automatic retry:

public int $tries = 3;           // 3 attempts
public int $backoff = 60;        // 60 seconds between attempts

Exponential backoff:

Attempt	Expected after	Total elapsed
1	Immediate	0 seconds
2	60 seconds	60 seconds
3	120 seconds	180 seconds

Handled errors

Authentication errors:

• Invalid credentials → Job terminated (no retry)
• Token expired → Regenerate token automatically

Network errors:

• Timeout → Automatic retry
• Connection refused → Automatic retry
• Rate limiting → Retry with backoff

Data errors:

• Invalid JSON → Log warning, continue with other workflows
• Missing required fields → Log warning, skip workflow

Error monitoring

Failed Jobs:

1. Navigate to Jobs → Failed Jobs
2. View detailed error
3. Click Retry to re-execute

Logs:

# View logs in real-time
lando logs -f

# Search for specific errors
lando logs -f | grep -i "error"

# Log file
lando exec appserver tail -f storage/logs/laravel.log

Performance

Implemented optimizations

Caching:

• OAuth tokens cached (1 hour)
• Reduces authentication calls

Pagination:

• 50 workflows per page
• Avoids timeout on large datasets

Background processing:

• Asynchronous jobs
• Does not block UI
• Parallel processing possible

Database optimization:

• Indexes on zuora_id, customer_id
• Foreign keys for integrity
• Cascade delete for cleanup

Typical metrics

Synchronization time:

Workflows	Estimated time	API calls
10	~5-10 seconds	1-3 pages
50	~20-30 seconds	1-5 pages
100	~40-60 seconds	1-10 pages

Influencing factors:

Factor	Impact
Zuora network latency	High
Workflow JSON size	Medium
Number of tasks per workflow	Medium
Database load	Low-Medium

Best practices

Synchronization frequency

Development:

• Manual when necessary
• Scheduler every hour for testing

Production:

• Scheduler every hour (default)
• Every 30 minutes for critical data
• Manual for immediate updates

Monitoring

Check regularly:

• Failed jobs (retry or investigate)
• Sync duration (performance issues?)
• Error logs (error patterns?)

Alerts:

• Email on failed jobs (configurable)
• Slack notifications (via Laravel)
• Monitoring tools (Sentry, Bugsnag)

Troubleshooting

Workflows not synchronized:

1. Verify customer credentials
2. Check queue worker is active
3. Verify failed jobs
4. Check logs for errors

Slow synchronization:

1. Verify Zuora network latency
2. Check database load
3. Increase workers if necessary
4. Consider additional cache

Missing data:

1. Verify complete JSON export
2. Check logs for warnings
3. Verify Zuora API permissions
4. Manual test with Postman

Zuora APIs used

GET /workflows

List all workflows:

GET https://rest.zuora.com/workflows?page=1&page_length=50
Authorization: Bearer {access_token}

Response:

{
  "data": [
    {
      "id": "wf_123",
      "name": "Subscription Workflow",
      "status": "Active",
      "createdAt": "2024-01-01T00:00:00Z",
      ...
    }
  ],
  "pagination": {
    "page": 1,
    "page_length": 50,
    "next_page": 2
  }
}

GET /workflows//export

Export complete JSON:

GET https://rest.zuora.com/workflows/{id}/export
Authorization: Bearer {access_token}

Response:

{
  "id": "wf_123",
  "name": "Subscription Workflow",
  "tasks": [
    {
      "id": "task_1",
      "name": "Send Email",
      "action_type": "Email",
      ...
    }
  ],
  "triggers": [...],
  "schedule": {...}
}

Next steps

Customer management

Customer organization management

Task management

Task extraction and management

Zuora API

Zuora API details

Troubleshooting

Problem resolution