Skip to content

Scheduler Troubleshooting

This guide covers common scheduler issues and how to resolve them in production.

Architecture

The scheduler uses a Postgres-backed schedule table as the single source of truth. A single scheduler instance polls enabled schedules and dispatches Celery tasks.

Important: Only one scheduler instance should run at a time. This is enforced operationally via deployment configuration (e.g., single replica), not via database locks.

Schedule auto-disable

The scheduler automatically disables schedules that cannot be executed safely.

Symptoms

  • Logs show Schedule <id> auto-disabled: <reason>. Manual intervention required.
  • The schedule's enabled flag is set to false

Causes

Reason Description
invalid task_type The task_type value is not a valid TaskType enum member
task scope mismatch The schedule scope conflicts with TaskType policy
invalid task_args The task_args JSON fails ScheduleTaskArgs validation
missing project metadata A project schedule references a project_id that no longer exists
invalid crontab The crontab expression cannot be parsed (must be 5 fields)
crontab due check failed The crontab schedule raised an exception during due-time calculation

Recovery steps

  1. Check scheduler logs for the specific reason.
  2. Fix the underlying data issue (e.g., correct the crontab, restore the project, or update task_args).
  3. Re-enable the schedule via the API or database.

Database connectivity

Symptoms

  • Logs show Schedule load failed; will retry: <error>
  • Tasks are not being dispatched

Causes

  • Postgres is unavailable or restarting
  • Network interruptions between scheduler and Postgres
  • Migrations have not run (schedule table missing)

Recovery steps

  1. Check Postgres connectivity from the scheduler container.
  2. Verify migrations have been applied.
  3. The scheduler will automatically retry on the next tick interval.