# Roadmap

Work that is agreed on but not done, in rough priority order. Items come
from the v2 design review and from the 1.x versus 2.0 evaluation; each
says what it fixes and roughly how big it is (S: one sitting, M: a focused
PR, L: multi-day with a design discussion first).

## Scheduling and workers

- **Block batching** [M]. A worker asks for `N` blocks in one round trip
  (`AcquireBlock { batch_size }` → `SendBlocks { blocks }`) and releases them
  as a batch. This is the primitive for a read → GPU → write pipeline inside
  one worker; today each `acquire_block()` is a synchronous round trip, so a
  worker can hold several blocks but never overlaps their latency.
- **Requeue failed blocks at the end of the queue** [S]. A block that fails
  goes straight back to the front of the ready queue, so a block that fails
  deterministically burns its retries, and possibly the worker restart
  budget, before the rest of the task has run. Appending it instead gives
  every other block a turn first and keeps a single bad block from
  abandoning a long task early.
- **Connection-aware worker accounting** [M]. daisy counts a worker as
  alive while its spawn function is running. A submit-and-return launcher
  (`sbatch` without `--wait`, `bsub` without `-K`, a bare `Popen`) therefore
  looks like a dead worker, gets respawned, and can abandon a task whose
  jobs are still queued. Tracking liveness by connection instead would make
  fire-and-forget launchers work and remove the hard requirement that spawn
  functions block.
- **Idle-worker release policy** [S, decision needed]. A worker whose task
  has no ready blocks is released after an idle period and respawned when
  work arrives, which costs a process start each time and, in the
  evaluation, turned a late failure into a restart-budget hit. Keeping
  workers until the task drains, or making the idle timeout per task, are
  the two candidates.
- **Retry backoff** [S]. `Task(retry_backoff=...)`, default off: a failed
  block waits `backoff * 2^retries` before becoming ready again, so transient
  failures against a shared resource do not retry in a storm.

## Multi-machine runs

- **A launcher and worker entry point** [L, RFC first]. Remote workers
  already work: the context reaches them through `DAISY_CONTEXT`, the
  server listens on all interfaces, and `daisy.Client()` connects back. What
  is missing is a packaged way to start them: a `python -m daisy.worker`
  entry point, a SLURM/LSF helper, and the deploy story for the user's
  process function and its imports. Workers also need a watchdog of their
  own, since the driver's Ctrl-C handler cannot reach a worker whose node
  hangs.

## Resume and inspection

- **Resume UX** [M]. The tracking store already makes re-runs skip done
  blocks and the pre-run log says how many remain. Still open: a helper that
  reports what is done and what failed before a re-run, and a dry-run mode
  that prints the planned block counts, ROIs and resource allocation
  without running anything.
- **Hand-launched workers without a spawn function** [M]. A run where the
  user starts every worker themselves and daisy only serves blocks, as 1.x
  allowed with `num_workers=0`. 2.0 refuses `max_workers=0` today.

## Deferred

- Worker init and cleanup hooks for block functions: the 0-arg worker
  function already gives a per-worker lifecycle, so this waits for a
  concrete need.
- A block prioritisation callback: FIFO until someone needs otherwise.