Symphony / Workspace cleanup timed job

Nightly workspace cleanup

A direct housekeeping timer that removes idle Symphony per-issue workspaces after their TTL while preserving unrelated state and reusable caches.

Cadence 07:00 host timeMode direct housekeepingScope ENG-123 style dirsOutput removed path + size

systemd timer->workspace scan->TTL filter->safe removal

Operating model

timed job

Symphony can preserve workspaces for retries and continuations, but old dehydrated directories become disk overhead. This timer prunes only issue-shaped directories under the configured workspaces root after the TTL.

07:00

Nightly start

systemd calendar

Default TTL

WORKSPACE_TTL_DAYS

Path regex

uppercase key plus number

Cache removal

workspace dirs only

KEYDelete only expired issue workspaces

The regex protects the root from broad deletion. Anything that is not shaped like an issue key is left alone, even if it is old.

System shape

click the nodes

This job is direct host maintenance. Its safety comes from a narrow root, an issue-key directory regex, and an mtime cutoff.

Cadence design

compare modes

Each timed job has a small set of modes that decide whether the run is healthy, degraded, or ready to create follow-up work. Use the controls to compare the operating contract.

Root

Verify the cleanup root

The script should operate only under the Symphony workspaces root.

WORKSPACES_ROOT defaults correctly
missing root exits 0
only first-level dirs considered
operator-owned side paths ignored

Cleanup safety threshold82%

Run manually with a temporary root or inspect candidates before enabling deletion.

Execution tracks

interactive

The tracks make each run inspectable. They separate inputs, outputs, and failure modes so reports do not collapse into vague status updates.

Per-issue directories

Workspace scope

Operate only on Symphony issue workspaces, not arbitrary files under the host.

Inputs

WORKSPACES_ROOT
directory names
mtime

Outputs

candidate list
no-op message
skipped paths

Cleanup risk map

priority map

Workspace cleanup is valuable only when the deletion boundary is obvious and auditable.

Workspace rootwrong root can target the wrong tree

96

Issue-key regexnon-workspace directories must be skipped

92

TTL mtime cutoffrecent handoff or retry directories must remain available

86

Max-depth searchnested state should not be discovered accidentally

80

Deletion loggingoperators need path and size evidence

72

Cache preservationreusable caches are outside this timer scope

62

This timer deliberately does not inspect Linear state. It relies on age and naming, while Symphony itself cleans terminal active workspaces during reconciliation.

Output rubric

ticket rules

The run should create follow-up work only when the output is concrete. Use the rubric to keep reports, bug tickets, and operational follow-ups separated.

Removed

Expired issue-shaped workspace directories were removed and logged with approximate size.

ENG-123 idle over 7d
MT-456 removed

Run contract

evidence rows

The job contract is a bounded deletion set plus journal evidence for every removed workspace.

Service

Direct timer

symphony-workspace-cleanup.timer starts workspace_ttl_cleanup.sh as the symphony user.

Boundary

Issue-shaped dirs only

Candidates must be first-level directories under WORKSPACES_ROOT, match the issue-key regex, and be older than the TTL.

WORKSPACES_ROOT

WORKSPACE_TTL_DAYS

Root exists or no-op message

Issue-key regex used

Candidate count

Removed path and size

No candidates older than TTL message

Timer status

Command plan

operator path

Use a manual run to inspect behavior, then enable the nightly timer.

Manual runsudo -u symphony WORKSPACE_TTL_DAYS=7 /opt/symphony/elixir/scripts/workspace_ttl_cleanup.sh

Timer installsudo systemctl enable --now symphony-workspace-cleanup.timer

List timersystemctl list-timers | grep symphony-workspace-cleanup

Read journaljournalctl -u symphony-workspace-cleanup.service -n 80 --no-pager

!Keep the regex narrow

Do not broaden the find expression unless the workspace naming contract changes in Symphony itself.

Rollout path

ship safely

Inspect the root scope

Confirm /var/lib/symphony/workspaces contains only per-issue workspace directories and expected support files.

Run manually candidate check

Use the default TTL and read the deletion or no-op output before enabling the timer.

Enable nightly cleanup systemd

Install the unit files, daemon-reload, and enable the timer.

Tune TTL under load capacity

Adjust WORKSPACE_TTL_DAYS only when retry and rework behavior proves the current window is too long or too short.