Overview
3xcode pyspark convert converts a single file by default, but pass --dir instead of a file path and it switches to bulk mode: every .sql file in the directory is discovered, planned, converted, validated, and auto-fixed through the same 5-phase pipeline, distributed across parallel workers. The whole run is tracked as a session: a durable record of which files completed, which failed, and where to pick up again if the CLI, your terminal, or your machine goes away mid-run.
Parallel by default
Bulk runs spread files across multiple workers automatically, with the worker count auto-detected from your machine's CPU and memory unless you override it.
Resumable by design
Every bulk run is a named session persisted to disk. An interrupted run (Ctrl-C, a closed laptop lid, a crashed process) picks up exactly where it left off, skipping files already completed.
Bulk Mode: --dir
Point --dir at a folder and 3XCode converts every .sql file inside it. Each file still goes through the full Discovery → Planning → Grouping → Conversion → Validation → Auto-Fix pipeline individually, but the CLI orchestrates the whole batch, shows a live multi-file progress display, and writes one audit report per file plus a session summary.
# Initialize a workspace once per project (creates input/, output/, .sessions/, logs/) 3xcode workspace init ./migration --name "warehouse-migration" # Convert every .sql file in a directory, 4 workers in parallel, hardest files first, # tracked under a named session for easy resume/status/cancel later 3xcode pyspark convert --dir ./sql/ -w 4 --schedule complex-first --session "batch-1" # --session sets a human-readable name, not the resumable ID -- look up the # real session ID (auto-generated, e.g. session-20260321-batch-1) with: 3xcode session list # Watch live progress at any point during the run 3xcode session status <session_id> --watch # If the run gets interrupted (Ctrl-C, crash, closed laptop), resume it later -- # already-completed files are skipped, only pending/failed files are re-run 3xcode pyspark convert --resume <session_id>
The --session flag names a session for readability, but the ID that --resume and session status expect is the auto-generated ID shown in the terminal output when the session starts. Run 3xcode session list at any time to look it up.
A bulk run requires an active workspace (workspace init or workspace switch) and a valid login/license: the same authentication and license checks that apply to a single-file conversion apply here too, since bulk mode still makes real LLM calls per file, it just fans them out.
Parallel Workers
The -w / --workers flag controls how many files convert concurrently during a bulk run.
Worker flag
| Flag | Default | Behavior |
|---|---|---|
-w N / --workers N | 0 | N parallel workers process files concurrently. `0` (the default) auto-detects a recommended worker count from your machine's CPU core count and available RAM. |
You can see the machine-specific auto-detected recommendation without starting a conversion by running 3xcode health, which reports CPU count, available and total RAM, and the recommended worker count for your system as part of its System Resources check. Pass an explicit -w value to override it: a higher count speeds up large batches on capable machines, a lower count keeps resource usage down on shared or constrained machines.
Scheduling Strategies
The --schedule flag controls the order in which files are handed to workers. Two strategies are available:
simple-first (default)
Simpler, faster files convert first, so you see completed output and early confidence scores sooner. Good default for mixed batches where you want quick feedback.
complex-first
The hardest, most time-consuming files are scheduled first, while workers are fresh and the full worker pool is available to absorb their longer runtimes. Reduces the chance a single large file becomes a straggler at the very end of the batch.
# Default: simple-first (quick wins land first) 3xcode pyspark convert --dir ./sql/ --session "batch-1" # Explicit: tackle the biggest, most complex files first 3xcode pyspark convert --dir ./sql/ --schedule complex-first --session "batch-1"
Session Naming & Resumability
Every bulk conversion (and every single-file conversion, for that matter) is tracked as a session with a unique ID. Naming your sessions makes it easy to find and resume them later.
Session-related conversion flags
| Flag | Default | Purpose |
|---|---|---|
--session NAME | None | Names the session for this bulk conversion, so it's easy to identify in `session list` and `session status`. |
--resume SESSION_ID | None | Resumes an interrupted or partially-completed session by ID, short-circuiting straight into resume logic: skips files already completed, retries pending/failed ones. |
-d / --detach | False | Runs the conversion as a detached background worker process that survives the CLI exiting. Returns immediately, printing the session ID. Capped at 3 concurrent detached workers machine-wide. |
A session's on-disk state lives under the active workspace's .sessions/ directory, so resumability survives closing your terminal, restarting your machine, or switching to a different terminal tab, as long as you're pointed at the same workspace, session resume and pyspark convert --resume find it.
Session Command Reference
The 3xcode session command group manages sessions directly, independent of starting a new conversion. Running bare 3xcode session with no subcommand defaults to session list.
session subcommands
| Command | Key flags | What it does |
|---|---|---|
session create | -n/--name, -d/--dir, --model | Creates a new session for a directory of SQL files without immediately starting the conversion run. |
session list | None | Table of sessions in the active workspace: ID, name, state (created/in progress/completed/partially completed/cancelled), file counts, cost, created date. Also the default action of bare `session`. |
session info <id> | None | Full session summary (name, state, model, file counts, cost, duration, success rate) plus a per-file table with status, cost, duration, and any error message. |
session resume <id> | --model, -v/--verbose | Resumes an interrupted or partially-completed session, converting only files that haven't completed yet. |
session status <id> | -w/--watch | Live status: per-status file counts, running worker's current phase/file/heartbeat, per-phase durations, and an ETA estimate. `--watch` tails live events in real time. |
session cancel <id> | -f/--force | Signals a running worker to stop cleanly between phases: completed files stay completed, in-progress files are marked interrupted, session state becomes cancelled. |
session rename <id> <new_name> | None | Renames a session without touching its ID or any converted output. |
session delete <id> | -f/--force | Deletes a session's input/output/log files. Refuses if a worker is currently running for it. Cancel first. |
session list-all | None | Lists currently-running sessions across every registered workspace on the machine, not just the active one. |
Workspace Organization
Sessions live inside a workspace, and 3xcode workspace init <path> --name <name> lays down the standard layout that every bulk run reads from and writes to:
input/
Source SQL files for the workspace's conversions, organized by session ID for bulk runs.
output/<session_id>/<stem>/
Per-session output, nested one level deeper per source file: generated PySpark modules under pyspark/*.py, JSON phase artifacts under reports/, a top-level *_audit.md and *_audit.json per file, and a CLI-compatible copy of the main pipeline file.
.sessions/
Durable session state and event logs: this is what makes session resume, session status, and session cancel work across process restarts.
logs/
Workspace-level logs. workspace status reports session counts, file counts, and the last session ID for the active workspace at a glance.
Use 3xcode workspace list to see every registered workspace on the machine and which one is active, workspace switch to move between projects, and workspace status for a quick summary of the active workspace's sessions and file counts. session status, session cancel, session delete, and session rename all resolve sessions across every registered workspace, not just the active one, so you don't need to switch workspaces just to check on or clean up a session elsewhere.