Reference
Configuration Reference
Complete key-by-key reference for global ~/.config/spotexec/config.toml, committed .spotexec.toml, tasks, hooks, and project-defined fleets.
spotexec splits configuration into two TOML files so credentials and hardware stay private while project tasks and caches travel with Git:
| File | Purpose | Committed to Git? |
|---|---|---|
~/.config/spotexec/config.toml | Your runners, SSH hosts, AWS accounts, regions, VPCs, and cost caps | Never |
.spotexec.toml (repo root) | How this project syncs, caches dependencies, builds containers, and runs tasks | Yes |
Precedence Order (Highest First)#
- CLI flags (
--timeout,-r,--sync,--pool, etc.) SPOTEXEC_*environment variables (SPOTEXEC_TIMEOUT,SPOTEXEC_SYNC,SPOTEXEC_RUNNER,SPOTEXEC_POOL,SPOTEXEC_PRIORITY,SPOTEXEC_JSON=1,SPOTEXEC_AGENT=1).spotexec.tomlin the repository root~/.config/spotexec/config.toml- Built-in defaults
(Exceptions: -r / SPOTEXEC_RUNNER overrides any pool; --json / SPOTEXEC_JSON=1 overrides --pretty; a task's explicit pool overrides SPOTEXEC_POOL.)
Global Config: ~/.config/spotexec/config.toml#
version = 1
default_pool = ["homelab", "aws"] # runners tried in priority order
# local_fallback = false # run locally when no runner is reachable
[runners.homelab]
type = "ssh"
hosts = ["me@192.168.1.50", "me@homelab.tailnet.ts.net"]
max_slots = 4
priority = 10
[runners.aws]
type = "aws-spot"
region = "us-east-2"
profile = "spotexec"
instance_types = ["c7a.4xlarge", "c6a.4xlarge", "m7a.4xlarge"]
max_slots = 8
max_instances = 4
min_instances = 0 # kept warm only while `spotexec daemon` runs
priority = 20
idle_timeout = "10m"
max_lifetime = "12h"
volume_gb = 64
hourly_cost = 0.25 # used by `spotexec monitor` for $/hr burn rate
bootstrap_script = """
apt-get update -o DPkg::Lock::Timeout=300 && apt-get install -y build-essential
"""
user_bootstrap_script = """
curl -fsSL https://mise.run | sh
"""
remote_path_setup = 'export PATH="$HOME/.local/bin:$HOME/.local/share/mise/shims:$PATH"'Common Runner Keys#
| Key | Default | Description |
|---|---|---|
type | (required) | "ssh", "aws-spot", or "sim" (local Docker simulator). |
max_slots | vCPUs ÷ 4 (min 1) | Concurrent isolated slots per machine (1–44). |
priority | 0 | Lower numbers run first; higher numbers receive overflow demand. |
work_root | ~/.cache/spotexec | Remote directory storing the helper binary, slots, and dependency caches. |
remote_path_setup | (none) | Shell snippet sourced before commands outside [container] (e.g. mise or nvm PATH). |
ssh_options | (none) | Extra -o Key=Value options passed to OpenSSH. |
hourly_cost | (none) | Estimated USD/hour per instance displayed in spotexec monitor. |
ssh Runner Keys#
| Key | Description |
|---|---|
host / hosts | Single SSH destination or list of routes to the same machine (user@host, hostname, or ~/.ssh/config alias), dialed in parallel with a 500ms preference window for earlier entries. |
user | SSH login username when omitted from host. |
identity_file | Path to private key (defaults to ~/.ssh/config resolution). |
aws-spot Runner Keys#
| Key | Default | Description |
|---|---|---|
region | AWS_REGION / us-east-1 | Target EC2 region (always set explicitly). |
profile | (env / default) | AWS credentials/config profile name (spotexec after spotexec setup). |
instance_types | c7a.4xlarge, c6a.4xlarge, m7a.4xlarge | Instance types tried in order on Spot capacity errors (must share one CPU architecture). |
arch | Derived from first type | "amd64" or "arm64" (Graviton). |
max_instances | 4 | Maximum concurrent VMs in this fleet. |
min_instances | 0 | Warm VM floor maintained while spotexec daemon is running. |
idle_timeout | "10m" | Duration without active slots before the VM powers itself off. |
max_lifetime | "12h" | Unconditional VM lifetime cap (must exceed boot_timeout). |
boot_timeout | "15m" | Maximum time allowed for first-boot scripts before the VM is marked failed. |
bootstrap_script | (none) | First-boot root bash -e script run after the watchdog is armed. |
user_bootstrap_script | (none) | First-boot login-shell script run as user after bootstrap_script. |
max_price | On-demand price | Maximum Spot bid in USD/hour as a string (e.g. "0.40"). |
on_demand_fallback | false | Fall back to On-Demand if all Spot pools return insufficient capacity. |
volume_gb | 64 | Root gp3 EBS volume size in GiB (minimum 8). |
ami | "auto" | "auto" (newest matching baked AMI, else Ubuntu 24.04), "base", or explicit "ami-...". |
user | "ubuntu" | SSH login user for the AMI. |
subnet_id, security_group_ids | Default VPC + spotexec-ssh | Custom public subnet and security groups. |
ssh_cidr | "0.0.0.0/0" | Allowed source CIDR for the managed spotexec-ssh group (Ed25519 key auth only). |
cache_bucket | "auto" | "auto" (spotexec-cache-<acct>-<region>), custom S3 bucket name, or "off". |
Project Config: .spotexec.toml#
Run spotexec init at the root of a repository to auto-generate .spotexec.toml for npm, pnpm, Yarn, Bun, Go, Rust, or Python projects:
version = 1
timeout = "5m" # per-command timeout (exits 124)
grace_period = "5s" # delay between SIGTERM and SIGKILL
queue_timeout = "10m" # max wait for a busy pool
env_allow = ["NODE_OPTIONS"] # local env vars forwarded via 0600 file
env = { TZ = "UTC" } # static env vars set on every remote run
pool = ["aws"] # optional override of global default_pool
[sync]
mode = "auto" # auto (write back) | in (discard remote edits) | none
writeback = "always" # always | on-success | never
exclude = [".env*", "*.pem"] # extra excludes on top of .gitignore & .spotexecignore
pull_artifacts = ["coverage/"] # gitignored outputs copied back automatically
[deps]
lockfiles = ["pnpm-lock.yaml"]
install = "pnpm install --frozen-lockfile"
cache_dirs = ["node_modules", "packages/*/node_modules"]
hydrate = "auto" # auto (reflink, else copy) | reflink | copy | hardlink
[container]
dockerfile = ".spotexec/Dockerfile"
[tasks.test]
run = "pnpm vitest run --minWorkers=1 --maxWorkers=$SPOTEXEC_CPUS {args}"
[tasks.typecheck]
run = "pnpm tsc --noEmit"
[tasks.verify]
parallel = ["typecheck", "test"]
fail_fast = false
[hook]
when_load_above = 0.6 # Claude Code hook offloads only when laptop is busy[tasks.<name>] Keys#
| Key | Description |
|---|---|
run | Shell command string. {args} expands positional arguments passed after sx <task>; if omitted, extra args are appended. |
parallel | List of non-parallel task names to fan out concurrently across separate slots (sx verify). |
fail_fast | Cancel sibling parallel tasks immediately when one fails. |
timeout, sync, exclusive | Per-task execution overrides. |
pool, priority | Route specific tasks (e.g. e2e) to larger fleets or higher queue priority. |
Sandboxed Project Fleets (extends)#
A repository can customize VM sizing or bootstrap scripts by extending one of your global runners—without ever being allowed to alter SSH targets, credentials, regions, or VPCs:
pool = ["ci-big"]
[runners.ci-big]
extends = "aws" # must reference a runner in ~/.config/spotexec/config.toml
instance_types = ["c7a.2xlarge", "m7a.2xlarge"]
max_slots = 2
max_instances = 3