# Configuration Reference

`spotexec` splits configuration into two TOML files so credentials and hardware stay private while project tasks and caches travel with Git:

| File | Purpose | Committed to Git? |
| --- | --- | --- |
| `~/.config/spotexec/config.toml` | Your runners, SSH hosts, AWS accounts, regions, VPCs, and cost caps | **Never** |
| `.spotexec.toml` (repo root) | How this project syncs, caches dependencies, builds containers, and runs tasks | **Yes** |

### Precedence Order (Highest First)

1. CLI flags (`--timeout`, `-r`, `--sync`, `--pool`, etc.)
2. `SPOTEXEC_*` environment variables (`SPOTEXEC_TIMEOUT`, `SPOTEXEC_SYNC`, `SPOTEXEC_RUNNER`, `SPOTEXEC_POOL`, `SPOTEXEC_PRIORITY`, `SPOTEXEC_JSON=1`, `SPOTEXEC_AGENT=1`)
3. `.spotexec.toml` in the repository root
4. `~/.config/spotexec/config.toml`
5. Built-in defaults

*(Exceptions: `-r` / `SPOTEXEC_RUNNER` overrides any pool; `--json` / `SPOTEXEC_JSON=1` overrides `--pretty`; a task's explicit `pool` overrides `SPOTEXEC_POOL`.)*

---

## Global Config: `~/.config/spotexec/config.toml`

```toml
version = 1
default_pool = ["homelab", "aws"]   # runners tried in priority order
# local_fallback = false            # run locally when no runner is reachable

[runners.homelab]
type = "ssh"
hosts = ["me@192.168.1.50", "me@homelab.tailnet.ts.net"]
max_slots = 4
priority = 10

[runners.aws]
type = "aws-spot"
region = "us-east-2"
profile = "spotexec"
instance_types = ["c7a.4xlarge", "c6a.4xlarge", "m7a.4xlarge"]
max_slots = 8
max_instances = 4
min_instances = 0                 # kept warm only while `spotexec daemon` runs
priority = 20
idle_timeout = "10m"
max_lifetime = "12h"
volume_gb = 64
hourly_cost = 0.25                # used by `spotexec monitor` for $/hr burn rate
bootstrap_script = """
apt-get update -o DPkg::Lock::Timeout=300 && apt-get install -y build-essential
"""
user_bootstrap_script = """
curl -fsSL https://mise.run | sh
"""
remote_path_setup = 'export PATH="$HOME/.local/bin:$HOME/.local/share/mise/shims:$PATH"'
```

### Common Runner Keys

| Key | Default | Description |
| --- | --- | --- |
| `type` | *(required)* | `"ssh"`, `"aws-spot"`, or `"sim"` (local Docker simulator). |
| `max_slots` | `vCPUs ÷ 4` (min `1`) | Concurrent isolated slots per machine (`1`–`44`). |
| `priority` | `0` | Lower numbers run first; higher numbers receive overflow demand. |
| `work_root` | `~/.cache/spotexec` | Remote directory storing the helper binary, slots, and dependency caches. |
| `remote_path_setup` | *(none)* | Shell snippet sourced before commands outside `[container]` (e.g. `mise` or `nvm` PATH). |
| `ssh_options` | *(none)* | Extra `-o Key=Value` options passed to OpenSSH. |
| `hourly_cost` | *(none)* | Estimated USD/hour per instance displayed in `spotexec monitor`. |

### `ssh` Runner Keys

| Key | Description |
| --- | --- |
| `host` / `hosts` | Single SSH destination or list of routes to the same machine (`user@host`, hostname, or `~/.ssh/config` alias), dialed in parallel with a 500ms preference window for earlier entries. |
| `user` | SSH login username when omitted from `host`. |
| `identity_file` | Path to private key (defaults to `~/.ssh/config` resolution). |

### `aws-spot` Runner Keys

| Key | Default | Description |
| --- | --- | --- |
| `region` | `AWS_REGION` / `us-east-1` | Target EC2 region (always set explicitly). |
| `profile` | *(env / default)* | AWS credentials/config profile name (`spotexec` after `spotexec setup`). |
| `instance_types` | `c7a.4xlarge`, `c6a.4xlarge`, `m7a.4xlarge` | Instance types tried in order on Spot capacity errors (must share one CPU architecture). |
| `arch` | Derived from first type | `"amd64"` or `"arm64"` (Graviton). |
| `max_instances` | `4` | Maximum concurrent VMs in this fleet. |
| `min_instances` | `0` | Warm VM floor maintained while `spotexec daemon` is running. |
| `idle_timeout` | `"10m"` | Duration without active slots before the VM powers itself off. |
| `max_lifetime` | `"12h"` | Unconditional VM lifetime cap (must exceed `boot_timeout`). |
| `boot_timeout` | `"15m"` | Maximum time allowed for first-boot scripts before the VM is marked failed. |
| `bootstrap_script` | *(none)* | First-boot root `bash -e` script run after the watchdog is armed. |
| `user_bootstrap_script` | *(none)* | First-boot login-shell script run as `user` after `bootstrap_script`. |
| `max_price` | On-demand price | Maximum Spot bid in USD/hour as a string (e.g. `"0.40"`). |
| `on_demand_fallback` | `false` | Fall back to On-Demand if all Spot pools return insufficient capacity. |
| `volume_gb` | `64` | Root `gp3` EBS volume size in GiB (minimum `8`). |
| `ami` | `"auto"` | `"auto"` (newest matching baked AMI, else Ubuntu 24.04), `"base"`, or explicit `"ami-..."`. |
| `user` | `"ubuntu"` | SSH login user for the AMI. |
| `subnet_id`, `security_group_ids` | Default VPC + `spotexec-ssh` | Custom public subnet and security groups. |
| `ssh_cidr` | `"0.0.0.0/0"` | Allowed source CIDR for the managed `spotexec-ssh` group (Ed25519 key auth only). |
| `cache_bucket` | `"auto"` | `"auto"` (`spotexec-cache-<acct>-<region>`), custom S3 bucket name, or `"off"`. |

---

## Project Config: `.spotexec.toml`

Run `spotexec init` at the root of a repository to auto-generate `.spotexec.toml` for npm, pnpm, Yarn, Bun, Go, Rust, or Python projects:

```toml
version = 1
timeout = "5m"                 # per-command timeout (exits 124)
grace_period = "5s"            # delay between SIGTERM and SIGKILL
queue_timeout = "10m"          # max wait for a busy pool
env_allow = ["NODE_OPTIONS"]   # local env vars forwarded via 0600 file
env = { TZ = "UTC" }           # static env vars set on every remote run
pool = ["aws"]                 # optional override of global default_pool

[sync]
mode = "auto"                  # auto (write back) | in (discard remote edits) | none
writeback = "always"           # always | on-success | never
exclude = [".env*", "*.pem"]   # extra excludes on top of .gitignore & .spotexecignore
pull_artifacts = ["coverage/"] # gitignored outputs copied back automatically

[deps]
lockfiles = ["pnpm-lock.yaml"]
install = "pnpm install --frozen-lockfile"
cache_dirs = ["node_modules", "packages/*/node_modules"]
hydrate = "auto"               # auto (reflink, else copy) | reflink | copy | hardlink

[container]
dockerfile = ".spotexec/Dockerfile"

[tasks.test]
run = "pnpm vitest run --minWorkers=1 --maxWorkers=$SPOTEXEC_CPUS {args}"

[tasks.typecheck]
run = "pnpm tsc --noEmit"

[tasks.verify]
parallel = ["typecheck", "test"]
fail_fast = false

[hook]
when_load_above = 0.6          # Claude Code hook offloads only when laptop is busy
```

### `[tasks.<name>]` Keys

| Key | Description |
| --- | --- |
| `run` | Shell command string. `{args}` expands positional arguments passed after `sx <task>`; if omitted, extra args are appended. |
| `parallel` | List of non-parallel task names to fan out concurrently across separate slots (`sx verify`). |
| `fail_fast` | Cancel sibling parallel tasks immediately when one fails. |
| `timeout`, `sync`, `exclusive` | Per-task execution overrides. |
| `pool`, `priority` | Route specific tasks (e.g. `e2e`) to larger fleets or higher queue priority. |

### Sandboxed Project Fleets (`extends`)

A repository can customize VM sizing or bootstrap scripts by extending one of your global runners—without ever being allowed to alter SSH targets, credentials, regions, or VPCs:

```toml
pool = ["ci-big"]

[runners.ci-big]
extends = "aws"                # must reference a runner in ~/.config/spotexec/config.toml
instance_types = ["c7a.2xlarge", "m7a.2xlarge"]
max_slots = 2
max_instances = 3
```