training-throttle

System-aware training throttle with 4 load levels. Detects CPU/GPU load and adapts batch size, worker count, and GPU memory allocation automatically.

Zero hard dependencies. Uses os.getloadavg out of the box; psutil is optional for richer load data.

Quickstart

from training_throttle import TrainingThrottle

throttle = TrainingThrottle()
state = throttle.check()
print(state.summary())              # [full] batch×1.00 workers=4 gpu=100% | fleet_load=0.12 zone=idle
batch = throttle.effective_batch_size(base_batch=64)  # → 64
workers = state.num_workers         # → 4
gpu_frac = state.gpu_fraction       # → 1.0

Load Zones

| Load | Level | Batch | Workers | GPU | Interval | |--------|----------|-------|---------|------|----------| | 0–30% | FULL | ×1.0 | 4 | 100% | 30s | | 30–60% | REDUCED | ×0.5 | 2 | 50% | 15s | | 60–85% | MINIMAL | ×0.25 | 1 | 25% | 10s | | 85–100%| PAUSED | ×0.0 | 0 | 0% | 5s |

API

| Method | Description | |--------|-------------| | TrainingThrottle(min_level, prefer_gpu, custom_load_fn) | Create a throttle instance | | .check() | Probe load and return ThrottleState | | .should_check() | Whether enough time elapsed for a new check | | .fleet_load() | Return composite 0–1 load value | | .effective_batch_size(base_batch) | Adjusted batch size ≥ 1 | | .wait_for_idle(timeout, poll) | Block until not PAUSED | | .history() | Full (timestamp, ThrottleState) list | | .summary() | One-line human-readable status | | ThrottleState.should_train | True unless PAUSED | | ThrottleState.to_dict() | Serialize to plain dict |

Options

# Skip GPU detection (CPU-only machines)
throttle = TrainingThrottle(prefer_gpu=False)

# Never run above REDUCED
throttle = TrainingThrottle(min_level=ThrottleLevel.REDUCED)

# Custom load function
throttle = TrainingThrottle(custom_load_fn=lambda: 0.42)

Ecosystem

Part of the SuperInstance project. Auto-registers as a superinstance.plugins entry-point when installed alongside plato-core.

License

MIT