# nvidiactl

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Go Report Card](https://goreportcard.com/badge/codeberg.org/mutker/nvidiactl)](https://goreportcard.com/report/codeberg.org/mutker/nvidiactl)
[![Version](https://img.shields.io/badge/version-0.4.0-blue.svg)](https://codeberg.org/mutker/nvidiactl/releases)

nvidiactl is a command-line tool providing automatic fan speed management and dynamic power limit adjustment for NVIDIA GPUs. It can optionally be run as a systemd service.

## Features

- 🌡️ **Automatic fan speed control** based on GPU temperature
- 📈 **Opt-in load-driven fan control** (`load_fan`) ramps fans from SM utilization before thermal lag (requires metrics)
- ⚡ **Dynamic power limit management** to balance performance and noise
- 🎛️ **Customizable temperature thresholds** and fan speed limits
- 🔁 **Hysteresis support** to prevent rapid fluctuations in fan speed
- 🚀 **Performance mode** to target maximum power limit and skip thermal power throttling
- 📊 **Monitoring mode** for observing fan speed, temperature, and power without applying changes
- 📝 **Structured logging** to stdout (captured by systemd journal when run as a service)
- 🔧 **Direct GPU interaction** using NVML (NVIDIA Management Library)
- 🖥️ **Standalone application** or systemd service functionality
- 🔍 **Debug logging** via `--log-level debug` for detailed troubleshooting
- 📈 **Metrics collection** in local database for advanced statistics

## Requirements

- **Root privileges** — required to control NVIDIA GPU settings
- **Go 1.26+** to build from source (`go.mod` sets `toolchain go1.26.0`; CI installs Go 1.23 and upgrades via that directive)
- NVIDIA driver with NVML support

## Installation

### Arch Linux (AUR)

Install the `nvidiactl-git` package from the AUR using your preferred AUR helper. For example, using `yay`:

```
yay -S nvidiactl-git
```

After installation, enable and start the systemd service:

```
sudo systemctl enable --now nvidiactl.service
```

For debug logging, add an override with `sudo systemctl edit nvidiactl`:

```
[Service]
ExecStart=
ExecStart=/usr/bin/nvidiactl --log-level debug
```

### Building from Source

1. Ensure Go 1.26 or later is installed.

2. Clone and build:

   ```
   git clone https://codeberg.org/mutker/nvidiactl.git
   cd nvidiactl
   go build -v -o nvidiactl ./cmd/nvidiactl
   ```

3. Install the binary and configuration:

   ```
   sudo cp nvidiactl /usr/local/bin/
   sudo cp nvidiactl.example.conf /etc/nvidiactl.conf
   sudo cp nvidiactl.service /etc/systemd/system/
   sudo systemctl daemon-reload
   sudo systemctl enable --now nvidiactl.service
   ```

The bundled unit file targets `graphical.target` and uses `PartOf=graphical-session.target`, so it is intended for desktop sessions with an NVIDIA GPU.

## Configuration

Configuration uses a TOML file at `/etc/nvidiactl.conf`, command-line flags, or `NVIDIACTL_*` environment variables. Flags take precedence over the config file; environment variables override file values after the file is loaded.

See [`nvidiactl.example.conf`](nvidiactl.example.conf) for a documented starting point. Common options:

| Key / flag | Default | Purpose |
|------------|---------|---------|
| `interval` / `--interval` | `2` | Seconds between control ticks |
| `temperature` / `--temperature` | `80` | Maximum target temperature (°C) |
| `fanspeed` / `--fanspeed` | `100` | Maximum fan speed (%) |
| `hysteresis` / `--hysteresis` | `4` | Temperature delta before fan changes |
| `auto_fan_hysteresis` / `--auto-fan-hysteresis` | `4` | Margin below `min_temperature` before re-enabling auto fan |
| `performance` / `--performance` | `false` | Target max power; skip thermal throttling |
| `monitor` / `--monitor` | `false` | Observe only; no GPU mutations |
| `log_level` / `--log-level` | `"info"` | `debug`, `info`, `warning`, or `error` |
| `metrics` / `--metrics` | `false` | Enable SQLite metrics collection |
| `load_fan` / `--load-fan` | `false` | Opt-in utilization-driven fan ramps (requires `metrics`) |
| `load_power` / `--load-power` | `false` | Opt-in utilization-driven proactive power (requires `metrics`) |
| `database` / `--database` | `/var/lib/nvidiactl/metrics.db` | Metrics database path |
| `restoration_factor` / `--restoration-factor` | `2` | Power restoration aggressiveness after cooldown |

### Advanced options

Also available in the example config and `--help`:

- `min_temperature` — temperature before fan control activates (default `50`)
- `max_power_limit_change`, `watts_per_degree`, `power_limit_hysteresis` — power tuning
- `power_limit_window_size` — moving-average window for power history
- `performance_pow_factor`, `normal_pow_factor` — fan curve exponents
- `load_fan`, `load_fan_threshold`, `load_fan_pow_factor`, `load_fan_hysteresis_percent`, `load_max_age_seconds` — load-driven fan tuning (see below)
- `load_power`, `load_power_floor_threshold`, `load_power_ceiling_threshold`, `load_power_floor_max_watts`, `load_power_ceiling_max_watts`, `load_power_hysteresis_percent`, `load_power_pow_factor` — load-driven power tuning (see below)
- `operation_timeout` — per-tick timeout in seconds (default `2`)
- `cleanup_timeout` — forced shutdown delay after second interrupt (default `5`)
- `--config` — explicit config file path

Environment variables use the `NVIDIACTL_` prefix with underscores, for example `NVIDIACTL_LOG_LEVEL=debug` or `NVIDIACTL_METRICS=true`.

Send `SIGHUP` to reload the config file and environment variables. CLI flags are not re-applied on reload. Reload stops sampler goroutines and invalidates the in-memory utilization cache; the next trend sample reseeds it.

### Load-driven fan control (`load_fan`, opt-in)

When `load_fan = true` and `metrics = true`, the 10s trend sampler feeds an in-memory utilization cache that the 2s control tick reads without extra NVML or SQLite calls. Fan targets merge as `max(temp_target, load_target)` once temperature is at or above `min_temperature` and a fan curve exists. Missing, stale, or fail-closed utilization falls back to temperature-only behavior identical to the default path.

Defaults: `load_fan_threshold = 50`, `load_fan_pow_factor = 1.0`, `load_fan_hysteresis_percent = 10`, `load_max_age_seconds = 10`. Config validation rejects `load_fan` without `metrics`.

Legacy config keys (`load_aware`, `util_fan_threshold`, etc.) are accepted with a deprecation log for one release cycle.

### Load-driven proactive power (`load_power`, opt-in)

When `load_power = true` and `metrics = true`, the same trend-sampler utilization cache drives proactive power limits on the 2s tick without extra NVML or SQLite reads. Low utilization applies a floor branch (lower watts); high utilization applies a ceiling branch (raise watts). Results merge with thermal `calculateLoopPowerLimit` output; thermal emergency reduction always wins over load ceiling. `performance_mode` bypasses load-driven branches.

Defaults: `load_power_floor_threshold = 30`, `load_power_ceiling_threshold = 50`, `load_power_floor_max_watts = 30`, `load_power_ceiling_max_watts = 30`, `load_power_hysteresis_percent = 10`, `load_power_pow_factor = 1.0` (reuses `load_max_age_seconds` staleness). Config validation rejects `load_power` without `metrics`. Independent of `load_fan`.

Legacy keys (`util_aware_power`, `util_power_*`, etc.) are accepted with a deprecation log.

## Usage

Run after configuring `/etc/nvidiactl.conf`:

```
sudo nvidiactl
```

Examples:

```
nvidiactl --temperature=85 --fanspeed=80 --performance
nvidiactl --metrics
nvidiactl --monitor
nvidiactl --config /path/to/nvidiactl.conf
nvidiactl --version
```

## Metrics inspection

When metrics collection is enabled (`metrics = true` in config or `--metrics`), the daemon writes control-loop samples to a SQLite database (schema version 7, default `/var/lib/nvidiactl/metrics.db`, mode `0640`). The v7 schema is a five-table shape (`schema_migrations`, `gpu`, `sample`, `event`, `policy`) with `schema_migrations` as the single source of truth; throttle (2s) and utilization trend (10s) rows are source-partitioned in `sample` without extra NVML calls on the control tick. Throttle rows carry fan and power merge telemetry (`fan_merge_reason`, `load_fan_target_percent`, `power_merge_reason`, `load_power_target_watts`). Pre-v7 databases (legacy v3/v4/v5/v6) are converted to v7 in a single transaction by `nvidiactl migrate`, with a `metrics_pre_v7_<unix_ms>.db` backup; ECC collection was dropped in v7 and legacy `metrics_ecc_v1` rows are not backfilled.

Create a `metrics` group and grant read access so tuners can inspect history without root:

```bash
sudo groupadd -f metrics
sudo usermod -aG metrics "$USER"
sudo chgrp metrics /var/lib/nvidiactl/metrics.db
sudo chmod 0640 /var/lib/nvidiactl/metrics.db
```

Override the database path with `database` in config, `NVIDIACTL_DATABASE`, or `--database` on stats commands.

Examples (no root required when the DB is group-readable):

```bash
nvidiactl stats
nvidiactl stats --format json
nvidiactl stats query --since 1h --format csv
nvidiactl stats schema
nvidiactl stats query --since 30m --bucket 2s --aggregate last --fields temp_current,fan_speed_delta_percent,util_sm_percent,fan_merge_reason,load_cache_quality,throttle_sw_thermal_slowdown
nvidiactl stats query --format prometheus-text --fields temp_current,power_draw_milliwatts,gpu_uuid
nvidiactl stats reset --confirm --database /var/lib/nvidiactl/metrics.db
```

Pass `--config /etc/nvidiactl.conf` to compute derived `policy_temp_headroom_celsius` from the configured max temperature. With v4 data, `stats` also joins `gpu_device` and nearest sampler rows for `temp_margin_celsius`, `power_headroom_watts`, throttle flags, and control-loop `fan_action_outcome` / `power_action_outcome`. Sampler failures write `tick_observation_quality=fail_closed` throttle rows (no silent gaps).

### Metrics sampler benchmark

Before adding sampler tables, measure NVML latency and mutex contention on your GPU:

```bash
sudo nvidiactl bench sampler
sudo nvidiactl bench sampler --format json --iterations 300 --contention-duration 60s
```

The report includes isolated tick/throttle/trend p99 latencies, TryLock failure rate under simulated tick contention, pass/fail gates, and a recommended `fail_closed` row protocol. Archive the JSON output when planning extended telemetry.

### Load-aware fan control benchmark

Verify load-fan merge behavior without a long daemon run:

```bash
# Synthetic (default): Decision #3 scenario matrix, no GPU workload, no root
nvidiactl bench load-fan --mode synthetic
nvidiactl bench load-fan --mode synthetic --format json > load-fan-bench-$(date +%F).json

# Live: real NVML + trend sampler observation window (requires root and NVIDIA GPU)
sudo nvidiactl bench load-fan --mode live --format json --duration 30s > load-fan-live-$(date +%F).json
```

**Synthetic mode** runs nine in-process scenarios (missing/stale/fail-closed util, temp floor, max-merge load_wins and temp_wins, hysteresis skip, symmetric manual gates, no fan curve) through the real control-loop decide/merge path with injected utilization cache snapshots. It asserts the tick NVML read profile is unchanged versus temperature-only control.

**Live mode** opens a real GPU controller, starts the metrics coordinator with an in-memory utilization cache, and polls control ticks for `--duration` (default 30s). The `live_util_observed` gate is informational: an idle GPU blocks that gate with an expected message while synthetic gates still determine `load_fan_gate_ready`.

Policy overrides mirror config knobs: `--load-fan`, `--load-fan-threshold`, `--load-fan-pow-factor`, `--util-hysteresis`, `--util-max-age`, and `--device`.

Archivable JSON includes GPU metadata, scenario results, gate pass/fail, and `load_fan_gate_ready` (true when synthetic verification passes, even if live util observation is blocked on an idle GPU):

```json
{
  "gpu_name": "synthetic (in-process)",
  "load_fan_gate_ready": true,
  "gates": [
    {"name": "synthetic_matrix_passed", "passed": true},
    {"name": "tick_nvml_read_set_unchanged", "passed": true}
  ],
  "summary": "Synthetic verification passed."
}
```

Use `stats` for post-hoc `fan_merge_reason` / `power_merge_reason` history; use `bench load-fan` and `bench load-power` for repeatable pre-ship or config-change verification.

### Util-aware power benchmark

```bash
nvidiactl bench load-power --mode synthetic
nvidiactl bench load-power --mode synthetic --format json > load-power-bench-$(date +%F).json
sudo nvidiactl bench load-power --mode live --format json --duration 30s
```

Synthetic mode runs eight in-process Decision #4 scenarios (util missing/stale/fail_closed, floor win, ceiling win, hysteresis skip, thermal override, performance_mode bypass) and asserts the tick NVML read profile is unchanged. Live mode uses an informational `live_power_observed` gate; `load_power_gate_ready` follows synthetic gates.

## Releasing

Release tags must be strict semver: `vMAJOR.MINOR.PATCH` (for example `v1.2.3`).

Before tagging, align:

1. **`VERSION`** — must match the tag without the `v` prefix
2. **`README.md`** — version badge must match
3. **`CHANGELOG.md`** — must contain a `## [MAJOR.MINOR.PATCH]` section with release notes

Forgejo CI (`.forgejo/workflows/ci.yaml`) validates these on tag push, builds with `-X main.Version=…`, and publishes release notes from the CHANGELOG section. Promote `[Unreleased]` entries to a versioned heading before tagging.

## Roadmap

- Add presets for fan and power limit adjustment curves that can be applied during runtime
- Enhance metrics collection with additional metrics: frequency, memory usage, processes, etc.
- Add objective-based (perf vs power vs noise) fan and power limit adjustment curves utilizing collected statistics

## Architecture

See [ARCHITECTURE.md](ARCHITECTURE.md) for detailed design documentation.

## Contributing

Contributions are welcome! Please feel free to submit a pull request or issue.

## License

This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.
