#piper-tts-training

A reproducible pipeline for fine-tuning a Piper VITS voice on a single rented (or local) NVIDIA GPU, then exporting it to ONNX for use with Piper / Home Assistant.

This repo is code only — the scripts, the environment recipe, and (importantly) the non-obvious fixes needed to get piper1-gpl training end-to-end. It ships no audio and no models: you bring your own dataset.

📋 Operational briefing: PIPER_TTS_TRAINING_CONTEXT.md — a Context Capsule with the bounds, gotchas, and calibration in one place.

#⚠️ Training data — read this first

This repo contains no data or trained models. Only train on audio you own or are licensed to use. Cloning a real, identifiable person's voice, or training on copyrighted recordings, without permission may infringe copyright, performers'/publicity rights, or trademarks depending on your jurisdiction. You are responsible for how you use this pipeline.

#What you need

• An NVIDIA GPU box, Ubuntu 22.04, ~40 GB free disk. (Developed on a DigitalOcean H100 Droplet with the "NVIDIA AI/ML Ready" image, but any CUDA box works — including an NVIDIA DGX Spark.)
• A dataset of (audio, transcript) pairs on the Hugging Face Hub, or adapt prepare_dataset.py to your source.
• Cost tip: a small single-speaker VITS voice cannot saturate a big GPU — training is bound by per-step latency, not compute (see Gotchas). A cheap 24–48 GB card (RTX 4000/6000 Ada, L40S) finishes in about the same wall-clock as an H100 for a fraction of the price. Don't overpay for the GPU.

#Quick start

git clone git@github.com:LewisSmallwood/piper-tts-training.git ~/piper-tts-training
cd ~/piper-tts-training
export WORKDIR=~/piper-tts-training

bash setup.sh                                    # env + framework + the fixes below
python3 prepare_dataset.py --dataset ORG/YOUR_DATASET   # -> dataset/wav + dataset/metadata.csv
VOICE_NAME=en_US-yourvoice-medium bash train.sh  # resumes from newest ckpt if present
VOICE_NAME=en_US-yourvoice-medium bash export.sh # -> output/en_US-yourvoice-medium.onnx(.json)

Fine-tuning from a base voice: download a base checkpoint (e.g. from rhasspy/piper-checkpoints on Hugging Face) into checkpoints/base.ckpt before the first train.sh — train.sh resumes from the newest checkpoint it finds and falls back to checkpoints/base.ckpt.

#Steps in detail

To tune the delivered voice, edit inference.noise_scale in the exported .onnx.json (≈0.4–0.5; lower = more consistent/robotic). It's read at inference — no re-export needed.

#Live dashboard (optional)

dashboard/ is a small React + Vite app that reads the training log and shows overall progress, epoch %, speed and ETA, auto-refreshing every 5 s. It reads over SSH (or a local file):

cd dashboard
npm install
HOST=root@<gpu-ip> LOG=~/piper-tts-training/training.log BASE_EPOCH=<resume-epoch> npm run dev
# open http://localhost:3000

Prefer the terminal? tail -f training.log works too. (HOST/LOG/BASE_EPOCH/MAX_EPOCH are all env-configurable; omit HOST to read a local log.)

#Gotchas (why this repo exists)

These are the things that silently break a from-scratch piper1-gpl fine-tune. All are handled by setup.sh / the scripts:

1. espeakbridge won't compile without the espeak-ng dev headers (libespeak-ng-dev, not just the espeak-ng runtime). And a non-editable wheel install drops the piper.train package. Working combo (in setup.sh): build the wheel once to compile espeakbridge.so, stash it, reinstall editable (restores piper.train), then copy the .so back into src/piper/. It's an abi3 build, so it's portable across Python 3.x.
2. datasets>=4.0 requires torchcodec to decode audio, or your prep yields 0 samples. Pin datasets<4.0 (uses soundfile). Done in setup.sh.
3. Pin torch<2.6 (this uses 2.5.1). Torch 2.6 changed torch.load to weights_only=True, which breaks resuming Lightning checkpoints unless you patch add_safe_globals.
4. Old checkpoints may carry a sample_bytes hyperparameter the current model rejects on resume (fit does not accept option 'model.sample_bytes'). See patches/checkpoint-compat.md for the one-line fix.
5. Export race: Lightning keeps only the top-1 checkpoint and renames it every epoch, so ls -t | head -1 can 404 mid-training. export.sh copies the checkpoint to a stable path before exporting.
6. A tiny model can't saturate a large GPU. Measured on an H100: ~16 s/epoch is irreducible serial per-sample latency (multiple backward passes + host↔device syncs) — GPU sawtooths 0→100 %, CPU 60–95 % idle, iowait ~0 %. Bigger batch, more dataloader workers, and mixed precision do not speed it up. It is not disk- or worker-bound. Pick a cheap GPU accordingly.

#Deploy to Piper / Home Assistant

1. Copy output/<voice>.onnx and output/<voice>.onnx.json into /share/piper/ on your Home Assistant host.
2. Restart the Piper add-on.
3. Settings → Voice Assistants → TTS → select your voice.

#Handing a run off to an NVIDIA DGX Spark

train.sh resumes from the newest checkpoint it finds, so this pipeline is built for handoffs: pause a fine-tune on one machine, copy the checkpoint to another, and pick up at the exact epoch you left off. That makes an NVIDIA DGX Spark (GB10 Grace-Blackwell, 128 GB unified memory, DGX OS) a natural home for a small TTS fine-tune — it's low-power and always-on, and (per the cost note above) a job like this doesn't need datacenter throughput, so a Spark finishes it perfectly well.

To resume on a Spark: scp your latest lightning_logs/**/epoch=*.ckpt across, drop it under lightning_logs/ in your WORKDIR, then run bash train.sh — it continues from that checkpoint.

Caveat (not yet verified on Spark): the DGX Spark is ARM64 (aarch64), whereas setup.sh pins the x86_64 CUDA build of torch==2.5.1. On a Spark, install the matching aarch64 CUDA PyTorch build in place of that one line — the rest of the setup and every gotcha below applies unchanged. Reports/PRs from Spark users very welcome.

#Renting a GPU

The pipeline works on any CUDA box, and cheaper options exist — Vast.ai (marketplace GPUs) or Enverge (rent a DGX Spark) — use whatever suits you. If you go with DigitalOcean (what this was built on), this referral gives you $200 in free credit (60 days) to start:

(Referral link — you get the $200 credit, I get $25 once you've spent $25. Entirely optional; the honest cost guidance above stands.)

#Acknowledgements

• Built on OHF-Voice/piper1-gpl (GPL-3.0) — the VITS training + inference framework, cloned by setup.sh.
• The reference dataset used while developing this pipeline was STCOMP/majel_tts_medium_clean — credit to STCOMP. Bring your own dataset that you have the rights to (see the disclaimer above).

#License

The scripts in this repo are MIT-licensed (see LICENSE). piper1-gpl, which setup.sh clones separately, is GPL-3.0 and retains its own license.