The tts tool

The tts broker tool (lever-tool-tts) reads agent messages aloud on the chat page with a natural neural voice that runs on your own host. It serves the chat page’s read-aloud when remote.voice.read_aloud is server. Agent text goes only to your host, never to a cloud speech service. Part of tools; see also the whisper tool, its speech-to-text counterpart.

What it does

For each part of a message the page reads aloud, the remote proxy sends the text to the tool’s Unix socket (-speak-socket, the same path as remote.voice.speak_socket). The tool runs sherpa-onnx’s sherpa-onnx-offline-tts once, as its child, with a speech model from lever’s pinned table, and answers the audio as WAV. The default model is Kokoro (kokoro-en-v0_19: 11 English voices, American and British, such as bf_emma and bm_george).

The socket is not reached through the broker. It is 0600 in a private 0700 directory, so only processes of your user can connect. There is no long-running engine and no TCP port: nothing listens but the socket and the tool’s MCP backend.

The guarantees: text and audio are never stored or logged. The jail cannot reach the socket, and the engine has no port. The tool runs the engine only with model files unpacked from an archive whose size and sha256 match lever’s table, each unchanged since, and only from a program file that no one but you or root can change. The engine gets a reduced environment and no stdin, and its output is discarded (it prints the text it speaks).

For agents the tool offers one read-only MCP operation, voices (the model, its voices and the default). The broker needs a tool to register at least one operation; voices gives an agent nothing to speak with, and no agent can call it without an obtain grant.

Requirements

• sherpa-onnx’s sherpa-onnx-offline-tts, which you install yourself (below). Install it outside the tree, owned by you or root and writable by no one else. Lever never builds or installs it.
• Disk space for the model: about 350 MB for kokoro-en-v0_19 and 80 MB for piper-en_GB-alba-medium, espeak-ng’s data included, and as much again while lever voice fetch unpacks it (the archives are 320 MB and 67 MB). The engine runs on the CPU.
• A sherpa-onnx release that runs Kokoro models (--kokoro-model in sherpa-onnx-offline-tts --help).
• Install the tool with make install. Releases ship it for darwin and linux.
• Remote access with the chat page and remote.voice on (read-aloud is part of it).

Install sherpa-onnx

Build it from source (CMake and a C++17 compiler), or unpack a prebuilt release archive for your platform from the project’s GitHub releases. From source:

git clone https://github.com/k2-fsa/sherpa-onnx
cd sherpa-onnx && git checkout <a release tag>
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF ..
make -j4 sherpa-onnx-offline-tts
sudo install -m 0755 bin/sherpa-onnx-offline-tts /usr/local/bin/

A static build (BUILD_SHARED_LIBS=OFF) is one file. With a shared build or a prebuilt archive, keep the program with its lib/ directory, for example under /opt/sherpa-onnx/, owned by root, and point -server at /opt/sherpa-onnx/bin/sherpa-onnx-offline-tts. Check it with sherpa-onnx-offline-tts --help. The tool checks the program file and its directory, not the libraries next to it: keep lib/ as safe from other users as the program. The tool resolves the program’s real path once at start and refuses it if it later resolves to another file; restart the tool (lever reload) after an upgrade.

1. Configure the tool

broker:
  tools:
    - name: tts
      command: [/home/you/.local/bin/lever-tool-tts,
                -tree, /home/you/my-instance/workspace,
                -models, /home/you/my-instance/.lever-state/voice-models,
                -server, /usr/local/bin/sherpa-onnx-offline-tts,
                -model, kokoro-en-v0_19,
                -voice, bf_emma,
                -speak-socket, /home/you/my-instance/.lever-state/tts/speak.sock]
      backend: 127.0.0.1:3213
      operations:
        - {name: voices}
remote:
  voice:
    enabled: true                                                       # with the rest of remote.voice
    socket: /home/you/my-instance/.lever-state/whisper/dictate.sock     # dictation (whisper tool)
    read_aloud: server
    speak_socket: /home/you/my-instance/.lever-state/tts/speak.sock     # the tool's -speak-socket

The broker adds -backend and -admin itself. Every other setting is a flag in command. Lever recognises the tool by its program’s base name, lever-tool-tts (also behind an env prefix), or a command that takes -speak-socket, and reads these flags only in its command:

Config load checks each of these (the model and voice against lever’s table, the numbers against their ranges) and refuses the program, the models and the socket inside the tree. The Kokoro voices of kokoro-en-v0_19 are af, af_bella, af_nicole, af_sarah, af_sky, am_adam, am_michael (American) and bf_emma, bf_isabella, bm_george, bm_lewis (British), speaker ids 0 to 10 in that order, as sherpa-onnx documents them.

Why kokoro-en-v0_19. It is the English-only Kokoro model: espeak-ng alone turns its text into phonemes, so sherpa-onnx needs only the model, its voices, its tokens and espeak-ng’s data, and every sherpa-onnx release that runs Kokoro runs it. sherpa-onnx’s newer multi-lingual Kokoro models (kokoro-multi-lang-v1_0 has eight British voices to this one’s four) also need a lexicon or language flag that depends on the voice, and carry Chinese dictionaries and rules; lever does not offer them yet.

2. Fetch the model

lever voice fetch kokoro-en-v0_19
lever voice fetch                  # the whisper tool's model, and the tts tool's

sherpa-onnx publishes each model as one archive among the assets of its tts-models release: the ONNX model, its tokens, Kokoro’s voice file and espeak-ng’s data. Lever pins the archive’s URL, size and sha256 (as GitHub records them for the asset) and downloads it only from github.com and GitHub’s release-asset hosts. It keeps the archive only if its size and sha256 match, and then unpacks it into <models>/<model>/: directories (0700) and regular files (0600) only, from below the archive’s top directory. A link, a device or a path outside that directory refuses the whole archive, and so does one without the files sherpa-onnx needs. Lever writes a manifest there with the size and sha256 of every file it unpacked, and deletes the archive.

If sherpa-onnx uploads a model’s archive again, its sha256 changes and lever voice fetch refuses it until a lever release pins the new one; a model already fetched keeps working.

Neither the tool nor the proxy ever downloads a model. Before every run the tool checks the program and the model: every file in the model’s directory must be one the manifest lists, of its size and sha256, so a file changed or added there is refused. The manifest itself is trusted as lever wrote it: it guards against a damaged, partial or changed model, not against a program running as you, which could rewrite the manifest too (as it could replace the engine itself). It hashes the files again whenever the metadata (inode, size, times, owner, mode) of anything in that directory changed since its last full check, so a model fetched later, or replaced, is checked at the next request. With no tts tool configured, lever voice fetch downloads into .lever-state/voice-models/; point -models at that absolute path.

Then run lever reload and check lever doctor: the tool’s own row, and the tts tool and tts model rows. tts tool asks the speech socket for its health (not running, not ready, or ready with the model, voice and limits); tts model checks every file of the -model in -models against the manifest of its verified archive, and that no other file is there. With read_aloud: server they check the tool that serves remote.voice.speak_socket; otherwise the first tts tool in broker.tools. The voice row says which read-aloud the chat page uses.

How a request runs

1. The page cleans the message for speech (markdown removed, a code block read as “code block skipped”, a link as its text or host name), splits it into parts of whole sentences, the first one short so the voice starts soon, and asks for one part at a time, the next while one plays.
2. The proxy checks the login, the request and the limits (below), then posts the text to the socket.
3. The tool cleans the text again: control and format characters and angle brackets become spaces, and no part starts with -. The text also goes to the engine after --, which ends its options, so it is never read as one. It refuses an empty text, one over -max-chars and a voice the model lacks.
4. It runs sherpa-onnx-offline-tts once, with a private output directory under -models, reads the WAV it wrote (16-bit PCM, mono, at the model’s rate), deletes the directory, and answers a WAV it writes itself from those samples. Nothing else of the engine’s file goes on.

One run at a time. A run gets 2 minutes; a request whose page went away kills its run. On Linux the run also dies with the tool when the tool is killed; on macOS a run left behind by a tool killed hard ends by itself when its one text is spoken, and a later start removes its output directory once it is ten minutes old.

The tool writes one line per request to tool-logs/tts.log: the length, the voice, the outcome and the latency. Never the text.

Limits

The proxy’s limits, per login, kept in memory (they start again when the proxy restarts): 40,000 characters an hour, 200,000 a day, and 1,200 requests an hour. Three requests at a time in all, of which a contact never takes the last, so an operator can always listen; one at a time per login; one with the engine at a time. A text the engine never got (the tool was down or refused it, or the page gave up while it waited) does not count. A login with voice: false gets no read-aloud at all.

Trust boundaries