# Antimatter

Antimatter is an evidence-driven reconstruction of the NEUTRINO v3.1.4
inference pipeline. It converts a Sinsy HTS full-context label into corrected
timing, frame-rate F0, a 100-bin mel-spectrogram, and a 48 kHz waveform using
the four ZUNKO ONNX models supplied with this research corpus.

This project is for interoperability and academic analysis. It is not an
official NEUTRINO implementation and does not redistribute model files.

## Workspace

The Rust tree is one Cargo workspace:

- `antimatter`: CLI, staged synth engine, and persistent studio child
- `amadeus`: egui editor, audition, and export workflow
- `crates/studio-protocol`: `antimatter.studio/v1` length-prefixed Bincode messages
- `crates/project-store`: SQLite `.amadeus` artifacts, FLAC, cache, and mix bus

Amadeus keeps Antimatter alive as a child process. Standard output is reserved
for length-prefixed Bincode control/events; generated arrays and audio stay in the shared SQLite
project instead of crossing stdio.

```bash
cargo run -p antimatter -- studio --stdio --project song.amadeus
cargo run -r --bin amadeus
```

## Usage

```bash
uv run python main.py score.lab --model-dir MODELS/ZUNKO --output-dir output
```

The command writes:

- `output/score.lab`: timing-corrected Sinsy label
- `output/score.f0`: headerless little-endian float32, one value per 10 ms frame
- `output/score.melspec`: headerless little-endian float32, frame-major, 100 bins
- `output/score.wav`: mono 16-bit PCM at 48 kHz

Use `--skip-timing` to retain the input label boundaries while still running
the pitch, acoustic, and vocoder models.

## Continuous PTM Score Conditioning

Use a phone-aligned score-pitch sidecar to condition the normal pipeline
without bypassing `t.bin`:

```bash
uv run python main.py score.lab \
  --score-pitch-sidecar score.pitch.json \
  --model-dir MODELS/ZUNKO --output-dir output
```

The sidecar supplies continuous Hz values per label. The same values reach
`t.bin`, `p.bin`, and `s.bin`; label durations are still refined by `t.bin`.
This avoids rounding PTM predictions to equal-tempered musical notes.

## Experimental Vietnamese Tone Correction

The Python route can conservatively reshape voiced Vietnamese rime contours
after `p.bin` and before `s.bin`:

```bash
uv run python main.py score.lab \
  --tone-correction conservative \
  --tone-profiles /path/to/tone-profiles.v1.json \
  --tone-correction-strength 0.35
```

Correction is off by default. It preserves frame count, unvoiced frames, and
the local median pitch while adding each tone's speaker-balanced female VIVOS
contrast relative to `ngang`. The `ngang` profile is therefore a no-op instead
of flattening valid sentence intonation. Enabled runs additionally write
`score.raw.f0` and `score.tone.tsv` for inspection.

Strength `1.0` applies the full measured corpus contrast. Values up to `3.0`
are accepted for deliberately expressive speech; the transform remains
bounded by the 120-cent local safety cap.

Use `--pitch-override baseline.f0` to render multiple variants from the exact
same headerless float32 p.bin result. This removes pitch-model randomness from
an A/B comparison; the acoustic and vocoder graphs can still vary between
processes.

Cephome singing labels carry their authored Vietnamese tonal swing in the
full-context `E` field. Interpolate those phone-level cents control points
after `p.bin` and before `s.bin` with:

```bash
uv run python main.py score.lab --label-tone-bend-strength 1.0 \
  --model-dir MODELS/ZUNKO --output-dir output/tone-bent
```

The run also writes `score.raw.f0` so the unbent p.bin result remains available
for a direct comparison. Strength zero is the default and preserves the old
pipeline.

The native Rust renderer treats the same metadata as an explicit post-pitch
plugin. When enabled and it finds valid `VIE` full-context control points within
±120 cents, it applies a note-anchored contour after `p.bin`: the configured
attack and release use a CSS-style cubic Bézier easing, the tail plateau returns
to the written note, and the full-strength contour is mean-centered in cents.
Short notes proportionally compress the envelope instead of producing a hard
step. Labels without that contract remain an exact no-op. The pitch-only Rust
command applies the same plugin, so its F0 artifact matches the contour supplied
to `s.bin` during a full render.

Pass `--vietnamese-tone-weight` to scale the authored cents as a percentage:

```bash
target/release/antimatter --vietnamese-tone-weight 68 score.lab
target/release/antimatter score.lab   # default: tone pass off (0%)
```

The default is `0` (the pass is opt-in). Direct CLI renders pass the strength as
`--vietnamese-tone-weight` (0–100 percent; the shape stays at its defaults).
Studio renders use the typed `vietnamese_tone_weight` and
`vietnamese_tone_shape` (compression, attack, release, plateau, and easing)
track settings; the legacy `PHOOK_VI_*` environment variables were removed.
Amadeus edits those values in the Pitch Inspector, not the action bar.

Install the optional chart dependencies and build a word-aligned F0 montage:

```bash
uv sync --extra charts
uv run --extra charts python sketch/plot_f0_montage.py \
  --label timed.lab --words word-pitches.tsv \
  --baseline baseline.f0 --corrected corrected.f0 \
  --output-dir output/charts
```

The chart uses a musical-note vertical scale, alternating word segments,
staggered word labels, and a separate cents-delta strip. It writes individual
ten-second sheets, a combined PNG/SVG montage, and `word-segments.tsv`.

## Reproducible Inspection

```bash
uv run python sketch/inspect_models.py --output sketch/models.json
uv run python sketch/probe_runtime.py
uv run python -m unittest discover -s tests -v
```

`inspect_models.py` records hashes, signatures, operators, and parameter
counts. `probe_runtime.py` uses controlled tensors to demonstrate the actual
runtime dimensions, including two misleading ONNX declarations: `t.bin`
returns `P+1` values although it declares `[b,p]`, and `p.bin` returns `F`
values for the intended batch size of one although it declares `[b]`.

## Reconstructed Contract

| Stage | Inputs | Output |
|---|---|---|
| Timing (`t.bin`) | phoneme ID, score F0, score duration, shared-onset ordinal | phoneme-boundary shifts |
| Pitch (`p.bin`) | ID, timed duration, score F0, score duration, ordinal, 1-based frame-to-phoneme map | F0 in Hz |
| Acoustic (`s.bin`) | pitch inputs plus frame F0 | `[1,F,100]` log mel-spectrogram |
| Vocoder (`v.bin`) | `[1,F,101]`: 100 mel bins followed by F0 | `F*480` waveform samples |

The model-facing phoneme IDs are one-based. ID zero is reserved for padding.
The 40 entries, in order, are:

```text
a b br by ch cl d dy e f g gy h hy i j k ky m my n N ny o p pau py r ry
s sh sil t ts ty u v w y z
```

Sinsy timestamps are integers in 100 ns units. The executable extracts the
central phoneme with `-(\w+?)\+` and the score note with `E:(\w+?)\]`.
Consecutive labels with the same score start receive ordinals 0 through 7.
Notes are converted to equal-tempered Hz and clamped to the singer's supported
MIDI range, 45 through 86; `xx` remains zero.

## Timing Reconstruction

The executable sends the complete label stream, including the leading and
trailing `pau`/`sil` context, to ONNX Runtime in one call. It uses the first
`P` timing values and ignores a terminal `P+1` value when the graph emits one.
Global boundary zero stays fixed. For each internal boundary `i`, the
implementation computes:

```text
new_start[i] = round_ms(max(
    old_start[i] + timing_output[i],
    new_start[i - 1] + 0.010
))
new_end[i - 1] = new_start[i]
```

This identifies the timing output as score-relative internal-boundary shifts,
not direct phoneme durations. The last score end is retained unless correction
would make the final duration non-positive.

Frame boundaries use positive `std::round(duration / 0.010)`. The resulting
alignment is one-based: frame value 1 selects phoneme zero, and so on.

## Full-Track Orchestration

Sinsy emits one continuous label stream, and NEUTRINO preserves that stream.
Timing, pitch, acoustic, and vocoder each run once over the complete track.
`pau`, `sil`, and `br` remain ordinary model inputs rather than becoming
application-side cut points. Waveform edge zeroing and quadratic fades apply
only at the beginning and end of the complete generated track.

The original example corpus in `/home/stdpi/repo/cephome/example` confirms the
raw formats. Its F0/mel files correspond
to an older 403-phone mono label, however, while the current full-context score
contains 413 phones. It is therefore an oracle for shape and serialization,
not a value-by-value regression fixture for the current score.

## Acoustic and Vocoder Evidence

The pitch graph pads encoder states with a zero row and gathers them with the
one-based frame alignment. Its tail adds a predicted log-F0 residual to score
log-F0, exponentiates base 10, and maps values below 40 Hz to zero. The
executable subsequently bounds voiced F0 at 2000 Hz.

The acoustic graph embeds `log10(F0 + 1e-5)` and produces 100 channels. The
executable clips these channels to `[-7,1]`.

In the vocoder, the first slice selects channels 0 through 99 and the second
selects the last channel. This proves the ordering `[mel[0:100], F0]`. The
graph uses a 960-sample transform window and 480-sample hop. Runtime probes
confirm output length `F*480`, consistent with the executable defaults of
48,000 Hz and 10 ms frames. The executable zeros 240 samples at each edge and
applies adjacent 240-sample quadratic fades; the POC reproduces this behavior.

## Training Hypothesis

The shared text and phoneme-count embeddings, Transformer-like encoder, note
conditioning, and duration expansion are consistent with a FastSpeech2-style
front end. The pitch model contains a stochastic 20-step flow/diffusion-like
predictor conditioned on score F0. The acoustic model combines an auxiliary
decoder with a 20-step diffusion decoder over 100-bin log-mel features. The
vocoder conditions a neural source/filter-style waveform generator on mel and
explicit F0, then performs overlap-add synthesis. This is an architectural
hypothesis derived from graph topology and initializer names, not recovered
training source code.

Likely supervision consists of aligned Sinsy score labels, observed phoneme
boundaries, extracted frame F0 with unvoiced zeros, 100-bin log-mel targets,
and 48 kHz waveform segments. The timing objective likely regresses internal
boundary displacement from score timing. Pitch likely learns residual log-F0
relative to score pitch, while acoustic diffusion learns conditional mel
generation and the vocoder learns waveform reconstruction from mel plus F0.

## Evidence Boundaries

The reconstruction is supported by `neutrino.exe.txt`, ONNX graph structure,
and controlled ONNX Runtime execution. Remaining differences from the original
may include undocumented command-line controls, output encoder options, and
the exact ONNX Runtime random-number sequence. The pitch, acoustic, and vocoder
graphs contain random operators, so repeated runs are intentionally not
bit-identical.
