[vrunk11]

Title: Fix the quality loss from the chroma calibration: leave the decoded samples unmodified and record the chroma gain in the metadata

Checklist

• I have searched the open pull requests to confirm this change has not already been submitted.
• My branch is up to date with the target branch. (Based on 6a5c3a1; to be confirmed against the target.)
• I have tested my changes and all existing tests pass. (Not ticked: 3 unit tests fail, identically on unmodified 6a5c3a1. See Testing.)
• I have updated documentation where necessary.
• My code follows the project's coding standards (see CONTRIBUTING.md).

Description

Problem. A decode looks worse than it should: the chroma level written to the .tbc and to the .cvbs is not what the fixed filter chain produces but what a per-disc calibration forces, and that calibration adds noise to the luminance. Nothing in the output says it was done, and it cannot be undone afterwards. This is the quality problem reported in decode-orc#N (a decode looks degraded when the .cvbs and .tbc paths are compared).

Cause. By default ld-decode runs a burst-driven "auto inverse-MTF chroma" calibration. It filters the samples with the inverse of the disc's MTF, with a strength tracked from the colour burst, until the burst reaches 20 IRE. That filter boosts a wide band, not only the chroma, so the luminance noise rises with it; and it is applied to the samples themselves, for both .tbc and .cvbs, with no record in the metadata.

Fix.

1. The calibration no longer runs by default. The samples are left as the fixed filter chain produces them. --auto_chroma_correction runs it and gives the previous output (checked below).
2. The gain a decoder needs to bring the burst to nominal is recorded: capture.chroma_gain in the .tbc.db (new nullable column, appended), and in capture_notes of the .cvbs.meta, the one field the format reserves for source-specific facts. The decoder, not the sample file, then decides what to do with it.
3. --no_mtf_servo stops the 2T MTF servo on its own. It still runs by default. Until now the only way to stop it was -m other than 1.0, which also changes the level.

What this fixes, measured on NTSC (details under Motivation): the extra luminance noise (black PSNR 1.2 to 2.1 dB, white SNR up to 2.9 dB) and the loss of the real differences in chroma level between discs.

What this changes for someone running ld-decode with no extra option: the chroma in the .tbc and the .cvbs is lower than before. On NTSC the burst is 15.5 to 17.0 IRE instead of about 20. PAL was not measured. To get the previous output, add --auto_chroma_correction.

Not fixed here: the chroma level also changed with the NTSC RF band-pass split of ca97e78 (the burst fell from 16.2 to 14.5 IRE on one capture), and the MTF servo changes the output (see Additional Notes). Neither is touched.

	Before	After this PR
Chroma of the decoded samples (.tbc and .cvbs)	Raised until the burst reaches its nominal level (calibration always on)	Left as the fixed filter chain produces it. Calibration only with --auto_chroma_correction
--deemp_strength X	Sets the strength and turns the calibration off	Unchanged: sets the strength; the calibration stays off, and wins over --auto_chroma_correction
.tbc.db	field_record.median_burst_ire only	Also capture.chroma_gain: the gain that brings the burst to nominal
.cvbs.meta	Nothing about the burst	capture_notes carries chroma_gain=...; median_burst_ire=...; nominal_burst_ire=... (the existing PAL note is kept in front)
2T MTF servo	On; the only way to stop it was -m other than 1.0	On by default, unchanged. New --no_mtf_servo stops it on its own

chroma_gain is the nominal burst level over the median of the fields' median_burst_ire: multiplying the chroma by it puts the burst at nominal. It is about 1.0 for a decode made with --auto_chroma_correction, and NULL when no field had a usable burst.

Motivation

Why the calibration degrades the output, in four points.

1. It adds noise to the luminance. The filter is a wide-band boost, so the luminance noise rises with the chroma. Measured below.
2. It forces every disc to the same level. Discs are not mastered to one chroma level; the calibration removes those differences from the samples, with a strength chosen per disc from 0.84 to 1.27 (table below).
3. It cannot be undone. Once the boost is in the samples, a decoder cannot recover the unboosted signal, whereas a gain given in the metadata can always be applied or left out.
4. It was not visible. The only way to turn it off was --deemp_strength, whose name suggests the fixed de-emphasis, and nothing in the output recorded that the calibration had run.

Evidence for point 2: burst level of five NTSC captures from different discs (ld-decode-testdata-ci, first 14 frames, -t 1 --tbc, this branch):

Capture	Burst, default (correction off)	chroma_gain recorded	Burst, --auto_chroma_correction	Strength the calibration chose
ntsc/ve-monitor	15.5	1.2877	19.7	1.223
ntsc/ve-snw-cut	15.7	1.2769	20.0	1.270
ntsc/issue176	16.6	1.2069	20.0	0.997
ntsc/ggv1016-side1-start	17.0	1.1730	20.1	0.842
ntsc/ggv-ntsc-mb-v2800	16.0	1.2498	19.9	1.115

Without the calibration the five discs differ by about 10 % (15.5 to 17.0 IRE); with it they all end at 19.7 to 20.1, so a real difference between the discs is gone from the samples.

The correction also has a cost in the luminance. On a nine-frame NTSC capture (not in the test data), default against --auto_chroma_correction, over its 19 fields:

• subcarrier amplitude: 1.31 times higher with the correction;
• black PSNR (the decoder's own metric): 1.2 to 2.1 dB lower, 1.6 dB on average;
• white SNR: 0.0 to 2.9 dB lower, 1.2 dB on average.

In the chroma band the signal and the noise are both multiplied by the same factor (burst 5622 to 7351, noise 1734 to 2267, measured on the left margin of that capture, which is not perfectly flat, so only the ratio is meaningful), so the chroma SNR does not change (10.2 dB both ways). The extra noise is in the luminance. A gain applied by a decoder to its separated chroma leaves the luminance as it is.

Related Issues

Addresses decode-orc#N: "compare a .tbc ... and a .cvbs in decode orc ... the .cvbs version is worse; expected: the result should look the same". At equal options the .tbc and the .cvbs that ld-decode writes carry the same chroma level (burst 14.98 and 14.92 IRE on one capture), so a difference between the two is a difference between the decodes (version or options), not between the formats. This PR corrects one source of such a difference, the chroma calibration; the parts listed under "Not fixed here" remain. decode-orc/decode-orc#330

Changes Made

• lddecode/main.py: --auto_chroma_correction and --no_mtf_servo; auto_deemp is now args.auto_chroma_correction and args.deemp_strength is None; the --deemp_strength help text corrected (it said 1.0; the NTSC default is 0.96).
• lddecode/decoder.py: mtf_servo option feeds mtf_2t_servo; capture.chroma_gain column and its upsert; the burst levels are collected as fields are committed (FieldInfo only keeps the last few fields); _chroma_gain() and _chroma_gain_note(); the CVBS note is set just before the writer closes.
• lddecode/metrics.py: chroma_gain_to_nominal(), pure, ignoring non-finite levels and levels at or below MIN_USABLE_BURST_IRE (5.0, the floor checkAutoDeemp already uses).
• cmake_modules/LdDecodeTests.cmake: the decodes that feed the VITS conformance checks (NTSC and PAL cvbs, the radius sweep, the capture cuts) and the PAL serial/threaded comparison pass --auto_chroma_correction, so they keep judging the corrected output.
• Docs: docs/user-guide/command.md, docs/technical/vits-servos.md, docs/technical/metadata-format.md.
• Tests: see Testing.

Testing

Unit lane, python -m pytest -q tests/unit: 2258 passed, 3 failed, 3 skipped. The 3 failures are identical on unmodified 6a5c3a1 in the same environment (Python 3.13, numpy 2.5.3, scipy 1.18.1, numba 0.68.0) and are unrelated to this change:
test_demod_half_rate.py::test_the_full_rate_delays_are_unchanged[PAL] (a 1e-8 tolerance), test_pulses_burst_lines.py::test_burst_levels_at_the_buffer_edges_and_on_bad_samples[NTSC], test_start_finder.py::test_sync_only_demodulation_matches_the_normal_sync_path.

• All existing tests pass (pytest --output-on-failure): see above, 3 pre-existing failures.
• Tested manually with:
  • NTSC capture, -t 1 --tbc: capture.chroma_gain is 1.3232 by default and 1.0001 with --auto_chroma_correction (burst 15.30 and 19.97 IRE). With the flag, the burst and the VIRS line 19 (19.8 and 21.3 IRE) match the previous default.
  • ve-snw-cut.ldf as CVBS (-l 6), as the CTest does: vits_conformance.py passes with --auto_chroma_correction (34 checks, 17 skipped) and fails 7 of 34 without it, which is why the CMake decodes above pass the flag. cvbs_verify.py passes in both cases, and the CVBS note is written (chroma_gain=1.2768; ...).
  • The five captures of the table above.
• New tests added for: chroma_gain_to_nominal (tests/unit/test_metrics_chroma_gain.py, 11 tests: median rather than mean, unusable levels ignored, no usable level gives None, per-system nominal); the options (tests/unit/test_main_cli_mapping.py: default off, flag arms it, --deemp_strength wins, --no_mtf_servo); the collection of burst levels at commit (tests/unit/test_decoder_output_stage.py). All marked unit; none touches the filesystem; no random data.

Not run: the CTest functional lanes, including the serial/threaded comparisons and the radius sweep (their decode command lines were edited as described but not executed); anything on PAL. mtf_servo reaching mtf_2t_servo in LDdecode.__init__ has no unit test (it needs a full decoder); it was exercised by decoding.

Screenshots (if applicable)

Not attached.

Additional Notes

• Default change. If keeping the default is preferred, the PR can be reduced to the metadata and the two flags, with the calibration staying on by default and the flag inverted into an opt-out; the CMake changes then go away. I can split it that way.
• Nothing reads chroma_gain yet. Reading the recorded value is a change for the decoders, not for this repository, and is the reason for recording it.
• How to use the value. The gain is computed from the burst, which is a single frequency. On the one frame I compared, applying it to the whole picture over-corrects the chroma by about 6 % compared with what the inverse-MTF filter does, because that filter boosts the burst more than the picture's chroma bandwidth. A decoder may want a slightly lower gain; this is a single frame and a single disc.
• Schema. The column is nullable and appended; user_version is not changed. The reader in ld-decode-tools (src/library/tbc/sqliteio.cpp) selects capture by column name and already tolerates an optional column there (blanking_16b_ire), so older readers ignore it.
• MTF servo, not changed here. On the nine-frame capture the servo moved the MTF level from 0.795 to 0.0 after frame 7 and the subcarrier rose by 9 % from frame 8 (SNR 0.1 to 0.4 dB lower). I have no evidence that this is worse for image quality, only that the output differs; that is for a separate discussion.