fix(mimic-iv): apply APS III axillary temperature +1 C correction - #2046
fix(mimic-iv): apply APS III axillary temperature +1 C correction#2046acnimma wants to merge 1 commit into
Conversation
Axillary temperatures run ~1 C low vs core. APACHE III / APS III raises them by 1 C before scoring. Aggregate first-day temperature min/max from vitalsign with that adjustment, then regenerate dialect copies.
|
axillary +1c correction matches the aps iii paper — nice. is the regenerated dialect sql included so bq/duckdb stay in sync? |
|
postgres + duckdb in PR thank you |
Chessing234
left a comment
There was a problem hiding this comment.
the main thing missing is evidence that this fires at all. how many first-day rows actually have a temperature_site matching '%axillary%'? the distinct values and counts of itemid 224642 on the demo would settle it — if the site is rarely charted alongside a temperature, or spelled differently, this is a no-op.
there's also a mismatch in what the correction keys off. vitalsign.temperature is an AVG over the charttime group while temperature_site is a MAX over that same group, so when two sites are charted at one charttime the averaged number carries a single site label that doesn't describe it, and the +1 either applies to a blend or gets skipped. applying the correction per-itemid inside vitalsign.sql, before the AVG, keys it to the actual reading.
vital_temp also re-aggregates the whole vitalsign table per stay instead of reading first_day_vitalsign. postgres-make-concepts.sql builds and indexes the firstday tables ahead of the score queries specifically so those "run in minutes rather than hours" — worth a timing before/after, or deriving the site-adjusted min/max in first_day_vitalsign so apsiii keeps reading one row per stay.
and the postgres copy doesn't reproduce: regenerating from this branch's bigquery source with the pinned sqlglot==30.11.0 from requirements-lock.txt gives a 34-line diff against what's committed. main reproduces byte-identically and the duckdb copy matches, so it looks like a different sqlglot version.
Chessing234
left a comment
There was a problem hiding this comment.
i checked the window first since that was my main worry, and it's fine — vital_temp uses the same intime - 6 hours to intime + 1 day bounds as first_day_vitalsign, so temperature stays consistent with the other vitals feeding the score. build order is fine too, measurement/vitalsign.sql runs well before score/apsiii.sql. applying the +1 per row before the min/max aggregation rather than to the aggregate is also the right way round.
two things though.
the postgres copy isn't transpiler output. running the repo's transpiler (sqlglot 30.11.0 from requirements-lock) over your bigquery source and diffing mimic-iv/concepts_postgres/score/apsiii.sql on this branch: sqlglot emits the whole header block as one line and the branch has it split across ~25. duckdb is byte-identical, and the same check against main is identical, so it comes from this branch. generated-up-to-date in transpile.yml diffs that folder.
the substantive one is temperature_site coverage. vitalsign pivots per charttime, and the site (itemid 224642) is charted much less often than the temperature itself. where the site is null at a given charttime, LOWER(ce.temperature_site) LIKE '%axillary%' evaluates to null and the row falls through to the uncorrected ELSE branch. so within a single stay you can end up taking the min over a mix of corrected and uncorrected readings from the same axillary probe, purely depending on whether the nurse charted the site on that row — and since temperature_min drives a 20-point band at the bottom of the scale, that's not a rounding-level concern.
could you show: the distinct values of temperature_site and their counts, what fraction of non-null temperature rows have a non-null site, and how many stays have both corrected and uncorrected axillary rows inside the first-day window? if the site is sparse, carrying it forward per stay_id (or per charttime block) would be more faithful than correcting only the rows where it happened to be charted. i'm also not certain %axillary% matches the actual spelling in the data — that's worth confirming rather than assuming.
Summary
vitalsignwith site adjustmentTest plan
pytest tests/test_transpile.pypasses locallyAPS III previously ignored temperature site. Axillary measurements are increased by 1 degree Celsius before scoring, matching APACHE III methodology. First-day temperature min/max are derived from vitalsign using the same charttime window as first_day_vitalsign.