fix(ledger): stop a source backfill dating decade-old facts as today

[idoshamun]

What

Two dating paths read a source backfill's crawl moment as if it meant something. Registering a source imports its whole archive at once, so a 2013 post and a 2026 post are both crawled this morning.

1. evidenceDerivedDate no longer dates a claim from an archive import

The fallback to post.createdAt fires when the post carries no publishedAt. Measured on prod today: of 12,468 claims dated evidence_crawled, 91.4% carry exactly the crawl date, and every claim on a pre-2025 article is wrong by that article's own age.

URL year	claims	year mismatch	avg drift
2021	3	3	+5.00
2022	15	15	+4.00
2023	8	8	+3.00
2024	18	18	+1.61
2025	776	72	+0.09
2026	907	7	−0.01

The direction is the sharp edge. M1 denies a pending candidate whose effectiveDate is older than 24 months. evidence_crawled sets it to today. So an undated candidate on an undated post is not merely misdated — it is misdated in the exact direction that defeats the guard meant to catch it, and enters the ledger reading as current. Confirmed live: 17565f7a (already corroborated) records a 2022 netsim-tools→netlab rename as 2026-08-11; seven .NET MAUI claims from a 2024/09 article all sit at 2025-12-11.

Note the 2026 row — on live-feed content the fallback is correct (drift −0.01 years). So this does not remove the fallback, it gates it: a post crawled within a day of its source being registered came in the opening archive sweep, and only that case goes undated. Undated is the honest outcome (playbook §10) and it leaves the row visible to the review lane rather than silently current.

2. extractClaims no longer tells bragi an archive post was published today

publishedDate was defaulting to new Date() whenever the payload carried no published_at. Bragi uses that field for exactly one job — resolving the relative expressions a post uses ("last month", "since March", "starting today") — so a 2016 post's relative dates resolved to this year.

Measured as unfired. Candidates on NULL-publishedAt posts carry a lower recent-date share than the control (2.28% in 2026-08 vs 7.46%), and the sharp test — dated this month while the URL says the article is old — returns zero rows. These are release-announcement posts that state their own dates, so published_date is rarely load-bearing. Fixing it as a forward guard, not a cleanup.

Decision worth a look

ARCHIVE_IMPORT_WINDOW_MS is one day. The 2026-08-23 imports each drained inside ninety seconds, so the window is deliberately generous; the only thing it costs is leaving the first day of a new source's genuinely fresh posts undated. Happy to tighten it, or to replace the heuristic with a real import marker if you'd rather yggdrasil flag a backfill explicitly.

Not in this PR

• The upstream cause. yggdrasil's RSS publisher parses the feed's <pubDate> into SourceEntry.PublishedAt and then drops it when building EntryData (workflows/source_engines/rss/publisher/publisher.go maps only URL/Title/Keywords/OriginID). The article engine can therefore only take whatever date the content cleaner scrapes off the page — which is why 19 of the 45 sources added on 2026-08-23 carry NULL publishedAt on every post while the other 26 do not. Carrying the feed date through would make both of the above moot for future imports.

• Running the repair. bin/repairCrawlDatedClaims.ts is in this PR (second commit) but is deliberately not run yet. It invents nothing — the recovered date comes off the post row — so it reaches exactly as far as the posts have been repaired, and the posts are fixed upstream. Order is: merge fix(date): recover publication dates the URL and JSON-LD already state newspaper#12 → ship dailydotdev/yggdrasil-cleaner-newspaper#54 → re-clean the affected posts → then run it. Running it today would find almost nothing.

  It is narrow on purpose, because it overwrites a date rather than filling an empty one: only evidence_crawled (an extracted date was read off the change itself, possibly by a reviewer; evidence_published is already better), only where the post now carries a publishedAt, and only where that date is earlier than the claim's — a post published after we crawled it is bad in a way this repair cannot name, and a repair that overwrites may only act on the cases it can name.

  Recovery tiers, measured: 913 posts / 1,674 claims carry the date in the URL (cross-checked 15/15 exact against live pages, no network needed); ~2,800 posts recoverable by page fetch (40-post sample, CI [2,028–3,529] — an estimate, not a measurement); only 724 posts / 715 claims are structurally unrecoverable, all daily.dev-generated collection/trends posts with no external URL.

[pulumi]

🍹 The Update (preview) for dailydotdev/api/prod (at ceddc82) was successful.

Resource Changes

    Name                                                    Type                           Operation
~   vpc-native-check-analytics-report-cron                  kubernetes:batch/v1:CronJob    update
~   vpc-native-update-current-streak-cron                   kubernetes:batch/v1:CronJob    update
~   vpc-native-hourly-notification-cron                     kubernetes:batch/v1:CronJob    update
~   vpc-native-expire-super-agent-trial-cron                kubernetes:batch/v1:CronJob    update
+   vpc-native-api-db-migration-9bb42ef8                    kubernetes:batch/v1:Job        create
~   vpc-native-user-profile-analytics-clickhouse-cron       kubernetes:batch/v1:CronJob    update
~   vpc-native-ws-deployment                                kubernetes:apps/v1:Deployment  update
~   vpc-native-update-tags-str-cron                         kubernetes:batch/v1:CronJob    update
~   vpc-native-private-deployment                           kubernetes:apps/v1:Deployment  update
~   vpc-native-daily-digest-cron                            kubernetes:batch/v1:CronJob    update
~   vpc-native-generate-search-invites-cron                 kubernetes:batch/v1:CronJob    update
~   vpc-native-deployment                                   kubernetes:apps/v1:Deployment  update
~   vpc-native-materialize-monthly-best-post-archives-cron  kubernetes:batch/v1:CronJob    update
~   vpc-native-rotate-daily-quests-cron                     kubernetes:batch/v1:CronJob    update
~   vpc-native-sync-subscription-with-cio-cron              kubernetes:batch/v1:CronJob    update
~   vpc-native-squad-posts-analytics-refresh-cron           kubernetes:batch/v1:CronJob    update
~   vpc-native-ledger-hygiene-cron                          kubernetes:batch/v1:CronJob    update
~   vpc-native-user-posts-analytics-refresh-cron            kubernetes:batch/v1:CronJob    update
~   vpc-native-validate-active-users-cron                   kubernetes:batch/v1:CronJob    update
~   vpc-native-bg-deployment                                kubernetes:apps/v1:Deployment  update
~   vpc-native-update-source-public-threshold-cron          kubernetes:batch/v1:CronJob    update
+   vpc-native-api-clickhouse-migration-9bb42ef8            kubernetes:batch/v1:Job        create
~   vpc-native-update-tag-materialized-views-cron           kubernetes:batch/v1:CronJob    update
~   vpc-native-personalized-digest-deployment               kubernetes:apps/v1:Deployment  update
~   vpc-native-materialize-yearly-best-post-archives-cron   kubernetes:batch/v1:CronJob    update
~   vpc-native-user-world-clickhouse-cron                   kubernetes:batch/v1:CronJob    update
~   vpc-native-user-profile-updated-sync-cron               kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-zombie-images-cron                     kubernetes:batch/v1:CronJob    update
~   vpc-native-update-achievement-rarity-cron               kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-gifted-plus-cron                       kubernetes:batch/v1:CronJob    update
~   vpc-native-post-analytics-clickhouse-cron               kubernetes:batch/v1:CronJob    update
~   vpc-native-personalized-digest-cron                     kubernetes:batch/v1:CronJob    update
~   vpc-native-post-analytics-achievements-cron             kubernetes:batch/v1:CronJob    update
~   vpc-native-interest-scheduled-run-cron                  kubernetes:batch/v1:CronJob    update
-   vpc-native-api-clickhouse-migration-2bef4ad0            kubernetes:batch/v1:Job        delete
~   vpc-native-clean-channel-highlights-cron                kubernetes:batch/v1:CronJob    update
~   vpc-native-update-views-cron                            kubernetes:batch/v1:CronJob    update
~   vpc-native-subscription-anniversary-achievements-cron   kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-old-notifications-cron                 kubernetes:batch/v1:CronJob    update
~   vpc-native-ledger-corroboration-cron                    kubernetes:batch/v1:CronJob    update
~   vpc-native-update-trending-cron                         kubernetes:batch/v1:CronJob    update
... and 21 other changes