Skip to content

Fix ISO8601 parsing with explicit languages - #1352

Merged
serhii73 merged 1 commit into
scrapinghub:masterfrom
serhii73:fix-360-iso8601-language-parser
Jul 23, 2026
Merged

Fix ISO8601 parsing with explicit languages#1352
serhii73 merged 1 commit into
scrapinghub:masterfrom
serhii73:fix-360-iso8601-language-parser

Conversation

@serhii73

Copy link
Copy Markdown
Collaborator

Fixes #360

Bug

dateparser.parse("2017-06-22", languages=["it"]) returns None, while the same string parses fine with languages=["en"] or with no languages at all.

There is also a silent variant: when the day is ≤ 12, a wrong date is returned instead of None:

>>> dateparser.parse("2017-06-10", languages=["it"])
datetime.datetime(2017, 10, 6, 0, 0)  # October 6 instead of June 10

Any language whose locale date order is day-first is affected (it, fr, de, …) — 253 of the 308 locales in the project data use DMY.

Root cause

Italian's locale date_order is DMY, so _parser's parse_number() in dateparser/parser.py tries to assign numeric tokens to components in day → month → year order:

  • 2017 only fits year (and, being a four-digit token, also sets skip_component = "year" for the remaining tokens),
  • 06 is assigned to day, because day precedes month in DMY,
  • 22 cannot be a month (> 12) and the year slot is skipped, so assignment fails with ValueError: Unable to parse: 22, and the result is None.

For 2017-06-10 the last token can be a month, so parsing "succeeds" — with month and day swapped.

Fix

A date string that starts with a four-digit year is big-endian (ISO 8601-style); no real-world notation puts the day between the year and the month (no locale in the project data uses YDM). So, when the year has already been consumed from a leading four-digit token and neither day nor month has been assigned yet, parse_number() now expects the remaining numeric components in month → day order.

The correction deliberately does not apply when the caller explicitly sets DATE_ORDER (checked via settings._mod_settings, the same mechanism date.py uses to decide whether the locale order applies), so explicit DATE_ORDER="DMY"/"YDM" behavior is unchanged. It also does not trigger for strings where the year is not the leading token (22-06-2017, 06 2017 10) or for two-digit years, and the existing PREFER_LOCALE_DATE_ORDER=False workaround keeps working.

Tests

Added test_iso_datestamp_format_should_parse_with_locale_date_order covering:

  • the exact case from Parsing valid ISO8601 date with languages = it fails #360 (2017-06-22, it — was None),
  • the silent month/day swap (2017-06-10, it — was October 6),
  • another DMY language (2017-06-22, fr),
  • a full ISO timestamp under a DMY locale, without the PREFER_LOCALE_DATE_ORDER=False workaround that the neighbouring existing test needs.

All four fail on current master and pass with the fix. Full suite: 24064 passed, 20 skipped, 1 xfailed. pre-commit (ruff, ruff-format) passes.

Related: #790 proposes a dedicated ISO parser; this PR instead makes a minimal correction to the existing token-assignment logic, which also fixes the silent month/day swap that occurs before any fallback parser would run.

@codecov

codecov Bot commented Jul 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.13%. Comparing base (762dfff) to head (7d74974).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1352   +/-   ##
=======================================
  Coverage   97.12%   97.13%           
=======================================
  Files         235      235           
  Lines        2924     2927    +3     
=======================================
+ Hits         2840     2843    +3     
  Misses         84       84           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@serhii73
serhii73 merged commit 315e396 into scrapinghub:master Jul 23, 2026
15 checks passed
@serhii73
serhii73 deleted the fix-360-iso8601-language-parser branch July 23, 2026 12:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parsing valid ISO8601 date with languages = it fails

2 participants