Skip to content

Support MySQL 8.4 tagged GTIDs in GtidSet - #182

Open
verbotenj wants to merge 1 commit into
osheroff:masterfrom
verbotenj:fix/mysql-8.4-tagged-gtids
Open

verbotenj wants to merge 1 commit into
osheroff:masterfrom
verbotenj:fix/mysql-8.4-tagged-gtids

Conversation

@verbotenj

Copy link
Copy Markdown

Problem

MySQL 8.4 (WL#15294) extends the GTID grammar with an optional user-defined tag:

uuid_set: uuid:[tag:]interval[:[tag:]interval]...
tag:      [a-zA-Z_][a-zA-Z0-9_]{0,31}

GtidSet's constructor assumes every colon-delimited token after the uuid is an interval and hands each to Long.parseLong. Reading a tagged set from @@global.gtid_executed or @@global.gtid_purged therefore throws:

java.lang.NumberFormatException: For input string: "mysqlsh"
	at java.base/java.lang.Long.parseLong(Long.java:709)
	at com.github.shyiko.mysql.binlog.GtidSet.<init>(GtidSet.java:61)
	at io.debezium.connector.mysql.gtid.MySqlGtidSet.<init>(MySqlGtidSet.java:42)
	at io.debezium.connector.mysql.jdbc.MySqlConnection.knownGtidSet(MySqlConnection.java:71)
	at io.debezium.connector.binlog.BinlogStreamingChangeEventSource.execute(BinlogStreamingChangeEventSource.java:232)

This is not an edge case on InnoDB Cluster: Group Replication requires gtid_mode=ON, so the GTID path is always taken, and there is no server setting that disables tag generation. Any downstream consumer of this library (Debezium, and through it Airbyte, Flink CDC, …) is unable to do CDC against such a server.

Reported downstream as airbytehq/airbyte#85846, which this change is intended to close (together with a Debezium dependency bump).

Change

Parsing. Tags are now recognised explicitly. MySQL treats a tag as part of the identity of a transaction group, so tagged and untagged transactions from the same server are tracked as separate UUIDSets, keyed internally by uuid[:tag]. A tag applies to every interval that follows it until the next tag, matching the server's rendering (<uuid>:1-5:tag_a:1-3:15-21:tag_b:8-52). toString() folds the groups back into the canonical single uuid_set form, so parse/render round-trips.

UUIDSet gains getTag() and isTagged(). getUUID() still returns the bare server uuid — DumpBinaryLogGtidCommand depends on that — and getUUIDSet(String) accepts either uuid or uuid:tag. Equality and isContainedWithin now discriminate on the tag, so a tagged set is not treated as contained within an untagged one with the same intervals.

Wire protocol. COM_BINLOG_DUMP_GTID has no field for tags. Sending tagged intervals under their bare uuid would tell the source we already hold untagged transactions we do not, and it would skip them — silent data loss. DumpBinaryLogGtidCommand therefore omits tagged sets, which is what the server itself expects from clients predating tagged GTIDs (MYSQL_RPL_SKIP_TAGGED_GTIDS, exercised by the server's own rpl_com_binlog_dump_gtids_skip_tagged test). The source re-sends those transactions rather than skipping them.

Encoding tagged GTIDs on the wire is deliberately left out of scope — it needs the 8.4 protocol extension, which is a larger change.

Untagged parsing, rendering and containment semantics are unchanged.

Tests

Added to GtidSetTest: tagged parsing and round-trip, mixed untagged+multi-tag sets on one server, tag/untagged distinctness, tag-aware containment, a guard that a bare interval is never mistaken for a tag, multi-server sets, and a regression guard on untagged behaviour. New DumpBinaryLogGtidCommandTest asserts the serialized uuid_set count and payload size for untagged, tagged, and mixed sets.

mvn test: 74/74 passing, no pre-existing tests modified.

MySQL 8.4 (WL#15294) extends the GTID grammar with an optional
user-defined tag:

    uuid_set: uuid:[tag:]interval[:[tag:]interval]...
    tag:      [a-zA-Z_][a-zA-Z0-9_]{0,31}

GtidSet's parser assumed every colon-delimited token after the uuid was
an interval and passed each one to Long.parseLong, so any tagged GTID
set read from @@global.gtid_executed or @@global.gtid_purged threw:

    java.lang.NumberFormatException: For input string: "mysqlsh"
        at com.github.shyiko.mysql.binlog.GtidSet.<init>(GtidSet.java:61)

This makes tag parsing explicit. MySQL treats a tag as part of the
identity of a transaction group, so tagged and untagged transactions
from one server are now tracked as separate UUIDSets keyed by
uuid[:tag], and toString folds them back into the canonical single
uuid_set rendering.

COM_BINLOG_DUMP_GTID has no field for tags. Rather than send tagged
intervals under their bare uuid - which would tell the source we already
hold untagged transactions we do not, causing it to skip them -
DumpBinaryLogGtidCommand now omits tagged sets. This mirrors the
server's own MYSQL_RPL_SKIP_TAGGED_GTIDS behaviour for clients that
predate tagged GTIDs; the source re-sends those transactions instead.

Untagged parsing, rendering and containment semantics are unchanged.

Signed-off-by: Ales Verbic <averbic@applause.com>
@osheroff

Copy link
Copy Markdown
Owner

not a fan -- if we don't support the tags and protocol extensions, and what the AI is saying that "tags are treated as part of the set identifier", how does a consumer of the library use this at all? the DUMP command, at the heart of everything, is going to fail weirdly -- it will just start from 0 in those GTID sets every time.

So sure, take a whack at the problem, but actually make it usable. Once it is I'll review the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants