Conversation
MySQL 8.4 (WL#15294) extends the GTID grammar with an optional
user-defined tag:
uuid_set: uuid:[tag:]interval[:[tag:]interval]...
tag: [a-zA-Z_][a-zA-Z0-9_]{0,31}
GtidSet's parser assumed every colon-delimited token after the uuid was
an interval and passed each one to Long.parseLong, so any tagged GTID
set read from @@global.gtid_executed or @@global.gtid_purged threw:
java.lang.NumberFormatException: For input string: "mysqlsh"
at com.github.shyiko.mysql.binlog.GtidSet.<init>(GtidSet.java:61)
This makes tag parsing explicit. MySQL treats a tag as part of the
identity of a transaction group, so tagged and untagged transactions
from one server are now tracked as separate UUIDSets keyed by
uuid[:tag], and toString folds them back into the canonical single
uuid_set rendering.
COM_BINLOG_DUMP_GTID has no field for tags. Rather than send tagged
intervals under their bare uuid - which would tell the source we already
hold untagged transactions we do not, causing it to skip them -
DumpBinaryLogGtidCommand now omits tagged sets. This mirrors the
server's own MYSQL_RPL_SKIP_TAGGED_GTIDS behaviour for clients that
predate tagged GTIDs; the source re-sends those transactions instead.
Untagged parsing, rendering and containment semantics are unchanged.
Signed-off-by: Ales Verbic <averbic@applause.com>
Owner
|
not a fan -- if we don't support the tags and protocol extensions, and what the AI is saying that "tags are treated as part of the set identifier", how does a consumer of the library use this at all? the DUMP command, at the heart of everything, is going to fail weirdly -- it will just start from 0 in those GTID sets every time. So sure, take a whack at the problem, but actually make it usable. Once it is I'll review the PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
MySQL 8.4 (WL#15294) extends the GTID grammar with an optional user-defined tag:
GtidSet's constructor assumes every colon-delimited token after the uuid is an interval and hands each toLong.parseLong. Reading a tagged set from@@global.gtid_executedor@@global.gtid_purgedtherefore throws:This is not an edge case on InnoDB Cluster: Group Replication requires
gtid_mode=ON, so the GTID path is always taken, and there is no server setting that disables tag generation. Any downstream consumer of this library (Debezium, and through it Airbyte, Flink CDC, …) is unable to do CDC against such a server.Reported downstream as airbytehq/airbyte#85846, which this change is intended to close (together with a Debezium dependency bump).
Change
Parsing. Tags are now recognised explicitly. MySQL treats a tag as part of the identity of a transaction group, so tagged and untagged transactions from the same server are tracked as separate
UUIDSets, keyed internally byuuid[:tag]. A tag applies to every interval that follows it until the next tag, matching the server's rendering (<uuid>:1-5:tag_a:1-3:15-21:tag_b:8-52).toString()folds the groups back into the canonical singleuuid_setform, so parse/render round-trips.UUIDSetgainsgetTag()andisTagged().getUUID()still returns the bare server uuid —DumpBinaryLogGtidCommanddepends on that — andgetUUIDSet(String)accepts eitheruuidoruuid:tag. Equality andisContainedWithinnow discriminate on the tag, so a tagged set is not treated as contained within an untagged one with the same intervals.Wire protocol.
COM_BINLOG_DUMP_GTIDhas no field for tags. Sending tagged intervals under their bare uuid would tell the source we already hold untagged transactions we do not, and it would skip them — silent data loss.DumpBinaryLogGtidCommandtherefore omits tagged sets, which is what the server itself expects from clients predating tagged GTIDs (MYSQL_RPL_SKIP_TAGGED_GTIDS, exercised by the server's ownrpl_com_binlog_dump_gtids_skip_taggedtest). The source re-sends those transactions rather than skipping them.Encoding tagged GTIDs on the wire is deliberately left out of scope — it needs the 8.4 protocol extension, which is a larger change.
Untagged parsing, rendering and containment semantics are unchanged.
Tests
Added to
GtidSetTest: tagged parsing and round-trip, mixed untagged+multi-tag sets on one server, tag/untagged distinctness, tag-aware containment, a guard that a bare interval is never mistaken for a tag, multi-server sets, and a regression guard on untagged behaviour. NewDumpBinaryLogGtidCommandTestasserts the serializeduuid_setcount and payload size for untagged, tagged, and mixed sets.mvn test: 74/74 passing, no pre-existing tests modified.