Skip to content

Instrument transient GitHub user refresh failures - #1492

Open
jt-schmidt wants to merge 1 commit into
Shopify:mainfrom
jt-schmidt:jt-schmidt/observe-github-user-refresh-timeouts
Open

Instrument transient GitHub user refresh failures#1492
jt-schmidt wants to merge 1 commit into
Shopify:mainfrom
jt-schmidt:jt-schmidt/observe-github-user-refresh-timeouts

Conversation

@jt-schmidt

Copy link
Copy Markdown

What

Adds targeted observability for transient GitHub network failures during Shipit::User#refresh_from_github!.

When a refresh fails with a known transient connection/timeout error (Faraday::ConnectionFailed, Faraday::TimeoutError, Net::OpenTimeout, Net::ReadTimeout), Shipit now:

  • emits an ActiveSupport::Notifications event: transient_github_refresh_error.shipit
  • logs a warning with non-email context: Shipit user id, GitHub id/login, exception class/message
  • re-raises the original exception so existing retry/error behavior is unchanged

Why

PI investigation of shop/issues#16591 / Vault issue 15928 found the current Observe error group is from Shipit::RefreshGithubUserJob timing out while opening a TCP connection to api.github.com:443.

This keeps the first change low risk: add signal before changing retry/error reporting behavior.

Test plan

  • Added coverage that Net::OpenTimeout during refresh_from_github! emits the notification, logs, and re-raises.
  • ruby -c app/models/shipit/user.rb
  • ruby -c test/models/users_test.rb

I could not run the full model test locally because the checkout is missing bundled native/test dependencies (Bundler::GemNotFound for sqlite3/ejson-rails/mysql2/etc.).

Refs: https://github.com/shop/issues/issues/16591

Assisted-By: devx/c427d7ec-0c56-4e55-a319-ae0556f7c78b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant