Skip to content

Keep track's optimistic metric update when not connected - #72

Open
ryanechternacht wants to merge 1 commit into
mainfrom
ryan/replicator-track-offline
Open

ryanechternacht wants to merge 1 commit into
mainfrom
ryan/replicator-track-offline

Conversation

@ryanechternacht

Copy link
Copy Markdown
Member

Problem

track enqueues the event and then optimistically adds its quantity to the matching metric on the cached company, via DataStream::Client#update_company_metrics. That second step only ran when @datastream_client&.connected? was true. In replicator mode connected? is the health endpoint's ready field, and in websocket mode it is the WebSocket connection state. So while the replicator reported not ready, or the WebSocket was disconnected, usage stopped counting locally and every metered limit stayed at the last value the replicator or stream wrote. The event itself was still enqueued to the API either way.

Why the gate existed

The condition arrived in #5 ("Ruby custom code"), the initial port of the hand-written layer, as if @datastream_client&.connected? && body[:company] with the comment "Update company metrics locally if DataStream is active and connected". The PR description says it follows the SDK spec with schematic-node as the primary reference, and its review has no discussion of the condition. #23 later touched the method (adding track options) without changing the condition, and has no discussion of it either.

It is a port of the Go and Node gate, and SchematicHQ/schematic-go#239 traces that back: Go introduced it when datastreamConnected was the only signal that DataStream was in use, a refactor translated it mechanically into IsConnected(), and replicator mode later redefined IsConnected() as readiness without revisiting the call site. Nothing in Ruby's history ties it to a correctness concern either.

Why removing it is safe

  • No double counting. The SDK never receives metric deltas. A full company message replaces the cached company, and a partial message's metrics go through Merge.upsert_metrics, which replaces each metric by (event_subtype, period, month_reset) with the server's value. In replicator mode the replicator writes whole company records to Redis. Whatever arrives next overwrites the optimistic value rather than adding to it, which is the same reconciliation that already happens between a bump and the next push while connected.
  • No write to a missing cache. update_company_metrics reads the company from the cache first and returns if it isn't there, so it only rewrites an entry that already exists. In replicator mode the keys use the last cache version the client knows.
  • The cache outlives the outage. A cached company is served again as soon as the replicator reports ready or the WebSocket reconnects, and Ruby doesn't re-request companies on reconnect, so usage tracked in between should already be in it.

The gate is dropped in websocket mode too, for the same reasons, and a single "DataStream is in use" condition is simpler than branching on mode.

Change

In SchematicClient#track, the metric update now runs when @datastream_client && body[:company], in both modes. Nothing else changes.

Related, not changed here

update_company_metrics rewrites the company and its lookup keys with the SDK's DataStream cache_ttl (24 hours by default), while the replicator writes with no TTL by default. It also rewrites the company when no metric matched the event. Both already happen today whenever track runs with the replicator ready. With this change they can also happen while the replicator is not ready, so during a very long outage a company that was tracked and then went idle for cache_ttl could drop out of Redis until the replicator resyncs. schematic-node#202 raised the same point; a follow-up could keep the existing expiry in replicator mode and skip the write when no metric matched.

Overlap

#71 (replicator cache-ready gate) edits the flag-check gates and the health poll in the same files but not track. The two branches merge cleanly.

Parity

This matches SchematicHQ/schematic-go#239. The same change is open for Node (schematic-node#202), Python (schematic-python#134), Java (schematic-java#115) and C# (schematic-csharp#202).

Tests

New SchematicClient - track metric update while not connected block in test/custom.test.rb:

  • Replicator mode, with /ready answering 503 {"ready": false, "cache_version": "v1"} and a Redis stand-in seeded the way the replicator writes a company (the entity under its ID key plus a lookup key). connected? is false, track with quantity 5 moves the cached metric from 10 to 15.
  • Replicator mode, not ready, tracking a company that isn't cached leaves Redis empty.
  • Websocket mode with no connection, a cached company's metric moves from 10 to 15 after track.

On main the replicator test reads the cache version from the rules engine, since main ignores a 503's body, so the test sets v1 explicitly. With #71 the 503 records v1 itself.

The two "moves the metric" tests fail on main (value stays 10) and pass with this change. Locally, in a ruby:3.3 container to match CI: bundle exec rubocop (1140 files, no offenses) and bundle exec ruby -I lib -I test test/custom.test.rb (193 runs, 397 assertions, 0 failures, 0 errors) pass.

🤖 Generated with Claude Code

track only bumped the cached company metric when the DataStream client
reported connected?, which in replicator mode is the replicator's
readiness. While the replicator reported not ready, or the WebSocket was
disconnected, usage stopped counting locally and metered limits stayed at
the last value the replicator or stream wrote.

Drop the connected? condition so the update runs whenever DataStream is
in use, in both modes. Server and replicator updates replace metric
values outright, so nothing is counted twice, and the update is a no-op
for companies that aren't cached.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant