Repository navigation
IGNITE-28938 Split MarshallableMessage: a message with a custom wire form needs no marshaller - #13427
Conversation
e79af2b to
afa9093
Compare
…form needs no marshaller MarshallableMessage meant two things at once: "call me before sending" and "I need a Marshaller". Six of its implementations use neither the marshaller nor the class loader - they only convert their own fields into the form that goes on the wire: GridCacheEntryInfo recalculates a TTL, the partition map messages copy and zip their maps, TcpDiscoveryNode wraps metrics into a message, ColocationGroup packs bits, GridNearTxPrepareRequest sets an expiry flag on its entries. Added CustomWireFormMessage with toWireForm() and fromWireForm(), moved those six to it and taught the generator and the factory provider about it. A class can implement both: the generated TcpDiscoveryNodeMarshaller keeps its Marshaller for the @Marshalled fields and calls the new hook next to it. Also fixed GridTestUtils.loadMarshaller: it picked the companion constructor by 'instanceof MarshallableMessage', while the generator emits one taking a Marshaller for 'marshallable || hasMarshalled'. It now asks the constructor itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…marshalling step Both interfaces were listed side by side wherever the code asks 'does this message have a step of its own' - in the NonMarshallableMessage check and when requiring the generated marshaller. Extracted CustomMarshallingMessage as their common parent, so those places ask once. The generator still tells the two apart: it emits marshal(marshaller) for one and toWireForm() for the other. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ion, drop unused imports, shorten javadocs Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ireFormMessage with codegen tests Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…CustomWireFormMessage A custom wire form is not marshalling: it is the message reshaping its own fields. It no longer goes into the generated marshaller; MessageMarshalling runs it as a step of its own, before marshalling and after unmarshalling. A message gets a marshaller only when it has something to marshal. CustomWireFormMessage is deprecated: computing inside a message belongs to the code that fills it in and reads it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…cs it made stale Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ling to marshalling only Marshalling must not run the step a message does itself. MessageWire is now the entry point callers and generated code use: it runs the message's own step on the way out, marshalling on the way in, and nothing else knows the order. MessageMarshalling is left with marshalling alone and is no longer visible outside the package. NonMarshallableMessage no longer conflicts with a custom wire form: the marker says no marshaller is generated, which says nothing about a step the message runs itself. Covered by a test. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The callers of MessageWire still spoke of marshalling: a boolean named unmarshal, private methods marshal and unmarshall, a GridIoMessage marked marshalled. Each covers both steps now, so each is named after the wire, not after one of the steps it runs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…hind toWireForm/fromWireForm never threw, so the checked exception went away along with the imports it kept alive. MessageProcessor and the other files the rework touched but did not need to change are back as they were. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The warning is the point of the deprecation: it is what tells a reader that this code depends on a message doing work of its own. Suppressing it in the very place that runs those messages hides exactly what should be visible. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
MarshalMode drove more than marshalling: nested messages go through MessageWire, cache objects through their own calls, collections through loops. It is the direction the generated code runs in, so it says OUT and IN now, and the helpers that serve both directions no longer carry marshal in their names. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The generated companion did both: it turned @Marshalled fields into bytes and walked the nested messages and cache objects. Only 28 of the 197 it made for core marshalled anything, so the name fitted almost none of them. The two are companions of their own now. MessageMarshaller keeps the bytes: @Marshalled fields and the marshal call a MarshallableMessage defines. MessageWireForm walks the fields. MessageWireGenerator and MessageMarshallerGenerator generate them over the shared ground of MessageWireCompanionGenerator, and MessageWire runs the three steps in order: the message's own step, marshalling, walking on the way out, and the reverse on the way in. Core now gets 28 marshallers instead of 197, and 187 wire forms. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…the split Four helpers sat in the shared base while only one subclass ever called them, and the base still described its work as marshalling although marshalling had moved out of it. The helpers went to the subclass that uses them and the javadoc now says what the code does: generating the code of a field in a direction. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…arshaller that no longer walks Two messages still made their marshaller call MessageWire: the elements they rebuild from wire arrays were skipped by the common walk, so the rebuild had to take them through the wire itself. Both halves of that are gone — the walk no longer skips those arrays, and the rebuild only puts the elements back. No marshaller calls MessageWire now, so the walking code moved to MessageWireFormGenerator, leaving the shared base at 281 lines from 526. Codegen now checks two lists, custom-wire-form-messages.txt and marshallable-messages.txt: a message that takes part in its own wiring by hand and is not listed is rejected at compile time. The lists name who does it today. They are meant to shrink, and when the wire form one empties, the deprecated interface goes away with the step that runs it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…the marshaller Nothing stopped a message from putting a wire form step inside marshal(): moving a field into the one that is actually sent compiles just as well as marshalling it. A list of allowed classes would only say who does it today. MessageProcessor now reads the method body instead. A MarshallableMessage whose marshal or unmarshal does something without ever touching the marshaller it is given is rejected: that is not marshalling. An empty body still passes — it does nothing at all, as GridEventStorageMessage.unmarshal does while its filters wait for IGNITE-28901. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Removing the lists left their leftovers: a resolved TypeElement nobody read, the constant behind it, and a local holding a component type the moved-out code used to need. Three more helpers turned out to serve one subclass each once the marshaller stopped walking, so they moved there. The shared base is down to 244 lines. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ut of the shared base The Tree API check on marshal() bodies is gone: it weighed more than an already large diff. The base still knew about @Marshalled — the kinds, the fields behind them, and the reading of them — while only the marshaller ever asked. It knows nothing about marshalling now, and the wire form no longer reads message fields it has no use for. What is left there is what both companions need: the context resolution, the message factory local, the NonMarshallableMessage skip. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
NonMarshallableMessage was the last thing about marshalling left in the base, and only the wire form ever asked about it: a message excluded from wiring has nothing to walk. It went there along with the type predicates, the loop depth and the MessageWire constant, all of which the marshaller never touched either. The base is 94 lines now and holds what both companions put at the top of a generated method — the cache object context and the message factory local — plus the source they accumulate. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Moving members between the generators left their fields at the bottom of the class and, twice, two javadocs stacked on one declaration. Constants, fields, constructor, methods — in that order now, and MessageProcessor keeps the javadoc it had. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…hat only changed its last byte Three parameters outlived their use when the marshaller stopped looking at the message fields: both generate methods took the ordered fields, and the collection rebuild took a field name. A regenerated serializer expectation differed from master by its trailing newline alone and is back as it was. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The marker was named when a message had one companion, so "no marshaller is generated" said it all. With the companions split it means something wider: the message needs nothing but its serializer, gets no companion of any kind, and nothing walks into a field of its type. That is what PlainMessage says, and it takes the last marshalling word out of the wire form generator. The interface is not in 2.18, so the rename costs nothing today; after 2.19 it would break plugins. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…s only where they are needed The context line ran into the code right after it, and every for-each carried a cast. A field of the message keeps its type arguments, so iterating it needs none — 256 loops lost the cast. Inside a loop the variable is declared by simple name and is raw, so the two nested loops keep theirs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three call sites, two lines each, and a boolean argument that read as ownStep(msg, true). The instanceof says what happens right where it happens now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
MessageWire.toWire called wireForm.toWire, which reads as recursion though the two are different things: one runs all three steps of a message, the other does one of them. The companion is now named after its work — walkOut, walkIn, walkInNio — and the entry point keeps the name every caller and the generated code already use. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ing helpers Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…res, one verb pair prepare/restore Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
4189a73 to
338e6da
Compare
…llingCalls, recursion stays in the walker Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…MessageMarshaller, SelfMarshallingMessage Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…imports and javadoc Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…the self-marshalling step is new Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…MessageFactoryProvider, untouched javadoc Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…-check, test the generated marshaller
…ing, dead annotation
# Conflicts: # modules/core/src/test/java/org/apache/ignite/spi/discovery/tcp/DiscoveryUnmarshalVulnerabilityTest.java
|
/runall 🎨 Checkstyle autofix: fixed 2 of 18 violation(s) in 2 file(s) — commit a644bab. 16 remain (javadoc/naming/wrapping need a human) — the Checkstyle suite may still fail. |
|
Ignite PR Checker verdict · RunAll build 9253822 · 147 suites ran, 0 reused
❌ 1 blocker(s) in 1 suite(s):
⏳ Auto re-run #1 in progress — 3 blocker + 1 broken suite(s) re-queued (attempt 1/2), ≈ settled by 02:35 MSK. This comment updates when they settle. |
# Conflicts: # modules/core/src/main/java/org/apache/ignite/internal/CoreMessagesProvider.java
| } | ||
|
|
||
| // TODO IGNITE-28920: move the expireTime rebase out of the marshalling hooks. | ||
| // TODO IGNITE-28920: move the expireTime rebase out of the self-marshalling step. |
There was a problem hiding this comment.
It seems, that https://issues.apache.org/jira/browse/IGNITE-28920 is fixed i your PR? If so, do we need this TODO?
There was a problem hiding this comment.
Good catch, but only half of IGNITE-28920 is done here.
The ticket asks for two things:
-
"drop the
MarshallableMessageinterface fromGridCacheEntryInfo" — this PR does it. The class now implementsSelfMarshallingMessage, whose methods take noMarshaller, so the class no longer claims to carry marshalled payload. -
"move the expireTime rebase out of the marshalling hooks" — this PR does not do it. The rebase still runs inside the message; only the hook changed its name (
marshal(Marshaller)->selfMarshal()).
So the TODO is still needed, and I reworded it to name the remaining part:
// TODO IGNITE-28920: the rebase still runs inside the message;
// move it to the code filling and reading the entry.I can also narrow the ticket description to part 2, so that the ticket and the TODO say the same thing. Tell me if you want that and I will update the ticket.
|
LGTM |
Possible compatibility issues. Please, check rolling upgrade casesThis PR modifies protected classes (with Order annotation). Affected files:
|
Problem
MarshallableMessagemeans two things at once: "call me before sending" and "I need aMarshaller". Six of its implementations use neither the marshaller nor the class loader — they only convert their own fields into the form that goes on the wire:GridCacheEntryInfo— turns an absolute TTL into a relative one and back;GridDhtPartitionsFullMessage,GridDhtPartitionsSingleMessage— copy and zip the partition maps;TcpDiscoveryNode— wraps metrics intoNodeMetricsMessage;ColocationGroup— packs assignments into bits;GridNearTxPrepareRequest— sets the expiry-transfer flag on its entries.They still received a
Marshallerin the generated companion, and "does this message need a marshaller" could not be told from its type.Solution
A new interface for the second meaning:
The six classes moved to it; the generator (
MessageProcessor,MessageMarshallerGenerator) andAbstractMarshallableMessageFactoryProviderhandle both interfaces. TheNonMarshallableMessagecheck covers the new one as well.A class may implement both: the generated
TcpDiscoveryNodeMarshallerkeeps itsMarshallerfor the@Marshalledfields and callstoWireForm()next to it, whileGridCacheEntryInfoMarshallernow has noMarshallerfield at all. Companions holding aMarshallerin core: 29 → 25.Also fixed
GridTestUtils.loadMarshaller: it picked the companion constructor byinstanceof MarshallableMessage, while the generator emits aMarshallerconstructor formarshallable || hasMarshalled— so the criterion was wrong for classes with@Marshalledfields and no interface. It now asks the constructor itself.No wire format change. Part of IGNITE-28935.
Verification
MessageProcessorTest— 39/39 (the expected diagnostic text is updated: the message now names both interfaces)MessageFactoryMarshallerInitializationTest— 3/3DirectMarshallingMessagesTest— 1/1IgniteCacheEntryListenerAtomicTest— 9/9CacheExchangeMergeTest— 35/35GridCacheAtomicNearEnabledFullApiSelfTest— 150/150TcpDiscoverySelfTest— 51 tests;testMulticastIpFinderfailed once on a local multicast flake and passes on a rerun-Pcheckstyle— clean🤖 Generated with Claude Code