Skip to content

validator: support distributed validators (beacon_committee_selections and sync_committee_selections) #832

Open

Description

@MegaRedHand

The validator client cannot run behind distributed validator middleware such as Obol Charon. Behind it, each validator client holds one share of a validator's key. It decides on its own whether the validator aggregates, from a selection proof signed with that share alone, and the answer is wrong.

Code references are at main @ 5d8628e9.

How a distributed validator works

The validator's key is split into shares across several operators. Each operator runs a normal validator client loaded with its share, pointed at the middleware, which sits in front of a beacon node and speaks the Beacon API to the client. The middleware combines a threshold of partial signatures into one signature under the validator's key.

That works for attestations and blocks with no change to the client: it signs with its share and the middleware does the rest. It does not work for aggregation. Whether a validator aggregates is decided by hashing its selection proof (the slot signed under the selection domain). A partial signature hashes to a different value than the combined one, so each client reaches its own, wrong, verdict.

The Beacon API has two endpoints for this. They are "implemented by a distributed validator middleware client", and "consensus clients need not support this endpoint and may return a 501" (ethereum/beacon-APIs @ 6bc8c7bd, apis/validator/beacon_committee_selections.yaml:4-11, apis/validator/sync_committee_selections.yaml:4-12).

  • POST /eth/v1/validator/beacon_committee_selections takes {validator_index, slot, selection_proof} with the partial proof and returns the combined one. Clients "must query this endpoint at the start of an epoch" for every validator with attester duties.
  • POST /eth/v1/validator/sync_committee_selections does the same for sync committee contributions, with subcommittee_index, "at the start of each slot".

What happens today

Neither endpoint is called, and ethlambda validator has no flag for a distributed setup (bin/ethlambda/src/validator.rs:19-169). The aggregation decision is made locally in three places:

  • Attestation aggregation. selection_for signs the slot and applies is_aggregator to the signature (crates/validator/src/aggregation_selection.rs:92-101, rule at :59-70).
    • At the start of an epoch, subscriptions::subscribe uses it to set is_aggregator on every committee subscription (crates/validator/src/subscriptions.rs:96-135). The beacon node decides from that flag whether to collect the committee's votes.
    • In each slot, AggregationService::aggregate uses it again to decide which duties aggregate, and puts the proof in the AggregateAndProof (crates/validator/src/aggregation.rs:108-140).
  • The proof cache. SigningContext remembers each selection proof by (pubkey, slot) so the subscription and the aggregation share one signature (crates/validator/src/signing.rs:87-88, sign_selection_proof at :597).
  • Sync committee contributions. SyncCommitteeService::aggregate signs a selection proof per (validator, subnet) at the contribution deadline and applies is_sync_committee_aggregator to it (crates/validator/src/sync_committee.rs:216-240, rule at :57).

Behind middleware, each operator's client would then:

  • claim aggregator duties it does not have, and miss the ones it has, in the subscriptions;
  • fetch and sign aggregates for slots where the validator was not selected, which the middleware cannot combine because the other operators decided differently;
  • put a partial proof in the AggregateAndProof, which no beacon node accepts.

The validator's attestations and blocks would still work. Its aggregation and sync contribution duties would not.

What other clients do

Client Flag Where it calls the endpoints Read at
Lighthouse --distributed (validator_client/src/cli.rs:194-200) beacon_committee_selections once an epoch with every duty's partial proof, retried at the next slot if it fails (validator_client/validator_services/src/duties_service.rs:172-227, :1741-1755). sync_committee_selections in validator_services/src/sync.rs:571-580. unstable @ 11813cfb
Prysm --distributed (cmd/validator/flags/flags.go:425-428) distributedSelector sends partial proofs to the middleware for attestations and sync committees (validator/client/aggregator_selector.go:202-281, :343-376). develop @ 0c1e3d24
Teku --Xobol-dvt-integration-enabled, hidden, off by default (teku/src/main/java/tech/pegasys/teku/cli/options/ValidatorClientOptions.java:77-87, validator/api/.../ValidatorConfig.java:73) validator/client/.../DvtAttestationAggregations.java:113 and duties/synccommittee/ObolDvtSyncAggregatorSelectionProofProvider.java:86. master @ 62a54b96
Nimbus --distributed, "Enable usage of Obol middleware (BETA)" (beacon_chain/conf.nim:1107-1110) beacon_chain/validator_client/selection_proofs.nim:153-178 and :436-461. unstable @ 9bed158a
Lodestar --distributed (packages/cli/src/cmds/validator/options.ts:442-445, wired at packages/validator/src/validator.ts:304, :343) packages/validator/src/services/attestationDuties.ts:212-215 and syncCommitteeDuties.ts:147-150. unstable @ 77088eb7

All five have the mode, and all five keep it off by default.

Proposal

Add an opt-in distributed mode, off by default.

  • Attester aggregation.
    • At the start of each epoch, after duties are fetched, sign every duty's selection proof as today, then send them in one beacon_committee_selections request (current epoch, then next epoch, as the spec allows).
    • Store the combined proofs the middleware returns in the SigningContext cache in place of the partial ones.
    • Compute is_aggregator from the combined proof, for the subscription and for the aggregation tick.
    • Put the combined proof in the AggregateAndProof.
  • Sync contributions.
    • At the start of each slot, sign the selection proofs for every sync duty and subnet and send them in one sync_committee_selections request.
    • Use the combined proofs at the contribution deadline.
  • Failure. If the middleware does not answer, aggregate for nobody. The verdict from the partial proof is wrong, so falling back to it only adds bad aggregates. Retry the request at the next slot, as Lighthouse does.
  • No change to the beacon node. The spec puts the endpoints on the middleware. ethlambda beacon does not serve them, and does not need to.

To decide

  • The flag's name. --distributed is what other clients use, and ethlambda keygen already has a --distributed flag with an unrelated meaning (naming lean keys by pubkey, bin/ethlambda/src/keygen.rs:60-63). They are different sub-commands, so they do not collide, but the shared name may confuse.
  • When to call. The spec asks for the start of the epoch for attester selections. Today the subscriptions are sent from a task spawned after the slot's block (crates/validator/src/lib.rs:472-500). The middleware call would go there, ahead of the subscription it feeds.
  • Other assumptions to check. Whether anything else in the client assumes its key is the validator's full key. Behind middleware the client signs with a share, so any check of a signature or proof must be against the share's pubkey. The verification of remote-signer answers (SigningContext::verified, signing.rs:188) is one place to confirm, for an operator who keeps the share in Web3Signer.
  • What to test against. A kurtosis devnet with Charon would show it end to end. Is there one we can run?

Tests

  • A mock that answers beacon_committee_selections with a proof chosen to select (or not select) the validator: the subscription's is_aggregator, the aggregation tick and the published AggregateAndProof all follow the returned proof.
  • The same for sync_committee_selections and the published ContributionAndProof.
  • With the mode off, no selections request is ever made.
  • A middleware error: the epoch's subscriptions go out with is_aggregator: false and no aggregate is attempted.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions