Skip to content

[python] Fix record and union default resolution - #4086

Open
xgbtxy wants to merge 1 commit into
apache:mainfrom
xgbtxy:fix/python-record-union-defaults
Open

xgbtxy wants to merge 1 commit into
apache:mainfrom
xgbtxy:fix/python-record-union-defaults

Conversation

@xgbtxy

@xgbtxy xgbtxy commented Nov 14, 2026

Copy link
Copy Markdown

What is the purpose of the change

Fix two coupled specification-compliance bugs in DatumReader._read_default_value (lang/py/avro/io.py) during schema evolution:

  1. Nested RecordSchema explicit null defaults: _read_default_value previously evaluated json_val = default_value.get(field.name); if json_val is None: json_val = field.default. When an outer record's default JSON object explicitly set an inner nullable field to null (e.g., "default": {"x": null} where inner field x has "type": ["int", "null"], "default": 99), json_val is None conflated the explicit null with an omitted key and silently overwrote None with 99. Conversely, omitting a required inner field (has_default == False) whose type started with "null" silently fabricated None instead of raising InvalidDefaultException.
  2. UnionSchema first-matching-branch resolution: Per the Apache Avro Specification (Schema Declaration — Records), "Default values for union fields correspond to the first schema that matches in the union." _read_default_value previously hardcoded field_schema.schemas[0], rejecting valid defaults matching subsequent union branches (including null in ["int", "null"]). Resolving against the first matching branch also requires strict JSON type guards on boolean, int/long (not isinstance(..., bool) and 32/64-bit bounds), EnumSchema symbol membership, and ArraySchema (not isinstance(..., (str, bytes, Mapping))) so earlier branches do not falsely coerce incompatible JSON defaults.

No JIRA issue: the ASF Jira tracker requires an account.

Verifying this change

This change added tests and can be verified as follows:

  • Added TestDefaultValueResolution in lang/py/avro/test/test_io.py covering explicit null in nested record defaults ({"x": None} vs {}), omitted inner fields with default: null, rejection of omitted required inner fields, union defaults matching non-first primitive/enum/array/record branches, and rejection of unmatched union defaults.
  • Verified locally with uv run --frozen ruff format --diff ., uv run --frozen ruff check ., uv run --frozen mypy avro/, and uv run --frozen python -m unittest avro.test.test_io (196 tests passing).

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? not applicable

Assisted by Google Antigravity.

In DatumReader._read_default_value, resolving a RecordSchema default
used default_value.get(field.name) and replaced any None result with
field.default. That silently overwrote explicit JSON null entries in
nested record defaults (e.g. {"x": null}) whenever the inner field
defined a non-null default, and fabricated None when a required inner
field was omitted.

In addition, UnionSchema defaults only tried schemas[0] instead of the
first matching branch required by the Avro Specification ("Default
values for union fields correspond to the first schema that matches in
the union"), which also prevented explicit null or non-first-branch
values in record and union defaults from resolving.

Check key membership and field.has_default explicitly for RecordSchema
defaults, and resolve UnionSchema defaults against the first matching
branch with strict JSON type checks on primitive, enum, and array
branches.

Assisted-by: Google Antigravity
Copilot AI balanced review requested due to automatic review settings November 14, 2026 15:06

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants