Skip to content

Accept combining characters and extenders in element and attribute names - #379

Open
wleeaf wants to merge 1 commit into
ruby:masterfrom
wleeaf:accept-combining-characters-in-names
Open

wleeaf wants to merge 1 commit into
ruby:masterfrom
wleeaf:accept-combining-characters-in-names

Conversation

@wleeaf

@wleeaf wleeaf commented Oct 24, 2026

Copy link
Copy Markdown

NCNAME_STR leaves COMBININGCHAR and EXTENDER as TODOs, so an element or attribute name fails to parse when a later character is a combining mark that isn't [:alpha:], or the middle dot. REXML builds and writes these names itself, but can't read them back:

Name master This PR Expat (Python xml.etree)
café (decomposed é) Invalid attribute name: <́> parses parses
col·leccio (Catalan middle dot) Invalid attribute name: <·leccio> parses parses
เจมส์ (Thai, U+0E4C) Invalid attribute name: <์> parses parses
क्षेत्र (Devanagari virama U+094D) Invalid attribute name: <्षेत्र> parses parses
பெயர் (Tamil virama U+0BCD) Invalid attribute name: <்> parses parses
xml = REXML::Element.new("col·leccio").to_s  # => "<col·leccio/>"
REXML::Document.new(xml)                           # master: REXML::ParseException

[4] NameChar includes CombiningChar and Extender. This change adds:

  • the combining marks (\p{M}) that [:alpha:] doesn't already include. Many vowel signs are Alphabetic, but tone marks, viramas and U+0300-U+036F aren't. Using an intersection keeps the character class free of duplicated ranges, so there is no -w warning.
  • U+00B7 and U+0387, the only Extender characters that aren't [:alpha:].

Names starting with a mark are still rejected, and no character that was accepted before is rejected now.

Tests: test_combining_character_in_name (element and attribute name) and test_extender_in_name in test/parse/test_element.rb. Both fail before the change and pass after it. bundle exec rake test passes (843 tests), as do the --enable-frozen-string-literal run and bundle exec rake warning:error rdoc. ruby -w shows no new warnings.

Performance: tag scanning with TAG_PATTERN alone is 1-5% slower in a micro-benchmark. Parsing a 5000-element document with 2 attributes per element showed no measurable difference (best of 15 runs: 0.172s vs 0.168s).

NCNAME_STR left COMBININGCHAR and EXTENDER as TODO, so names whose non-first characters are combining marks that are not [:alpha:] (U+0301 in a decomposed "cafe\u0301", the Thai U+0E4C, the Devanagari virama U+094D, the Tamil U+0BCD) or the middle dot U+00B7 ("col\u00B7leccio") raised ParseException, although REXML::Element.new accepts these names and writes them. Add the combining marks that [:alpha:] does not already include, and the two Extender characters that are not [:alpha:].
Copilot AI balanced review requested due to automatic review settings October 24, 2026 23:05

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants