Skip to content

Spark: Support backslash escapes in string literals - #2989

Open
andygrove wants to merge 2 commits into
apache:mainfrom
andygrove:spark-backslash-escape
Open

andygrove wants to merge 2 commits into
apache:mainfrom
andygrove:spark-backslash-escape

Conversation

@andygrove

Copy link
Copy Markdown
Member

Dialects: Spark

SparkSqlDialect does not process backslash escapes in string literals, so SELECT '\thello' parses to a backslash followed by thello, and SELECT 'I\'m' fails to parse. Spark unescapes these (string literals, unescapeSQLString), as DatabricksDialect already does here.

This enables supports_string_literal_backslash_escape for Spark. It also enables ignores_wildcard_escapes, because Spark keeps the backslash in \% and \_, as MySQL does.

Not covered: \uXXXX, \UXXXXXXXX and octal escapes, and \a / \f, which Spark reads as a / f. The tokenizer handles those the same way for every dialect.

Found through apache/datafusion#21516.

This PR was written with AI assistance (Claude Code). I reviewed the change and the tests.

andygrove and others added 2 commits November 16, 2026 17:34
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added the Spark label Nov 17, 2026
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.29%. Comparing base (5201fb1) to head (596d248).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2989   +/-   ##
=======================================
  Coverage   83.29%   83.29%           
=======================================
  Files          42       42           
  Lines       35372    35378    +6     
  Branches    35372    35378    +6     
=======================================
+ Hits        29463    29469    +6     
  Misses       2456     2456           
  Partials     3453     3453           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants