Skip to content

Add Firebird - #3059

Draft
alexey-milovidov wants to merge 2 commits into
mainfrom
firebird
Draft

alexey-milovidov wants to merge 2 commits into
mainfrom
firebird

Conversation

@alexey-milovidov

Copy link
Copy Markdown
Member

Adds Firebird 5.0.4, a row-oriented relational database, installed from the official release tarball (Ubuntu packages only Firebird 3.0), in the default SuperServer mode.

  • String columns are VARCHAR(n) CHARACTER SET NONE (raw bytes; the data has invalid UTF-8), sized just above the longest value of each column. Firebird limits the declared row size to 64 KB, and its sort, used for every GROUP BY and ORDER BY, pads VARCHAR keys to their declared length: on 1M rows, grouping by short values took 20.8 s as VARCHAR(100) and 490 s as VARCHAR(8000).
  • Firebird has no COPY, and external tables need fixed-width records, so loader.cpp parses hits.tsv and inserts with the IBatch API in embedded mode, with 4 threads (more were slower). A Python driver loop managed only 220 rows/s. Forced writes are off during the load and switched back on afterwards; the primary key (CounterID, EventDate, UserID, EventTime, WatchID) is added after the load using parallel workers.
  • Queries are the PostgreSQL ones with FETCH/OFFSET syntax, CHAR_LENGTH, SUBSTRING ... SIMILAR in place of REGEXP_REPLACE (Q29), DATEADD/DATEDIFF in place of DATE_TRUNC (Q43), and GROUP BY positions instead of aliases.

Expected: single-threaded queries; Q34 and Q35 (GROUP BY URL over 100M rows, ~800 GB of 8 KB sort keys) should fail for lack of disk space.

The official ARM64 binaries of 5.0.4 are built with -p (gprof instrumentation; _mcount took 40% of the load's CPU time). It was removed on Firebird master in FirebirdSQL/firebird@5231e27746 but remains in v5.0-release; x86-64 is unaffected.

Validated locally (aarch64, private server instance) on a 1M-row sample: load and primary key work, all queries run, and the answers of the rewritten Q29 and Q43 match clickhouse-local running the original queries. Timings on this host are not meaningful (contended disk).

🤖 Generated with Claude Code

Firebird 5.0.4 from the official release tarball, loaded with a small
C++ loader using the batch API in embedded mode, with the PostgreSQL
queries adapted to the Firebird dialect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

The run of firebird on c6a.4xlarge did not produce results.

Logs:

In embedded mode the user defaults to the OS user (`firebird`), which has
no rights on the table created by SYSDBA, so the first EC2 run failed with
`no permission for SELECT access to TABLE HITS`. Calling `exit` from a
failing thread also shut down the engine under the other threads, which
then crashed; use `_exit` instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alexey-milovidov
alexey-milovidov deployed to benchmark-approval November 4, 2026 23:15 — with GitHub Actions Active
@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

The run of firebird on c6a.4xlarge did not produce results.

Logs:

@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

The run of firebird on c6a.4xlarge did not produce results.

Logs:

@alexey-milovidov alexey-milovidov added the machine:c6a.metal PR benchmark machine override: c6a.metal (192 vCPU, 384 GB, AMD) label Nov 6, 2026
@alexey-milovidov
alexey-milovidov deployed to benchmark-approval November 6, 2026 00:26 — with GitHub Actions Active

This branch was successfully deployed

1 active deployment
benchmark-approval — fd6a5e89 Deployed Nov 6, 2026 by alexey-milovidov via launch #1059
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

machine:c6a.metal PR benchmark machine override: c6a.metal (192 vCPU, 384 GB, AMD)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant