Skip to content
View orca-zhang's full-sized avatar
🎖️
<coolcode />
🎖️
<coolcode />

Organizations

@bilibili @ez8-co @orcastor @benchplus @IceWhaleTech @agentex-ai

Block or report orca-zhang

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
orca-zhang/README.md

Orca Zhang

Distributed Systems · Storage · AI Infrastructure

15 years of engineering · Tech Leader / Architect

简体中文 · Email

I design, build and operate infrastructure for distributed storage, enterprise data protection, community services and AI inference. I learn new domains quickly and take systems from core implementation through rollout and long-term operations, with a focus on performance, availability and cost.

Engineering track record

Distributed storage & enterprise data protection

  • Backup & disaster recovery: Optimized the core storage engine to deliver +100% overall system performance, bringing a commercial backup and disaster-recovery system to a world-leading performance level.
  • Enterprise file sharing: Led core engine development for an enterprise file-sharing platform built on a fourth-generation distributed filesystem. Its vendor earned 8 consecutive years of Gartner recognition, as the only Asia-Pacific vendor listed in its category.
  • Distributed object storage: Contributed to object storage serving 1.5 PB+ of data, optimized Paxos / Gossip / LevelDB based storage, and led an archival storage service from zero to launch.
  • I/O optimization: Raised 4 KB file transfer throughput from 3–4 to 13,000+ files/s on N100 / SSD / gigabit networking; custom raw-disk reading improved filesystem operations 5–6× and backup audits 1.9–2.2×.

Community services & communications

  • Services built to last: Delivered private messaging, inbox services and activity feeds that have operated reliably in production for years.
  • Live-streaming architecture: Reworked services to reach million-scale QPS on a single endpoint, with zero production incidents during the migration.
  • Community platform from zero: Led architecture and delivery of 10+ microservices in 3 months, supporting 130+ applications and reducing new-business launch cycles from 3 weeks to 1 week.
  • Push infrastructure: Unified 3 push systems serving 100M+ daily PV, with APNs delivery within 3 seconds at a 10M-user scale. Raised delivery from 20% to 50% (+30 percentage points) and reduced push costs from CNY 1M to CNY 50K per month (-95%). Among the first in the industry to deliver end-to-end business-funnel visibility and rapid fault isolation, and recall / replacement across all push channels.
  • Push-driven engagement: Raised push-triggered app-launch UV / DAU from 11.8% to a peak of 15.38%, keeping the ratio at or above 13%.

Security & production infrastructure

  • Security platforms: Built and maintained anti-fraud, compliance-detection and real-time risk-control systems; delivered account-system internationalization and regulatory compliance changes.
  • Core infrastructure ownership: Maintained shared services, core libraries and storage infrastructure; supported C10K connection handling, monolith-to-microservice evolution, unit testing and CI/CD.
  • Availability & connectivity: Optimized P2P / relay connectivity and hardened software distribution with multiple sources, dual-stack networking and DNS / IP failover.

AI inference, native runtimes & model access

  • Native inference stack delivered: Released Orca L0, a standalone MIT-licensed Level Zero runtime and kernel compiler adapter, integrated into 125B MoE inference on A770 16 GB + system RAM. Host execution does not link SYCL, Unified Runtime or oneMKL.
  • 671B model deployment: Optimized ktransformers to run the complete DeepSeek R1 671B model at 28 tok/s, single request; Q2_K_XL on dual EPYC 9275F, 1 TB RAM and one RTX 4090 48 GB.
  • Performance gains over the original SYCL backend: With fixed 2048 input / 128 output tokens on Windows / A770, Orca L0 improves decode throughput by 40.78%–49.31% and prefill throughput by 52.72%–54.55% across Code2K / Prose2K. These compare the November 4 SYCL and November 8 L0 versions across measurement cohorts on the same machine, including cache and MTP policy differences. Conditions and raw evidence.
  • Deployment & embedding efficiency: Trimmed the Intel SYCL inference distribution from 10.6 GB to 59 MB. Distilled BGE-M3 into a 4-layer INT8 model, 2.27 GB → 150 MB, with measured local embedding latency 200 → 20 ms/text.
  • LLM protocol interoperability: Delivered a gateway translating 4 client protocols into 4 upstream formats, handling third-party relay differences in tool schemas, argument payloads, call IDs and streaming events.

Selected work

📂 filens

A local file CLI for Agents and a pure frontend preview engine, sharing a Rust / WASM conversion core. Files stay on your device. Try it online.

  • Evidence for Agents: The CLI covers 349 file extensions, with local reading, OCR, conversion and nested-archive search. Outputs include original passages, document structure, available page coordinates and JSON / Markdown.
  • Complex formats, local previews: The browser covers 450 file extensions, including Office, modern iWork, PDF, CAD / STEP, 3D, RAW and panoramic photos / video, through direct preview or local WASM conversion. Coverage and limits.
  • Integration and offline use: Vue / JavaScript APIs, iOS / Android SDKs and a DeepSeek Harness plugin; the CLI can export self-contained HTML with the original file and preview engine included.
  • Load by capability: Split modules load on demand, or use the all-in-one JS build. Preview 0.3.7's main single-JS runtime is 3.40 MB Brotli, including Vue, styles, workers and WASM; optional RAR / 7z assets add 0.36 MB. Build measurements.

Local inference for a 125B MoE model on Intel Arc A770.

  • GPU and system memory: 16 GB VRAM + system RAM runs Qwen3.8-Flash-Next, keeping hot experts in VRAM and using XMX compute and MTP speculative decoding.
  • Native backends: Orca L0 is preferred; Orca Direct is experimental. Direct encodes GPU commands and submits through Windows D3DKMT, without SYCL, UR, Level Zero or Intel's GPU user-mode driver at execution time. Paths and status.

Orca L0 throughput gains over the original SYCL backend:

Scenario Decode throughput gain Prefill throughput gain
Code2K +40.78%–41.01% +54.35%–54.55%
Prose2K +49.12%–49.31% +52.72%–54.42%

The table compares the corrected November 4, 2026 SYCL build with the November 8 Orca L0 configuration: Windows / A770, fixed 2048 input / 128 output tokens, measured after warmup. Results come from separate measurement cohorts on the same machine and reflect whole-version gains, including version, cache capacity and MTP policy differences. Conditions and raw data.

Experimental Orca Direct prefill results: A separate November 4–5 cross-cohort comparison records 71.26%–72.85% / 70.08%–71.53% gains over SYCL for Code2K / Prose2K, using the frozen builds and configurations from that stage. Direct measurements.

A local-first Go Agent runtime with 19 MB idle memory. The design centers on observable execution, controlled budgets and evidence before promotion.

  • Make protocol compatibility operational: Convert across OpenAI Chat Completions, Responses, Anthropic Messages and Gemini; normalize relay-specific tool schemas, empty arguments, call / result IDs and SSE events. Compatibility handling sits alongside provider verification, failover and circuit breakers. Schema normalization.
  • Evaluate the whole execution path: One candidate and a fixed baseline connect selector, execution, budget and cutover-readiness gates, keeping routing decisions and execution results comparable.
  • Native capabilities before model calls: Route voice, OCR, PDF, document conversion and media work through local tools; use model reasoning where the workflow needs it.
  • Treat web access as stateful execution: HTTP → proxy extraction → browser fallback, challenge detection, cookie / session reuse and multiple browser engines handle real-site failures.
  • Controlled skill evolution: Failures and repeated grounded lessons produce candidate revisions; follow-up evaluation, human promotion and lineage-checked rollback govern adoption. Harness and evolution.

⚙️ ZimaOS-AI

An experimental Go control plane for local inference, developed alongside Blue.

  • Reusable engine integration: Go SDK and service entry points, native backend bindings and hardware discovery provide a shared foundation for local model execution.
  • Evaluation tools already built: Task-level Harness cases and runners for tool calling and grounded QA, alongside performance / quality comparisons for model and runtime candidates.
  • Roadmap: Network-attached GPUs; Harness-guided dynamic quantization tuned to local hardware, memory and task-quality budgets; an AI BMC suite; transparent device sleep / wake orchestration.

🐋 orcas

Lightweight content-addressable object storage with client-side data processing and batching by default. Work in progress.

  • End-to-end encryption: User-controlled keys; the SDK encrypts each chunk before upload and decrypts it on read.
  • Compression that earns its cost: Skip recognized compressed formats; test the first chunk and disable compression if it does not shrink. Compress before encrypting.
  • Staged content deduplication: File size and header XXH3 filter candidates before full XXH3 / SHA-256 checks; content references enable instant uploads and reuse across versions.
  • Batching through the storage path: Batch reference checks and metadata updates, deduplicate within an upload batch, pack small files and chunk large ones to reduce round trips and per-file I/O.

Systems libraries & design decisions

💉 yapi

  • Bridge process and bitness boundaries: A common calling interface across x86 / x64 / WOW64, including cross-bitness remote calls.
  • Local-style API, explicit ABI work: Marshal multiple arguments and 64-bit results behind ordinary call syntax; header-only, with no helper DLL or external LIB.

🐞 emock

  • Mock six function families with one macro: Virtual, non-virtual, static, global, variadic and template functions, without adding mock interfaces to production classes.
  • Resolve overloads and patch safely: Windows PDB symbol matching plus near-jump trampolines address overloaded methods and avoid long-jump overwrites of short x64 functions.
  • A handle and a mapped view are different owners: Inspect both open handles and memory-mapped views to handle locks that survive closing a file handle, entirely in user mode.
  • Handle Windows edge cases: Reserved / malformed names, trailing dots or spaces, UNC paths and cross-bitness operations; avoid scan stalls on blocking special handles.
  • Small client, complete transport path: Header-only C++ query / write client with HTTP, UDP and batch writes, under 300 LOC, using standard and system libraries.
  • One HTTP body parser: A Duff's-device-style state machine shares body handling between Content-Length and chunked responses. Implementation.
  • Optimize the application's full path: Store object pointers directly to avoid serialization; preallocated nodes, compact index-based links and slot reuse reduce allocation and GC pressure.
  • Cheap sharding: Power-of-two buckets reduce lock contention and use bitmask selection. ecache2 adds generic keys: integers select shards directly; paired integer keys use XOR.
  • Keep expiry work predictable: A shared 100 ms clock, calibrated every second, avoids per-access clock calls; lazy expiry and slot reuse avoid periodic full-cache cleanup.
  • Protect the hot set: LRU-2 promotes reused entries to a separate queue so scans do not displace hot data. Optional Redis invalidation narrows stale-data windows. Comparative benchmarks cover throughput, allocations, GC pauses and hit rate.

🏎️ borm / zorm

  • Performance comes from the data path: reflect2 data binding, SQL / metadata reuse and shape-aware call-site caches reduce repeated reflection, query construction and allocations.
  • Measured reuse optimization: zorm's published reuse-path microbenchmark reports 10,320 → 1,200 ns/op (8.6×) and 15 → 2 allocs/op against its original implementation. Benchmark details · ORM comparisons.

✂️ cad2x

  • Extract the CAD engine, trim the desktop stack: Derived from LibreCAD; deeply trimmed QtCore / QtGui and removed QtWidgets, QtPrintSupport, GUI platform, OpenGL and Boost dependencies.
  • CAD conversion in ~2.9 MB: DWG / DXF → DXF / PDF / PNG / SVG in a Linux ARM64 executable after strip + UPX, with font assets separate. Lazy TTF / TTC loading reuses system fonts.

🔬 ezpp

  • Measure functions, blocks and object lifetimes: Header-only C++ instrumentation with RAII scopes and atomic statistics, supporting recursion and multiple threads, with runtime print / save / clear controls.
  • An optimization workhorse: Built for internal bottleneck diagnosis; now used in Strata-A770 to isolate verifier, MTP, expert execution and cache costs, guiding fusion and scheduling experiments. Gains are checked with profiling disabled. Applied profiling.

Hardware design & hands-on work

🌡️ Four-input NTC fan controller

Four-input NTC controller features and front/back design renderings Four-input NTC fan cable assembly and wiring illustration

Board design renderings and cable assembly illustration. Click to enlarge.

A fully analog fan controller designed for compute-card cooling modifications and DIY builds.

  • Four probes, highest temperature first: Accepts up to four 50 kΩ B3950 NTC probes and uses the highest temperature to drive one fan-control output at approximately 25 kHz PWM.
  • Small, with no firmware to flash: Roughly 11 × 10 mm, running on regulated 12 V, with no MCU. The four-layer board has 12 surface-mount components on the front and wiring pads on the back; the 100 × 100 mm, 72-board panel, BOM, Gerbers and placement files are complete.
  • Fixed temperature curve: Nominally about 30% duty at low temperatures, ramping from around 33°C to 100% duty around 61°C. The curve comes from behavioral simulation; first-board measurement and calibration remain pending. PWM duty is separate from actual fan RPM.

Hands-on GPU systems

NVIDIA Tesla and AMD Instinct MI50 cooling, power adaptation and hardware modifications, followed through driver, runtime and inference-framework bring-up. These projects deepen my practical understanding of power delivery, thermals, embedded control and high-speed interconnects.

🧰 Workshop & AI hardware

Lately I've been using AI to help design circuit boards and have equipped my own hardware workshop:

  • Electronics assembly and repair: SMT pick-and-place machine, reflow oven, rework station, high-frequency hot-air gun and more.
  • Mechanical prototyping and machining: 3D printer, CNC machine, table saw and more.

I want to go deeper into AI hardware, bringing code, circuits, machining and assembly together to make something different.

❄️ Compute-card cooling shop

I also run a Xianyu shop specializing in cooling modifications for low- and mid-range compute cards. In this niche, I'd place my shop among China's top five. Different ideas have led to more rigorous test data, better value and better performance.

  • After-sales service: One-year returns and exchanges, including free replacements with new units.
  • Overseas orders: International customers are welcome to get in touch. I can arrange professional, reliable forwarding; the business currently reaches Southeast Asia, Australia, Canada and beyond.

More open source

Project Highlights
idgen Snowflake IDs; Redis / local fallback, NTP synchronization, rollback protection and JS-safe 53-bit IDs.
LRUCache · Go / C++ / JS Three implementations; the C++ version avoids copying through key overloading.
xpjson C++11 JSON library; about 1,500 LOC, no external dependencies, TMP, SSO and COW.

Tech stack

C++ · Go · Python · Rust / WASM · TypeScript / JavaScript · Vue · Linux / Windows

Beyond code

A Beef Wellington I learned to make in 2022

A Beef Wellington I learned to make in 2022. Someday I'd like to open a small Wuxi snack shop, serving bubble wontons, steamed buns, magnolia cakes, hot and sour soup, and Suzhou-style noodles.

Contact

If my repositories interest you beyond their star counts, I'd love the chance to talk “face to face” about the know-how behind them: the design tradeoffs, lessons learned and ideas behind the code.

I know many of you enjoy cool code and cool things as much as I do, and are using, studying and appreciating these projects. Come say hello, or let's build something different together.

For technical discussions, open-source collaboration or hardware modification inquiries, email orca.zhang@yahoo.com.

GitHub statistics

Most used languages GitHub statistics

GitHub Roast score card

Pinned Loading

  1. ez8-co/yapi ez8-co/yapi Public

    💉 全能进程注入器 [Yet Another Process Injector] that reduce differences between x64, wow64 and x86 processes.

    C++ 294 90

  2. ecache ecache Public

    🦄【轻量级本地内存缓存】🤏代码少于300行⌚️30s接入🚀高性能、极简设计、并发安全🌈支持LRU 和 LRU-2模式 🦖支持分布式一致性 [ecache] Extremely easy, ultra fast, concurrency-safe and support distributed consistency. Similar to bigcache, cachego, freecac…

    Go 265 29

  3. orcastor/cad2x-converter orcastor/cad2x-converter Public

    🗜️【CAD转换命令行工具】支持从DXF、DWG转换成DXF、PDF、PNG、SVG格式 [cad2x] Minimal stand-alone CLI tool convert CAD files (DXF / DWG) to other formats (DXF / PDF / PNG / SVG)

    C++ 228 36

  4. influxdb-cpp influxdb-cpp Public

    💜 C++ client for InfluxDB.

    C++ 182 84

  5. ez8-co/emock ez8-co/emock Public

    🐞 下一代C/C++跨平台mock库 (Next generation cross-platform mock library for C/C++)

    C++ 141 37

  6. borm borm Public

    【🔥今日热门】🏎️ 更好的ORM库 (Better ORM library that is simple, fast and self-mockable for Go)

    Go 119 19