Repository navigation
test failures on s390x (big-endian) #2983
Description
When running version 95 tests on a big-endian machine like s390x, I get the following test error:
./check.py --binaryen-bin s390x-redhat-linux-gnu/bin
warning: no mozjs found (did not check native wasm support nor asm.js validation)
warning: no interpreter provided (did not test spec interpreter validation)
...
.. fuzz-exec_all-features.wast
executing: /builddir/build/BUILD/binaryen-version_95/s390x-redhat-linux-gnu/bin/wasm-opt --fuzz-exec --all-features split.wast --print
executing: /builddir/build/BUILD/binaryen-version_95/s390x-redhat-linux-gnu/bin/wasm-opt --fuzz-exec --all-features split.wast --print --debug
executing: /builddir/build/BUILD/binaryen-version_95/s390x-redhat-linux-gnu/bin/wasm-opt --fuzz-exec --all-features split.wast --print
executing: /builddir/build/BUILD/binaryen-version_95/s390x-redhat-linux-gnu/bin/wasm-opt --fuzz-exec --all-features split.wast --print
executing: /builddir/build/BUILD/binaryen-version_95/s390x-redhat-linux-gnu/bin/wasm-opt --fuzz-exec --all-features split.wast --print --debug
executing: /builddir/build/BUILD/binaryen-version_95/s390x-redhat-linux-gnu/bin/wasm-opt --fuzz-exec --all-features split.wast --print
executing: /builddir/build/BUILD/binaryen-version_95/s390x-redhat-linux-gnu/bin/wasm-opt --fuzz-exec --all-features split.wast --print
executing: /builddir/build/BUILD/binaryen-version_95/s390x-redhat-linux-gnu/bin/wasm-opt --fuzz-exec --all-features split.wast --print --debug
executing: /builddir/build/BUILD/binaryen-version_95/s390x-redhat-linux-gnu/bin/wasm-opt --fuzz-exec --all-features split.wast --print
executing: /builddir/build/BUILD/binaryen-version_95/s390x-redhat-linux-gnu/bin/wasm-opt --fuzz-exec --all-features split.wast --print
executing: /builddir/build/BUILD/binaryen-version_95/s390x-redhat-linux-gnu/bin/wasm-opt --fuzz-exec --all-features split.wast --print --debug
executing: /builddir/build/BUILD/binaryen-version_95/s390x-redhat-linux-gnu/bin/wasm-opt --fuzz-exec --all-features split.wast --print
executing: /builddir/build/BUILD/binaryen-version_95/s390x-redhat-linux-gnu/bin/wasm-opt --fuzz-exec --all-features split.wast --print
executing: /builddir/build/BUILD/binaryen-version_95/s390x-redhat-linux-gnu/bin/wasm-opt --fuzz-exec --all-features split.wast --print --debug
executing: /builddir/build/BUILD/binaryen-version_95/s390x-redhat-linux-gnu/bin/wasm-opt --fuzz-exec --all-features split.wast --print
incorrect output, diff:
--- /builddir/build/BUILD/binaryen-version_95/test/passes/fuzz-exec_all-features.txt
+++ actual
@@ -66,7 +66,7 @@
[fuzz-exec] calling unaligned_notify
[trap unaligned atomic operation]
[fuzz-exec] calling wrap_cmpxchg
-[LoggingExternalInterface logging 42]
+[LoggingExternalInterface logging 704643072]
[fuzz-exec] calling oob_notify
[trap final > memory: 18446744073709551512 > 65514]
(module
@@ -135,7 +135,7 @@
[fuzz-exec] calling unaligned_notify
[trap unaligned atomic operation]
[fuzz-exec] calling wrap_cmpxchg
-[LoggingExternalInterface logging 42]
+[LoggingExternalInterface logging 704643072]
[fuzz-exec] calling oob_notify
[trap final > memory: 18446744073709551512 > 65514]
[fuzz-exec] comparing aligned_for_size
Traceback (most recent call last):
File "/builddir/build/BUILD/binaryen-version_95/./check.py", line 403, in <module>
sys.exit(main())
File "/builddir/build/BUILD/binaryen-version_95/./check.py", line 386, in main
TEST_SUITES[test]()
File "/builddir/build/BUILD/binaryen-version_95/scripts/test/wasm_opt.py", line 75, in test_wasm_opt
shared.fail_if_not_identical_to_file(actual, expected_file)
File "/builddir/build/BUILD/binaryen-version_95/scripts/test/shared.py", line 348, in fail_if_not_identical_to_file
fail_if_not_identical(actual, f.read(), fromfile=expected_file)
File "/builddir/build/BUILD/binaryen-version_95/scripts/test/shared.py", line 337, in fail_if_not_identical
fail(actual, expected, fromfile=fromfile)
File "/builddir/build/BUILD/binaryen-version_95/scripts/test/shared.py", line 332, in fail
fail_with_error("incorrect output, diff:\n\n%s" % diff_str)
File "/builddir/build/BUILD/binaryen-version_95/scripts/test/shared.py", line 320, in fail_with_error
raise Exception(msg)
Exception: incorrect output, diff:
--- /builddir/build/BUILD/binaryen-version_95/test/passes/fuzz-exec_all-features.txt
+++ actual
@@ -66,7 +66,7 @@
[fuzz-exec] calling unaligned_notify
[trap unaligned atomic operation]
[fuzz-exec] calling wrap_cmpxchg
-[LoggingExternalInterface logging 42]
+[LoggingExternalInterface logging 704643072]
[fuzz-exec] calling oob_notify
[trap final > memory: 18446744073709551512 > 65514]
(module
@@ -135,7 +135,7 @@
[fuzz-exec] calling unaligned_notify
[trap unaligned atomic operation]
[fuzz-exec] calling wrap_cmpxchg
-[LoggingExternalInterface logging 42]
+[LoggingExternalInterface logging 704643072]
[fuzz-exec] calling oob_notify
[trap final > memory: 18446744073709551512 > 65514]
[fuzz-exec] comparing aligned_for_size
Activity
rathann commentedon Sep 15, 2021
Still reproducible with 102 release.
kripken commentedon Sep 15, 2021
Is there a way to debug something like this without big-endian hardware?
If not, at least reducing it on your side would be helpful - a tiny testcase could make this easy to figure out. The fastest way to get a reduced testcase might be to run
./scripts/fuzz_opt.py, and if it fails quickly, to follow the instructions it gives for reducing what it found.rathann commentedon Sep 17, 2021
@kripken You can reach out to one of Fedora s390x arch guys, Dan Horák to get SSH access to a s390x machine.
You can also emulate s390x using QEMU.
I'll follow your instructions to reduce the testcase, too. Stay tuned.
rathann commentedon Sep 20, 2021
Details
original.wasm.gz
t.wasm.gz
w.wasm.gz
I hope this is what you need.
kripken commentedon Sep 20, 2021
Size 8 is the smallest possible wasm file, so it's failing even to load that..?
What error do you get when you do
wasm-opt w.wasmon that last wasm file?rathann commentedon Sep 21, 2021
No error:
kripken commentedon Sep 21, 2021
Oh, then something went wrong with the reduction. When the reducer ran such a command, it failed for it - perhaps you are in a different directory or otherwise it can't find
wasm-opt. Running the fuzzer script from the binaryen directory is what I normally do (./scripts/fuzz_opt.py) and it's possible you need to set up the PATH otherwise, or pass--binariesto the reduction script so it finds them.You can run this to see:
That runs the fuzzer script with the seed that failed, and gives it the
w.wasmfile that the reduction arrived it. If that succeeds, try(without the file at the end, it will run the entire testcase).
kapouer commentedon Feb 12, 2025
Note that practically all big-endian architectures have the same test failure (hppa, powerpc, ppc64, sparc64, s390x).
Is there something preventing binaryen to run on a big-endian host ?
kripken commentedon Feb 12, 2025
I can't think of any big-endian specific issue. We don't do any low-level bit manipulation of that sort, in general. But perhaps we have undefined behavior somewhere, that's my best guess.
rathann commentedon Feb 25, 2025
Well, this is still reproducible with 122:
Note that treating the little-endian bit-representation of
42as big-endian gives us exactly704643072. See in Python:tlively commentedon Feb 25, 2025
Looks like
ShellExternalInterface::Memory::{set,get}do not behave correctly on big-endian architectures. They should be reversing the bytes before storing and after loading values.sertonix commentedon Jan 23, 2026
If that is the case I would suspect issues in code using
CtorEvalExternalInterface::{doStore,doLoad}as well.In general a lot of
memcpyandreinterpret_castdo not take endianess into account. Andv128values probably needs special care by swapping endianess when converting to and from normal integer types (haven't found the code if that is done anywhere).Edit: also
static_castIs there any opinion on ether using something with
std::byteswapon big-endian systems or using endian independent code like(b[1] << 8) | b[0]? (Bit shifting seems to already be used in some places)kripken commentedon Jan 26, 2026
If we need to fix places that way, that sounds good to do, of course. Where specifically were you looking?
sertonix commentedon Jan 27, 2026
I was looking for places which converts pointers to integers to pointers to integers of different width (like
uint8_t[2] a = {1, 0}; uint16_t *v = &a;). Unfortunately that can come in many different ways so there is no specific way to look for them but memcpy, casts, read and write operations are good places to start.12 remaining items
Add and use readLE/writeLE helpers
[Big Endian] Add and use readLE/writeLE helpers (#8470)
sertonix commentedon May 21, 2026
Is it needed for performance that in
src/wasm/literal.cppthe union members are accessed directly (through.i32) instead of the wrappers that have an assert (like.geti32())? Still need to finish running tests but I think there are a few places where the accessed union member does not match the type and the assert would help with noticing such bugs more easily.Fix i64x2 shift on big-endian
Fix i64x2 shift on big-endian
kripken commentedon May 21, 2026
Using the wrappers is probably fine, though it would be worth measuring first.
Fix i64x2 shift on big-endian (#8748)
sertonix commentedon May 21, 2026
5 test failure left and it's not getting easier 😅
sertonix commentedon May 21, 2026
I think the assumption that
(i32 & 1)is the low bit of ashared_ptris broken. On s390x it's roughly in the middle of the pointer (and likely set to 1).binaryen/src/literal.h
Lines 53 to 54 in cfc3d50
binaryen/src/literal.h
Lines 311 to 314 in cfc3d50
Fixing that seems to require a larger change. The only portable way I can think of to have multiple union members be accessible without UB would be to use a struct like the following. The good news is that the union is always 128-bits so there shouldn't be size increase (on >= 64-bit systems).
Edit: Oh, shared_ptr is already 128-bit
(just an idea, not functional)
Is there any other idea and/or opinion on how the issue can be fixed?
sertonix commentedon May 21, 2026
My idea doesn't look good.
Moving the externref payloads to GCData like before c3bc803 seems better.
CC @tlively
tlively commentedon May 22, 2026
I guess one option is to change which bit distinguishes externref payloads depending on the platform so that it always happens to line up with the low bit of the shared_ptr. This should be possible no matter where that low bit ends up being because we can use the
v128field if we have to. But that's really gross and probably brittle.I agree that moving externref payloads to the GCData seems better. If we have an externref Literal, we can look at its GCData and if it contains a single i32 Literal, we know it is a boxed externref payload. Otherwise it should contain some reference-typed Literal and we know it is an externalized reference.
sertonix commentedon May 22, 2026
You mean something like this? The concept works on s390x but the patch is broken somewhere since it triggers an assert on all arches. Help with doing this right would be appreciated
sertonix commentedon May 22, 2026
I think I tracked down the last test failures. In javascript binaryen uses typed arrays such as
Int32Arraywhich depend on the host endianess. For exactly that reason MDN recommends the usage ofDataViewwhen dealing with WASM memory: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/DataView#endiannessI verified the concept with
test/unit/input/asyncify.js. UsingDataVieweverywhere seems to require large changes (seegit grep -E '\b(((Big)?(Ui|I)nt|Float)(16|32|64)Array|HEAP(U|F|)(8|16|32|64))\b').Edit:
Atomics.addseems to use host endianess without an option to change that. It may need to be replaced with something likeAtomics.compareExchange.Patch for test_asyncify_js
tlively commentedon May 23, 2026
I uploaded #8765 to fix the problem with externref representation.
Fix asyncify.js test on big-endian