WasmFiddle | repository contains the WasmFiddle website source code | Binary Executable Format library

Ah, the joys of microbenchmarking :-)

V8 has two compilers for Wasm: a non-optimizing baseline compiler that produces code really fast, and an optimizing compiler that takes quite a bit longer to produce code, but that code is typically about twice as fast. When a module is loaded, current versions first compile all functions with the baseline compiler. Once that's done, execution can start, and optimized compilation jobs are scheduled to run in the background. When an optimized compilation job is complete, the respective function's code is swapped, and the next invocation of the function will use it. (The details here will very likely change in the future, but the general principle will remain.) That way, typical applications get both good startup latency, and good peak performance.

But, as with any heuristic or strategy, you can craft a case where it gets it wrong...

In your benchmark, each function is called only once. In the fast cases, optimizing kernel finishes before init returns. In the slow cases, kernel is called before its optimized compilation job is done, so its baseline version runs. Apparently when embedding V8 directly, you reliably get the latter scenario, whereas when running via WasmFiddle in Chrome, you get the former most of the time, but not always.

I can't explain why your custom embedding runs are even slower than the slow case in Chrome; I'm not seeing that on my machine (OTOH, in Chrome, I'm seeing an even bigger delta: about 1100ms for a fast run and 4400ms for a slow run); however I used the d8 shell instead of compiling my own embedding. One thing that's different is that when measuring with time on the command line, you include process startup and initialization, which the Date.now() calls around main() don't include. But that should only account for 10-50 milliseconds or so, not for a 3.6s → 5.0s difference.

While this situation might look quite unfortunate for your microbenchmark, it is generally working as intended, i.e. not a bug, and hence unlikely to change on V8's side. There are several things you can do to make the benchmark more reflective of real-world behavior (assuming this one doesn't exactly represent some real application you have):

• execute functions multiple times; you'll see that the first run will be slower (or, depending on function size and module size and number of available CPU cores and scheduling luck, the first few runs)

• wait a bit before calling the hottest functions, e.g. by doing