bench | CLI to manage Multi-tenant deployments for Frappe apps | Continuous Deployment library

Parsing CSV files correctly while supporting several data types (eg. floating-point numbers, integers, strings) and possibly ill-formed input files is clearly not easy, and doing so efficiently is actually pretty hard. Moreover, decoding UTF-8 strings is also much slower than reading directly ASCII strings. This is the reasons why most CSV libraries are pretty slow. Not to mention wrapping library in Python could introduce pretty big overheads regarding the input types (especially string).

Hopefully, if you need to read a CSV file containing a square matrix of integers that is assumed to be correctly formed, then you can write a much faster specific code dedicated to your needs (which does not care about floating-point numbers, strings, UTF-8, header decoding, error handling, etc.).

That being said, any call to a basic CPython function tends to introduce a huge overhead. Even a simple call to open+read is relatively slow (the binary mode is significantly faster than the text mode but unfortunately not so fast). The trick is to use Numpy to load the whole binary file in RAM with np.fromfile. This function is extremely fast: it just read the whole file at once, put its binary content in a raw memory buffer and return a view on it. When the file is in the operating system cache or a high-throughput NVMe SSD storage device, it can load the file at the speed of several GiB/s.

One the file is loaded, you can decode it with Numba (or Cython) so the decoding can be nearly as fast as a native code. Note that Numba does not support well/efficiently strings/bytes. Hopefully, the function np.fromfile produces a contiguous byte array and Numba can compute it very quickly. You can know the size of the matrix by just reading the first line and counting the number of comma. Then you can fill the matrix very efficiently by decoding integer on-the-fly, packing them in a flatten matrix and just consider end-of-line characters as regular separators. Note that \r and \n can both appear in the file since the file is read in binary mode.

Here is the resulting implementation:

When you unroll the for loop, this is what it would look like:

You can use a dictionary to achieve the behavior you want

Here's a small code snippet -

The duplicate index values in the result can't be explained by the posted code.

In the code:

libstdc++'s std::string_view::find_first_of looks something like:

Try this for cart

I haven't tested your full setup with electrumx and the ethereum stuff present in your docker-compose file, but regarding your problem, the following steps worked properly, and I think it will do as well in your complete setup.

I ran with docker a bitcoin node based in the ulamlabs/bitcoind-custom-regtest:latest image you provided:

The difference is that if the generating function is already known in the benchmarked function, the generator is inlined and the involved Int-s are unboxed as well. If the generating function is the benchmark parameter, it cannot be inlined.

From the benchmarking perspective the second version is the correct one, since in normal usage we want the generating function to be inlined.

You need to keep Rst in the testbench high for a longer amount of time. The 1st posedge of Clk happens at time 200ns, and that is when you release the reset. You need to keep the reset asserted until after the 1st posedge of the clock in order to properly reset your RegFile, since it is a synchronous reset.

This testbench change allows RegFile to be reset to known values: