aho-corasick | Aho–Corasick string matching algorithm in C | Learning library

The following will do a threaded search for a word in a dictionary, one thread per word. You would need to read your 1000000 word file into memory to do it this way.

Reqwest lists OpenSSL as a requirement on Linux due to it using native-tls, which depends on openssl. You need to install the pkg-config and libssl-dev packages:

if I understood correctly, you are attempting to use std::unordered_map, std::string and std::set in device code. I'm not an expert on Intel-specific oneAPI SYCL extensions, but in pure SYCL 1.2.1 this is not allowed and I would be surprised if this works in DPC++.

The SYCL 1.2.1 spec does not really define how SYCL interacts with the standard library. While some implementations may be able to make some guarantees about certain well-defined portions of the standard library working in devie code as an extension (commonly e.g. std:: math functions), this is not universally guaranteed across SYCL implementations. Additionally supporting STL containers in device code (which is not required by the SYCL spec) I would imagine to be particularly difficult and I've never heard of a SYCL implementation supporting that. This is because containers typically employ mechanisms unsupported in SYCL device code because they require runtime support, for example throwing exceptions. Because on, say, a GPU there's no C++ runtime, such mechanisms cannot work in SYCL.

It is also important to understand that this is not really a SYCL-specific limitation, but a common restriction among heterogeneous programming models. Other heterogeneous programming models such as CUDA impose similar restrictions for similar reasons.

Another difficulty with containers in kernels is that STL data structures are usually not really designed for the massively parallel SIMT execution model on a SYCL device, making them prone to race conditions.

The final probem is the one you have already identified: You are copying pointers to host memory. Since you are on oneAPI DPC++, the easiest solution to work with pointer-based data structures is to use the Intel SYCL extension of unified shared memory (USM) which can be used to generate pointers that are valid both on host and device. There is also a USM allocator that could be passed to containers if they were supported in device code.

I will assume you got your results according to the example you linked.

Any way to reuse the memory occupied by the short life result string?

No.

Or just some other solution.

If you could change the code that uses the String objects generated by this method to accept a CharSequence instead. Then you could pass it the StringBuilder instance in builder, and avoid the toString() call.

The problem is that you wouldn't be able to prevent something from casting CharSequence to StringBuilder and mutating it. (But if the code is not security critical, you could ignore that. It would be hard to do that by accident, especially if you use the CharSequence interface type when passing the StringBuilder around.)

The other problem is that the caller will actually be getting the same object each the time with different state. It wouldn't be able to keep the state ... unless it called toString() on it.

But you may be worrying unnecessarily about performance. The GC is relatively good at dealing with short-lived objects. Assuming that an object is unreachable on the first GC cycle after it is created, it won't ever be marked or copied, and the cost of deleting it will be zero. To a first approximation, it is the reachable objects in the "from" space that will cost you.

I would first do some profiling and GC monitoring. Only go down the path of changing your code as above if there is clear evidence that the short lived strings are causing a performance problem.

(My intuition is that 400 short term strings per second should not be a problem, assuming that 1) they are not huge and 2) you picked a GC that is suitable for your use-case.)