cpp-httplib | A C++ header-only HTTP/HTTPS server and client library | HTTP library

Thanks! I was able to send HTTPS responses using HttpClient. I also copy-pasted "json.hpp" into my project from a header-only json library, since Windows.Data.Json wasn't being recognized by VS for some reason.

I successfully was able to get a HTTPS response and parse the data !

Try this. The definition must be before including the library.

Many modern allocators, including the one in glibc 2.17 that you are using, use multiple arenas (a structure which tracks free memory regions) in order to avoid contention between threads which want to allocate at the same time.

Memory freed back to one arena isn't available to be allocated by another arena (unless some type of cross-arena transfer is triggered).

By default, glibc will allocate new arenas every time a new thread makes an allocation, until a predefined limit is hit (which defaults to 8 * number of CPUs) as you can see by examining the code.

One consequence of this is that memory allocated then freed on a thread may not be available to other threads since they are using separate areas, even if that thread isn't doing any useful work.

You can try setting the glibc malloc tunable glibc.malloc.arena_max to 1 in order to force all threads to the same arena and see if it changes the behavior you were observing.

Note that this has everything to do with the userspace allocator (in libc) and nothing to do with the OS allocation of memory: the OS is never informed that the memory has been freed. Even if you force a single arena, it doesn't mean that the userspace allocator will decide to inform the OS: it may simply keep the memory around to satisfy a future request (there are tunables to adjust this behavior also).

However, in your test using a single arena should be enough to prevent the constantly increasing memory footprint since the memory is freed before the next thread starts, and so we expect it to be reused by the next task, which starts on a different thread.

Finally, it is worth pointing out that what happens is highly dependent on exactly how threads are notified by the condition variable: presumably Linux uses a FIFO behavior, where the most recently queued (waiting) thread will be the last to be notified. This causes you to cycle through all the threads as you add tasks, causing many arenas to be created. A more efficient pattern (for a variety of reasons) is a LIFO policy: use the most recently enqueued thread for the next job. This would cause the same thread to be repeatedly reused in your test and "solve" the problem.

Final note: many allocators, but not the on in the older version of glibc that you are using, also implement a per-thread cache which allows the allocation fast path to proceed without any atomic operations. This can produce a similar effect to the use of multiple arenas, and which keeps scaling with the number of threads.

If you want to deal with Django through Json objects, try to send objects as Json and return objects as Json.

Following the README on cpp-httplib github:

• add #define CPPHTTPLIB_OPENSSL_SUPPORT,
• link libssl and libcrypto.

So the code would be so:

It's gcc's known particular feature, its std::thread implementation is built upon pthreads so it requires specifying -pthread to correctly link programs with threads.