memory-profiler | A memory profiler for Linux | Monitoring library

Which python module can I use to calculate the amount of memory spent executing a function? I'm using memory_profiler, but it shows the amount of memory spent by each line of the algorithm, in this case I want one that shows the total amount spent.

I have the following code from the example folder with the exception that I added @profile. I am just trying to make this example run because in my code which is more complex I have the same error and I would like to know how much memory is used on each line.

SYSTEM:

Python: 3.9

memory-profiler: 0.58

OS: Manjaro

CODE:

My memory usage on a Django DRF API project increases over time and RAM is getting filled once I reach 50+ API calls.

So far I tried

• loaded all models, class variable upfront
• used memory profiler, cleaned code as possible to reduce variable usage
• added garbage collection : gc.disable() at beginning and gc.enable() at end of code
• added ctypes malloc.trim() at end of code etc
• setting gunicorn max-requests limit ( this results in more model loading / response time at that moment)

Any suggestions on how to free up memory at the end of each request ?

So the issue I am having is that I want to measure the memory consumption and the time for a cell line in iPython on Google Colab, where the line is returning an object too. I am doing this with memory-profiler, so I can store all results of the benchmark in a file automatically.

I found out that memit -o gives me a MemitResult object, but the issue is that the object assignment gives an undefined m̀odel:

NameError: name 'model' is not defined

A short MWE:

I am trying to install a memory profiler (https://github.com/arnaud-lb/php-memory-profiler) on my EC2 instance (LAMP stack runnign php 7.2) as we are running in to memory allocation errors. During the installation, I get the following error:

• checking for judy lib... yes, shared
• checking for judy files in default path... not found
• configure: error: Please reinstall the judy distribution
• ERROR: `/tmp/pear/install/memprof/configure --with-php-config=/usr/bin/php-config' failed

For the life of me, I cannot figure out how to reinstall the judy library. I've tried both pecl (No releases available for package "pecl.php.net/libjudy") and yum (No package libjudy available.)

I've searched for ways to install it and have come up empty.

Anyone have any advice?

Thanks in advance.

p.s. I have also asked this question of the memory profile developer.

I have a web application which uses Google's Datastore, and has been running out of memory after enough requests.

I have narrowed this down to a Datastore query. A minimum PoC is provided below, a slightly longer version which includes memory measuring is on Github.