memstat | Fast memory statistics and better out-of-band GC

i want dump golang memstat

I am trying to measure the evolution of the number of heap-allocated objects before and after I call a function. I am forcing runtime.GC() and using runtime.ReadMemStats to measure the number of heap objects I have before and after.

The problem I have is that I sometimes see unexpected heap growth. And it is different after each run.

A simple example below, where I would always expect to see a zero heap-objects growth.

https://go.dev/play/p/FBWfXQHClaG

I'm trying to send kubernetes' logs with Filebeat and Logstash. I do have some deployment on the same namespace.

I tried the suggested configuration for filebeat.yml from elastic in this [link].(https://raw.githubusercontent.com/elastic/beats/7.x/deploy/kubernetes/filebeat-kubernetes.yaml)

So, this is my overall configuration:

filebeat.yml

I should probably explain why would I want that first.

I understand in Go substring(s[i:j]) and string.Split and some other string operations work in-place: the resulting substrings share the same memory block of the original string.

For example I read a large string, parse and get a few substrings from it, which will be kept in the long run in a server program, they will "hold" the large memory block from GC, wasting memory. I assume if I could make a copy of those substrings and keep those copies instead, GC could free that large string.

But I can't find a string copy mechanism in Go, I tried converting it to []byte then string again, memory usage dropped roughly 3/4 in my particular use case.

But this doesn't feel right: 1st, it introduces two copy operations. 2nd, since I never really write to that byte slice, I suspect it might got optimized out in release builds.

I can't imagine this hasn't been asked before, but my search doesn't yield any relevant results, or is there some better practices to do these kinds of things in Go?

BTW I tried to append an empty string(+"") to it, memory consumption doesn't drop, I assume it got optimized out even in test builds.

For measuring memory usage, I call runtime.GC() then runtime.ReadMemStats() and compare MemStats.Alloc, which seems pretty consistent in my tests.

nginx.yaml

From previous questions on the topic I've found two ways of getting the memory usage:

1. echo "::memstat" | mdb -k
2. a=$(/usr/sbin/prtconf | /usr/bin/awk '/Memory/ {print $3*1024}'); vmstat 1 2 | tail -1 | awk "{print 100 - (\$5 / $a) * 100}"

I opted for the first one since I'd rather rely on existing tooling rather than calculating it on my own. However, on some systems the output of "::memstat" is formatted slightly differently than others (Header starts with "Usage Type/Subtype" instead of "Page Summary" etc). This is a bit of an issue since I'm parsing it in code.

I could just fix the parser to deal with both variants, hoping that there are no other variants in hiding.

Could anyone suggest a better way that won't be as brittle?

So basically what I want to do is delete all keys from the database if its size is less than 1GB. I want to do it from a PHP script with follwoing code.

I am struggling with a php that has memory problem.

The script dies with the message: PHP Fatal error: Allowed memory size of 2097152 bytes exhausted (tried to allocate 3027408 bytes) in test.php on line 9

I created a small test script:

On my mac I am running nginx in a docker file and filebeat in a docker file.