memstats | Live memory profiling monitor for Go | Monitoring library

i want dump golang memstat

I am trying to measure the evolution of the number of heap-allocated objects before and after I call a function. I am forcing runtime.GC() and using runtime.ReadMemStats to measure the number of heap objects I have before and after.

The problem I have is that I sometimes see unexpected heap growth. And it is different after each run.

A simple example below, where I would always expect to see a zero heap-objects growth.

https://go.dev/play/p/FBWfXQHClaG

I'm trying to send kubernetes' logs with Filebeat and Logstash. I do have some deployment on the same namespace.

I tried the suggested configuration for filebeat.yml from elastic in this [link].(https://raw.githubusercontent.com/elastic/beats/7.x/deploy/kubernetes/filebeat-kubernetes.yaml)

So, this is my overall configuration:

filebeat.yml

I should probably explain why would I want that first.

I understand in Go substring(s[i:j]) and string.Split and some other string operations work in-place: the resulting substrings share the same memory block of the original string.

For example I read a large string, parse and get a few substrings from it, which will be kept in the long run in a server program, they will "hold" the large memory block from GC, wasting memory. I assume if I could make a copy of those substrings and keep those copies instead, GC could free that large string.

But I can't find a string copy mechanism in Go, I tried converting it to []byte then string again, memory usage dropped roughly 3/4 in my particular use case.

But this doesn't feel right: 1st, it introduces two copy operations. 2nd, since I never really write to that byte slice, I suspect it might got optimized out in release builds.

I can't imagine this hasn't been asked before, but my search doesn't yield any relevant results, or is there some better practices to do these kinds of things in Go?

BTW I tried to append an empty string(+"") to it, memory consumption doesn't drop, I assume it got optimized out even in test builds.

For measuring memory usage, I call runtime.GC() then runtime.ReadMemStats() and compare MemStats.Alloc, which seems pretty consistent in my tests.

nginx.yaml

So basically what I want to do is delete all keys from the database if its size is less than 1GB. I want to do it from a PHP script with follwoing code.

On my mac I am running nginx in a docker file and filebeat in a docker file.

Problem

In the code below, I have a couple of Go-routines and the one I'm facing issues with is the "calculateMemoryUsage()" Go-routine, where the avg memory usage is to be calculated every half a second. For some reason, the total keeps giving me weird values. Here's the Print Logs:

Context: I have been struggling this whole week to get this stack up and running: filebeat -> kafka -> logstash -> elasticsearch - kibana, each one in its own docker (you will find around 3 or 4 other questions mine here without answer resulted from different tentatives). I have decided to downsize the stack and then move block by block untill i reach a final docker-compose. Then I tried the simplest stack I can imagine to push forward the simplest log I can imagine and I am facing the issue mentioned above in my question.

Issue: I am trying to run straight from command line three docker containers: filebeat, elasticsearch and kibana. When I try to start kibana I get "No living connections". I am following carefully the answer provide in another stackoverflow question. Any clue why I am not able to connect from Kibana container to Elasticsearch container?

Here are all three docker commands: