stream-to-array | readable stream 's data into a single array | Runtime Evironment library

I have a huge object that serves as a map with 2,7 million keys. I attempt to write the object to the file system, in order to persist it and not recompute it every time I need it. At another step, I need to read the object again. I need to have access to the entire object in memory, as it needs to serve as a map.
For writing, I convert the object to an array and stream it to the file system with the function below. The reason why I convert it to an array first is that it seems to be significantly faster to stream an array instead of an object. The writing part takes about a minute, which is fine. The output file has a size of 4,8GB.
The problem I'm facing is when attempting to read the file. For this, I create a read stream and parse the content. However, for some reason, I seem to be hitting some sort of memory limit. I used various different approaches for reading and parsing, and they all seem to work up until around 50% of the data is read (at this time the node process on my machine occupies 6GB memory, which is slightly below the limit I set). From then, the reading time significantly increases by factor 10, probably because node is close to using the maximum allocated memory limit (6144MB). It feels like I'm doing something wrong.
The main thing that I don't understand is why writing is not a problem, while reading is, even though during the write step, the entire array is kept in memory as well. I'm using node v8.11.3.

So to summarize:

• I have a large object I need to persist to the file system as an array using streams
• Writing works fine
• Reading works until around 50% of the data is read, then reading time increases significantly

How can I read the file more performantly?

I tried various libraries, such as stream-to-array, read-json-stream, JSONStream

example of an object to write:

I am using 'ssh2-sftp-client' to do this job. So basically it works fine if I copy the file to /tmp/ folder of Lambda first and then upload it to s3. I want to point the read stream to s3 without saving in the lambda /tmp/ folder.

So basically: