gcs | GURPS Character Sheet | Game Engine library

I have a react application (Node back end) running on Heroku (free option) connecting to a MongoDB running on Atlas (also free option). When I connect the application from my local machine to the Atlas DB all is fine and data retrieved (all 108 K records) in about 10 seconds, smaller amounts (4-500 records) of data in much less time. The same request from the application running on Heroku to the Atlas DB fails. The application running on Heroku can retrieve a small number of records (1-10) from the same collection of (108 K records), in less than a second. As soon as I try to retrieve a couple of hundred records the system fails. Below are the logs. I included the section of the logs that show a successful retrieval of 1 record and then failing on the request for about 450 records.

I have three questions:

1. What is the cause of the issue?
2. Is there a work around in the free option of Heroku?
3. If there is no work around in the free option, what Heroku pay level will I need to get to and what steps will I need to take to get this working? I will probably upgrade in the future but want to prove all is working before going in that direction.

Logs:

My program grabs ~70 pages of 1000 items from an API and bulk-inserts it into a SQLite database using Sequelize. After looping through a few times, the memory usage of node goes up to around 1.2GB and and then eventually crashes the program with this error: FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory. I've tried using delete for all of the big variables that I use for the response of the API call and stuff with variable = undefined and then global.gc(), however I still get huge amounts of memory usage and eventually it crashes. Would increasing the memory cap of Node.js help? Or would the memory usage of it just keep increasing until it hits the next cap?

Here's the full output of the error:

The executor for the project gitlab-runner is docker. I try to run docker-in-docker and I get the following Error from pipeline:

ERROR: Job failed (system failure): Error response from daemon: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: process_linux.go:508: setting cgroup config for procHooks process caused: resulting devices cgroup doesn't match target mode: unknown (docker.go:385:0s)

I followed this guide: https://www.digitalocean.com/community/tutorials/how-to-set-up-a-continuous-deployment-pipeline-with-gitlab-ci-cd-on-ubuntu-18-04 and after I read the docs of gitlab CI/CD and gitlab-runner, but I can't find out how to solve this problem.

This is currently my config.toml file:

I'm new to beam so the whole triggering stuff really confuse me. I have files that are uploaded regularly to gcs to a path that looks something like this: node-///files_parts and I need to write something that would trigger when all 8 parts of a file exist.

Their names are something like that: file_1_part_1, file_1_part_2, file_2_part_1, file_2_part_2 (there could be multiple files parts in the same dir but if its a problem I could ask for it to change).

Is there any way to create this trigger? and if not what do you suggest I could do instead?

Thanks!

I am trying to write a parquest schema for my json message that needs to be written back to a GCS bucket using apache_beam

My json is like below:

I am having problem with the memory when I try to start my react app with npm start. The error says

I have a nodejs script which was working fine on nodejs 12. I got a new macbook air on which I installed nodejs LTS 14. The scripts was not working as intended so I have downgraded it to nodejs 12 LTS. Now I'm getting an error for process out of memory. I have also tried using --max-oud-size to increase the memory allocation.

export NODE_OPTIONS=--max_old_space_size=4096

It didn't work. Following is the stack trace for the error

This is extended question to Can we send data from Google cloud storage to SFTP server using GCP Cloud function?

I want to use Google Colab free TPU with a custom dataset, that's why I need to upload it to GCS. I created bucket in GCS and uploaded dataset.

Also I read that there are two classes of operations with data in GCS: operation class A and operation class B [reference].

My questions are: does accessing dataset from GCS in Google Colab fall in one of these operation classes? What is average price you pay for using GCS for Colab TPU?