ping-service | Building ping service with the gRPC framework

I'm learning unit tests, and I've covered pretty much the basics.

Now I'm trying to test my scraper. My initial function involves multiple scrapers, but i wanted to check if each scraper works properly. The output of the test (to test be successful) should be if the return has an [{},{},...,{}] - array of objects. I really cant know what is inside of the objects(data is always different), but it's always a array of objects.

Now what my question is: How to properly do this? I've tried in multiple ways, and my test always fails. Closes that I've been is this solution:

So I've finished my backend and frontend part of the project.

Now big aspect of my project is scraper function, which is implemented in the backend side of the code. Right now, I need to open VS code every day, and run a function which will trigger the scrapers. Now I've researched about, and Azure has a function apps which has a scheduled function.

Now what I want is: I want just to call a file inside my Azure repo. My backend and frontend are in the different repos, and I want to run file scraping-service.js inside scraping folder in order to scrape data and insert the data into the db.

Now normally I run pipeline with azure-service.yml which has its own configuration for running my project. Is any way to implement this function to run just scraping-service.js at certain time of the day?

Thanks!

So I've been working on a scraper project.

Now I've implemented many things but I've been stuck on this one thing.

So first let me explain workflow: Scrapers are called in scraping-service module, where I wait for the promise of the functions called to be resolved. Data is fetched in scrapers, and passed to the data_functions object where data is: merged, validated and inserted into DB.

Now here is the code:

scraping-service

I've been working on a scraper function.

Now about the app: there are two scrapers, that scrape data about apartments from two pages. Now when data is finished scraping it's been passed to mergeData function, where the goal is to merge all the arrays of objects, that are passed from scrapers - into one single array with all objects from the scrapers, and then to be passed to insert function, to be inserted into the DB.

Now here is one of the scrapers

I am trying to run a springboot microservice docker image. It fetches the DB connection properties from a Config server. However it is unable to connect to the container.

Error :

Summary

in our Kubernetes-cluster we introduced a HPA whit memory and cpu limits. Right now we do not understand why we have 2 replicas of one service.

The service in question uses 57% / 85% Memory and has 2 replicas instead of one. We think that it is because when you sum up the memory of both pods it is more than 85% but it would not be if there would be only one pod. So is this preventing it from scale down? What can we do here?

We also observe a peak in memory usage when we deploy a service. We are using spring-boot services in aks (azure) and think it may scales up there and never down. Do we miss something or has anyone a suggestion?

Helm

hpa:

I'm super new to node and coding in general so please forgive me. I am trying to use weld scraping service, but I am using npm. To run, it says to use a yarn command: yarn dev # development . Is there a way I can run the same command with npm? I tried npm install dev # development, but it threw an error for the #. I tried downloading the contents of the repository first and then running, and got the same result. I tried researching what # means in yarn, or in terminal, and the only thing I found is in a shell script it shows that you are the superuser?

I am having an issue with my dockerized webapp. Essentially I have a docker-compose yaml file and when the file is ran, everything seems to boot up smoothly. I can see that the container is running using docker ps. I can see that it is listening to the correct ports, however when I try to connect via my browser, it refuses to connect.

My yaml file

I am trying to fix a calculator for excel which extracts the most recent international surcharge rate for TNT from this website: https://www.tnt.com/express/en_nz/site/shipping-services/fuel-surcharges-apac.html As you can see the 15.75% is the latest surcharge rate.

The screenshot I have uploaded is the particular p tag where I want to extract the content inside "15.75%". Webpage screenshot

I have the following VBA code to test I am getting the correct elements: