scraping-service | REST API for scraping dynamic websites using Node | Scraper library

So I've finished my backend and frontend part of the project.

Now big aspect of my project is scraper function, which is implemented in the backend side of the code. Right now, I need to open VS code every day, and run a function which will trigger the scrapers. Now I've researched about, and Azure has a function apps which has a scheduled function.

Now what I want is: I want just to call a file inside my Azure repo. My backend and frontend are in the different repos, and I want to run file scraping-service.js inside scraping folder in order to scrape data and insert the data into the db.

Now normally I run pipeline with azure-service.yml which has its own configuration for running my project. Is any way to implement this function to run just scraping-service.js at certain time of the day?

Thanks!

So I've been working on a scraper project.

Now I've implemented many things but I've been stuck on this one thing.

So first let me explain workflow: Scrapers are called in scraping-service module, where I wait for the promise of the functions called to be resolved. Data is fetched in scrapers, and passed to the data_functions object where data is: merged, validated and inserted into DB.

Now here is the code:

scraping-service

I've been working on a scraper function.

Now about the app: there are two scrapers, that scrape data about apartments from two pages. Now when data is finished scraping it's been passed to mergeData function, where the goal is to merge all the arrays of objects, that are passed from scrapers - into one single array with all objects from the scrapers, and then to be passed to insert function, to be inserted into the DB.

Now here is one of the scrapers

I'm super new to node and coding in general so please forgive me. I am trying to use weld scraping service, but I am using npm. To run, it says to use a yarn command: yarn dev # development . Is there a way I can run the same command with npm? I tried npm install dev # development, but it threw an error for the #. I tried downloading the contents of the repository first and then running, and got the same result. I tried researching what # means in yarn, or in terminal, and the only thing I found is in a shell script it shows that you are the superuser?