self-hosted | Deploy Revolt using Docker | Continuous Deployment library

Personally I have found that deploying my background services as ASP.NET Core applications works well, because we can:

1. Deploy /health endpoints using Healthchecks
2. Use IHostedService for running the background services

Hub allows you to load data from anywhere. Hub works locally, on Google Cloud, MinIO, AWS as well as Activeloop storage (no servers needed!). So, it allows you to load data and directly stream datasets from cloud storage to ML workflows.

You can find more information about storage authentication in the Hub docs.

Then, Hub allows you to stream data to PyTorch or TensorFlow with simple dataset integrations as if the data were local since you can connect Hub datasets to ML frameworks.

One builtin method would be to use system hooks. For example, you can create a hook that responds to user_create events and adds the user to the project.

Another way may be just to run a scheduled CI pipeline that scripts this or similar automation (e.g. cron job on the server or whatever).

You can use the users list API to enumerate all current users in your GitLab instance (requires admin privileges). You can also use the project membership API to enumerate all members of the project. You can compare the two results to find any users that need to be added.

Pseudocode:

Go to the self hosted runner directory, list folders, you should see the bin and externals directory with the version suffix bin.2.288.1, externals.2.288.1 - most current should have symlink with full directory. Also you can check version of each component listener, plugin-host,worker in /bin directory json files. There's no need to force updating self hosted runner - it should update automatically to the latest. One scenario in which you will need to update the runners manually is inside docker container - you can use this script:

https://github.com/GarnerCorp/github-actions-runner/blob/cb604618a600bd5174f3dbc8d51d15eee6585c68/scripts/install-runner

to test locally you need to make use of mocks which can simulate having an oracle network. Because you're working locally, a Chainlink node doesn't know about your local blockchain, so you can't actually do proper VRF requests. Note you can try deploy a local Chainlink node and a local blockchain and have them talk, but it isn't fully supported yet so you may get mixed results. Anyway, as per the hardhat starter kit that you linked, you can set the defaultNetwork to be 'hardhat' in the hardhat.config.js file, then when you deploy and run the integration tests (yarn test-integration), it will use mocks to mock up the VRF node, and to test the requesting of a random number. See the test here, and the mock contracts and linktoken get deployed here

ok, I resolved by converting all my PowerShell scripts to bash. Now I can script within a decorator that can run in any agent from any agent pool. The only setback is that different OS can have different bash versions and a mismatch of the code functions, so I still needed to make kind of a switch to determine what is the os and run a particular bash script.

I suggest three solutions

1. Create another pipeline to run the clean up tasks on agents - you can also add demand for specific agents (See here https://docs.microsoft.com/en-us/azure/devops/pipelines/process/demands?view=azure-devops&tabs=yaml) by Agent.Name -equals [Your Agent Name]. You can set frequency to minutes, hours, as you like using cron pattern. As while this pipeline will be running and taking up the agent, the agent being cleaned will not be available for other jobs. Do note that you can trigger this pipeline from another pipeline, but if both are using the same agents - they can just get deadlocked.

2. Create a template containing scripts tasks having all clean up logic and use it at the end of every job (which you have discounted).

3. Rather than using static VM's for agent hosting, use Azure scaleset for Self hosted agents - everytime agents are scaled down they are gone and when scaled up they start fresh. This saves a lot of money in terms of sitting agents not doing anything when no one is working. We use this option and went away from static agents. We have also used packer to create the VM image/vhd overnight to update it with patches, softwares required, and docker images cached. ref: https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/scale-set-agents?view=azure-devops

However, since I do not own the repositories of which the pull request is incoming

That means you would not be able to modify anything in that repository though: no commit status (2012) or even check API (2017).

You would need to be a collaborator on that repository to do anything on it.

After countless hours working on this, it seems that ultimately the issue was that our internal Web Application Firewall was blocking some part of the transfer of artefacts to the server, or the response back from it. With the WAF reconfigured not to block traffic from the machine running the GitLab Runner, the artefacts are successfully uploaded and the job succeeds.

This would have been significantly easier to diagnose if the logging from GitLab was better. As per my comment on this issue, it should be possible to see the content of the response from the GitLab server after uploading artefacts, even when the response code is 200.

What's strange - and made diagnosing the issue even harder - is that when I worked through the issue with the admin of our GitLab instance, digging through logs and running it in debug mode, the artefact upload process was uploading something successfully. We could see, for example, the GitLab Runner's log had been uploaded to the server. Clearly the WAF's blocking was selective and didn't block everything in both directions.