kaniko | Build Container Images In Kubernetes | Continuous Deployment library

The reason why the rules: condition on $CI_RUNNER_TAGS does not work is because rules: are evaluated at the time that the pipeline is created, and simply determines whether the job is included in the created pipeline. Because runners can only pick up jobs after they are created, this condition cannot be evaluated.

want my pipeline to pick up the runner based on availability

Unfortunately, the only mechanism by which you can control the runner used is the job tags. Pipelines can't pick their runners, partly because runners receive jobs through a "pull" mechanism -- all runners periodically poll GitLab for available jobs (matching their own tags, if applicable) -- runners choose jobs, not the other way around. So this wouldn't be possible.

What you might be able to do, however, is assign both runners to your project as shared runners, allowing them to pick up untagged jobs (or use a shared tag). You can modify the script logic based on the runner that does pick up the job.

For example, you might use the value of $CI_RUNNER_TAGS in your script: to determine what the job should do, based on which runner is running the job. Then you can have just one job instead of two.

Based on the comments, my suggestion is to add $HOME/go/bin to the path and use the default GOPATH.

Go mod depends on the bin directory inside the GOPATH. It installs new packages there. The go binary itself can actually reside somewhere else. If you follow these install instruction https://go.dev/doc/install, go itself will actually be in /usr/local/go but the GOPATH is still $HOME/go.

I would also recommend, not involving apt in this at all. This looks like trouble in the form of conflicts with different installations.

I finally solved it using the trick shown in this article.

Yes, it was a cop out. ;-) I ran aws cli in my windows PC.

This ended up being an issue with how the Kubernetes runner itself was configured inside of the runner configuration toml. The default container image we were using for our runners required a modification to the PATH environment variable so we were using the environment configuration setting to do this as outlined here. It seems that this PATH variable did not include the busybox shell defined in the kaniko debug image. We have since moved that PATH change inside our Docker image where it should've been in the first place and things are working as expected.

secondly: changes in job1/* should only trigger stage1's specific job in child pipeline. changes in job2/* however should trigger stage1 and then stage2 sequentially (depends) but with the variable defined in .gitlab-ci:

Just use the version that MassTransit depends upon, which is much earlier than v13. Upgrading past that without the proper assembly redirects is likely causing your issue.

Upgrading to Kaniko 1.7.0 might solve your problem: https://github.com/GoogleContainerTools/kaniko/pull/1735

It's not a perfect replacement but you can use one of the following instead:

I'm not sure what's the big picture so I'll add various options

For remote automation tasks from a flask web server we're using paramiko for its simplicity and quick setup, though you might prefer something like pyinfra for large projects or subprocess for small local tasks.

1. Paramiko - a bit more hands-on\manual than shutit, run commands over the ssh protocol.

example:

I'd recommend you do the shell stuff in a shell script, using the positional parameters as Benjamin suggests: