CMD-S | A simple save plugin for WordPress | Content Management System library

This issue appears to no longer be happening. I upgraded to Raspbian bullseye, and healthchecks have been running for a week straight, without issues.

We've had some problems with Airflow in Docker so we're trying to move away from it at the moment.

Some suggestions:

1. Set the version of openpyxl to a specific version in requirements.txt
2. Add openpyxl twice to requirements.txt
3. Create a requirements.in file with your main components, and create a requirements.txt off that using pip-compile. This will add subcomponents too
4. Try specifying a python version as well

Hopefully one of these steps will help.

Try this :

LETS GOOOOOOOOOO!
PAN COMIDO!
DU GATEAU!
Finally got it to work :). So the main issue was the fact that I didn't have all the required packages. So I tried doing just pip install configparser in the container and this actually helped for one of the DAGs I had to run. However this didn't seem sustainable nor practical so I decided to just go ahead with the Dockerfile method in effect extending the image. I believe this was the way they called it. So here's my Dockerfile \

Sounds like you are heading in the right direction (with one general comment at the end though).

1. Yes, you need to push image to container registry and refer to it via public (or private if you authenticate) tag. The tag in this case is usally the registry/name:tag. For example you can see one of the CI images of Airlfow here: https://github.com/apache/airflow/pkgs/container/airflow%2Fmain%2Fci%2Fpython3.9 - the purpose is a bit different (we use it for our CI builds) but the mechanism is the same: you build it locally, tag with the "registry/image:tag" docker build . --tag registry/image:tag and run docker push registry/image:tag. Then whenever you refer to it from your docker compose, via registry/image:tag, docker compose/swarm will pull the right image. Just make sure you make unique TAGs when you build your images to know which image you push (and account for future images).

2. Env files should be fine and they will distribute across the instances, but locally mounted volumes will not. You either need to have some shared filesystem (like NFS, maybe EFS if you use AWS) where the DAGs are stored, or use some other synchronization method to distribute the DAGs. It can be for example git-sync - which has very nice properties especially if you use Git to store the DAG files, or baking DAGs into the image (which requires to re-push images when they change). You can see different options explained in our Helm Chart https://airflow.apache.org/docs/helm-chart/stable/manage-dags-files.html

3. You cannot use localhost you need to set it to a specific host and make sure your broker URL is reachable from all instances. This can be done either by assining specific IP address/DNS name to your 'broker' instance and opening up the right ports in firewalls (make sure you control where you can reach thsoe ports from) and maybe even employing some load-balancing.

I do not know DockerSwarm well enough how difficult or easy it is to set it all up, but nonestly, that's kind of a lot of work - it seems - to do it all manually.

I would strongly, really strongly encourage you to use Kubernetes and the Helm Chart which Airlfow community develops: https://airflow.apache.org/docs/helm-chart/stable/index.html . There a lot of issues and necessary configurations either solved in the K8S (scaling, shared filesystems - PVs, networking and connectiviy, resource management etc. etc.) or by our Helm (Git-Sync side containers, broker configuration etc.)

If you add an environment variable named AIRFLOW_VAR_CONFIG_BUCKET to the list under environment:, it should be accessible by Airflow. Sounds like you're doing that correctly.

Two things to note:

• Variables (& connections) set via environment variables are not visible in the Airflow UI. You can test if they exist by executing Variable.get("config_bucket") in code.
• The Airflow scheduler/worker (depending on Airflow executor) require access to the variable while running a task. Adding a variable to the webserver is not required.

At a push you could enable listener with MQTT over Websockets as the protocol and then use a basic curl get request to check it the broker is up.

e.g. add this to the mosquitto.conf

For this issue: " Failed to fetch log file from worker. [Errno -3] Temporary failure in name resolution"

Looks like the worker's hostname is not being correctly resolved. The web program of the master needs to go to the worker to fetch the log and display it on the front-end page. This process is to find the host name of the worker. Obviously, the host name cannot be found, Therefore, add the host name to IP mapping on the master's vim /etc/hosts

1. You need to have the image that's going to be used in all your containers except message broker, meta database and worker monitor. Following is the Dockerfile.

2.If using LocalExecutor, the scheduler and the webserver must be on the same host.

Docker file:

Based on the update to your question, you're trying to run the mysqladmin ping command inside the container. mysqladmin is attempting to connect as the root user, but authenticating to your database server requires a password.

You can provide a password to mysqladmin by:

• Using the -p command line option
• Using the MYSQL_PWD environment variable
• Creating a credentials file

If we move the root password out of your image, and instead set it at runtime, we can write your docker-compose.yml file like this: