celery | Go client library for calling Celery tasks | Pub Sub library

i use celery in django ,
i add a task to my project and get error, but before add this task my project is work good.

I'm running the code below as part of a Celery task.

I'm developing an python-django app running in docker containers (django, celery, postgres, redist...etc). It runs on Windows 10 with WSL2-Debian & Docker Desktop.

During my work I need to observe the consoles of all those containers, so I can monitor apps behavior, like when you run docker-compose up so you got all of them live.

When you click on the container within windowed Docker Desktop app you can see the container's console output, but not actual - it looks like it works until some point of time and there are no updates of the consoles output. I remember it was working live just prior to a two or three Docker Desktop updates, and I'm sure it was real time there, but not now.

Did I change a setting or Docker Desktop was bugged?

PS. When I start my containers with docker-compose up (without -d) I can observe live logs on my shell console, but not in Docker Desktop anymore.

Any help how to restore Docker Desktop live console view?

I have a nested collection that I want to transform, pulling some keys "up a level" and discarding some other keys.

Every item in the collection has an allergens property.

How can I properly kill celery tasks running on containers inside a kubernetes environment? The structure of the whole application (all written in Python) is as follows:

1. A SDK that makes requests to our API;

2. A Kubernetes structure with one pod running the API and other pods running celery containers to deal with some long-running tasks that can be triggered by the API. These celery containers autoscale.

Suppose we call a SDK method that in turn makes a request to the API that triggers a task to be run on a celery container. What would be the correct/graceful way to kill this task if need be? I am aware that celery tasks have a revoke() method, but I tried using this approach and it did not work, even using terminate=True and signal=signal.SIGKILL (maybe this has something to do with the fact that I am using Azure Service Bus as a broker?)

Perhaps a mapping between a celery task and its corresponding container name would help, but I could not find a way to get this information as well.

Any help and/or ideas would be deeply appreciated.

I have a dockerized Django project and everything works fine because Celery keeps displaying runserver logs instead of celery logs.

Here's my docker-compose.yml:

I am trying to introduce dynamic workflows into my landscape that involves multiple steps of different model inference where the output from one model gets fed into another model.Currently we have few Celery workers spread across hosts to manage the inference chain. As the complexity increase, we are attempting to build workflows on the fly. For that purpose, I got a dynamic DAG setup with Celeryexecutor working. Now, is there a way I can retain the current Celery setup and route airflow driven tasks to the same workers? I do understand that the setup in these workers should have access to the DAG folders and environment same as the airflow server. I want to know how the celery worker need to be started in these servers so that airflow can route the same tasks that used to be done by the manual workflow from a python application. If I start the workers using command "airflow celery worker", I cannot access my application tasks. If I start celery the way it is currently ie "celery -A proj", airflow has nothing to do with it. Looking for ideas to make it work.

I am trying to use Celery to create periodic tasks in my application. However, I cannot see the outputs of the periodic task that I wrote.

The backend is on a Windows-based redis-server. The server is up and running.

project/celery.py

I'm having different python programs doing long polling at different machines, and am thinking of a queuing based mechanism to manage the load and provide an async job functionality.

These programs are standalone, and aren't part of any framework.

I'm primarily thinking about Celery due to its ability for multi-processing and sharing tasks across multiple celery workers. Is celery a good choice here, or am I better off simply using an event based system with RabbitMQ directly?