celery-cloudwatch | Uploads results of Celery tasks to AWS CloudWatch | Cloud Storage library
kandi X-RAY | celery-cloudwatch Summary
kandi X-RAY | celery-cloudwatch Summary
Uploads results of Celery tasks to AWS CloudWatch.
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
- Monitor Celery events
- Upload a log event
- Get the sequence token associated with a log stream
- Calculate the maximum number of bytes allocated to the virtual machine
- Gets the timestamp in milliseconds
- Run the monitor
- Entry point for celery
celery-cloudwatch Key Features
celery-cloudwatch Examples and Code Snippets
Community Discussions
Trending Discussions on celery-cloudwatch
QUESTION
Current setup: celery running on docker containers (with our product's code) on an EC2 node, creating and processing tasks. Our backend/broker is Redis, running in AWS' elasticache.
Goal: being able to see the queue size at any given time (similar to flower's monitoring), hopefully through AWS CloudWatch, but not needed. The content of the tasks isn't pertinent, as I am familiar with making a backup of the redis instance, and can parse the backup using local tools to do any analysis needed. Short lived historical data is highly preferred (CloudWatch goes back 2 weeks, and has granularity of 1 min datapoints, this is quite nice).
Based on how I'm aware Flower works, Flower wouldn't be feasible to use due to the amount of security groups/restrictions that we currently have in place. Additionally flower is only monitoring while you're on the page, so there is no historical data saved.
Elasticache already has built in CloudWatch for number of items in redis. This seems to me the best route to achieve the goal. However currently the queue represents one item in redis (no matter how many tasks are in the queue). Here is a sample of the redis backup parsed to json:
...ANSWER
Answered 2019-Feb-06 at 22:56To see the queue length of a queue using a redis broker, just use llen
in redis. e.g., llen celery
.
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install celery-cloudwatch
You can use celery-cloudwatch like any standard Python library. You will need to make sure that you have a development environment consisting of a Python distribution including header files, a compiler, pip, and git installed. Make sure that your pip, setuptools, and wheel are up to date. When using pip it is generally recommended to install packages in a virtual environment to avoid changes to the system.
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page