Celery (software)
Stable release | 5.3.4
/ September 3, 2023 |
---|---|
Repository | |
Written in | Python |
Platform | Cross-platform |
Available in | Python |
Type | Message-oriented middleware |
License | BSD License |
Website | docs |
Celery is an open source asynchronous task queue or job queue which is based on distributed message passing. While it supports scheduling, its focus is on operations in real time.[1]
Overview
The execution units, called tasks, are executed concurrently on one or more worker nodes using multiprocessing, eventlet[2] or gevent.[3] Tasks can execute asynchronously (in the background) or synchronously (wait until ready). Celery is used in production systems, for services such as Instagram, to process millions of tasks every day.[1]
Technology
Celery is written in Python, but the protocol can be implemented in any language. It can also operate with other languages using webhooks.[4] There is also a Ruby-Client called RCelery,[5] a PHP client,[6] a Go client,[7] a Rust client,[8] and a Node.js client.[9]
The recommended message brokers are RabbitMQ or Redis. In comparison with RabbitMQ, Redis represents a good start. However, if there is a valid reason and Redis is no longer meeting the requirements for the project, it is simple to switch to RabbitMQ. Additionally, MongoDB, Amazon SQS, CouchDB, IronMQ, and databases (using SQLAlchemy or the Django ORM) are supported in status experimental.[10]
See also
References
- ^ a b "Celery: Distributed Task Queue". Archived from the original on 2019-06-13. Retrieved 2016-01-14.
- ^ "Eventlet Networking Library". eventlet.net. Retrieved 2022-09-21.
- ^ "What is gevent? — gevent 21.12.1.dev0 documentation". www.gevent.org. Retrieved 2022-09-21.
- ^ "HTTP Callback Tasks (Webhooks) — Celery 3.1.23 documentation". docs.celeryproject.org. Archived from the original on 2016-10-30. Retrieved 2016-08-17.
- ^ "leapfrogonline/rcelery". GitHub. Retrieved 2016-08-17.[permanent dead link]
- ^ "gjedeer/celery-php". GitHub. Retrieved 2016-08-17.
- ^ "gocelery/gocelery". GitHub. Retrieved 2018-08-06.
- ^ rusty-celery/rusty-celery, Rusty Celery, 2022-10-09, retrieved 2022-10-09
- ^ "mher/node-celery". GitHub. Retrieved 2016-08-17.
- ^ "Backends and Brokers — Celery 5.2.7 documentation". docs.celeryproject.org. Archived from the original on 2022-06-23. Retrieved 2022-06-23.
External links
- All articles with dead external links
- Articles with dead external links from November 2019
- Articles with permanently dead external links
- Articles with short description
- Short description matches Wikidata
- Official website different in Wikidata and Wikipedia
- Free system software
- Free software programmed in Python
- Python (programming language) software
- Concurrent programming libraries
- Inter-process communication
- Message-oriented middleware
- Software using the BSD license
- All stub articles
- Free and open-source software stubs