Task #4163: [ce, ee] celery refactor + upgrade
Bring celery workers up to shapen with initializing pyramid + pylons stack
It seems currently celery workers are not properly initialized and the stack is breaking on some tasks that require pyramid elements
#1 Updated by Daniel D over 4 years ago
Celery task do not have access to pylons/pyramid elements such as:
- url generator
- request context
- user that triggered a task
Problems caused by this:
- create_repo at the moment triggers a CreateRepoEvent which requires the request context/user and url generator to create the url for the repo
#3 Updated by Daniel D over 4 years ago
After investigation our options are:
Subclass celery.Task and make it pass the request object to celery worker, proxied and reapplied as a pyramid request context when the task is run, this will require upgrading celery as the version we run 2.2.10 lacks the capability to support this - although there seem to be some conflicts with ampq/kombu and paster here.
Use pyramid_celery - this will require rewriting the entire celery configuration/deployment system we have now.
In both cases we will also need to hack pylons.url threadlocal until it is replaced with pyramid request.route_url.
#4 Updated by Redmine Integration over 4 years ago
pullrequest created, https://internal-code.rhodecode.com/rhodecode-enterprise-ce/pull-request/2518, (status: under_review)
#5 Updated by Redmine Integration over 4 years ago
- Status changed from In Progress to Resolved
#6 Updated by Redmine Integration over 4 years ago
pullrequest merged, https://internal-code.rhodecode.com/rhodecode-enterprise-ce/pull-request/2518, (status: approved)