Task #4139
closed
Task #4163: [ce, ee] celery refactor + upgrade
Bring celery workers up to shapen with initializing pyramid + pylons stack
0%
Description
It seems currently celery workers are not properly initialized and the stack is breaking on some tasks that require pyramid elements
Updated by Daniel D over 8 years ago
Celery task do not have access to pylons/pyramid elements such as:
- url generator
- request context
- user that triggered a task
Problems caused by this:
- create_repo at the moment triggers a CreateRepoEvent which requires the request context/user and url generator to create the url for the repo
Updated by Daniel D over 8 years ago
After investigation our options are:
Subclass celery.Task and make it pass the request object to celery worker, proxied and reapplied as a pyramid request context when the task is run, this will require upgrading celery as the version we run 2.2.10 lacks the capability to support this - although there seem to be some conflicts with ampq/kombu and paster here.
Use pyramid_celery - this will require rewriting the entire celery configuration/deployment system we have now.
In both cases we will also need to hack pylons.url threadlocal until it is replaced with pyramid request.route_url.
Updated by Redmine Integration over 8 years ago
pullrequest created, https://internal-code.rhodecode.com/rhodecode-enterprise-ce/pull-request/2518, (status: under_review)
Updated by Redmine Integration over 8 years ago
- Status changed from In Progress to Resolved
Updated by Redmine Integration over 8 years ago
pullrequest merged, https://internal-code.rhodecode.com/rhodecode-enterprise-ce/pull-request/2518, (status: approved)
Updated by Marcin Kuzminski [CTO] over 8 years ago
- Status changed from Resolved to Feedback
Updated by Daniel D over 8 years ago
- Status changed from Feedback to Closed
This particular problem has been fixed, a new task has been made here: https://issues.rhodecode.com/issues/4163 for planning the refactor of celery.