Task #4139: Bring celery workers up to shapen with initializing pyramid + pylons stack - RhodeCode CE/EE - RhodeCode - issues

Actions

Task #4139

closed

Task #4163: [ce, ee] celery refactor + upgrade

Status:

Closed

Priority:

Normal

Assignee:

Category:

Target version:

Start date:

31.07.2016

Due date:

% Done:

Estimated time:

Sorting:

Commit Number:

Description

It seems currently celery workers are not properly initialized and the stack is breaking on some tasks that require pyramid elements

Actions

Celery task do not have access to pylons/pyramid elements such as:

Problems caused by this:

create_repo at the moment triggers a CreateRepoEvent which requires the request context/user and url generator to create the url for the repo

Actions

Actions

After investigation our options are:

Subclass celery.Task and make it pass the request object to celery worker, proxied and reapplied as a pyramid request context when the task is run, this will require upgrading celery as the version we run 2.2.10 lacks the capability to support this - although there seem to be some conflicts with ampq/kombu and paster here.
Use pyramid_celery - this will require rewriting the entire celery configuration/deployment system we have now.

In both cases we will also need to hack pylons.url threadlocal until it is replaced with pyramid request.route_url.

Actions