Project

General

Profile

Task #4169

Task #4163: [ce, ee] celery refactor + upgrade

re-architecture celery support

Added by Marcin Kuzminski [staff] about 3 years ago. Updated almost 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
-
Target version:
Start date:
12.08.2016
Due date:
% Done:

0%

Estimated time:
Sorting:
Commit Number:

Description

Currently celery is a mess. We need to re-think how to handle the celery workers inside our application.

we should plan around:

  • start with fresh approach to celery, so we cen remove all the magic code we have now
  • upgrade celery to latest version
  • port our wrapper (ResultWrapper) on latest celery tasks code. This is to handle fallback execution (means execute sync when exceptions, or celery is down, or misconfigured connection to celery)
  • think about paster celeryd, probably we should create new executable, or we keep it this way for backward compat
  • We should design our new celery to make sure it has appl required registry items, as well as the pylons stuff so each task plays out nicely.

Few gotachas from old code.

  • ResultWrapper to handle taks nicer than standard celery
  • DBSession cleanups, on execution (the @dbsession decorator)

History

#1 Updated by Daniel D about 3 years ago

  • Parent task set to #4163

#2 Updated by Marcin Kuzminski [staff] almost 3 years ago

  • Target version changed from v4.4 to v4.5

#3 Updated by Marcin Kuzminski [staff] almost 3 years ago

  • Target version changed from v4.5 to v4.6

#4 Updated by Marcin Kuzminski [staff] over 2 years ago

  • Target version changed from v4.6 to v4.7

#5 Updated by Marcin Kuzminski [staff] over 2 years ago

  • Target version changed from v4.7 to v4.10

#6 Updated by Marcin Kuzminski [staff] almost 2 years ago

  • Target version changed from v4.10 to v4.11

#7 Updated by Marcin Kuzminski [staff] almost 2 years ago

  • Status changed from New to In Progress

#8 Updated by Redmine Integration almost 2 years ago

  • Status changed from In Progress to Resolved

#9 Updated by Redmine Integration almost 2 years ago

pullrequest created by marcink (status: under_review). https://internal-code.rhodecode.com/rhodecode-enterprise-ce/pull-request/2926

Also available in: Atom PDF