Task #4163: [ce, ee] celery refactor + upgrade
re-architecture celery support
Currently celery is a mess. We need to re-think how to handle the celery workers inside our application.
we should plan around:
- start with fresh approach to celery, so we cen remove all the magic code we have now
- upgrade celery to latest version
- port our wrapper (ResultWrapper) on latest celery tasks code. This is to handle fallback execution (means execute sync when exceptions, or celery is down, or misconfigured connection to celery)
- think about paster celeryd, probably we should create new executable, or we keep it this way for backward compat
- We should design our new celery to make sure it has appl required registry items, as well as the pylons stuff so each task plays out nicely.
Few gotachas from old code.
- ResultWrapper to handle taks nicer than standard celery
- DBSession cleanups, on execution (the @dbsession decorator)