RhodeCode CE/EE

We're having trouble sometimes when trying to create or fork repositories. The reports I've heard are that it either never finishes or gets a 404 error. It seems like it does work at least some of the time, though.

We're running Community Edition 4.4 using Celery, RabbitMQ, and PostgreSQL.

I checked our Celery log file and we're seeing errors like this when we try to create/fork (this one is a fork, but the details are basically the same for create except for the task name):

[2016-10-17 11:19:46,408: ERROR/MainProcess] Task rhodecode.lib.celerylib.tasks.create_repo_fork[d71a3d33-d803-43cf-aa6d-0a6d48c5e935] raised exception: SQLAlchemyError('(DatabaseError) SSL error: decryption failed or bad record mac\n',)
Traceback (most recent call last):
  File "/opt/rhodecode/store/32ay02r6m59qvxk7gvybpfsqmmhd810v-python2.7-celery-2.2.10/lib/python2.7/site-packages/celery/execute/trace.py", line 34, in trace
    return cls(states.SUCCESS, retval=fun(*args, **kwargs))
  File "/opt/rhodecode/store/2nrqs5ylil69bsjb09j699ydahcxvvfn-python2.7-rhodecode-enterprise-ce-4.4.0/lib/python2.7/site-packages/rhodecode/lib/celerylib/__init__.py", line 115, in __call__
    ip_addr=proxy_data['auth_user']['ip_addr'])
  File "/opt/rhodecode/store/2nrqs5ylil69bsjb09j699ydahcxvvfn-python2.7-rhodecode-enterprise-ce-4.4.0/lib/python2.7/site-packages/rhodecode/lib/auth.py", line 796, in __init__
    self.propagate_data()
  File "/opt/rhodecode/store/2nrqs5ylil69bsjb09j699ydahcxvvfn-python2.7-rhodecode-enterprise-ce-4.4.0/lib/python2.7/site-packages/rhodecode/lib/auth.py", line 851, in propagate_data
    anon_user = self.anonymous_user = User.get_default_user(cache=True)
  File "/opt/rhodecode/store/2nrqs5ylil69bsjb09j699ydahcxvvfn-python2.7-rhodecode-enterprise-ce-4.4.0/lib/python2.7/site-packages/rhodecode/model/db.py", line 810, in get_default_user
    user = User.get_by_username(User.DEFAULT_USER, cache=cache)
  File "/opt/rhodecode/store/2nrqs5ylil69bsjb09j699ydahcxvvfn-python2.7-rhodecode-enterprise-ce-4.4.0/lib/python2.7/site-packages/rhodecode/model/db.py", line 691, in get_by_username
    return q.scalar()
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2429, in scalar
    ret = self.one()
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2398, in one
    ret = list(self)
  File "/opt/rhodecode/store/2nrqs5ylil69bsjb09j699ydahcxvvfn-python2.7-rhodecode-enterprise-ce-4.4.0/lib/python2.7/site-packages/rhodecode/lib/caching_query.py", line 102, in __iter__
    return self.get_value(createfunc=lambda:
  File "/opt/rhodecode/store/2nrqs5ylil69bsjb09j699ydahcxvvfn-python2.7-rhodecode-enterprise-ce-4.4.0/lib/python2.7/site-packages/rhodecode/lib/caching_query.py", line 121, in get_value
    ret = cache.get_value(cache_key, createfunc=createfunc)
  File "/opt/rhodecode/store/l7bv0fc6pv8fmmgmdbh39hzn4iskq3hj-python2.7-Beaker-1.7.0/lib/python2.7/site-packages/beaker/cache.py", line 317, in get
    return self._get_value(key, **kw).get_value()
  File "/opt/rhodecode/store/l7bv0fc6pv8fmmgmdbh39hzn4iskq3hj-python2.7-Beaker-1.7.0/lib/python2.7/site-packages/beaker/container.py", line 378, in get_value
    v = self.createfunc()
  File "/opt/rhodecode/store/2nrqs5ylil69bsjb09j699ydahcxvvfn-python2.7-rhodecode-enterprise-ce-4.4.0/lib/python2.7/site-packages/rhodecode/lib/caching_query.py", line 103, in <lambda>
    list(Query.__iter__(self)))
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2441, in __iter__
    return self._execute_and_instances(context)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2456, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 841, in execute
    return meth(self, multiparams, params)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 322, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 938, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1070, in _execute_context
    context)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1271, in _handle_dbapi_exception
    exc_info
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1063, in _execute_context
    context)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 442, in do_execute
    cursor.execute(statement, parameters)
DatabaseError: (DatabaseError) SSL error: decryption failed or bad record mac

This seems like it could be related to this issue: https://github.com/celery/celery/issues/1564