Project

General

Profile

Actions
Copy link

Bug #4166

open

[ce] Rhodecode crashing after MySQL error

Added by Mihai Nica over 7 years ago. Updated over 7 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
-
Category:
-
Target version:
v4.3
Start date:
12.08.2016
Due date:
% Done:

0%

Estimated time:
(Total: 0.00 h)
Sorting:
Commit Number:
Affected Version:

Description

Rhodecode Enterprise crashes with a MySQL error after a while.
Already checked the pool_recycle, and it's set.
Celery is not enabled.

Here is the information in the error log:

2016-08-11 08:48:37.672 ERROR [rhodecode.lib.caches] Failed to commit on cache key update
Traceback (most recent call last):
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/lib/caches.py", line 216, in __exit__
    Session().commit()
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 788, in commit
    self.transaction.commit()
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 384, in commit
    self._prepare_impl()
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 364, in _prepare_impl
    self.session.flush()
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 1985, in flush
    self._flush(objects)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2103, in _flush
    transaction.rollback(_capture_exception=True)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2067, in _flush
    flush_context.execute()
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 372, in execute
    rec.execute(self)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 526, in execute
    uow
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 65, in save_obj
    mapper, table, insert)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 602, in _emit_insert_statements
    execute(statement, params)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 841, in execute
    return meth(self, multiparams, params)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 322, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 938, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1005, in _execute_context
    None, None)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1271, in _handle_dbapi_exception
    exc_info
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 999, in _execute_context
    conn = self._revalidate_connection()
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 354, in _revalidate_connection
"Can't reconnect until invalid "
StatementError: Can't reconnect until invalid transaction is rolled back (original cause: InvalidRequestError: Can't reconnect until invalid transaction is rolled back) u'INSERT INTO cache_invalidation (cache_key, cache_args, cache_active) VALUES (%s, %s, %s)' [{'cache_active': True, 'cache_args': u'wsys/linux-tools/backup', 'cache_key': u'vcs-14268wsys/linux-tools/backup'}]
2016-08-11 08:48:38.000 ERROR [rhodecode.config.middleware] failed to fetch settings
Traceback (most recent call last):
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/config/middleware.py", line 174, in error_handler
    rc_config = SettingsModel().get_all_settings()
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/model/settings.py", line 234, in get_all_settings
    return _compute()
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/model/settings.py", line 217, in _compute
    for result in q
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/lib/caching_query.py", line 105, in __iter__
    return Query.__iter__(self)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2441, in __iter__
    return self._execute_and_instances(context)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2454, in _execute_and_instances
    close_with_result=True)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2445, in _connection_from_session
    **kw)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 880, in connection
    execution_options=execution_options)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 885, in _connection_for_bind
    engine, execution_options)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 326, in _connection_for_bind
    conn = bind.contextual_connect()
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1910, in contextual_connect
    self.pool.connect(),
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/pool.py", line 338, in connect
    return _ConnectionFairy._checkout(self)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/pool.py", line 645, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/pool.py", line 442, in checkout
    dbapi_connection = rec.get_connection()
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/pool.py", line 506, in get_connection
    self.connection = self.__connect()
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/pool.py", line 539, in __connect
    connection = self.__pool._creator()
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 96, in connect
    connection_invalidated=invalidated
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 90, in connect
    return dialect.connect(*cargs, **cparams)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 377, in connect
    return self.dbapi.connect(*cargs, **cparams)
  File "/opt/rhodecode/store/70w1dh4xc8q6m8avi9d90c5qn614cbaz-python2.7-MySQL-python-1.2.5/lib/python2.7/site-packages/MySQLdb/__init__.py", line 81, in Connect
    return Connection(*args, **kwargs)
File "/opt/rhodecode/store/70w1dh4xc8q6m8avi9d90c5qn614cbaz-python2.7-MySQL-python-1.2.5/lib/python2.7/site-packages/MySQLdb/connections.py", line 193, in __init__
    super(Connection, self).__init__(*args, **kwargs2)

We are running:

rhodecode@vcs:~$ rccontrol status

 - NAME: community-1
 - STATUS: RUNNING
 - TYPE: Community
 - VERSION: 4.2.1
 - VCS: vcsserver-1
 - URL: http://0.0.0.0:5000

 - NAME: vcsserver-1
 - STATUS: RUNNING
 - TYPE: VCSServer
 - VERSION: 4.2.1
 - URL: http://127.0.0.1:10001

And MySQL 5.5 on Ubuntu Linux 14.04.

Subtasks 1 (1 open — 0 closed)

Bug #4173: [ce, ee] mysql recycle pool timeout not workingResolvedDaniel D12.08.2016

Actions
Actions
Copy link
 #1

Updated by Marcin Kuzminski [CTO] over 7 years ago

  • Target version set to v4.4

We put a special code for concurrency problem we found here: https://code.rhodecode.com/rhodecode-enterprise-ce/files/bf576ea02764f8fca47a4c14d81dceffe9b1cd83/rhodecode/lib/caches.py#L217

It might happen on concurrent pushes that cache invalidation was already set, thus in this case we don't care much on the IntegrityError. However the error above could potentially indicate that there's a other problem happening.

{{rum(66, Mihai Nica)}} is there a way to reproduce this issue somehow on your side ?

  • maybe there are relevant bits in logs that would help us isolate it ?
  • are there any errors on mysql side ?

One possibility is that we should catch: StatementError together with IntegrityError

Actions
Copy link
 #4

Updated by Marcin Kuzminski [CTO] over 7 years ago

{{rum(45, Daniel D)}} did we test for the case of db crash that the new session cleanups works around this issue ?

Actions
Copy link
 #5

Updated by Marcin Kuzminski [CTO] over 7 years ago

  • Status changed from New to In Progress
Actions
Copy link
 #8

Updated by Daniel D over 7 years ago

Apart from the expected error message when the db is down, Rhodecode will connect as normal once the db is up again.

Actions
Copy link
 #9

Updated by Mihai Nica over 7 years ago

I didn't manage to find a way to reproduce the problem.
As for MySQL specific errors, didn't see any.

As for the log contents, this is error trace I get once the app crashes (all the requests before were successful).

2016-08-16 23:09:17.991 INFO  [rhodecode.lib.middleware.request_wrapper] IP: 10.0.0.1 Request to /bc/BCADSRV/git-receive-pack time: 2.307s
[2016-08-17 08:15:21 +0000] [14263] [ERROR] Error handling request /wsys/linux-tools/backup
Traceback (most recent call last):
  File "/opt/rhodecode/store/rnklx8i8kclsz24q0nbfd959f9nlz1m8-python2.7-gunicorn-19.6.0/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 135, in handle
    self.handle_request(listener, req, client, addr)
  File "/opt/rhodecode/store/rnklx8i8kclsz24q0nbfd959f9nlz1m8-python2.7-gunicorn-19.6.0/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 176, in handle_request
    respiter = self.wsgi(environ, resp.start_response)
  File "/opt/rhodecode/store/miwpp0jyk99zbccj6vzwv4axjayp41cs-python2.7-Paste-2.0.2/lib/python2.7/site-packages/paste/gzipper.py", line 34, in __call__
    response.gzip_start_response)
  File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/middleware.py", line 80, in __call__
    config.environ = environ
  File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/__init__.py", line 22, in __setattr__
    self.load_wsgi_environ(value)
  File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/__init__.py", line 51, in load_wsgi_environ
    result = mapper.routematch(path)
  File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/mapper.py", line 692, in routematch
    result = self._match(url, environ)
  File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/mapper.py", line 646, in _match
    domain_match)
  File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/route.py", line 578, in match
    not self.conditions['function'](environ, result):
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/config/routing.py", line 151, in check_group
    by_name_match = repo_group_model.get_by_group_name(repo_group_name)
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/model/repo_group.py", line 76, in get_by_group_name
    return repo.scalar()
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2429, in scalar
    ret = self.one()
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2398, in one
    ret = list(self)
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/lib/caching_query.py", line 105, in __iter__
    return Query.__iter__(self)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2441, in __iter__
    return self._execute_and_instances(context)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2456, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 841, in execute
    return meth(self, multiparams, params)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 322, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 938, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1070, in _execute_context
    context)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1271, in _handle_dbapi_exception
    exc_info
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1063, in _execute_context
    context)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 442, in do_execute
    cursor.execute(statement, parameters)
  File "/opt/rhodecode/store/70w1dh4xc8q6m8avi9d90c5qn614cbaz-python2.7-MySQL-python-1.2.5/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
    self.errorhandler(self, exc, value)
  File "/opt/rhodecode/store/70w1dh4xc8q6m8avi9d90c5qn614cbaz-python2.7-MySQL-python-1.2.5/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
OperationalError: (OperationalError) (2006, 'MySQL server has gone away') 'SELECT groups.group_id AS groups_group_id, groups.group_name AS groups_group_name, groups.group_parent_id AS groups_group_parent_id, groups.group_description AS groups_group_description, groups.enable_locking AS groups_enable_locking, groups.user_id AS groups_user_id, groups.created_on AS groups_created_on \nFROM groups \nWHERE groups.group_name = %s ORDER BY group_name' ('wsys/linux-tools/backup',)
[2016-08-17 08:15:23 +0000] [14263] [ERROR] Error handling request /favicon.ico
Traceback (most recent call last):
  File "/opt/rhodecode/store/rnklx8i8kclsz24q0nbfd959f9nlz1m8-python2.7-gunicorn-19.6.0/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 135, in handle
    self.handle_request(listener, req, client, addr)
  File "/opt/rhodecode/store/rnklx8i8kclsz24q0nbfd959f9nlz1m8-python2.7-gunicorn-19.6.0/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 176, in handle_request
    respiter = self.wsgi(environ, resp.start_response)
  File "/opt/rhodecode/store/miwpp0jyk99zbccj6vzwv4axjayp41cs-python2.7-Paste-2.0.2/lib/python2.7/site-packages/paste/gzipper.py", line 34, in __call__
    response.gzip_start_response)
  File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/middleware.py", line 80, in __call__
    config.environ = environ
  File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/__init__.py", line 22, in __setattr__
    self.load_wsgi_environ(value)
  File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/__init__.py", line 51, in load_wsgi_environ
    result = mapper.routematch(path)
  File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/mapper.py", line 692, in routematch
    result = self._match(url, environ)
  File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/mapper.py", line 646, in _match
    domain_match)
  File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/route.py", line 578, in match
    not self.conditions['function'](environ, result):
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/config/routing.py", line 151, in check_group
    by_name_match = repo_group_model.get_by_group_name(repo_group_name)
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/model/repo_group.py", line 76, in get_by_group_name
    return repo.scalar()
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2429, in scalar
    ret = self.one()
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2398, in one
    ret = list(self)
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/lib/caching_query.py", line 105, in __iter__
    return Query.__iter__(self)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2441, in __iter__
    return self._execute_and_instances(context)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2456, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 841, in execute
    return meth(self, multiparams, params)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 322, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 938, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1005, in _execute_context
    None, None)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1271, in _handle_dbapi_exception
    exc_info
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 999, in _execute_context
    conn = self._revalidate_connection()
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 354, in _revalidate_connection
    "Can't reconnect until invalid "
StatementError: Can't reconnect until invalid transaction is rolled back (original cause: InvalidRequestError: Can't reconnect until invalid transaction is rolled back) u'SELECT groups.group_id AS groups_group_id, groups.group_name AS groups_group_name, groups.group_parent_id AS groups_group_parent_id, groups.group_description AS groups_group_description, groups.enable_locking AS groups_enable_locking, groups.user_id AS groups_user_id, groups.created_on AS groups_created_on \nFROM groups \nWHERE groups.group_name = %s ORDER BY group_name' [immutabledict({})]
2016-08-17 08:15:24.093 INFO  [rhodecode.lib.middleware.request_wrapper] IP: 188.25.232.52 Request to /favicon.ico time: 0.086s
[2016-08-17 10:41:51 +0000] [14263] [ERROR] Error handling request /
Traceback (most recent call last):
  File "/opt/rhodecode/store/rnklx8i8kclsz24q0nbfd959f9nlz1m8-python2.7-gunicorn-19.6.0/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 135, in handle
    self.handle_request(listener, req, client, addr)
  File "/opt/rhodecode/store/rnklx8i8kclsz24q0nbfd959f9nlz1m8-python2.7-gunicorn-19.6.0/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 176, in handle_request
    respiter = self.wsgi(environ, resp.start_response)
  File "/opt/rhodecode/store/miwpp0jyk99zbccj6vzwv4axjayp41cs-python2.7-Paste-2.0.2/lib/python2.7/site-packages/paste/gzipper.py", line 31, in __call__
    return self.application(environ, start_response)
  File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/middleware.py", line 131, in __call__
    response = self.app(environ, start_response)
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/lib/middleware/https_fixup.py", line 46, in __call__
    return self.application(environ, custom_start_response)
  File "/opt/rhodecode/store/incd2pyifnx1i11anwh308gxp16rp957-python2.7-pyramid-1.6.1/lib/python2.7/site-packages/pyramid/router.py", line 223, in __call__
    response = self.invoke_subrequest(request, use_tweens=True)
  File "/opt/rhodecode/store/incd2pyifnx1i11anwh308gxp16rp957-python2.7-pyramid-1.6.1/lib/python2.7/site-packages/pyramid/router.py", line 198, in invoke_subrequest
    response = handle_request(request)
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/tweens.py", line 63, in pylons_compatibility_tween
    auth_user = get_auth_user(environ)
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/lib/base.py", line 350, in get_auth_user
    ip_addr=ip_addr)
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/lib/auth.py", line 796, in __init__
    self.propagate_data()
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/lib/auth.py", line 851, in propagate_data
    anon_user = self.anonymous_user = User.get_default_user(cache=True)
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/model/db.py", line 809, in get_default_user
    user = User.get_by_username(User.DEFAULT_USER, cache=cache)
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/model/db.py", line 690, in get_by_username
    return q.scalar()
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2429, in scalar
    ret = self.one()
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2398, in one
    ret = list(self)
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/lib/caching_query.py", line 102, in __iter__
    return self.get_value(createfunc=lambda:
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/lib/caching_query.py", line 121, in get_value
    ret = cache.get_value(cache_key, createfunc=createfunc)
  File "/opt/rhodecode/store/l7bv0fc6pv8fmmgmdbh39hzn4iskq3hj-python2.7-Beaker-1.7.0/lib/python2.7/site-packages/beaker/cache.py", line 317, in get
    return self._get_value(key, **kw).get_value()
  File "/opt/rhodecode/store/l7bv0fc6pv8fmmgmdbh39hzn4iskq3hj-python2.7-Beaker-1.7.0/lib/python2.7/site-packages/beaker/container.py", line 378, in get_value
    v = self.createfunc()
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/lib/caching_query.py", line 103, in <lambda>
    list(Query.__iter__(self)))
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2441, in __iter__
    return self._execute_and_instances(context)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2456, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 841, in execute
    return meth(self, multiparams, params)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 322, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 938, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1005, in _execute_context
    None, None)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1271, in _handle_dbapi_exception
    exc_info
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 999, in _execute_context
    conn = self._revalidate_connection()
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 354, in _revalidate_connection
    "Can't reconnect until invalid "
StatementError: Can't reconnect until invalid transaction is rolled back (original cause: InvalidRequestError: Can't reconnect until invalid transaction is rolled back) u'SELECT users.firstname AS users_firstname, users.email AS users_email, users.user_data AS users_user_data, users.user_id AS users_user_id, users.username AS users_username, users.password AS users_password, users.active AS users_active, users.admin AS users_admin, users.lastname AS users_lastname, users.last_login AS users_last_login, users.extern_type AS users_extern_type, users.extern_name AS users_extern_name, users.api_key AS users_api_key, users.inherit_default_permissions AS users_inherit_default_permissions, users.created_on AS users_created_on \nFROM users \nWHERE users.username = %s' [immutabledict({})]
2016-08-17 10:55:44.441 INFO  [rhodecode.lib.middleware.request_wrapper] IP: 10.0.0.1 Request to /favicon.ico time: 0.011s
2016-08-17 10:55:44.483 INFO  [rhodecode.lib.base] IP: 10.0.0.1 User: <AuthUser('id:2[redecs] ip:10.0.0.1 auth:True')> accessed /wsys/linux-tools/backup [summary.index]
2016-08-17 10:55:46.387 INFO  [rhodecode.lib.auth] user <AuthUser('id:2[redecs] ip:10.0.0.1 auth:True')> authenticating with:RHODECODE_AUTH IS authenticated on func SummaryController:index
2016-08-17 10:55:51.065 INFO  [rhodecode.lib.middleware.request_wrapper] IP: 10.0.0.1 Request to /wsys/linux-tools/backup time: 6.583s
[2016-08-17 10:55:51 +0000] [14265] [ERROR] Error handling request /js/rhodecode/routes.js?ver=398d8f3b
Traceback (most recent call last):
  File "/opt/rhodecode/store/rnklx8i8kclsz24q0nbfd959f9nlz1m8-python2.7-gunicorn-19.6.0/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 135, in handle
    self.handle_request(listener, req, client, addr)
  File "/opt/rhodecode/store/rnklx8i8kclsz24q0nbfd959f9nlz1m8-python2.7-gunicorn-19.6.0/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 176, in handle_request
    respiter = self.wsgi(environ, resp.start_response)
  File "/opt/rhodecode/store/miwpp0jyk99zbccj6vzwv4axjayp41cs-python2.7-Paste-2.0.2/lib/python2.7/site-packages/paste/gzipper.py", line 34, in __call__
    response.gzip_start_response)
  File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/middleware.py", line 80, in __call__
    config.environ = environ
  File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/__init__.py", line 22, in __setattr__
    self.load_wsgi_environ(value)
  File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/__init__.py", line 51, in load_wsgi_environ
    result = mapper.routematch(path)
  File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/mapper.py", line 692, in routematch
    result = self._match(url, environ)
  File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/mapper.py", line 646, in _match
    domain_match)
  File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/route.py", line 578, in match
    not self.conditions['function'](environ, result):
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/config/routing.py", line 151, in check_group
    by_name_match = repo_group_model.get_by_group_name(repo_group_name)
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/model/repo_group.py", line 76, in get_by_group_name
    return repo.scalar()
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2429, in scalar
    ret = self.one()
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2398, in one
    ret = list(self)
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/lib/caching_query.py", line 105, in __iter__
    return Query.__iter__(self)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2441, in __iter__
    return self._execute_and_instances(context)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2456, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 841, in execute
    return meth(self, multiparams, params)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 322, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 938, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1070, in _execute_context
    context)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1271, in _handle_dbapi_exception
    exc_info
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1063, in _execute_context
    context)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 442, in do_execute
    cursor.execute(statement, parameters)
  File "/opt/rhodecode/store/70w1dh4xc8q6m8avi9d90c5qn614cbaz-python2.7-MySQL-python-1.2.5/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
    self.errorhandler(self, exc, value)
  File "/opt/rhodecode/store/70w1dh4xc8q6m8avi9d90c5qn614cbaz-python2.7-MySQL-python-1.2.5/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
OperationalError: (OperationalError) (2006, 'MySQL server has gone away') 'SELECT groups.group_id AS groups_group_id, groups.group_name AS groups_group_name, groups.group_parent_id AS groups_group_parent_id, groups.group_description AS groups_group_description, groups.enable_locking AS groups_enable_locking, groups.user_id AS groups_user_id, groups.created_on AS groups_created_on \nFROM groups \nWHERE groups.group_name = %s ORDER BY group_name' ('js/rhodecode/routes.js',)
[2016-08-17 10:55:51 +0000] [14268] [ERROR] Error handling request /js/rhodecode/base/keyboard-bindings.js?ver=398d8f3b
Traceback (most recent call last):
  File "/opt/rhodecode/store/rnklx8i8kclsz24q0nbfd959f9nlz1m8-python2.7-gunicorn-19.6.0/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 135, in handle
    self.handle_request(listener, req, client, addr)
  File "/opt/rhodecode/store/rnklx8i8kclsz24q0nbfd959f9nlz1m8-python2.7-gunicorn-19.6.0/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 176, in handle_request
    respiter = self.wsgi(environ, resp.start_response)
  File "/opt/rhodecode/store/miwpp0jyk99zbccj6vzwv4axjayp41cs-python2.7-Paste-2.0.2/lib/python2.7/site-packages/paste/gzipper.py", line 34, in __call__
    response.gzip_start_response)
  File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/middleware.py", line 80, in __call__
    config.environ = environ
  File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/__init__.py", line 22, in __setattr__
    self.load_wsgi_environ(value)
  File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/__init__.py", line 51, in load_wsgi_environ
    result = mapper.routematch(path)
  File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/mapper.py", line 692, in routematch
    result = self._match(url, environ)
  File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/mapper.py", line 646, in _match
    domain_match)
  File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/route.py", line 578, in match
    not self.conditions['function'](environ, result):
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/config/routing.py", line 151, in check_group
    by_name_match = repo_group_model.get_by_group_name(repo_group_name)
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/model/repo_group.py", line 76, in get_by_group_name
    return repo.scalar()
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2429, in scalar
    ret = self.one()
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2398, in one
    ret = list(self)
  File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/lib/caching_query.py", line 105, in __iter__
    return Query.__iter__(self)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2441, in __iter__
    return self._execute_and_instances(context)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2456, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 841, in execute
    return meth(self, multiparams, params)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 322, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 938, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1070, in _execute_context
    context)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1271, in _handle_dbapi_exception
    exc_info
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1063, in _execute_context
    context)
  File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 442, in do_execute
    cursor.execute(statement, parameters)
  File "/opt/rhodecode/store/70w1dh4xc8q6m8avi9d90c5qn614cbaz-python2.7-MySQL-python-1.2.5/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
    self.errorhandler(self, exc, value)
  File "/opt/rhodecode/store/70w1dh4xc8q6m8avi9d90c5qn614cbaz-python2.7-MySQL-python-1.2.5/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
Actions
Copy link
 #10

Updated by Marcin Kuzminski [CTO] over 7 years ago

  • Status changed from In Progress to Resolved
  • Target version changed from v4.4 to v4.3

We released 4.3.1 which should address this issue, and recycle RhodeCode connections properly

Actions
Copy link

Also available in: Atom PDF