https://issues.rhodecode.com/https://issues.rhodecode.com/favicon.ico?16960560042016-08-12T12:04:02ZRhodeCode - issuesRhodeCode CE/EE - Bug #4166: [ce] Rhodecode crashing after MySQL errorhttps://issues.rhodecode.com/issues/4166?journal_id=233512016-08-12T12:04:02ZMarcin Kuzminski [CTO]marcin@rhodecode.com
<ul><li><strong>Target version</strong> set to <i>v4.4</i></li></ul><p>We put a special code for concurrency problem we found here: <a href="https://code.rhodecode.com/rhodecode-enterprise-ce/files/bf576ea02764f8fca47a4c14d81dceffe9b1cd83/rhodecode/lib/caches.py#L217" class="external">https://code.rhodecode.com/rhodecode-enterprise-ce/files/bf576ea02764f8fca47a4c14d81dceffe9b1cd83/rhodecode/lib/caches.py#L217</a></p>
<p>It might happen on concurrent pushes that cache invalidation was already set, thus in this case we don't care much on the IntegrityError. However the error above could potentially indicate that there's a other problem happening.</p>
<p>{{rum(66, Mihai Nica)}} is there a way to reproduce this issue somehow on your side ?</p>
<ul>
<li>maybe there are relevant bits in logs that would help us isolate it ?</li>
<li>are there any errors on mysql side ?</li>
</ul>
<p>One possibility is that we should catch: <code>StatementError</code> together with <code>IntegrityError</code></p>
RhodeCode CE/EE - Bug #4166: [ce] Rhodecode crashing after MySQL errorhttps://issues.rhodecode.com/issues/4166?journal_id=233782016-08-13T02:04:01ZRedmine Integration
<ul></ul><p>pullrequest created, <a href="https://internal-code.rhodecode.com/rhodecode-enterprise-ce/pull-request/2563" class="external">https://internal-code.rhodecode.com/rhodecode-enterprise-ce/pull-request/2563</a>, (status: under_review)</p>
RhodeCode CE/EE - Bug #4166: [ce] Rhodecode crashing after MySQL errorhttps://issues.rhodecode.com/issues/4166?journal_id=234082016-08-16T00:52:24ZRedmine Integration
<ul></ul><p>pullrequest created, <a href="https://internal-code.rhodecode.com/rhodecode-enterprise-ce/pull-request/2569" class="external">https://internal-code.rhodecode.com/rhodecode-enterprise-ce/pull-request/2569</a>, (status: under_review)</p>
RhodeCode CE/EE - Bug #4166: [ce] Rhodecode crashing after MySQL errorhttps://issues.rhodecode.com/issues/4166?journal_id=234172016-08-16T20:12:58ZMarcin Kuzminski [CTO]marcin@rhodecode.com
<ul></ul><p>{{rum(45, Daniel D)}} did we test for the case of db crash that the new session cleanups works around this issue ?</p>
RhodeCode CE/EE - Bug #4166: [ce] Rhodecode crashing after MySQL errorhttps://issues.rhodecode.com/issues/4166?journal_id=234182016-08-16T20:17:41ZMarcin Kuzminski [CTO]marcin@rhodecode.com
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li></ul> RhodeCode CE/EE - Bug #4166: [ce] Rhodecode crashing after MySQL errorhttps://issues.rhodecode.com/issues/4166?journal_id=234252016-08-16T22:02:30ZRedmine Integration
<ul></ul><p>Commit <a href="https://internal-code.rhodecode.com/rhodecode-enterprise-ce/changeset/d3c760652e972e4712de97e05a2b963534c834fb" class="external">https://internal-code.rhodecode.com/rhodecode-enterprise-ce/changeset/d3c760652e972e4712de97e05a2b963534c834fb</a> references this issue</p>
RhodeCode CE/EE - Bug #4166: [ce] Rhodecode crashing after MySQL errorhttps://issues.rhodecode.com/issues/4166?journal_id=234272016-08-16T22:02:40ZRedmine Integration
<ul></ul><p>pullrequest merged, <a href="https://internal-code.rhodecode.com/rhodecode-enterprise-ce/pull-request/2569" class="external">https://internal-code.rhodecode.com/rhodecode-enterprise-ce/pull-request/2569</a>, (status: approved)</p>
RhodeCode CE/EE - Bug #4166: [ce] Rhodecode crashing after MySQL errorhttps://issues.rhodecode.com/issues/4166?journal_id=234282016-08-17T04:21:18ZDaniel Ddaniel@rhodecode.com
<ul></ul><p>Apart from the expected error message when the db is down, Rhodecode will connect as normal once the db is up again. </p>
RhodeCode CE/EE - Bug #4166: [ce] Rhodecode crashing after MySQL errorhttps://issues.rhodecode.com/issues/4166?journal_id=234382016-08-17T11:09:44ZMihai Nica
<ul></ul><p>I didn't manage to find a way to reproduce the problem.<br>
As for MySQL specific errors, didn't see any.</p>
<p>As for the log contents, this is error trace I get once the app crashes (all the requests before were successful).</p>
<pre><code>2016-08-16 23:09:17.991 INFO [rhodecode.lib.middleware.request_wrapper] IP: 10.0.0.1 Request to /bc/BCADSRV/git-receive-pack time: 2.307s
[2016-08-17 08:15:21 +0000] [14263] [ERROR] Error handling request /wsys/linux-tools/backup
Traceback (most recent call last):
File "/opt/rhodecode/store/rnklx8i8kclsz24q0nbfd959f9nlz1m8-python2.7-gunicorn-19.6.0/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 135, in handle
self.handle_request(listener, req, client, addr)
File "/opt/rhodecode/store/rnklx8i8kclsz24q0nbfd959f9nlz1m8-python2.7-gunicorn-19.6.0/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 176, in handle_request
respiter = self.wsgi(environ, resp.start_response)
File "/opt/rhodecode/store/miwpp0jyk99zbccj6vzwv4axjayp41cs-python2.7-Paste-2.0.2/lib/python2.7/site-packages/paste/gzipper.py", line 34, in __call__
response.gzip_start_response)
File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/middleware.py", line 80, in __call__
config.environ = environ
File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/__init__.py", line 22, in __setattr__
self.load_wsgi_environ(value)
File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/__init__.py", line 51, in load_wsgi_environ
result = mapper.routematch(path)
File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/mapper.py", line 692, in routematch
result = self._match(url, environ)
File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/mapper.py", line 646, in _match
domain_match)
File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/route.py", line 578, in match
not self.conditions['function'](environ, result):
File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/config/routing.py", line 151, in check_group
by_name_match = repo_group_model.get_by_group_name(repo_group_name)
File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/model/repo_group.py", line 76, in get_by_group_name
return repo.scalar()
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2429, in scalar
ret = self.one()
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2398, in one
ret = list(self)
File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/lib/caching_query.py", line 105, in __iter__
return Query.__iter__(self)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2441, in __iter__
return self._execute_and_instances(context)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2456, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 841, in execute
return meth(self, multiparams, params)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 322, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 938, in _execute_clauseelement
compiled_sql, distilled_params
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1070, in _execute_context
context)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1271, in _handle_dbapi_exception
exc_info
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1063, in _execute_context
context)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 442, in do_execute
cursor.execute(statement, parameters)
File "/opt/rhodecode/store/70w1dh4xc8q6m8avi9d90c5qn614cbaz-python2.7-MySQL-python-1.2.5/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
self.errorhandler(self, exc, value)
File "/opt/rhodecode/store/70w1dh4xc8q6m8avi9d90c5qn614cbaz-python2.7-MySQL-python-1.2.5/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
OperationalError: (OperationalError) (2006, 'MySQL server has gone away') 'SELECT groups.group_id AS groups_group_id, groups.group_name AS groups_group_name, groups.group_parent_id AS groups_group_parent_id, groups.group_description AS groups_group_description, groups.enable_locking AS groups_enable_locking, groups.user_id AS groups_user_id, groups.created_on AS groups_created_on \nFROM groups \nWHERE groups.group_name = %s ORDER BY group_name' ('wsys/linux-tools/backup',)
[2016-08-17 08:15:23 +0000] [14263] [ERROR] Error handling request /favicon.ico
Traceback (most recent call last):
File "/opt/rhodecode/store/rnklx8i8kclsz24q0nbfd959f9nlz1m8-python2.7-gunicorn-19.6.0/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 135, in handle
self.handle_request(listener, req, client, addr)
File "/opt/rhodecode/store/rnklx8i8kclsz24q0nbfd959f9nlz1m8-python2.7-gunicorn-19.6.0/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 176, in handle_request
respiter = self.wsgi(environ, resp.start_response)
File "/opt/rhodecode/store/miwpp0jyk99zbccj6vzwv4axjayp41cs-python2.7-Paste-2.0.2/lib/python2.7/site-packages/paste/gzipper.py", line 34, in __call__
response.gzip_start_response)
File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/middleware.py", line 80, in __call__
config.environ = environ
File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/__init__.py", line 22, in __setattr__
self.load_wsgi_environ(value)
File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/__init__.py", line 51, in load_wsgi_environ
result = mapper.routematch(path)
File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/mapper.py", line 692, in routematch
result = self._match(url, environ)
File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/mapper.py", line 646, in _match
domain_match)
File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/route.py", line 578, in match
not self.conditions['function'](environ, result):
File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/config/routing.py", line 151, in check_group
by_name_match = repo_group_model.get_by_group_name(repo_group_name)
File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/model/repo_group.py", line 76, in get_by_group_name
return repo.scalar()
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2429, in scalar
ret = self.one()
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2398, in one
ret = list(self)
File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/lib/caching_query.py", line 105, in __iter__
return Query.__iter__(self)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2441, in __iter__
return self._execute_and_instances(context)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2456, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 841, in execute
return meth(self, multiparams, params)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 322, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 938, in _execute_clauseelement
compiled_sql, distilled_params
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1005, in _execute_context
None, None)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1271, in _handle_dbapi_exception
exc_info
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 999, in _execute_context
conn = self._revalidate_connection()
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 354, in _revalidate_connection
"Can't reconnect until invalid "
StatementError: Can't reconnect until invalid transaction is rolled back (original cause: InvalidRequestError: Can't reconnect until invalid transaction is rolled back) u'SELECT groups.group_id AS groups_group_id, groups.group_name AS groups_group_name, groups.group_parent_id AS groups_group_parent_id, groups.group_description AS groups_group_description, groups.enable_locking AS groups_enable_locking, groups.user_id AS groups_user_id, groups.created_on AS groups_created_on \nFROM groups \nWHERE groups.group_name = %s ORDER BY group_name' [immutabledict({})]
2016-08-17 08:15:24.093 INFO [rhodecode.lib.middleware.request_wrapper] IP: 188.25.232.52 Request to /favicon.ico time: 0.086s
[2016-08-17 10:41:51 +0000] [14263] [ERROR] Error handling request /
Traceback (most recent call last):
File "/opt/rhodecode/store/rnklx8i8kclsz24q0nbfd959f9nlz1m8-python2.7-gunicorn-19.6.0/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 135, in handle
self.handle_request(listener, req, client, addr)
File "/opt/rhodecode/store/rnklx8i8kclsz24q0nbfd959f9nlz1m8-python2.7-gunicorn-19.6.0/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 176, in handle_request
respiter = self.wsgi(environ, resp.start_response)
File "/opt/rhodecode/store/miwpp0jyk99zbccj6vzwv4axjayp41cs-python2.7-Paste-2.0.2/lib/python2.7/site-packages/paste/gzipper.py", line 31, in __call__
return self.application(environ, start_response)
File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/middleware.py", line 131, in __call__
response = self.app(environ, start_response)
File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/lib/middleware/https_fixup.py", line 46, in __call__
return self.application(environ, custom_start_response)
File "/opt/rhodecode/store/incd2pyifnx1i11anwh308gxp16rp957-python2.7-pyramid-1.6.1/lib/python2.7/site-packages/pyramid/router.py", line 223, in __call__
response = self.invoke_subrequest(request, use_tweens=True)
File "/opt/rhodecode/store/incd2pyifnx1i11anwh308gxp16rp957-python2.7-pyramid-1.6.1/lib/python2.7/site-packages/pyramid/router.py", line 198, in invoke_subrequest
response = handle_request(request)
File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/tweens.py", line 63, in pylons_compatibility_tween
auth_user = get_auth_user(environ)
File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/lib/base.py", line 350, in get_auth_user
ip_addr=ip_addr)
File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/lib/auth.py", line 796, in __init__
self.propagate_data()
File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/lib/auth.py", line 851, in propagate_data
anon_user = self.anonymous_user = User.get_default_user(cache=True)
File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/model/db.py", line 809, in get_default_user
user = User.get_by_username(User.DEFAULT_USER, cache=cache)
File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/model/db.py", line 690, in get_by_username
return q.scalar()
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2429, in scalar
ret = self.one()
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2398, in one
ret = list(self)
File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/lib/caching_query.py", line 102, in __iter__
return self.get_value(createfunc=lambda:
File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/lib/caching_query.py", line 121, in get_value
ret = cache.get_value(cache_key, createfunc=createfunc)
File "/opt/rhodecode/store/l7bv0fc6pv8fmmgmdbh39hzn4iskq3hj-python2.7-Beaker-1.7.0/lib/python2.7/site-packages/beaker/cache.py", line 317, in get
return self._get_value(key, **kw).get_value()
File "/opt/rhodecode/store/l7bv0fc6pv8fmmgmdbh39hzn4iskq3hj-python2.7-Beaker-1.7.0/lib/python2.7/site-packages/beaker/container.py", line 378, in get_value
v = self.createfunc()
File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/lib/caching_query.py", line 103, in <lambda>
list(Query.__iter__(self)))
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2441, in __iter__
return self._execute_and_instances(context)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2456, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 841, in execute
return meth(self, multiparams, params)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 322, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 938, in _execute_clauseelement
compiled_sql, distilled_params
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1005, in _execute_context
None, None)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1271, in _handle_dbapi_exception
exc_info
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 999, in _execute_context
conn = self._revalidate_connection()
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 354, in _revalidate_connection
"Can't reconnect until invalid "
StatementError: Can't reconnect until invalid transaction is rolled back (original cause: InvalidRequestError: Can't reconnect until invalid transaction is rolled back) u'SELECT users.firstname AS users_firstname, users.email AS users_email, users.user_data AS users_user_data, users.user_id AS users_user_id, users.username AS users_username, users.password AS users_password, users.active AS users_active, users.admin AS users_admin, users.lastname AS users_lastname, users.last_login AS users_last_login, users.extern_type AS users_extern_type, users.extern_name AS users_extern_name, users.api_key AS users_api_key, users.inherit_default_permissions AS users_inherit_default_permissions, users.created_on AS users_created_on \nFROM users \nWHERE users.username = %s' [immutabledict({})]
2016-08-17 10:55:44.441 INFO [rhodecode.lib.middleware.request_wrapper] IP: 10.0.0.1 Request to /favicon.ico time: 0.011s
2016-08-17 10:55:44.483 INFO [rhodecode.lib.base] IP: 10.0.0.1 User: <AuthUser('id:2[redecs] ip:10.0.0.1 auth:True')> accessed /wsys/linux-tools/backup [summary.index]
2016-08-17 10:55:46.387 INFO [rhodecode.lib.auth] user <AuthUser('id:2[redecs] ip:10.0.0.1 auth:True')> authenticating with:RHODECODE_AUTH IS authenticated on func SummaryController:index
2016-08-17 10:55:51.065 INFO [rhodecode.lib.middleware.request_wrapper] IP: 10.0.0.1 Request to /wsys/linux-tools/backup time: 6.583s
[2016-08-17 10:55:51 +0000] [14265] [ERROR] Error handling request /js/rhodecode/routes.js?ver=398d8f3b
Traceback (most recent call last):
File "/opt/rhodecode/store/rnklx8i8kclsz24q0nbfd959f9nlz1m8-python2.7-gunicorn-19.6.0/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 135, in handle
self.handle_request(listener, req, client, addr)
File "/opt/rhodecode/store/rnklx8i8kclsz24q0nbfd959f9nlz1m8-python2.7-gunicorn-19.6.0/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 176, in handle_request
respiter = self.wsgi(environ, resp.start_response)
File "/opt/rhodecode/store/miwpp0jyk99zbccj6vzwv4axjayp41cs-python2.7-Paste-2.0.2/lib/python2.7/site-packages/paste/gzipper.py", line 34, in __call__
response.gzip_start_response)
File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/middleware.py", line 80, in __call__
config.environ = environ
File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/__init__.py", line 22, in __setattr__
self.load_wsgi_environ(value)
File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/__init__.py", line 51, in load_wsgi_environ
result = mapper.routematch(path)
File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/mapper.py", line 692, in routematch
result = self._match(url, environ)
File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/mapper.py", line 646, in _match
domain_match)
File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/route.py", line 578, in match
not self.conditions['function'](environ, result):
File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/config/routing.py", line 151, in check_group
by_name_match = repo_group_model.get_by_group_name(repo_group_name)
File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/model/repo_group.py", line 76, in get_by_group_name
return repo.scalar()
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2429, in scalar
ret = self.one()
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2398, in one
ret = list(self)
File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/lib/caching_query.py", line 105, in __iter__
return Query.__iter__(self)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2441, in __iter__
return self._execute_and_instances(context)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2456, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 841, in execute
return meth(self, multiparams, params)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 322, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 938, in _execute_clauseelement
compiled_sql, distilled_params
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1070, in _execute_context
context)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1271, in _handle_dbapi_exception
exc_info
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1063, in _execute_context
context)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 442, in do_execute
cursor.execute(statement, parameters)
File "/opt/rhodecode/store/70w1dh4xc8q6m8avi9d90c5qn614cbaz-python2.7-MySQL-python-1.2.5/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
self.errorhandler(self, exc, value)
File "/opt/rhodecode/store/70w1dh4xc8q6m8avi9d90c5qn614cbaz-python2.7-MySQL-python-1.2.5/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
OperationalError: (OperationalError) (2006, 'MySQL server has gone away') 'SELECT groups.group_id AS groups_group_id, groups.group_name AS groups_group_name, groups.group_parent_id AS groups_group_parent_id, groups.group_description AS groups_group_description, groups.enable_locking AS groups_enable_locking, groups.user_id AS groups_user_id, groups.created_on AS groups_created_on \nFROM groups \nWHERE groups.group_name = %s ORDER BY group_name' ('js/rhodecode/routes.js',)
[2016-08-17 10:55:51 +0000] [14268] [ERROR] Error handling request /js/rhodecode/base/keyboard-bindings.js?ver=398d8f3b
Traceback (most recent call last):
File "/opt/rhodecode/store/rnklx8i8kclsz24q0nbfd959f9nlz1m8-python2.7-gunicorn-19.6.0/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 135, in handle
self.handle_request(listener, req, client, addr)
File "/opt/rhodecode/store/rnklx8i8kclsz24q0nbfd959f9nlz1m8-python2.7-gunicorn-19.6.0/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 176, in handle_request
respiter = self.wsgi(environ, resp.start_response)
File "/opt/rhodecode/store/miwpp0jyk99zbccj6vzwv4axjayp41cs-python2.7-Paste-2.0.2/lib/python2.7/site-packages/paste/gzipper.py", line 34, in __call__
response.gzip_start_response)
File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/middleware.py", line 80, in __call__
config.environ = environ
File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/__init__.py", line 22, in __setattr__
self.load_wsgi_environ(value)
File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/__init__.py", line 51, in load_wsgi_environ
result = mapper.routematch(path)
File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/mapper.py", line 692, in routematch
result = self._match(url, environ)
File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/mapper.py", line 646, in _match
domain_match)
File "/opt/rhodecode/store/i1ax7221bg9k9c9fd3pv3mrlnsvxc11f-python2.7-Routes-1.13/lib/python2.7/site-packages/routes/route.py", line 578, in match
not self.conditions['function'](environ, result):
File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/config/routing.py", line 151, in check_group
by_name_match = repo_group_model.get_by_group_name(repo_group_name)
File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/model/repo_group.py", line 76, in get_by_group_name
return repo.scalar()
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2429, in scalar
ret = self.one()
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2398, in one
ret = list(self)
File "/opt/rhodecode/store/0lg6aqzs7b4bs1x12s8i6ib5hqa285rj-python2.7-rhodecode-enterprise-ce-4.2.1/lib/python2.7/site-packages/rhodecode/lib/caching_query.py", line 105, in __iter__
return Query.__iter__(self)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2441, in __iter__
return self._execute_and_instances(context)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2456, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 841, in execute
return meth(self, multiparams, params)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 322, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 938, in _execute_clauseelement
compiled_sql, distilled_params
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1070, in _execute_context
context)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1271, in _handle_dbapi_exception
exc_info
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1063, in _execute_context
context)
File "/opt/rhodecode/store/rvbznw1hks4ix1pnlfmpfwh9sqmm287d-python2.7-SQLAlchemy-0.9.9/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 442, in do_execute
cursor.execute(statement, parameters)
File "/opt/rhodecode/store/70w1dh4xc8q6m8avi9d90c5qn614cbaz-python2.7-MySQL-python-1.2.5/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
self.errorhandler(self, exc, value)
File "/opt/rhodecode/store/70w1dh4xc8q6m8avi9d90c5qn614cbaz-python2.7-MySQL-python-1.2.5/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
</code></pre> RhodeCode CE/EE - Bug #4166: [ce] Rhodecode crashing after MySQL errorhttps://issues.rhodecode.com/issues/4166?journal_id=235022016-08-23T23:35:33ZMarcin Kuzminski [CTO]marcin@rhodecode.com
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Resolved</i></li><li><strong>Target version</strong> changed from <i>v4.4</i> to <i>v4.3</i></li></ul><p>We released 4.3.1 which should address this issue, and recycle RhodeCode connections properly</p>