Project

General

Profile

Actions

Bug #5651

open

Pull requests can get stuck if the diff is too large (it was created by mistake but we can't open it to delete it)

Added by Yechen Qiao over 3 years ago. Updated almost 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
Start date:
12.01.2021
Due date:
% Done:

0%

Estimated time:
Sorting:
Commit Number:
Affected Version:

Description

Affected Version: 4.22.0 (There is no such version in the affected version list so I picked the closest one)

It's a new year. So our copyright statement has been updated with 2021 on it. If a developer forget to merge before PR, then this causes huge amount of diffs and causes the following blowup:

2021-01-12 10:26:50.935 [23364] ERROR [rhodecode.config.middleware] error occurred handling this request for path: /internalproducts/sapiofoundations/sapiofoundations-clientside/pull-request/900
Traceback (most recent call last):
File "/opt/rhodecode/store/da2il1f83gci4333j0sd256nji9nlhpa-python2.7-pyramid-1.10.4/lib/python2.7/site-packages/pyramid/tweens.py", line 41, in excview_tween
response = handler(request)
File "/opt/rhodecode/store/da2il1f83gci4333j0sd256nji9nlhpa-python2.7-pyramid-1.10.4/lib/python2.7/site-packages/pyramid/router.py", line 148, in handle_request
registry, request, context, context_iface, view_name
File "/opt/rhodecode/store/da2il1f83gci4333j0sd256nji9nlhpa-python2.7-pyramid-1.10.4/lib/python2.7/site-packages/pyramid/view.py", line 667, in call_view
response = view_callable(context, request)
File "/opt/rhodecode/store/da2il1f83gci4333j0sd256nji9nlhpa-python2.7-pyramid-1.10.4/lib/python2.7/site-packages/pyramid/config/views.py", line 188, in attr_view
return view(context, request)
File "/opt/rhodecode/store/da2il1f83gci4333j0sd256nji9nlhpa-python2.7-pyramid-1.10.4/lib/python2.7/site-packages/pyramid/config/views.py", line 214, in predicate_wrapper
return view(context, request)
File "/opt/rhodecode/store/da2il1f83gci4333j0sd256nji9nlhpa-python2.7-pyramid-1.10.4/lib/python2.7/site-packages/pyramid/viewderivers.py", line 436, in rendered_view
result = view(context, request)
File "/opt/rhodecode/store/da2il1f83gci4333j0sd256nji9nlhpa-python2.7-pyramid-1.10.4/lib/python2.7/site-packages/pyramid/viewderivers.py", line 132, in _class_view
response = getattr(inst, attr)()
File "/opt/rhodecode/store/cnpzq6z5xjxzxw2m3d1cfb5yhpazscn5-python2.7-rhodecode-enterprise-ce-4.22.0/lib/python2.7/site-packages/rhodecode/lib/auth.py", line 2507, in local_wrapper
return wrapper(func, *args, **kwds)
File "/opt/rhodecode/store/cnpzq6z5xjxzxw2m3d1cfb5yhpazscn5-python2.7-rhodecode-enterprise-ce-4.22.0/lib/python2.7/site-packages/rhodecode/lib/auth.py", line 1823, in __wrapper
return func(*fargs, **fkwargs)
File "/opt/rhodecode/store/cnpzq6z5xjxzxw2m3d1cfb5yhpazscn5-python2.7-rhodecode-enterprise-ce-4.22.0/lib/python2.7/site-packages/rhodecode/lib/auth.py", line 2507, in local_wrapper
return wrapper(func, *args, **kwds)
File "/opt/rhodecode/store/cnpzq6z5xjxzxw2m3d1cfb5yhpazscn5-python2.7-rhodecode-enterprise-ce-4.22.0/lib/python2.7/site-packages/rhodecode/lib/auth.py", line 1897, in __wrapper
return func(*fargs, **fkwargs)
File "/opt/rhodecode/store/cnpzq6z5xjxzxw2m3d1cfb5yhpazscn5-python2.7-rhodecode-enterprise-ce-4.22.0/lib/python2.7/site-packages/rhodecode/apps/repository/views/repo_pull_requests.py", line 442, in pull_request_show
force_shadow_repo_refresh=force_refresh)
File "/opt/rhodecode/store/cnpzq6z5xjxzxw2m3d1cfb5yhpazscn5-python2.7-rhodecode-enterprise-ce-4.22.0/lib/python2.7/site-packages/rhodecode/model/pull_request.py", line 2164, in validate
force_shadow_repo_refresh=force_shadow_repo_refresh)
File "/opt/rhodecode/store/cnpzq6z5xjxzxw2m3d1cfb5yhpazscn5-python2.7-rhodecode-enterprise-ce-4.22.0/lib/python2.7/site-packages/rhodecode/model/pull_request.py", line 1694, in merge_status
pull_request, force_shadow_repo_refresh=force_shadow_repo_refresh)
File "/opt/rhodecode/store/cnpzq6z5xjxzxw2m3d1cfb5yhpazscn5-python2.7-rhodecode-enterprise-ce-4.22.0/lib/python2.7/site-packages/rhodecode/model/pull_request.py", line 1757, in _try_merge
pull_request, target_vcs, target_ref)
File "/opt/rhodecode/store/cnpzq6z5xjxzxw2m3d1cfb5yhpazscn5-python2.7-rhodecode-enterprise-ce-4.22.0/lib/python2.7/site-packages/rhodecode/model/pull_request.py", line 1821, in _refresh_merge_state
Session().commit()
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 1036, in commit
self.transaction.commit()
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 503, in commit
self._prepare_impl()
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 482, in _prepare_impl
self.session.flush()
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2496, in flush
self._flush(objects)
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2637, in _flush
transaction.rollback(_capture_exception=True)
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 69, in __exit
_
exc_value, with_traceback=exc_tb,
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2597, in flush
flush_context.execute()
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
rec.execute(self)
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 589, in execute
uow,
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 236, in save_obj
update,
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 995, in _emit_update_statements
statement, multiparams
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 984, in execute
return meth(self, multiparams, params)
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 293, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1103, in _execute_clauseelement
distilled_params,
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1288, in _execute_context
e, statement, parameters, cursor, context
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1482, in _handle_dbapi_exception
sqlalchemy_exception, with_traceback=exc_info[2], from
=e
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
cursor, statement, parameters, context
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 588, in do_execute
cursor.execute(statement, parameters)
File "/opt/rhodecode/store/qy6y6npssdfx02kg1cg66lgazr98lx1l-python2.7-mysql-python-1.2.5/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
self.errorhandler(self, exc, value)
File "/opt/rhodecode/store/qy6y6npssdfx02kg1cg66lgazr98lx1l-python2.7-mysql-python-1.2.5/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
DataError: (_mysql_exceptions.DataError) (1406, "Data too long for column 'last_merge_metadata' at row 1")
[SQL: UPDATE pull_requests SET last_merge_org_rev=%s, last_merge_other_rev=%s, merge_status=%s, last_merge_metadata=%s WHERE pull_requests.pull_request_id = %s]
[parameters: ('35076a5bccfe3d1904b164ca7afb23cff83d3900', '20798a817a6c488286d204c6be30d6ec17b74a57', 2, '{"unresolved_files": "\n* conflict: U gwtcps_attachmentViewer/src/main/java/com/velox/gwt/client/plugins/attachmentviewertask/callbacks/AttachmentVi ... (132434 characters truncated) ... ion.java\n * conflict: U workflowbuilderCore/src/main/java/com/velox/gwt/client/plugins/workflowbuilderlite/tag/WorkflowBuilderWorkflowOption.java"}', 900L)]
(Background on this error at: http://sqlalche.me/e/9h9h)

Actions #1

Updated by Yechen Qiao over 3 years ago

On pull request workflow: pull request has the yellow status "merging" and when attempting to open throws a 503 on web browser.

Actions #2

Updated by Marcin Kuzminski [CTO] over 3 years ago

I think we for 4.23.X optimized the metadata column and it shouldn't be so big that it would not fit into the DB. But that PR should be either deleted by API, or Merged by API. The 503 error is a timeout from worker, or your HTTP server that waits too long for rendering.
Alternative would be to use the rccontrol ishell to interact with this pr manually.

Actions #3

Updated by Marcin Kuzminski [CTO] over 3 years ago

  • Affected Version v4.22 added
  • Affected Version deleted (v4.20)
Actions #4

Updated by Yechen Qiao over 3 years ago

Hi,

Thanks for the quick response!

I have followed your suggestion above.

I can confirm this is also causing issue for the latest RhodeCode 4.23.2. I have updated it and I am still seeing the trace. I've attached the long stacktrace at the end of this message.

I tried to delete the PR via rccontrol. However, it seems to be failing. Maybe delete is not implemented for this object at database level?
toDelete = PullRequest.get(900)
PullRequest.delete(toDelete)

The following error happens when I hit enter.
UnmappedInstanceError Traceback (most recent call last)
/opt/rhodecode/store/5vmp7ipy3w2m4qii4fymlz8amzc9xd3k-python2.7-rhodecode-enterprise-ce-4.23.2/lib/python2.7/site-packages/rhodecode/lib/rc_commands/ishell.pyc in ()
----> 1 PullRequest.delete(toDelete)

/opt/rhodecode/store/5vmp7ipy3w2m4qii4fymlz8amzc9xd3k-python2.7-rhodecode-enterprise-ce-4.23.2/lib/python2.7/site-packages/rhodecode/model/db.pyc in delete(cls, id_)
293 def delete(cls, id_):
294 obj = cls.query().get(id_)
--> 295 Session().delete(obj)
296
297 @classmethod

/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/session.pyc in delete(self, instance)
2013 except exc.NO_STATE as err:
2014 util.raise_(
-> 2015 exc.UnmappedInstanceError(instance), replace_context=err,
2016 )
2017

/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/util/compat.pyc in raise_(exception, with_traceback, replace_context, from_)

2021-01-12 18:29:21.981 [13523] ERROR [rhodecode.config.middleware] error occurred handling this request for path: /internalproducts/sapiofoundations/sapiofoundations-clientside/pull-request/900
Traceback (most recent call last):
File "/opt/rhodecode/store/da2il1f83gci4333j0sd256nji9nlhpa-python2.7-pyramid-1.10.4/lib/python2.7/site-packages/pyramid/tweens.py", line 41, in excview_tween
response = handler(request)
File "/opt/rhodecode/store/da2il1f83gci4333j0sd256nji9nlhpa-python2.7-pyramid-1.10.4/lib/python2.7/site-packages/pyramid/router.py", line 148, in handle_request
registry, request, context, context_iface, view_name
File "/opt/rhodecode/store/da2il1f83gci4333j0sd256nji9nlhpa-python2.7-pyramid-1.10.4/lib/python2.7/site-packages/pyramid/view.py", line 667, in call_view
response = view_callable(context, request)
File "/opt/rhodecode/store/da2il1f83gci4333j0sd256nji9nlhpa-python2.7-pyramid-1.10.4/lib/python2.7/site-packages/pyramid/config/views.py", line 188, in attr_view
return view(context, request)
File "/opt/rhodecode/store/da2il1f83gci4333j0sd256nji9nlhpa-python2.7-pyramid-1.10.4/lib/python2.7/site-packages/pyramid/config/views.py", line 214, in predicate_wrapper
return view(context, request)
File "/opt/rhodecode/store/da2il1f83gci4333j0sd256nji9nlhpa-python2.7-pyramid-1.10.4/lib/python2.7/site-packages/pyramid/viewderivers.py", line 436, in rendered_view
result = view(context, request)
File "/opt/rhodecode/store/da2il1f83gci4333j0sd256nji9nlhpa-python2.7-pyramid-1.10.4/lib/python2.7/site-packages/pyramid/viewderivers.py", line 132, in _class_view
response = getattr(inst, attr)()
File "/opt/rhodecode/store/5vmp7ipy3w2m4qii4fymlz8amzc9xd3k-python2.7-rhodecode-enterprise-ce-4.23.2/lib/python2.7/site-packages/rhodecode/lib/auth.py", line 2507, in local_wrapper
return wrapper(func, *args, **kwds)
File "/opt/rhodecode/store/5vmp7ipy3w2m4qii4fymlz8amzc9xd3k-python2.7-rhodecode-enterprise-ce-4.23.2/lib/python2.7/site-packages/rhodecode/lib/auth.py", line 1823, in __wrapper
return func(*fargs, **fkwargs)
File "/opt/rhodecode/store/5vmp7ipy3w2m4qii4fymlz8amzc9xd3k-python2.7-rhodecode-enterprise-ce-4.23.2/lib/python2.7/site-packages/rhodecode/lib/auth.py", line 2507, in local_wrapper
return wrapper(func, *args, **kwds)
File "/opt/rhodecode/store/5vmp7ipy3w2m4qii4fymlz8amzc9xd3k-python2.7-rhodecode-enterprise-ce-4.23.2/lib/python2.7/site-packages/rhodecode/lib/auth.py", line 1897, in __wrapper
return func(*fargs, **fkwargs)
File "/opt/rhodecode/store/5vmp7ipy3w2m4qii4fymlz8amzc9xd3k-python2.7-rhodecode-enterprise-ce-4.23.2/lib/python2.7/site-packages/rhodecode/apps/repository/views/repo_pull_requests.py", line 448, in pull_request_show
force_shadow_repo_refresh=force_refresh)
File "/opt/rhodecode/store/5vmp7ipy3w2m4qii4fymlz8amzc9xd3k-python2.7-rhodecode-enterprise-ce-4.23.2/lib/python2.7/site-packages/rhodecode/model/pull_request.py", line 2160, in validate
force_shadow_repo_refresh=force_shadow_repo_refresh)
File "/opt/rhodecode/store/5vmp7ipy3w2m4qii4fymlz8amzc9xd3k-python2.7-rhodecode-enterprise-ce-4.23.2/lib/python2.7/site-packages/rhodecode/model/pull_request.py", line 1687, in merge_status
pull_request, force_shadow_repo_refresh=force_shadow_repo_refresh)
File "/opt/rhodecode/store/5vmp7ipy3w2m4qii4fymlz8amzc9xd3k-python2.7-rhodecode-enterprise-ce-4.23.2/lib/python2.7/site-packages/rhodecode/model/pull_request.py", line 1750, in _try_merge
pull_request, target_vcs, target_ref)
File "/opt/rhodecode/store/5vmp7ipy3w2m4qii4fymlz8amzc9xd3k-python2.7-rhodecode-enterprise-ce-4.23.2/lib/python2.7/site-packages/rhodecode/model/pull_request.py", line 1814, in _refresh_merge_state
Session().commit()
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 1036, in commit
self.transaction.commit()
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 503, in commit
self._prepare_impl()
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 482, in _prepare_impl
self.session.flush()
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2496, in flush
self._flush(objects)
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2637, in _flush
transaction.rollback(_capture_exception=True)
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 69, in __exit
_
exc_value, with_traceback=exc_tb,
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2597, in flush
flush_context.execute()
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
rec.execute(self)
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 589, in execute
uow,
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 236, in save_obj
update,
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 995, in _emit_update_statements
statement, multiparams
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 984, in execute
return meth(self, multiparams, params)
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 293, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1103, in _execute_clauseelement
distilled_params,
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1288, in _execute_context
e, statement, parameters, cursor, context
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1482, in _handle_dbapi_exception
sqlalchemy_exception, with_traceback=exc_info[2], from
=e
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
cursor, statement, parameters, context
File "/opt/rhodecode/store/d9q26gqyfrarins50zxxq1c611alijr2-python2.7-sqlalchemy-1.3.15/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 588, in do_execute
cursor.execute(statement, parameters)
File "/opt/rhodecode/store/qy6y6npssdfx02kg1cg66lgazr98lx1l-python2.7-mysql-python-1.2.5/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
self.errorhandler(self, exc, value)
File "/opt/rhodecode/store/qy6y6npssdfx02kg1cg66lgazr98lx1l-python2.7-mysql-python-1.2.5/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
DataError: (_mysql_exceptions.DataError) (1406, "Data too long for column 'last_merge_metadata' at row 1")

Actions #5

Updated by Yechen Qiao about 3 years ago

Can confirm this is fixed at 4.24.1.

Delete is working.
I can also delete the broken pull request directly from UI. It no longer throws exception at those diffs.

Actions #6

Updated by Redmine Integration almost 3 years ago

  • Status changed from New to Resolved
Actions

Also available in: Atom PDF