Bug #5622

Upgrade from 4.18.3 to 4.19.3 broke all PRs

Added by Sebastian Elsner about 2 years ago. Updated almost 2 years ago.

Target version:
Start date:
Due date:
% Done:


Estimated time:
Commit Number:
Affected Version:


After the upgrade I get a Server Error 500 and basically identical tracebacks looking like this. I revered the container back afterwards.

Exceptions Tracker - Exception ID: 139841427643760
Exception `CommitDoesNotExistError` generated on UTC date: 2020-06-15T18:42:16.671343
Request: GET http://rhodecodekit.rise:10020/libpython/pull-request/448 Mozilla/5.0 (X11; Linux x86_64; rv:72.0) Gecko/20100101 Firefox/72.0

Traceback (most recent call last):
  File "/opt/rhodecode/store/da2il1f83gci4333j0sd256nji9nlhpa-python2.7-pyramid-1.10.4/lib/python2.7/site-packages/pyramid/", line 41, in excview_tween
    response = handler(request)
  File "/opt/rhodecode/store/da2il1f83gci4333j0sd256nji9nlhpa-python2.7-pyramid-1.10.4/lib/python2.7/site-packages/pyramid/", line 148, in handle_request
    registry, request, context, context_iface, view_name
  File "/opt/rhodecode/store/da2il1f83gci4333j0sd256nji9nlhpa-python2.7-pyramid-1.10.4/lib/python2.7/site-packages/pyramid/", line 667, in _call_view
    response = view_callable(context, request)
  File "/opt/rhodecode/store/da2il1f83gci4333j0sd256nji9nlhpa-python2.7-pyramid-1.10.4/lib/python2.7/site-packages/pyramid/config/", line 188, in attr_view
    return view(context, request)
  File "/opt/rhodecode/store/da2il1f83gci4333j0sd256nji9nlhpa-python2.7-pyramid-1.10.4/lib/python2.7/site-packages/pyramid/config/", line 214, in predicate_wrapper
    return view(context, request)
  File "/opt/rhodecode/store/da2il1f83gci4333j0sd256nji9nlhpa-python2.7-pyramid-1.10.4/lib/python2.7/site-packages/pyramid/", line 436, in rendered_view
    result = view(context, request)
  File "/opt/rhodecode/store/da2il1f83gci4333j0sd256nji9nlhpa-python2.7-pyramid-1.10.4/lib/python2.7/site-packages/pyramid/", line 132, in _class_view
    response = getattr(inst, attr)()
  File "/opt/rhodecode/store/avpzwi69pbzni741f4hwp8isizcm8m8g-python2.7-rhodecode-enterprise-ce-4.19.3/lib/python2.7/site-packages/rhodecode/lib/", line 2498, in local_wrapper
    return wrapper(func, *args, **kwds)
  File "/opt/rhodecode/store/avpzwi69pbzni741f4hwp8isizcm8m8g-python2.7-rhodecode-enterprise-ce-4.19.3/lib/python2.7/site-packages/rhodecode/lib/", line 1814, in __wrapper
    return func(*fargs, **fkwargs)
  File "/opt/rhodecode/store/avpzwi69pbzni741f4hwp8isizcm8m8g-python2.7-rhodecode-enterprise-ce-4.19.3/lib/python2.7/site-packages/rhodecode/lib/", line 2498, in local_wrapper
    return wrapper(func, *args, **kwds)
  File "/opt/rhodecode/store/avpzwi69pbzni741f4hwp8isizcm8m8g-python2.7-rhodecode-enterprise-ce-4.19.3/lib/python2.7/site-packages/rhodecode/lib/", line 1888, in __wrapper
    return func(*fargs, **fkwargs)
  File "/opt/rhodecode/store/avpzwi69pbzni741f4hwp8isizcm8m8g-python2.7-rhodecode-enterprise-ce-4.19.3/lib/python2.7/site-packages/rhodecode/apps/repository/views/", line 605, in pull_request_show
    hide_whitespace_changes, diff_context)
  File "/opt/rhodecode/store/avpzwi69pbzni741f4hwp8isizcm8m8g-python2.7-rhodecode-enterprise-ce-4.19.3/lib/python2.7/site-packages/rhodecode/apps/repository/views/", line 221, in _get_diffset
    hide_whitespace_changes, diff_context)
  File "/opt/rhodecode/store/avpzwi69pbzni741f4hwp8isizcm8m8g-python2.7-rhodecode-enterprise-ce-4.19.3/lib/python2.7/site-packages/rhodecode/model/", line 1787, in get_diff
    hide_whitespace_changes=hide_whitespace_changes, diff_context=diff_context)
  File "/opt/rhodecode/store/avpzwi69pbzni741f4hwp8isizcm8m8g-python2.7-rhodecode-enterprise-ce-4.19.3/lib/python2.7/site-packages/rhodecode/model/", line 1794, in _get_diff_from_pr_or_version
  File "/opt/rhodecode/store/avpzwi69pbzni741f4hwp8isizcm8m8g-python2.7-rhodecode-enterprise-ce-4.19.3/lib/python2.7/site-packages/rhodecode/lib/vcs/backends/hg/", line 478, in get_commit
    raise CommitDoesNotExistError(msg)
CommitDoesNotExistError: Commit 8df8c32ad7d2581f630729f8e7f4d7df7e979863 does not exist for `.__shadow_repo_2_pr-448`


community.log.tar.gz (2.07 MB) community.log.tar.gz Sebastian Elsner, 16.06.2020 18:39

Updated by Marcin Kuzminski [CTO] about 2 years ago

  • Status changed from New to In Progress
  • Priority changed from Normal to High


Does the pull request open, or it's raising the 500 error? If it opens, please try force update.

If it fails to open, could you try to update the PR using the API ?

We assume new PRs work ok?


Updated by Sebastian Elsner about 2 years ago

The PR does not open, it raises 500.

New PRs seem to work

I cannot get the force update working, I have not used the API before

root@rhodecodekittest:~# .rccontrol/community-1/profile/bin/rhodecode-api --instance-name=community-1 update_pull_request pullrequestid:448
ERROR: Failed to connect to backend at:``, make sure it is running and accepting connections.
Traceback (most recent call last):
  File "/opt/rhodecode/store/8qrsy8va6pmh2ys9jkq69va511dan444-python2.7-rhodecode-tools-1.4.0/lib/python2.7/site-packages/rhodecode_tools/lib/", line 176, in _api_call
  File "/opt/rhodecode/store/wz9gh57g4zk6aklc1b6d996l50az2kkx-python2.7-requests-2.22.0/lib/python2.7/site-packages/requests/", line 116, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/opt/rhodecode/store/wz9gh57g4zk6aklc1b6d996l50az2kkx-python2.7-requests-2.22.0/lib/python2.7/site-packages/requests/", line 60, in request
    return session.request(method=method, url=url, **kwargs)
  File "/opt/rhodecode/store/wz9gh57g4zk6aklc1b6d996l50az2kkx-python2.7-requests-2.22.0/lib/python2.7/site-packages/requests/", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/opt/rhodecode/store/wz9gh57g4zk6aklc1b6d996l50az2kkx-python2.7-requests-2.22.0/lib/python2.7/site-packages/requests/", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/opt/rhodecode/store/wz9gh57g4zk6aklc1b6d996l50az2kkx-python2.7-requests-2.22.0/lib/python2.7/site-packages/requests/", line 514, in send
    raise SSLError(e, request=request)
SSLError: HTTPSConnectionPool(host='', port=10020): Max retries exceeded with url: /_admin/api (Caused by SSLError(SSLEOFError(8, u'EOF occurred in violation of protocol (_ssl.c:726)'),))

I have googled the error, but got stuck at "its a python bug" or "your ssl is not on" (which it is). Do you have an idea?


Updated by Marcin Kuzminski [CTO] about 2 years ago

If you connect directly to the 10020 port you cannot use HTTPS, simply do HTTP call to that IP.

Also, could you provide a full detailed logs, and exception logged into the logs for reference ?

We have to investigate this issue in depth, since we cannot reproduce this on our set of 100s of test PRs we used to validate the 4.19.X changes.


Updated by Sebastian Elsner about 2 years ago

Updating PR with "update_commits" via the API works, so that is a workaround.

We have a very basic installation, Database ist SQLite, no fancy settings or workflows.

I attached the logs of community-1. Oldest entry is 2 days ago. In the middle there is some fuzz about the host not being found, this is because I worked on clone of the container rhode is running in and the IP of that clone changed, so there are errors until I fixed the ".ini"s of rhode. Pls ignore. Migration was done this morning, the 16th, you can see the startup in the logs.


Updated by Redmine Integration almost 2 years ago

  • Status changed from In Progress to Resolved

Updated by Sebastian Elsner almost 2 years ago

Thank you for the "workaround fix" in the last release :)

Also available in: Atom PDF