Bug #5633
open
Moderately large pull requests fail because inefficient use of reviewer_data_json column in pull_requests table
0%
Description
When we are trying to merge sufficiently large number of changes, internal service error occurs while attempting to make the pull requests.
On further investigation, it is because reviewer_data_json is too large.
But most of the space taken is due to user's base64 encoded images. The text in there easily fits if I remove the image data.
Is there a purpose why those user images are stored there? If not, would it be better to simply not store user images at all and parse them in real-time? :)
Updated by Yechen Qiao over 4 years ago
"But most of the space taken is due to user's base64 encoded images." to clarify this seems to be user photo retrieved from rhodecode, which seems to not need to be stored.
Updated by Marcin Kuzminski [CTO] over 4 years ago
- Status changed from New to In Progress
Hi,
The reason why those images are there is that there are two backends for the user images: gravatar, which is a link to the user gravatar, or the built-in initials gravatar. In the case of reviewer data we store only the LINK to the image, but because Previously RhodeCode didn't have a way to store artifacts, the links in case of the second backend were actually built base64 images.
Since now we have an artifact store, i guess it would be a good time to replace the on-the-fly generated base64 svg images to an actually stored link in the artifacts.
But it's odd that the DB cannot fit the data in that column, maybe there's some problem with the DB and the column limit ? I'm guessing you're using MySQL as a backend which has those odd limitations for some columns.
Updated by Yechen Qiao over 4 years ago
Yes exactly! we are using MariaDB. The column has TEXT type. If we increase to mediumtext manually store is fine but when retrieve it causes extreme slowdown to load the closed tab of the pull request list in project. Until eventually this simply became an Ajax error when loading it. We had to revert back by altering the column to have no reviewer info for the offending PRs.
Interestingly after doing so reviewer pane still seems to have all the info it had before?
Updated by Marcin Kuzminski [CTO] over 4 years ago
I Think you'd need to stop all traffic from RhodeCode (stop it completly) and then do the column alter
Updated by Redmine Integration about 4 years ago
- Status changed from In Progress to Resolved
Commit b5299f6ded6a by Marcin Kuzminski marcin@rhodecode.com on stable branch changed this issue.
https://code.rhodecode.com/rhodecode-enterprise-ce/changeset/b5299f6ded6a6201fce67f03369db52de0016fb1