Bug #5633
open
Moderately large pull requests fail because inefficient use of reviewer_data_json column in pull_requests table
Added by Yechen Qiao over 3 years ago.
Updated over 3 years ago.
Description
When we are trying to merge sufficiently large number of changes, internal service error occurs while attempting to make the pull requests.
On further investigation, it is because reviewer_data_json is too large.
But most of the space taken is due to user's base64 encoded images. The text in there easily fits if I remove the image data.
Is there a purpose why those user images are stored there? If not, would it be better to simply not store user images at all and parse them in real-time? :)
"But most of the space taken is due to user's base64 encoded images." to clarify this seems to be user photo retrieved from rhodecode, which seems to not need to be stored.
- Status changed from New to In Progress
Hi,
The reason why those images are there is that there are two backends for the user images: gravatar, which is a link to the user gravatar, or the built-in initials gravatar. In the case of reviewer data we store only the LINK to the image, but because Previously RhodeCode didn't have a way to store artifacts, the links in case of the second backend were actually built base64 images.
Since now we have an artifact store, i guess it would be a good time to replace the on-the-fly generated base64 svg images to an actually stored link in the artifacts.
But it's odd that the DB cannot fit the data in that column, maybe there's some problem with the DB and the column limit ? I'm guessing you're using MySQL as a backend which has those odd limitations for some columns.
Yes exactly! we are using MariaDB. The column has TEXT type. If we increase to mediumtext manually store is fine but when retrieve it causes extreme slowdown to load the closed tab of the pull request list in project. Until eventually this simply became an Ajax error when loading it. We had to revert back by altering the column to have no reviewer info for the offending PRs.
Interestingly after doing so reviewer pane still seems to have all the info it had before?
I Think you'd need to stop all traffic from RhodeCode (stop it completly) and then do the column alter
- Status changed from In Progress to Resolved
Also available in: Atom
PDF
Updated by 
Updated by 
Updated by