Project

General

Profile

Task #4120

[ce] replace get_repo_nodes api

Added by Daniel D about 3 years ago. Updated over 1 year ago.

Status:
New
Priority:
Normal
Assignee:
Category:
-
Target version:
Start date:
24.07.2016
Due date:
% Done:

0%

Estimated time:
Sorting:
Commit Number:

Description

The current get_repo_nodes api loads the entire repo in one go, for a large repo (gbs) this will result in an output at least the size of the repo, which can cause memory issues, currently issue #4114 adds a max_file_bytes to work around this issue for the full text search indexer but a more long term solution is required:

Splitting get_repo_nodes into 2 actions, one to get the nodes, another to get node data should solve this, also since getting node content one by one would be slow it would be a good idea to allow batch node retrieval.

The methods required would go along the lines of:

get_repo_nodes(repo_name, version, node_paths)
=>[
{'path': '/README', 'size': 434},
{'path': '.gitconfig', 'size': 53},
{'path': '/some/file', 'size': 5223},
{'path': '/another/file', 'size': 6433}
]

get_node_content('somerepo', 'tip', ['/README', '/some/file']) =>  contents of those 2 files only

History

#1 Updated by Marcin Kuzminski [staff] about 3 years ago

  • Target version changed from v4.4 to v4.5

#2 Updated by Marcin Kuzminski [staff] almost 3 years ago

  • Target version changed from v4.5 to v4.6

#3 Updated by Marcin Kuzminski [staff] over 2 years ago

  • Target version changed from v4.6 to v4.7

#4 Updated by Marcin Kuzminski [staff] over 2 years ago

  • Target version changed from v4.7 to v4.8

#5 Updated by Marcin Kuzminski [staff] over 2 years ago

  • Target version changed from v4.8 to v4.10

#6 Updated by Marcin Kuzminski [staff] about 2 years ago

  • Target version changed from v4.10 to v4.11

#7 Updated by Marcin Kuzminski [staff] over 1 year ago

  • Target version changed from v4.11 to v4.13

Also available in: Atom PDF