simon987/od-database
1
0
Fork 0
mirror of https://github.com/simon987/od-database.git synced 2025-12-16 08:09:04 +00:00
Code Issues Projects Releases Wiki Activity

Should fix memory usage problem when crawling

Browse Source
This commit is contained in:
Simon
2018-06-14 23:36:54 -04:00
parent 9aed18c2d2
commit adb94cf326
6 changed files with 46 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
search/search.py
View File

@@ -107,7 +107,8 @@ class ElasticSearchEngine(SearchEngine):
if len(docs) >= import_every:
self._index(docs)
docs.clear()
self._index(docs)
if docs:
self._index(docs)
def _index(self, docs):
print("Indexing " + str(len(docs)) + " docs")