simon987/od-database
1
0
Fork 0
mirror of https://github.com/simon987/od-database.git synced 2025-12-13 23:09:01 +00:00
Code Issues Projects Releases Wiki Activity

Crawler no longer crashes when website has no files

Browse Source
This commit is contained in:
Simon
2018-07-15 10:46:48 -04:00
parent e18ded7ac1
commit 112400886e
3 changed files with 15 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
tasks.py
View File

@@ -63,9 +63,9 @@ class TaskManager:
def complete_task(self, file_list, task, task_result, crawler_name):
if file_list:
self.search.delete_docs(task_result.website_id)
self.search.delete_docs(task_result.website_id)
if file_list:
def iter_lines():
with open(file_list, "r") as f: