simon987/od-database
1
0
Fork 0
mirror of https://github.com/simon987/od-database.git synced 2025-12-14 07:09:03 +00:00
Code Issues Projects Releases Wiki Activity

Crawl tasks are now fetched by the crawlers instead of pushed by the server

Browse Source
This commit is contained in:
Simon
2018-07-14 17:31:18 -04:00
parent d9e9f53f92
commit fe1d29aaea
20 changed files with 376 additions and 749 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
crawl_server/run.py Normal file
View File

@@ -0,0 +1,8 @@
from crawl_server.task_manager import TaskManager
import time
import config
tm = TaskManager(config.CRAWL_SERVER_PROCESSES)
while True:
time.sleep(1)