Simon cc4c70f400 Request content is read all at once

2018-08-11 13:05:24 -04:00

..

crawled

Should fix memory usage problem when crawling

2018-06-14 23:36:54 -04:00

__init__.py

Crawl tasks are now fetched by the crawlers instead of pushed by the server

2018-07-14 17:31:18 -04:00

crawler.py

Fix for negative sizes

2018-07-25 11:37:12 -04:00

remote_ftp.py

Logging and bugfix for http crawler

2018-06-25 14:36:16 -04:00

remote_http.py

Request content is read all at once

2018-08-11 13:05:24 -04:00

run.py

Temporarily removed logger in async methods (https://stackoverflow.com/questions/37907350)

2018-07-15 10:35:13 -04:00

task_manager.py

exceptions during push_result are logged instead of raised

2018-07-21 10:45:17 -04:00