10 Commits

Author SHA1 Message Date
Simon
d7ce1670a8 Logging and bugfix for http crawler 2018-06-25 14:36:16 -04:00
Simon
f6ee338c0f Removed unused statement 2018-06-24 18:24:41 -04:00
Simon
e11343de23 More FTP crawler bug fixes 2018-06-24 18:05:30 -04:00
Simon
ab35ce96cc FTP crawler bug fixes 2018-06-24 16:44:21 -04:00
Simon
8e937e69c0 Should fix some FTP errors 2018-06-24 13:50:55 -04:00
Simon
073551df3c Attempt to handle looping directories 2018-06-21 11:54:40 -04:00
Simon
dd93d40a55 Small bugfix for ftp crawler 2018-06-20 21:56:38 -04:00
Simon
7923647ea3 Made the ftp crawler work with the latest changes 2018-06-18 15:46:03 -04:00
Simon
1bd58468eb Bug fixes for FTP crawler 2018-06-13 15:54:45 -04:00
Simon
d61fd75890 Tasks can now be queued from the web interface. Tasks are dispatched to the crawl server(s) 2018-06-12 13:44:03 -04:00