10 Commits

Author	SHA1	Message	Date
Simon	d7ce1670a8	Logging and bugfix for http crawler	2018-06-25 14:36:16 -04:00
Simon	f6ee338c0f	Removed unused statement	2018-06-24 18:24:41 -04:00
Simon	e11343de23	More FTP crawler bug fixes	2018-06-24 18:05:30 -04:00
Simon	ab35ce96cc	FTP crawler bug fixes	2018-06-24 16:44:21 -04:00
Simon	8e937e69c0	Should fix some FTP errors	2018-06-24 13:50:55 -04:00
Simon	073551df3c	Attempt to handle looping directories	2018-06-21 11:54:40 -04:00
Simon	dd93d40a55	Small bugfix for ftp crawler	2018-06-20 21:56:38 -04:00
Simon	7923647ea3	Made the ftp crawler work with the latest changes	2018-06-18 15:46:03 -04:00
Simon	1bd58468eb	Bug fixes for FTP crawler	2018-06-13 15:54:45 -04:00
Simon	d61fd75890	Tasks can now be queued from the web interface. Tasks are dispatched to the crawl server(s)	2018-06-12 13:44:03 -04:00