16 Commits

Author	SHA1	Message	Date
simon987	b9f25630b4	Switch to postgresql, finish minimum viable task_tracker/ws_bucket integration	2019-03-27 19:34:05 -04:00
simon987	00e3fd7340	Remove task tracking	2019-03-09 13:26:05 -05:00
simon987	7f857d641f	Change ES settings, big refactor, removed recaptcha	2019-01-13 12:48:39 -05:00
Simon	6e80791264	Search filter	2018-11-16 16:49:23 -05:00
Simon	fe1d29aaea	Crawl tasks are now fetched by the crawlers instead of pushed by the server	2018-07-14 17:31:18 -04:00
Simon	5383ad6aea	Searches are not saved to database	2018-06-27 15:29:50 -04:00
Simon	5fd00f22af	Task logs now stored on main server	2018-06-24 20:32:02 -04:00
Simon	14d384e366	Decentralised crawling should work in theory + temporary fix for going further than the maximum 10k results elasticsearch allows by default	2018-06-21 19:44:27 -04:00
Simon	7400bdc2a9	Added admin blacklist control in dashboard	2018-06-20 11:28:06 -04:00
Simon	d61fd75890	Tasks can now be queued from the web interface. Tasks are dispatched to the crawl server(s)	2018-06-12 13:44:03 -04:00
Simon	a25976d24a	Generate and delete API tokens	2018-06-09 12:41:28 -04:00
Simon	dc0cde61a0	Basic admin page	2018-06-08 11:40:54 -04:00
Simon	4f6d7f32ad	Option to turn off SSL, moved secret keys to config.py, switched to sqlite WAL mode to avoid locked database problems	2018-06-07 10:33:35 -04:00
Simon	270ab1335a	Added reply to comments option, fixed some bugs	2018-06-02 17:26:15 -04:00
Simon	bb872a9248	Changed from mime to extension for graph and added script to clear invalid websites	2018-05-31 10:51:59 -04:00
Simon	ad645490f6	Initial commit	2018-05-28 20:35:04 -04:00