6 Commits

Author	SHA1	Message	Date
Simon	d849227798	barebones crawl_server microservice	2018-06-11 19:00:43 -04:00
Simon	0304c98a31	Added basic ftp spider for scrapy	2018-06-10 14:12:55 -04:00
Simon	4b1fce309c	Mark reddit post as crawled even if too small to comment	2018-06-06 16:01:07 -04:00
Simon	270ab1335a	Added reply to comments option, fixed some bugs	2018-06-02 17:26:15 -04:00
Simon	bb872a9248	Changed from mime to extension for graph and added script to clear invalid websites	2018-05-31 10:51:59 -04:00
Simon	ad645490f6	Initial commit	2018-05-28 20:35:04 -04:00