Simon 4b1fce309c Mark reddit post as crawled even if too small to comment

2018-06-06 16:01:07 -04:00

scrapy_od_database

Added requirement

2018-06-03 10:49:11 -04:00

static

Added blacklist feature (untested)

2018-06-06 10:17:30 -04:00

templates

File size format for export

2018-06-06 11:28:44 -04:00

test

Initial commit

2018-05-28 20:35:04 -04:00

__init__.py

Initial commit

2018-05-28 20:35:04 -04:00

.gitignore

Updated link

2018-06-06 13:23:23 -04:00

app.py

SSL support pt.2

2018-06-06 11:35:55 -04:00

blacklist.txt

Added blacklist feature (untested)

2018-06-06 10:17:30 -04:00

clean_invalid_websites.py

Changed from mime to extension for graph and added script to clear invalid websites

2018-05-31 10:51:59 -04:00

database.py

Added blacklist feature (untested)

2018-06-06 10:17:30 -04:00

export.sh

Exports are downloadable

2018-06-06 11:12:24 -04:00

export.sql

Changed from mime to extension for graph and added script to clear invalid websites

2018-05-31 10:51:59 -04:00

init_script.sql

Added reply to comments option, fixed some bugs

2018-06-02 17:26:15 -04:00

od_util.py

Added blacklist feature (untested)

2018-06-06 10:17:30 -04:00

queue_reddit_links.py

/r/test -> /r/opendirectories

2018-06-06 15:31:35 -04:00

reddit_bot.py

Minor adjustments

2018-06-06 14:15:06 -04:00

requirements.txt

Added requirement

2018-06-03 10:49:11 -04:00

scrapy.cfg

Renamed package (again) and removed unused files

2018-05-31 08:30:08 -04:00

task.py

Mark reddit post as crawled even if too small to comment

2018-06-06 16:01:07 -04:00

update_website.py

Initial commit

2018-05-28 20:35:04 -04:00

Description

No description provided

bootstrapelasticsearchscraping

MIT 2 MiB

Languages

Python 43%

CSS 29.4%

HTML 19.2%

JavaScript 8.3%