simon987/od-database

Fork 0

mirror of https://github.com/simon987/od-database.git synced 2025-04-10 14:06:45 +00:00

Go to file

Simon Fortier 92a91606ea

Create LICENSE

2018-06-07 10:36:26 -04:00

scrapy_od_database

Added requirement

2018-06-03 10:49:11 -04:00

static

Added blacklist feature (untested)

2018-06-06 10:17:30 -04:00

templates

File size format for export

2018-06-06 11:28:44 -04:00

test

Initial commit

2018-05-28 20:35:04 -04:00

__init__.py

Initial commit

2018-05-28 20:35:04 -04:00

.gitignore

Updated link

2018-06-06 13:23:23 -04:00

app.py

Option to turn off SSL, moved secret keys to config.py, switched to sqlite WAL mode to avoid locked database problems

2018-06-07 10:33:35 -04:00

blacklist.txt

Added blacklist feature (untested)

2018-06-06 10:17:30 -04:00

clean_invalid_websites.py

Changed from mime to extension for graph and added script to clear invalid websites

2018-05-31 10:51:59 -04:00

database.py

Added blacklist feature (untested)

2018-06-06 10:17:30 -04:00

export.sh

Exports are downloadable

2018-06-06 11:12:24 -04:00

export.sql

Changed from mime to extension for graph and added script to clear invalid websites

2018-05-31 10:51:59 -04:00

init_script.sql

Option to turn off SSL, moved secret keys to config.py, switched to sqlite WAL mode to avoid locked database problems

2018-06-07 10:33:35 -04:00

LICENSE

Create LICENSE

2018-06-07 10:36:26 -04:00

od_util.py

Added blacklist feature (untested)

2018-06-06 10:17:30 -04:00

queue_reddit_links.py

Should fix problem when comment/post with subdir doesn't exist pt. 2

2018-06-06 19:40:08 -04:00

README.md

Option to turn off SSL, moved secret keys to config.py, switched to sqlite WAL mode to avoid locked database problems

2018-06-07 10:33:35 -04:00

reddit_bot.py

Minor adjustments

2018-06-06 14:15:06 -04:00

requirements.txt

Added requirement

2018-06-03 10:49:11 -04:00

scrapy.cfg

Renamed package (again) and removed unused files

2018-05-31 08:30:08 -04:00

task.py

Mark reddit post as crawled even if too small to comment

2018-06-06 16:01:07 -04:00

update_website.py

Initial commit

2018-05-28 20:35:04 -04:00

README.md

OD-Database

Suggestions/concerns/PRs are welcome

Installation

Assuming you have Python 3 and git installed:

git clone https://github.com/simon987/od-database
cd od-database
pip3 install -r requirements.txt

Create /config.py and fill out the parameters. Empty config:

CAPTCHA_SITE_KEY = ""
CAPTCHA_SECRET_KEY = ""
FLASK_SECRET = ""
USE_SSL = True

Running

python3 app.py

You should be able to connect with your browser at https://localhost:12345

*Note: To use SSL you have to put the appropriate certificates in /certificates/