simon987/od-database

Fork 0

mirror of https://github.com/simon987/od-database.git synced 2025-04-18 18:06:44 +00:00

Go to file

Simon Fortier 460357f183

Create README.md

2018-06-07 10:01:33 -04:00

scrapy_od_database

Added requirement

2018-06-03 10:49:11 -04:00

static

Added blacklist feature (untested)

2018-06-06 10:17:30 -04:00

templates

File size format for export

2018-06-06 11:28:44 -04:00

test

Initial commit

2018-05-28 20:35:04 -04:00

__init__.py

Initial commit

2018-05-28 20:35:04 -04:00

.gitignore

Updated link

2018-06-06 13:23:23 -04:00

app.py

SSL support pt.2

2018-06-06 11:35:55 -04:00

blacklist.txt

Added blacklist feature (untested)

2018-06-06 10:17:30 -04:00

clean_invalid_websites.py

Changed from mime to extension for graph and added script to clear invalid websites

2018-05-31 10:51:59 -04:00

database.py

Added blacklist feature (untested)

2018-06-06 10:17:30 -04:00

export.sh

Exports are downloadable

2018-06-06 11:12:24 -04:00

export.sql

Changed from mime to extension for graph and added script to clear invalid websites

2018-05-31 10:51:59 -04:00

init_script.sql

Added reply to comments option, fixed some bugs

2018-06-02 17:26:15 -04:00

od_util.py

Added blacklist feature (untested)

2018-06-06 10:17:30 -04:00

queue_reddit_links.py

Should fix problem when comment/post with subdir doesn't exist pt. 2

2018-06-06 19:40:08 -04:00

README.md

Create README.md

2018-06-07 10:01:33 -04:00

reddit_bot.py

Minor adjustments

2018-06-06 14:15:06 -04:00

requirements.txt

Added requirement

2018-06-03 10:49:11 -04:00

scrapy.cfg

Renamed package (again) and removed unused files

2018-05-31 08:30:08 -04:00

task.py

Mark reddit post as crawled even if too small to comment

2018-06-06 16:01:07 -04:00

update_website.py

Initial commit

2018-05-28 20:35:04 -04:00

README.md

OD-Database

Suggestions/concerns/PRs are welcome

Installation

Assuming you have Python 3 and git installed:

git clone https://github.com/simon987/od-database
cd od-database
pip3 install -r requirements.txt

Running

python3 app.py

You should be able to connect with your browser at https://localhost:12345

*Note: For now, you need to have the appropriate SSL certificates (cert.pem & privkey.pem) in /certificates. There will be an option to disable SSL very soon