simon987/od-database
1
0
Fork 0
mirror of https://github.com/simon987/od-database.git synced 2025-12-11 14:08:51 +00:00
Code Issues Projects Releases Wiki Activity

Commit Graph

  • 3df667deb4 total size #26 master simon987 2020-01-31 11:26:40 -05:00
  • e8f0f96148 More bug fixes... simon 2020-01-25 10:49:35 -05:00
  • ae0fb9b1a6 Docker tweaking & bug fixes simon 2020-01-25 10:13:50 -05:00
  • 2b2ef5eac7 Update task_tracker images simon 2020-01-25 09:50:40 -05:00
  • b950b1f488 Bug fixes simon 2020-01-25 09:40:39 -05:00
  • 0881e8c40e docker tweaks simon 2020-01-22 16:57:00 -05:00
  • 853e38e46b Merge pull request #25 from simon987/docker-wip simon987 2020-01-22 16:04:51 -05:00
  • c61f51cb08 add kibana & update README.md docker-wip simon 2020-01-22 16:04:53 -05:00
  • 7f121d2ac0 docker-compose setup (wip) simon 2019-11-13 20:36:09 -05:00
  • df8ab7727b docker-compose setup (wip) simon 2019-11-13 13:03:43 -05:00
  • 31877283b3 Bug fixes, ES7 simon 2019-06-14 13:27:41 -04:00
  • 41ba6a35a4 Add mass import utility simon987 2019-04-06 19:25:49 -04:00
  • 2c7e71cde1 Fix crontab simon987 2019-04-06 15:34:30 -04:00
  • d30a17c331 tweak uwsgi config simon987 2019-04-06 15:24:06 -04:00
  • 6843900ec6 Change max assign time simon987 2019-04-06 12:50:28 -04:00
  • bfb59c5336 Install crontab on deploy simon987 2019-04-06 10:56:23 -04:00
  • b8b531f511 Move recrawl task to cron job simon987 2019-04-06 10:50:15 -04:00
  • 0c3d0b38e6 Don't use multiprocessing for recrawl task simon987 2019-04-06 09:21:02 -04:00
  • 06ae89f4d2 Only queue http tasks (temp) simon987 2019-04-06 09:07:17 -04:00
  • 310f343423 Deploy fix attempt simon987 2019-04-04 21:21:32 -04:00
  • 0f1c0df91a Compress deploy step simon987 2019-04-04 20:46:15 -04:00
  • df0cd26724 Add build badge simon987 2019-03-30 11:22:28 -04:00
  • 364208f94f Add linguist manual overrides simon987 2019-03-30 11:21:27 -04:00
  • 5b680be770 Update readme simon987 2019-03-30 09:16:54 -04:00
  • e02d08ca62 bugfix simon987 2019-03-28 21:04:38 -04:00
  • f35cc9dd5b jenkins tweaks simon987 2019-03-28 20:54:27 -04:00
  • 2046b36f9a Bug fixes simon987 2019-03-28 20:29:34 -04:00
  • d69ed65a0c Rewrite export.py, add diagram simon987 2019-03-27 22:09:08 -04:00
  • b9f25630b4 Switch to postgresql, finish minimum viable task_tracker/ws_bucket integration simon987 2019-03-27 19:34:05 -04:00
  • b170f9bfd8 Redirect stderr to file simon987 2019-03-27 17:40:59 -04:00
  • 019feecb03 Jenkins tweaks simon987 2019-03-24 21:40:57 -04:00
  • 841394ebac Jenkins tweaks simon987 2019-03-24 20:33:37 -04:00
  • 4ffe805b8d Use task_tracker for task tracking simon987 2019-03-24 20:21:43 -04:00
  • 00e3fd7340 Remove task tracking simon987 2019-03-09 13:26:05 -05:00
  • 6000e46ad7 compatibility fix for python 3.5 simon987 2019-02-03 09:15:04 -05:00
  • ae73e03067 remove pycache simon987 2019-02-03 09:07:59 -05:00
  • add0bfa2b3 update gitignore simon987 2019-02-03 09:03:34 -05:00
  • 204b82b71f Fix captcha part 2: don't store captcha answer in session cookie simon987 2019-02-03 09:01:21 -05:00
  • e8965497d4 Fix captcha part 1: more human readable (but less cool) simon987 2019-02-03 08:47:27 -05:00
  • 32c1c861ad hotfix attempt 3 pt. 2 simon987 2019-02-02 11:48:09 -05:00
  • 59b1d249ba hotfix attempt 3 simon987 2019-02-02 11:45:46 -05:00
  • 0ff6ea1682 hotfix attempt 2 simon987 2019-02-02 10:58:09 -05:00
  • 8ced4859f3 hotfix attempt 1 simon987 2019-02-02 09:17:59 -05:00
  • dd3775cecd Fix ES Settings simon987 2019-01-13 13:52:49 -05:00
  • 7f857d641f Change ES settings, big refactor, removed recaptcha simon987 2019-01-13 12:48:39 -05:00
  • d905c3efd5 Removed crawl_server module simon 2019-01-05 09:10:44 -05:00
  • 38dfb657ed Merge pull request #13 from terorie/proper-dl Simon Fortier 2018-12-14 22:49:03 -05:00
  • 1dc775fafd Merge pull request #12 from terorie/td-align-css Simon Fortier 2018-12-14 22:46:54 -05:00
  • 0519d1fbea Proper downloads page terorie 2018-12-14 19:58:56 +01:00
  • 32568971a8 Add td-numeric right-align in minified CSS terorie 2018-12-14 19:21:52 +01:00
  • 1ac3b97d7e Crawl stats: time format + sorting (#10) terorie 2018-12-14 15:30:06 +01:00
  • e89eb6e3e0 Fixes #9 Simon 2018-12-06 10:05:35 -05:00
  • d4fd764536 import problem temp fix Simon 2018-12-02 12:42:57 -05:00
  • a33d4f2dca retry on index fail/timeout Simon 2018-11-22 10:52:42 -05:00
  • 5782a45524 Maybe fix last PR Simon 2018-11-18 12:27:20 -05:00
  • fb6a1821ae Merge pull request #7 from terorie/master Simon Fortier 2018-11-18 12:23:14 -05:00
  • 812a9c4113 Exclusive /api/task/upload operation terorie 2018-11-18 18:21:21 +01:00
  • 750940d148 error handling for delete Simon 2018-11-18 11:19:34 -05:00
  • 254d6ea44e More debug logging Simon 2018-11-18 11:15:23 -05:00
  • 98f43f817a revert task queuing pt. 3 ._. Simon 2018-11-18 11:13:51 -05:00
  • 801d056da8 revert task queuing pt. 2 Simon 2018-11-18 11:11:38 -05:00
  • 4ce807c8a0 revert task queuing Simon 2018-11-18 11:10:44 -05:00
  • 6e491513bf Merge remote-tracking branch 'origin/master' Simon 2018-11-18 11:07:45 -05:00
  • 64ce9379c3 Replace delete_by_query with bulk delete (cleanup) Simon 2018-11-18 11:07:34 -05:00
  • 2f6ae3cb35 Replace delete_by_query with bulk delete Simon 2018-11-18 11:07:18 -05:00
  • 1812ec932d Completed tasks queue Simon 2018-11-17 21:36:29 -05:00
  • 876a511b54 Lowered search timeout Simon 2018-11-17 12:08:04 -05:00
  • 372c10b5ab Nicer search logging Simon 2018-11-17 12:04:59 -05:00
  • d8df91a0d6 app.py small cleanup + some logging Simon 2018-11-17 11:53:41 -05:00
  • a6c421c4a6 Flask logging disabled Simon 2018-11-17 11:20:46 -05:00
  • 4996de6aa9 Logging for search and better error handling Simon 2018-11-17 11:19:09 -05:00
  • edf1849bac Create new rescan task when no queued tasks pt2 Simon 2018-11-16 23:01:28 -05:00
  • 4c51598441 Get queued tasks temporarily returns only non-ftp websites Simon 2018-11-16 22:59:26 -05:00
  • e3f3a9cf7f Minified css & js, added about section in homepage Simon 2018-11-16 20:19:03 -05:00
  • 53317ab606 forgot submodule Simon 2018-11-16 16:53:25 -05:00
  • 6e80791264 Search filter Simon 2018-11-16 16:49:23 -05:00
  • a461b22ffc Stats page crawl server table improvement Simon 2018-11-15 13:46:21 -05:00
  • fc3de06c35 Next/Prev button works without captcha Simon 2018-11-15 12:54:31 -05:00
  • db26e851a4 API endpoint to cancel task Simon 2018-10-26 18:13:47 -04:00
  • 1d3318f6e2 Create new rescan task when no queued tasks Simon 2018-09-29 12:01:02 -04:00
  • 2ef60a05a5 proper user agent for pycurl Simon 2018-09-29 11:38:52 -04:00
  • 0c423ee9a9 Update README.md Simon Fortier 2018-09-20 19:41:55 -04:00
  • fff013f253 Update README.md Simon Fortier 2018-09-20 19:31:26 -04:00
  • bbd5c7694c Fixed typo in title Simon 2018-09-13 17:17:55 -04:00
  • 85437b1ef9 Merge remote-tracking branch 'origin/master' Simon 2018-09-06 19:46:56 -04:00
  • 53db765856 Static export file now managed by nginx Simon 2018-09-06 19:38:07 -04:00
  • 8b13de4a6b Re-init curl handle on error Simon 2018-08-25 16:46:53 -04:00
  • faa9ac3ccb Closing curl handle manually just to make sure Simon 2018-08-23 12:48:15 -04:00
  • dff4125c9f Bugfix post-pycurl update pt. 3 (Sorry!) Simon 2018-08-23 12:47:17 -04:00
  • 25e1e58828 Bugfix post-pycurl update pt. 3 (Sorry!) Simon 2018-08-23 12:46:00 -04:00
  • 6ffc43601b Bugfix post-pycurl update pt. 2 Simon 2018-08-23 12:40:13 -04:00
  • 484a0baf9d Bugfix post-pycurl update Simon 2018-08-23 12:37:27 -04:00
  • cadaf14c1b Small bugfix Simon 2018-08-23 12:12:23 -04:00
  • 54b4d2d5b4 removed debug lines Simon 2018-08-23 12:02:07 -04:00
  • d42be56dee More debug info Simon 2018-08-23 11:59:13 -04:00
  • 8dc8627f78 Update README.md Simon Fortier 2018-08-23 11:51:48 -04:00
  • 85c3aa918d replaced requests by pycurl Simon 2018-08-23 11:47:09 -04:00
  • 8f218f3c9d Bug fix for pages buttons pt.2 Simon 2018-08-16 13:24:00 -04:00
  • a2327bac7c Bug fix for pages buttons Simon 2018-08-16 13:13:34 -04:00
  • 6d27cbca02 xz -> lzma for export Simon 2018-08-15 11:32:36 -04:00