simon987/od-database
1
0
Fork 0
mirror of https://github.com/simon987/od-database.git synced 2025-04-10 05:56:43 +00:00
Code Issues Projects Releases Wiki Activity
305 Commits 2 Branches 0 Tags
Commit Graph

101 Commits

Author SHA1 Message Date
simon
e8f0f96148 More bug fixes... 2020-01-25 12:54:26 -05:00
simon
ae0fb9b1a6 Docker tweaking & bug fixes 2020-01-25 10:16:24 -05:00
simon
7f121d2ac0 docker-compose setup (wip) 2019-11-13 20:36:09 -05:00
simon
df8ab7727b docker-compose setup (wip) 2019-11-13 13:03:43 -05:00
simon987
4ffe805b8d Use task_tracker for task tracking 2019-03-24 20:23:05 -04:00
simon987
204b82b71f Fix captcha part 2: don't store captcha answer in session cookie 2019-02-03 09:01:21 -05:00
simon987
7f857d641f Change ES settings, big refactor, removed recaptcha 2019-01-13 12:48:39 -05:00
terorie
0519d1fbea
Proper downloads page 2018-12-14 20:15:37 +01:00
terorie
1ac3b97d7e Crawl stats: time format + sorting (#10) 
* Nicer stats
* Fix right align
* No leading day zeros
* Fix right-align padding
 2018-12-14 09:30:06 -05:00
Simon
e89eb6e3e0 Fixes #9 2018-12-06 10:05:35 -05:00
Simon
5782a45524 Maybe fix last PR 2018-11-18 12:27:20 -05:00
terorie
812a9c4113
Exclusive /api/task/upload operation 2018-11-18 18:21:21 +01:00
Simon
98f43f817a revert task queuing pt. 3 ._. 2018-11-18 11:13:51 -05:00
Simon
801d056da8 revert task queuing pt. 2 2018-11-18 11:11:38 -05:00
Simon
4ce807c8a0 revert task queuing 2018-11-18 11:10:44 -05:00
Simon
1812ec932d Completed tasks queue 2018-11-17 21:36:29 -05:00
Simon
372c10b5ab Nicer search logging 2018-11-17 12:04:59 -05:00
Simon
d8df91a0d6 app.py small cleanup + some logging 2018-11-17 11:53:41 -05:00
Simon
a6c421c4a6 Flask logging disabled 2018-11-17 11:20:46 -05:00
Simon
4c51598441 Get queued tasks temporarily returns only non-ftp websites 2018-11-16 22:59:26 -05:00
Simon
6e80791264 Search filter 2018-11-16 16:49:23 -05:00
Simon
db26e851a4 API endpoint to cancel task 2018-10-26 18:13:47 -04:00
Simon
1d3318f6e2 Create new rescan task when no queued tasks 2018-09-29 12:01:02 -04:00
Simon
85437b1ef9 Merge remote-tracking branch 'origin/master' 2018-09-06 19:46:56 -04:00
Simon
53db765856 Static export file now managed by nginx 2018-09-06 19:38:07 -04:00
Simon
6d27cbca02 xz -> lzma for export 2018-08-15 11:32:36 -04:00
Simon
c92f2f4937 Should fix export problem 2018-08-14 12:21:34 -04:00
Simon
c29af180c5 Captcha for searches 2018-08-10 12:46:40 -04:00
Simon
8ffd9179d2 Increased stats timeout value 2018-08-09 14:26:22 -04:00
Simon
f729b462f0 od_util can be used when od-database is a submodule part 2 2018-08-08 23:31:50 -04:00
Simon
458641654c Minimal configuration for reddit comment callback 2018-08-08 21:24:55 -04:00
Simon
65738b0e70 Small fix 2018-07-25 12:05:07 -04:00
Simon
fbbe952e4d Stats are generated in background and stored to file instead of on-demand 2018-07-24 20:29:25 -04:00
Simon
df5b01dc83 Bug when directory is empty with new file upload (server side) 2018-07-17 18:28:50 -04:00
Simon
898ffcf410 File upload is made in small chunks 2018-07-17 17:52:17 -04:00
Simon
73afebec28 Added API commands 2018-07-17 13:12:20 -04:00
Simon
cf96d1697d Fixed bug when submitting 2018-07-16 20:34:42 -04:00
Simon
3b3661cae2 Typo 2018-07-15 21:23:49 -04:00
Simon
0227684a53 Added API commands 2018-07-15 21:21:57 -04:00
Simon
8a19fa0ce7 Decreased bulk enqueue limit 2018-07-15 15:31:21 -04:00
Simon
c35491cb15 Multi threading for bulk enqueue 2018-07-15 12:50:26 -04:00
Simon
08c3e119f0 Typo 2018-07-15 10:52:04 -04:00
Simon
112400886e Crawler no longer crashes when website has no files 2018-07-15 10:46:48 -04:00
Simon
fe1d29aaea Crawl tasks are now fetched by the crawlers instead of pushed by the server 2018-07-14 17:31:18 -04:00
Simon
d9e9f53f92 Website should stay online even if elasticsearch is down / timing out 2018-07-12 12:06:45 -04:00
Simon
290322dfa7 Indexing is a bit gentler on server resources and some pages have been memory cached 2018-07-12 11:26:20 -04:00
Simon
1b743e7aba Updated README and catpchas are now toggled from the config 2018-07-10 22:57:52 -04:00
Simon
d138db8f06 Added filter to check if a website can be scanned from its parent directory 2018-07-10 10:14:23 -04:00
Simon
f226b82f5a Increased website count per page in /website/ 2018-07-08 10:44:25 -04:00
Simon
711e8282ef 'Go to random website' button, and navigation in the website list 2018-07-08 10:42:14 -04:00