5 Commits

Author SHA1 Message Date
Richard Patel
b1c40767e0
Remember scanned URLs 2018-10-28 17:07:30 +01:00
Richard Patel
c196b6f20d
Better config 2018-10-28 14:19:09 +01:00
Richard Patel
1c33346f45
Fix crawl descent 2018-10-28 03:06:18 +01:00
Richard Patel
a507110787
Add stats interval parameter 2018-10-28 02:47:20 +02:00
Richard Patel
79f540bf29
Scheduler 2018-10-28 02:40:12 +02:00