9 Commits

Author	SHA1	Message	Date
Richard Patel	ffde1a9e5d	Timeout and results saving	2018-11-15 20:14:31 +01:00
Richard Patel	a268c6dbcf	Reduce WaitQueue usage	2018-11-12 00:38:22 +01:00
Richard Patel	add6581804	Add resource stats logging	2018-11-05 22:41:17 +01:00
Richard Patel	fa37d45378	Remove too many crawler block ... More logging	2018-10-28 18:17:04 +01:00
Richard Patel	b1c40767e0	Remember scanned URLs	2018-10-28 17:07:30 +01:00
Richard Patel	ddfdce9d0f	Refactor a bit	2018-10-28 13:43:45 +01:00
Richard Patel	faad19f121	more stuff	2018-10-28 03:41:16 +01:00
Richard Patel	a507110787	Add stats interval parameter	2018-10-28 02:47:20 +02:00
Richard Patel	79f540bf29	Scheduler	2018-10-28 02:40:12 +02:00