14 Commits

Author	SHA1	Message	Date
Richard Patel	a8c27b2d21	Hash links	2018-11-06 02:01:53 +01:00
Richard Patel	ed5e35f005	Performance improvements	2018-11-06 00:34:22 +01:00
Richard Patel	77cb45dbec	Detect directory symlinks	2018-10-28 18:37:18 +01:00
Richard Patel	bfd7302be8	Add urfave/cli app	2018-10-28 17:59:46 +01:00
Richard Patel	b1c40767e0	Remember scanned URLs	2018-10-28 17:07:30 +01:00
Richard Patel	ddfdce9d0f	Refactor a bit	2018-10-28 13:43:45 +01:00
Richard Patel	79f540bf29	Scheduler	2018-10-28 02:40:12 +02:00
Richard Patel	3fb4d4bde9	More logs	2018-10-27 17:25:32 +02:00
Richard Patel	76c8c13d49	Use finite state machine	2018-10-27 16:55:00 +02:00
Richard Patel	442a2cf8a7	Compare finite state machine and Regex	2018-10-27 16:53:45 +02:00
Richard Patel	9e090d109d	Header state machine	2018-10-27 16:29:10 +02:00
Richard Patel	d748be72cd	File HEAD requests	2018-10-27 16:22:01 +02:00
Richard Patel	2844d344ec	Working listing	2018-10-27 15:00:20 +02:00
Richard Patel	f2d2b620fa	Simple queue crawler	2018-10-27 04:08:32 +02:00