diff --git a/.github/stress.png b/.github/stress.png new file mode 100644 index 0000000..aebd67e Binary files /dev/null and b/.github/stress.png differ diff --git a/README.md b/README.md index ad77bc6..a76b8b7 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,12 @@ -# oddb Go crawler +# oddb Go crawler 🚀 > by terorie 2018 :P + + * Crawls HTTP open directories (standard Web Server Listings) + * Gets name, path, size and modification time of all files + * Soon: Will work as a crawler for [OD-Database](https://github.com/simon987/od-database)! + +Stress test crawling [pandoradir](https://github.com/terorie/pandoradir) +on an average laptop (~10K requests per second, 4 connections): +![image](.github/stress.png) + +Memory usage is being optimized :P