simon987/Architeuthis
1
0
Fork 0
mirror of https://github.com/simon987/Architeuthis.git synced 2025-12-19 19:24:50 +00:00
Code Issues Projects Releases Wiki Activity

Config glob & hierarchy, fixes

Browse Source
This commit is contained in:
simon
2019-05-30 10:03:33 -04:00
parent 69f28f1ff7
commit 98c7f12cf1
5 changed files with 70 additions and 56 deletions
Download Patch File Download Diff File Expand all files Collapse all files

23
README.md
View File

@@ -34,18 +34,25 @@ and error handling. Built for automated web scraping.
"url": "http://p1.exemple.com:8080"
}
],
"hosts": {
"*": {
"every": "750ms",
"burst": 5,
"headers": {}
"hosts": [
{
"host": "*",
"every": "500ms",
"burst": 25,
"headers": {
"User-Agent": "Some user agent",
"X-Test": "Will be overwritten"
}
},
"reddit.com": {
{
"host": "*.reddit.com",
"every": "2s",
"burst": 2,
"headers": {"User-Agent": "mybot_v0.1"}
"headers": {
"X-Test": "Will overwrite default"
}
}
}
]
}
```