simon987/od-database-crawler
1
0
Fork 0
mirror of https://github.com/terorie/od-database-crawler.git synced 2025-12-16 08:19:06 +00:00
Code Issues Projects Releases Wiki Activity

Add User-Agent parameter

Browse Source
This commit is contained in:
Richard Patel
2018-11-18 14:24:04 +01:00
parent 6dbec8c789
commit a71157b4d8
3 changed files with 14 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
config.yml
View File

@@ -47,3 +47,6 @@ crawl:
# Time before discarding a network request
timeout: 10s
# Crawler User-Agent
user-agent: "Mozilla/5.0 (X11; od-database-crawler) Gecko/20100101 Firefox/52.0"