sirius/gcloud-lab
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

change user agent for scrapy

Browse source
This commit is contained in:
sirius0xdev 2026-03-23 01:16:51 +00:00
parent d56bd1ab15
commit 25b5c2bd7b
1 changed files with 2 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
apps/base/customer1/news_bot/scrapy-configmap.yaml
View file

@ -12,7 +12,7 @@ data:
# Scrapy Settings
LOG_LEVEL: "INFO"
USER_AGENT: "Mozilla/5.0 (Windows NT 10.0; Win64; x64) NewsScraper/1.0"
CONCURRENT_REQUESTS: "50"
USER_AGENT: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/26.0 Safari/605.1.15"
CONCURRENT_REQUESTS: "100"
# Proxy Endpoint (The URL part only)
PROXY_ENDPOINT: ""