最近写一个sina爬虫,然后爬了半天之后访问不上给我封掉了-.-,于是加上代理功能

  1. 代理
1
2
3
4
5
proxies={
    'http': 'http://127.0.0.1:8087',
    'https': 'http://127.0.0.1:8087',
}
requests.get(url,timeout=60,proxies=proxies)
  1. 证书
1
2
3
4
5
6
默认开启
requests.get(url, verify=False)
requests.get('https://github.com', verify='certfile')

s = requests.Session()
s.verify = 'certfile'
  1. 重定向
    1
    response = request.get(url, allow_redirects=False)
  2. 请求头
    1
    2
    3
    4
    s = requests.Session()
    s.headers.update(dict)

    requests.get(url,headers=dict)