pythonp爬取网页请求超时。获取整个响应

阅读 78

2022-10-31


未优化:

import requests
import eventlet

data=[]
websites=['http://google.com', 'http://bbc.co.uk']
for w in websites:
r= requests.get(w, verify=False)

​​https://eventlet.net/doc/modules/timeout.html​​

​​https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings​​

最后优化:

import requests
import eventlet

data=[]
websites=['http://google.com', 'http://bbc.co.uk']

eventlet.monkey_patch()
for w in websites:
try:
with eventlet.Timeout(15):#设置超时时间15s
r= requests.get(w, verify=False)
#逻辑代码
pass

except:
print("响应超时")
continue #跳出本次循环

print("程序执行完毕")

 

精彩评论(0)

0 0 举报