2024-06-26 14:10:25 发布
网友
我试图找到一种方法来知道网页是否存在。有很多方法,如httlib2、urlparse和using请求。但在我的情况下,如果网页不存在,网站会将我重定向到主页 例如 https://www.thenews.com.pk/latest/category/sports/2015-09-21
有什么方法可以抓住它吗
您提到的URL提供了一个重定向返回代码(307),您可以捕获该代码。请看这里:
$ curl -i https://www.thenews.com.pk/latest/category/sports/2015-09-21 HTTP/1.1 307 Temporary Redirect Date: Sun, 26 Mar 2017 10:13:39 GMT Content-Type: text/html; charset=UTF-8 Transfer-Encoding: chunked Connection: keep-alive Set-Cookie: __cfduid=ddcd246615efb68a7c72c73f480ea81971490523219; expires=Mon, 26-Mar-18 10:13:39 GMT; path=/; domain=.thenews.com.pk; HttpOnly Set-Cookie: bf_session=b02fb5b6cc732dc6c3b60332288d0f1d4f9f7360; expires=Sun, 26-Mar-2017 11:13:39 GMT; Max-Age=3600; path=/; HttpOnly Expires: Thu, 19 Nov 1981 08:52:00 GMT Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Pragma: no-cache Location: https://www.thenews.com.pk/ X-Cacheable: YES X-Varnish: 654909723 Age: 0 Via: 1.1 varnish X-Age: 0 X-Cache: MISS Access-Control-Allow-Origin: * Server: cloudflare-nginx CF-RAY: 345956a8be8a7289-AMS
您可以检查最后的url是否是您被重定向到的那个,以及是否有任何history的重定向
url
history
>>> import requests >>> target_url = "https://www.thenews.com.pk/latest/category/sports/2015-09-21" >>> response = requests.get(target_url) >>> response.history[0].url u'https://www.thenews.com.pk/latest/category/sports/2015-09-21' >>> response.url u'https://www.thenews.com.pk/' >>> response.history and response.url == 'https://www.thenews.com.pk/' != target_url True
您提到的URL提供了一个重定向返回代码(307),您可以捕获该代码。请看这里:
您可以检查最后的
url
是否是您被重定向到的那个,以及是否有任何history
的重定向相关问题 更多 >
编程相关推荐