获取URL时发生java Jsoup HTTP错误。进行申请后处理时，状态=403

Question

我一直在寻找这个问题，假设这个问题是通过用户代理解决的，但事实并非如此
我要做的是从请愿书中取出cookies，这就是代码

note: i'm try to do the petition to https webpage

/*obtiene cookies de la peticion*/
        Connection.Response res = Jsoup.connect(liga).header("Content-Type","text/html;charset=UTF-8")
                .cookie("TALanguage", "ALL")
                .data("mode", "filterReviews")
                .data("filterRating", "")
                .data("filterSegment", "")
                .data("filterSeasons", "")
                .data("filterLang", "ALL")
                .referrer(liga)         
                .header("X-Requested-With", "XMLHttpRequest")
                .header("X-Puid",xpuid)
                .data("returnTo",returnTo)
                .userAgent("Mozilla/5.0 (Windows; U; WindowsNT 5.1; en-US; rv1.8.1.6) Gecko/20070725 Firefox/2.0.0.6")                           
                .method(Method.POST)
                .execute();

        doc = res.parse();


        Map<String, String> cookies = res.cookies();

程序在.execute();行失败，日志中出现以下错误：

org.jsoup.HttpStatusException: HTTP error fetching URL. Status=403, URL=https://somepage.html

    at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:459)
    at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:434)
    at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:181)
    at mx.oeste.crawler.htmlunit.obtenerComentarios(htmlunit.java:82)
    at mx.oeste.crawler.htmlunit.main(htmlunit.java:40)

Answer 1

尝试将内容类型标题设置为“application/x-www-form-urlencoded”，如下所示：

Connection.Response res = Jsoup.connect(liga)
                               .header("Content-Type","application/x-www-form-urlencoded")
                               .cookie("TALanguage", "ALL")
                               .data("mode", "filterReviews")
                               .data("filterRating", "")
                               .data("filterSegment", "")
                               .data("filterSeasons", "")
                               .data("filterLang", "ALL")
                               .referrer(liga)         
                               .header("X-Requested-With", "XMLHttpRequest")
                               .header("X-Puid",xpuid)
                               .data("returnTo",returnTo)
                               .userAgent("Mozilla/5.0 (Windows; U; WindowsNT 5.1; en-US; rv1.8.1.6) Gecko/20070725 Firefox/2.0.0.6")                           
                               .method(Method.POST)
                               .execute();

如果不起作用，请尝试在手动从请愿书中获取cookie的同时监视您喜爱的浏览器。您可以使用开发人员工具监视浏览器