为什么apache java http库不能处理ContentEncoding:none的站点？

Question

我试图通过代理发送GET，有些站点的头是：Content-Encoding: none，这会导致Apache抛出异常。我想知道这是否是预期的行为，以及我是否应该将其视为一个bug：

Caused by: org.apache.http.HttpException: Unsupported Content-Coding: none
at org.apache.http.client.protocol.ResponseContentEncoding.process(ResponseContentEncoding.java:98)
at org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:139)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:199)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:85)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:186)
... 10 more

我的代码：

public CloseableHttpResponse getViaProxy(String url, String ip, int port, String username,
                                         String password) {
    CloseableHttpClient httpClient;
    if (username == null) {
        httpClient = HttpClients.custom()
                .setSSLSocketFactory(getCustomSslConnectionSocketFactory())
                .build();
    } else {
        CredentialsProvider credsProvider = new BasicCredentialsProvider();
        credsProvider.setCredentials(
                new AuthScope(ip, port),
                new UsernamePasswordCredentials(username, password));
        httpClient = HttpClients.custom()
                .setDefaultCredentialsProvider(credsProvider)
                .setSSLSocketFactory(getCustomSslConnectionSocketFactory())
                .build();
    }

    RequestConfig config = RequestConfig.custom()
            .setProxy(new HttpHost(ip, port))
            .build();
    HttpGet httpGet = new HttpGet(url);
    httpGet.setConfig(config);

    return httpClient.execute(httpGet);
}

我指的是这里的错误。这种方法似乎只支持带有内容编码的标题：gzip、deflate或identity

http://hc.apache.org/httpcomponents-client-ga/httpclient/xref/org/apache/http/client/protocol/ResponseContentEncoding.html

Answer 1

HTTP协议规范将gzip、compress、deflate和identity定义为有效的内容编码方案。应该使用identity来表示不需要进行内容转换

Answer 2

如果只想处理内容编码为none的请求，可以使用HttpClent的custom方法，如下所示：

CloseableHttpClient httpClient = HttpClients.custom().disableContentCompression().build();

此代码禁用两个拦截器RequestAcceptEncoding和ResponseContentEncoding