java在确认内容类型后如何将URL保存到文件？

Question

如果文件是特定的内容类型，我正在尝试从URL下载文件。URL可以提供html或pdf页面，我只想保存pdf文件。我的尝试如下：

HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("HEAD");
connection.connect();
String contentType = connection.getContentType();

if (contentType.equals("application/pdf")) {
      org.apache.commons.io.FileUtils.copyURLToFile(url, file);
}

contentType被正确提取，但是对copyURLToFile(url,file);的调用导致以下异常：

java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(Unknown Source)
at com.sun.net.ssl.internal.ssl.InputRecord.readFully(Unknown Source)
at com.sun.net.ssl.internal.ssl.InputRecord.read(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(Unknown Source)
at com.sun.net.ssl.internal.ssl.AppInputStream.read(Unknown Source)
at java.io.BufferedInputStream.fill(Unknown Source)
at java.io.BufferedInputStream.read1(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
at java.io.FilterInputStream.read(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(Unknown Source)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1025)
at org.apache.commons.io.IOUtils.copy(IOUtils.java:999)
at org.apache.commons.io.FileUtils.copyURLToFile(FileUtils.java:848)

如果我删除了获取contentType的代码行，并且只调用了copyURLToFile(url,file)，那么文件将被成功下载并保存。我是否以某种方式错误处理了我的HttpURLConnection，导致我的连接被重置

我还注意到，如果在if(contentType.equals("application/pdf")行上设置一个断点并等待几秒钟，那么对copyURLToFile的调用将成功，而不会重置连接。我引入了某种总是失败的比赛条件吗

Answer 1

为什么不在你读了头之后试着关闭连接呢

   HttpURLConnection connection = (HttpURLConnection) url.openConnection();
   connection.setRequestMethod("HEAD");
   connection.connect();
   String contentType = connection.getContentType();
   connection.close();

然后，FileUtils应该打开一个新连接，您的问题可能会得到解决

Answer 2

您应该使用开放连接来读取数据：

org.apache.commons.io.IOUtils.copy(connection.getInputStream(), new FileOutputStream(file));

不需要打开另一个连接，可能是服务器重置了连接

编辑：没有更改请求方法设置，但使用GET，这对我来说很有用：

public static void main(String args[]) throws IOException{
    URL url = new URL("http://www.google.com");
    HttpURLConnection connection = (HttpURLConnection) url.openConnection();
    connection.setRequestMethod("GET");
    String contentType = connection.getContentType();
    System.out.println("content-type: " + contentType);
    IOUtils.copy(connection.getInputStream(), new FileOutputStream("/temp/test.html"));
}

编辑：或者，如果您想先用HEAD请求检查页眉：

URL url = new URL("http://www.google.com");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("HEAD");
String contentType = connection.getContentType();
System.out.println("content-type: " + contentType);
connection.disconnect();
connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
IOUtils.copy(connection.getInputStream(), new FileOutputStream("/temp/test.html"));
connection.disconnect();