java如何防止块数据处理的while循环中的代码重复？

Question

我正在读取一个文件，收集处理过的行，然后在收集的每个数据块之后，将它们分批写入（例如，文件或数据库）

当循环终止（=文件被完全读取）时，我必须再次调用编写器。否则我就抓不到最后一块了

问：我是否可以改进代码以防止重复额外的write（）调用

List<String> collect = new ArrayList<>();

String line;
while ((line = reader.read()) != null) {
    String processed = processline(line);
    collect.add(processed);

    //write each x chunks to file
    if (collect.size() % 1000 == 0) {
        writer.write(collect);
        collect = new ArrayList<>();
    }
}

//can I prevent repetition here?
if (!collect.isEmpty()) {
    writer.write(collect);
}

Answer 1

你可以这样做，但你仍然会有一个额外的行得到最后一块

List<String> collect = new ArrayList<>();

String line;
do {
   line = reader.read();
   if(line != null) {
        String processed = processline(line);
        collect.add(processed);

       //write each x chunks to file
       if (collect.size() % 1000 == 0) {
            writer.write(collect);
            collect = new ArrayList<>();
       }
   }else{
       writer.write(collect);
   }
} while(line != null);

Answer 2

以下是我的建议：

    List<String> collect = new ArrayList<>();

    String line = reader.read();
    String nextLine;

    while (line != null) {

        nextLine = reader.read();

        String processed = processline(line);
        collect.add(processed);

        //write each x chunks to file
        if (collect.size() % 1000 == 0 || nextLine == null) {
            writer.write(collect);
            collect = new ArrayList<>();
            line = nextLine;
            continue;
        }

        line = nextLine;     

    }

Answer 3

将缓冲逻辑封装在一个单独的类中（因为这就是您正在做的，缓冲）。但是，当缓冲区太大时，您总是需要写入，当您完成读取时，您总是需要写入

class BufferingWriter implements Closeable {
    private List<String> buffer = new ArrayList<>(1000);
    private MyWriter writer;

    public void write(String line) {
        buffer.add(line);
        if (buffer.size() >= 1000) {
            flush();
        }
    }

    public void flush() {
        writer.write(buffer);
        buffer.clear();
    }

    @Override
    public void close() throws IOException {
        flush();
        // TBD: Pass the close call onto MyWriter if that is possible
        // or otherwise flag this writer as closed
    }
}

List<String> collect = new ArrayList<>();
try (BufferingWriter bwriter = new BufferingWriter(writer)) {
    String line;
    while ((line = reader.read()) != null) {
        String processed = processline(line);
        bwriter.write(line);
    }
}

Answer 4

做一个边做边做的循环怎么样

List<String> collect = new ArrayList<>();

String line;
do {
   line = reader.read();
   if(line != null) {
      String processed = processline(line);
      collect.add(processed);
   }

   if (collect.size() % 1000 == 0 
       || (line == null && !collect.isEmpty())) { // end of file
      writer.write(collect);
      collect = new ArrayList<>();
   } 

} while(line != null);

Answer 5

也许是这样的？不确定这是否是一种进步

List<String> collect = new ArrayList<>();

for (;;) {
    String line = reader.read();
    boolean done = line == null;
    if (!done) {
        String processed = processline(line);
        collect.add(processed);
    }

    //write each x chunks to file
    if ((collect.size() % 1000 == 0) || done) {
        if (!collect.isEmpty()) {
            writer.write(collect);
        }
        if (done) {
            break;
        }
        collect = new ArrayList<>();
    }
}