处理键盘中断和文件写入

Question

我有一个应用程序，它可以监控来自web的一些数据，并几乎每秒钟编写一次文件。这个应用程序全天候运行，但有时我必须关闭它。我使用键盘中断异常停止应用程序时，我想，但有一次，应用程序没有完成文件写入，我得到了一个损坏的文件。是什么让我修复了文件，以便让应用程序正常工作时，我再次打开它

这是一段代码：

while True:
    try:
        conns = [http.client.HTTPSConnection(C.REQUEST_HOST) for k in range(C.num_coins)]

        for k in range(C.num_coins):
            conns[k].request("GET", C.REQUEST_PATH_TICKER.format(C.coin_types[k]))

        responses = [conns[k].getresponse().read() for k in range(C.num_coins)]
        responses_json_new = [
            json.loads(responses[k].decode(), object_pairs_hook=OrderedDict)
            for k in range(C.num_coins)
        ]
        responses_json_new = [responses_json_new[k]["ticker"] for k in range(C.num_coins)]

        for k in range(C.num_coins):
            if responses_json_new[k]["last"] != responses_json_old[k]["last"]:

                date_ticker = datetime.datetime.fromtimestamp(
                    int(responses_json_new[k]["date"])
                ).strftime("%Y-%m-%d")

            with open(files_path[k] + date_ticker + ".json", "a") as f:
                json.dump(responses_json_new[k], f)
                f.write("\n")

            responses_json_old[k] = responses_json_new[k]
            print(
                out.format(
                    C.coin_types[k],
                    float(responses_json_new[k]["vol"]),
                    float(responses_json_new[k]["last"]),
                    datetime.datetime.fromtimestamp(
                        responses_json_new[k]["date"]
                    ).strftime("%Y-%m-%d %H:%M:%S"),
                )
            )

        currentDate, files_path = updateFilePath(currentDate, files_path)
        time.sleep(1)

    except KeyboardInterrupt:
        print("\nInterrupted!\n")
        for k in range(C.num_coins):
            if conns[k]:
                conns[k].close()
        sys.exit()
    except Exception as e:
        print("\nERROR:", sys.exc_info()[0])
        ut.exception_log(e, traceback.format_exc())
        ut.close_waiting(conns, wait=30)
        continue

    finally:
        for k in range(C.num_coins):
            if conns[k]:
                conns[k].close()

我需要一些关于如何处理这件事的建议

谢谢大家

Answer 1

.dump()可以（显然）将输出写成几个块，如果它足够大的话

首先要使用.dumps()编码为JSON字符串，然后使用.write()将其编码为文件

为了证明输出可以写在多个块中，让我们在Docker容器中启动strace，看看得到了什么

(host) $ docker run -it ubuntu:18.04 bash
# apt install strace python3
# cat > ex1.py

import json, sys
payload = [[x] * x for x in range(5, 10)] * 250
json.dump(payload, sys.stdout)

# strace python3 ex1.py >/dev/null

(uninteresting lines redacted)
write(1, "[[5, 5, 5, 5, 5], [6, 6, 6, 6, 6"..., 8194) = 8194
write(1, ", 6, 6], [7, 7, 7, 7, 7, 7, 7], "..., 8194) = 8194
write(1, "], [8, 8, 8, 8, 8, 8, 8, 8], [9,"..., 8194) = 8194
write(1, "[9, 9, 9, 9, 9, 9, 9, 9, 9], [5,"..., 4168) = 4168

# cat > ex2.py

import json, sys
payload = [[x] * x for x in range(5, 10)] * 250
pl2 = json.dumps(payload)
sys.stdout.write(pl2)

# strace python3 ex2.py 

(uninteresting lines redacted)
write(1, "[[5, 5, 5, 5, 5], [6, 6, 6, 6, 6"..., 28750) = 28750