Python比D快？？IO操作似乎慢了很多。。。发生什么事？

Question

我为我正在修的计算机系统课程（使用nand2tetris课程）编写了一个虚拟机语言给汇编翻译程序。我最初是用Python编写的，但是由于我在学习D，所以我想我应该翻译它。从语法上讲，Python不太难。我假设D作为一种性能语言并经过编译，至少和Python一样快，在一个大文件中，速度会快得多。但事实恰恰相反！尽管有相同的算法，当我构建一个非常大的文件进行编译时，D的执行速度始终比python慢一些。在一个大约500000行长的文件中，python通常需要大约2.6秒来完成，而D通常需要3秒。这不是一个巨大的差距，但值得注意的是python的速度会更快。在

我不想说我太天真了，认为python实际上比D快；但是，在这个例子中，至少在直觉上D并没有更快。如果您能在我的d代码中提供一些性能下降的可能原因，我将不胜感激。我认为瓶颈可能在IO操作上，但我不确定。在

源代码如下。细节并不那么重要；分配了一些汇编语言模板，然后对虚拟机语言进行了线性传递，将每条指令转换为等效的汇编代码块。在

编辑：在使用dmd -O -release -inline -m64重新编译D代码后，D将在输入上以2.20秒的时间显示出胜利者。然而，问题仍然存在，为什么在几乎相同的代码中，D的执行速度似乎比python慢。在

编辑2:使用下面的建议，我从使用简单的字符串列表切换到使用appender!string()，并显著地缩短了时间。但是值得一提的是，如果您在appender中有一堆字符串，不要使用如下命令将它们写入文件：

auto outputfile = File("foo.txt","w");
foreach(str; my_appender.data)
   outputfile.write(str);

相反，你可以写一些东西，比如：

^{pr2}$

第二个示例将使您的性能比使用简单的string[]略有提高。但是使用第一个方法给我带来了巨大的性能冲击，使执行时间增加了一倍。在

改为appender!string()，前面提到的大文件的编译花费了大约2.75秒（相对于Python的2.8），而原始文件的编译花费了大约3秒。这样做，同时使用dmd中的优化标志，总编译时间为1.98秒！：）

Python：

#!/usr/bin/python

import sys

operations_dict = {"add":"+", "sub":"-",
                   "and":"&", "or":"|",
                   "not":"!", "neg":"-",
                   "lt":"JLT", "gt":"JGT",
                   "eq":"JEQ", "leq":"JLE",
                   "geq":"JGE"}

vars_dict = {"this":("THIS","M"),
             "that":("THAT","M"),
             "argument":("ARG","M",),
             "local":("LCL","M",),
             "static":("f.%d","M",),
             "temp":("TEMP","A",)}

start = "@SP\nAM=M-1\n"
end = "@SP\nM=M+1\n"

binary_template = start + "D=M\n\
@SP\n\
AM=M-1\n\
M=M%sD\n" + end

unary_template = start + "M=%sM\n" + end

comp_template = start + "D=M\n\
@SP\n\
AM=M-1\n\
D=M-D\n\
@COMP.%d.TRUE\n\
D;%s\n\
@COMP.%d.FALSE\n\
0;JMP\n\
(COMP.%d.TRUE)\n\
@SP\n\
A=M\n\
M=-1\n\
@SP\n\
M=M+1\n\
@COMP.%d.END\n\
0;JMP\n\
(COMP.%d.FALSE)\n\
@SP\n\
A=M\n\
M=0\n" + end + "(COMP.%d.END)\n"

push_tail_template = "@SP\n\
A=M\n\
M=D\n\
@SP\n\
M=M+1\n"

push_const_template = "@%d\nD=A\n" + push_tail_template

push_var_template = "@%d\n\
D=A\n\
@%s\n\
A=%s+D\n\
D=M\n" + push_tail_template

push_staticpointer_template = "@%s\nD=M\n" + push_tail_template

pop_template = "@%d\n\
D=A\n\
@%s\n\
D=%s+D\n\
@R13\n\
M=D\n\
@SP\n\
AM=M-1\n\
D=M\n\
@R13\n\
A=M\n\
M=D\n"

pop_staticpointer_template = "@SP\n\
AM=M-1\n\
D=M\n\
@%s\n\
M=D"


type_dict = {"add":"arithmetic", "sub":"arithmetic",
                   "and":"arithmetic", "or":"arithmetic",
                   "not":"arithmetic", "neg":"arithmetic",
                   "lt":"arithmetic", "gt":"arithmetic",
                   "eq":"arithmetic", "leq":"arithmetic",
                   "geq":"arithmetic", 
                   "push":"memory", "pop":"memory"}

binary_ops = ["add", "sub", "and", "or"]
unary_ops = ["not", "neg"]
comp_ops = ["lt", "gt", "eq", "leq", "geq"]


op_count = 0
line_count = 0
output = ["// Assembly file generated by my awesome VM compiler\n"]

def compile_operation(op):
    global line_count

    if (op[0:2] == "//") or (len(op.split()) == 0):
        return ""

    # print "input: " + op
    operation = op.split()[0]
    header = "// '" + op +  "' (line " + str(line_count) + ")\n"
    line_count += 1

    if type_dict[operation] == "arithmetic":
        return header + compile_arithmetic(op)
    elif type_dict[operation] == "memory":
        return header + compile_memory(op)

def compile_arithmetic(op):
    global op_count
    out_string = ""
    if op in comp_ops:
        out_string += comp_template % (op_count, operations_dict[op], op_count, \
            op_count, op_count, op_count, op_count)
        op_count += 1
    elif op in unary_ops:
        out_string += unary_template % operations_dict[op]
    else:
        out_string += binary_template % operations_dict[op]
    return out_string

def compile_memory(op):
    global output
    instructions = op.split()
    inst = instructions[0]
    argtype = instructions[1]
    val = int(instructions[2])
    if inst == "push":
        if argtype == "constant":
            return push_const_template % val
        elif argtype == "static":
            return push_staticpointer_template % ("f." + str(val))
        elif argtype == "pointer":
            if val == 0:
                return push_staticpointer_template % ("THIS")
            else:
                return push_staticpointer_template % ("THAT")
        else:
            return push_var_template % (val, vars_dict[argtype][0], vars_dict[argtype][1])
    elif inst == "pop":
        if argtype != "constant":
            if argtype == "static":
                return pop_staticpointer_template % ("f." + str(val))
            elif argtype == "pointer":
                if val == 0:
                    return pop_staticpointer_template % "THIS"
                else:
                    return pop_staticpointer_template % "THAT"
            else:
                return pop_template % (val, vars_dict[argtype][0], vars_dict[argtype][1])

def main():
    global output

    if len(sys.argv) == 1:
        inputfname = "test.txt"
    else:
        inputfname = sys.argv[1]
    outputfname = inputfname.split('.')[0] + ".asm"

    inputf = open(inputfname)
    output += ["// Input filename: %s\n" % inputfname]
    for line in inputf.readlines():
        output += [compile_operation(line.strip())]

    outputf = open(outputfname, 'w')
    for outl in output:
        outputf.write(outl)

    outputf.write("(END)\n@END\n0;JMP");
    inputf.close()
    outputf.close()
    print "Output written to " + outputfname


if __name__ == "__main__":
    main()

D：

import std.stdio, std.string, std.conv, std.format, std.c.stdlib;

string[string] operations_dict, type_dict;
string[][string] vars_dict;
string[] arithmetic, memory, comp_ops, unary_ops, binary_ops, lines, output;
string start, end, binary_template, unary_template,
        comp_template, push_tail_template, push_const_template,
        push_var_template, push_staticpointer_template,
        pop_template, pop_staticpointer_template;
int op_count, line_count;

void build_dictionaries() {
    vars_dict = ["this":["THIS","M"],
                 "that":["THAT","M"],
                 "argument":["ARG","M"],
                 "local":["LCL","M"],
                 "static":["f.%d","M"],
                 "temp":["TEMP","A"]];

    operations_dict = ["add":"+", "sub":"-",
                   "and":"&", "or":"|",
                   "not":"!", "neg":"-",
                   "lt":"JLT", "gt":"JGT",
                   "eq":"JEQ", "leq":"JLE",
                   "geq":"JGE"];

    type_dict = ["add":"arithmetic", "sub":"arithmetic",
                   "and":"arithmetic", "or":"arithmetic",
                   "not":"arithmetic", "neg":"arithmetic",
                   "lt":"arithmetic", "gt":"arithmetic",
                   "eq":"arithmetic", "leq":"arithmetic",
                   "geq":"arithmetic", 
                   "push":"memory", "pop":"memory"];

    binary_ops = ["add", "sub", "and", "or"];
    unary_ops = ["not", "neg"];
    comp_ops = ["lt", "gt", "eq", "leq", "geq"];
}

bool is_in(string s, string[] list) {
    foreach (str; list)
        if (str==s) return true;
    return false;
}

void build_strings() {
    start = "@SP\nAM=M-1\n";
    end = "@SP\nM=M+1\n";

    binary_template = start ~ "D=M\n"
    "@SP\n"
    "AM=M-1\n"
    "M=M%sD\n" ~ end;

    unary_template = start ~ "M=%sM\n" ~ end;

    comp_template = start ~ "D=M\n"
    "@SP\n"
    "AM=M-1\n"
    "D=M-D\n"
    "@COMP.%s.TRUE\n"
    "D;%s\n"
    "@COMP.%s.FALSE\n"
    "0;JMP\n"
    "(COMP.%s.TRUE)\n"
    "@SP\n"
    "A=M\n"
    "M=-1\n"
    "@SP\n"
    "M=M+1\n"
    "@COMP.%s.END\n"
    "0;JMP\n"
    "(COMP.%s.FALSE)\n"
    "@SP\n"
    "A=M\n"
    "M=0\n" ~ end ~ "(COMP.%s.END)\n";

    push_tail_template = "@SP\n"
    "A=M\n"
    "M=D\n"
    "@SP\n"
    "M=M+1\n";

    push_const_template = "@%s\nD=A\n" ~ push_tail_template;

    push_var_template = "@%s\n"
    "D=A\n"
    "@%s\n"
    "A=%s+D\n"
    "D=M\n" ~ push_tail_template;

    push_staticpointer_template = "@%s\nD=M\n" ~ push_tail_template;

    pop_template = "@%s\n"
    "D=A\n"
    "@%s\n"
    "D=%s+D\n"
    "@R13\n"
    "M=D\n"
    "@SP\n"
    "AM=M-1\n"
    "D=M\n"
    "@R13\n"
    "A=M\n"
    "M=D\n";

    pop_staticpointer_template = "@SP\n"
    "AM=M-1\n"
    "D=M\n"
    "@%s\n"
    "M=D";
}

void init() {
    op_count = 0;
    line_count = 0;
    output = ["// Assembly file generated by my awesome VM compiler\n"];
    build_strings();
    build_dictionaries();
}

string compile_operation(string op) {
    if (op.length == 0 || op[0..2] == "//")
        return "";
    string operation = op.split()[0];
    string header = "// '" ~ op ~  "' (line " ~ to!string(line_count) ~ ")\n";
    ++line_count;

    if (type_dict[operation] == "arithmetic")
        return header ~ compile_arithmetic(op);
    else
        return header ~ compile_memory(op);
}

string compile_arithmetic(string op) {
    if (is_in(op, comp_ops)) {
        string out_string = format(comp_template, op_count, operations_dict[op], op_count, 
            op_count, op_count, op_count, op_count);
        op_count += 1;
        return out_string;
    } else if (is_in(op, unary_ops))
        return format(unary_template, operations_dict[op]);
    else
        return format(binary_template, operations_dict[op]);
}

string compile_memory(string op) {
    string[] instructions = op.split();
    string inst = instructions[0];
    string argtype = instructions[1];
    int val = to!int(instructions[2]);
    if (inst == "push") {
        if (argtype == "constant") {
            return format(push_const_template, val);
        } else if (argtype == "static")
            return format(push_staticpointer_template, ("f." ~ to!string(val)));
        else if (argtype == "pointer")
            if (val == 0)
                return format(push_staticpointer_template, "THIS");
            else
                return format(push_staticpointer_template, "THAT");
        else
            return format(push_var_template, val, vars_dict[argtype][0], vars_dict[argtype][1]);
    } else {
        if (argtype != "constant") {
            if (argtype == "static")
                return format(pop_staticpointer_template, ("f." ~ to!string(val)));
            else if (argtype == "pointer") {
                if (val == 0)
                    return format(pop_staticpointer_template, "THIS");
                else
                    return format(pop_staticpointer_template, "THAT");
            }
            else
                return format(pop_template, val, vars_dict[argtype][0], vars_dict[argtype][1]);
        } else {
            return "";
        }
    }
}

void main(string args[]) {
    init();
    if (args.length < 2) {
        writefln("usage: %s <filename>", args[0]);
        exit(0);
    }
    string inputfname = args[1];
    string outputfname = args[1].split(".")[0] ~ ".asm";

    auto inputf = File(inputfname, "r");
    output ~= format("// Input filename: %s\n", inputfname);
    foreach (line; inputf.byLine) {
        output ~= compile_operation(to!string(line).strip);
    }
    inputf.close();

    auto outputf = File(outputfname, "w");
    foreach (outl; output)
        outputf.write(outl);

    outputf.write("(END)\n@END\n0;JMP");
    outputf.close();
    writeln("Compilation successful. Output written to " ~ outputfname);
}

Answer 1

对于output变量，使用Appender（docs）：

import std.array : appender;

void main() {
   auto output = appender!string("// Assembly file generated by my awesome VM compiler\n");
   //...
   output.put(format("// Input filename: %s\n", inputfname));
   foreach (line; inputf.byLine) {
       output.put(compile_operation(line.to!string().strip()));
   }
   //...
   outputf.write(output.data());
   //...
}

另外，我建议您将type_dict更改为类似int[string]的内容，并将其与整数常量一起使用。在

^{pr2}$

使用canFind方法（docs）代替自定义is_in。甚至可以尝试SortedRange（docs）。在

Answer 2

您可能希望尝试使用std.array.appender作为输出，而不是在主函数中使用数组串联，因为appender可以最大限度地减少分配的数量，并且通常对追加进行了优化。您也可以尝试使用reserve。在

//Note: untested code
import std.array;
auto output = appender!string();

void init() {
   ...
   output.put("// Assembly file generated by my awesome VM compiler\n");
   ...
}

void main() {
   ...
   output.put(format("// Input filename: %s\n", inputfname));
   foreach (line; inputf.byLine) {
       output.put(compile_operation(to!string(line).strip));
   }
   ...
   foreach (outl; output.data)
       outputf.write(outl);
   ...
}