我想把系统日志文件解析成csv格式

Question

not running?' Jan 23 16:54:16 pfsense php: rc.start_packages: The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'

Jan 23 16:54:19 pfsense php: rc.start_packages: [FreeRADIUS]: XMLRPC sync is starting with timeout 150 seconds.

Jan 23 16:54:19 pfsense php: rc.start_packages: [FreeRADIUS]: XMLRPC Sync with '20.20.20.2' aborted due to the following error(s): Misconfigured Replication Target Port.

Jan 23 16:54:19 pfsense php: rc.start_packages: [FreeRADIUS]: XMLRPC sync is ending.

Jan 23 16:54:19 pfsense php: rc.start_packages: The command '/usr/local/etc/rc.d/radiusd.sh stop' returned exit code '1', the output was 'radiusd not running?'

Jan 23 16:54:21 pfsense php: rc.start_packages: [FreeRADIUS]: XMLRPC sync is starting with timeout 150 seconds.

我想通过python将上述系统日志文件的数据解析成csv文件。首先，我尝试了以下代码

    import csv
    import itertools 

    with open('system.log', 'r') as in_file:
        stripped = (line.strip() for line in in_file)
        lines = (line for line in stripped if line)
        grouped = zip(*[lines] * 7)
        with open('system.csv', 'w') as out_file:
            writer = csv.writer(out_file)
            writer.writerow(('month', 'day', 
    'time','pfsense','type','package','comment'))
            writer.writerows(grouped)

邮件头很好，但文件没有真正转换成csv。因此，我将该文件转换为分隔文本文件，并使用以下代码进行解析。

^{pr2}$

以上代码运行良好。但我需要一个脚本，可以将原始日志文件数据转换为带有标题的csv文件。

Answer 1

Question: ... I need a script that can convert the raw log file data into csv file with headers.

只读写一行数据。
考虑以下示例：

with open('system.csv', 'w') as out_file,
    open('system.log', 'r') as in_file:

    writer = csv.writer(out_file)
    writer.writerow(['month', 'day','time', 'pfsense', 'type', 'package', 'comment'])

    for line in in_file:
        columns = line[:-1].split(' ')
        columns[6] = ' '.join(columns[6:])
        writer.writerow(columns[:7])

Output:
month,day,time,pfsense,type,package,comment Jan,23,16:54:16,pfsense,php:,rc.start_packages:,The command... (omitted for brevity) Jan,23,16:54:19,pfsense,php:,rc.start_packages:,[FreeRADIUS]: XMLRPC... (omitted for brevity) Jan,23,16:54:19,pfsense,php:,rc.start_packages:,[FreeRADIUS]: XMLRPC... (omitted for brevity) Jan,23,16:54:19,pfsense,php:,rc.start_packages:,[FreeRADIUS]: XMLRPC... (omitted for brevity) Jan,23,16:54:19,pfsense,php:,rc.start_packages:,The command... (omitted for brevity) Jan,23,16:54:21,pfsense,php:,rc.start_packages:,[FreeRADIUS]: XMLRPC... (omitted for brevity)

用Python:3.4.2测试