Python将混合格式文本文件的输出解析为键值对字典

Question

我正在尝试分析来自EMC OneFS CLI的命令的输出。我们正在尝试收集库存数据，由于我们使用的是旧版本的API，因此无法使用REST SDK。在

相反，我们决定用“hacky”的方式来完成，并解析CLI命令的输出。在

在给定文本格式变化的情况下，如何在Python中可靠地解析这些数据？请注意，空白的数量各不相同，其中一些空格用:分隔，而另一些用=分隔。在

在这方面我有一个不错的开始，我能够将每一行拆分成一个列表，并处理=符号嵌套在括号中的情况。在

我仍然被困在其他方面：

• 将第一列作为“主机名”
• 为每个包含所有键值对的主机名构建一个单独的字典（有些用:分隔，有些用=分隔）

需要解析的原始输出：

visi-91:   SerNo: S123456789P
visi-91:  Config: 912-111-231
visi-91:   Class: storage
visi-91:  Series: x_series
visi-91: Product: IQ 32000x-ssd
visi-91: Comps:  Chassis CPU     Motherboard     NVRam   FlashDrive      Disk_Controller Disk_Expander   Power_Supplies 
visi-91:   HWGen: MaunaLoa (MaunaLoa Hardware)
visi-91: Chassis: ISI36 (Isilon (CIDesigns) 36-Bay Chassis)
visi-91:     CPU: GenuineIntel (2.33GHz, stepping 0x0001067a)
visi-91:    PROC: Dual-proc, Quad-core
visi-91:     RAM: 17150812160 Bytes
visi-91:    Mobo: SupMicX7DBU (SuperMicro X7DBU Motherboard)
visi-91:   NVRam: MT25208 (Mellanox 25208 based Card) (492MB card) (size 515899392B)
visi-91: FlshDrv: None (No physical dongle supported) ((null))
visi-91:  DskCtl: LSI3081E-HBA (LSI SAS3081E SAS Controller -- HBA) (8 ports)
visi-91:  DskExp: VIT7156 (Vitesse 7156 Disk Expander)
visi-91: PwrSupl: PS1 (type=Emerson, fw=v.01.00.00)
visi-91: PwrSupl: PS2 (type=Emerson, fw=v.01.00.00)
visi-91: ChasCnt: 1 (Single-Chassis System)
visi-91:   NetIF: em*4,ib*2 (4x Intel, 2x IB)
visi-91:  IBType: ISI25208 (Isilon MT25208-based IB Card)
visi-91:  LCDver: NoriVFD (Noritake VFD)
visi-91:     IMB: Board Version 0x20f
visi-91: Power Supplies OK
visi-91: CPU Operation (raw 0x882F0000)  = Normal
visi-91: CPU Speed Limit                 = 100.00%
visi-91: Chassis Fan1 (ISI F1)           = 7469
visi-91: Chassis Fan2 (ISI F2)           = 7552
visi-91: Chassis Fan3 (ISI F3)           = 7387
visi-91: Power Supply 1 Fan              = 5504
visi-91: Power Supply 2 Fan              = 5792
visi-91: Temp Until CPU Throttle (CPU 0) = 46.0
visi-91: Temp Until CPU Throttle (CPU 1) = 42.0
visi-91: Temp Front Panel                = 17.0
visi-91: Temp Chassis 2 (ISI T2)         = 22.0
visi-91: Temp System                     = 25.0
visi-92:   SerNo: S123456785K
visi-92:  Config: 912-111-231
visi-92:   Class: storage
visi-92:  Series: x_series
visi-92: Product: IQ 32000x-ssd
visi-92: Comps:  Chassis CPU     Motherboard     NVRam   FlashDrive      Disk_Controller Disk_Expander   Power_Supplies 
visi-92:   HWGen: MaunaLoa (MaunaLoa Hardware)
visi-92: Chassis: ISI36 (Isilon (CIDesigns) 36-Bay Chassis)
visi-92:     CPU: GenuineIntel (2.33GHz, stepping 0x0001067a)
visi-92:    PROC: Dual-proc, Quad-core
visi-92:     RAM: 17150812160 Bytes
visi-92:    Mobo: SupMicX7DBU (SuperMicro X7DBU Motherboard)
visi-92:   NVRam: MT25208 (Mellanox 25208 based Card) (492MB card) (size 515899392B)
visi-92: FlshDrv: None (No physical dongle supported) ((null))
visi-92:  DskCtl: LSI3081E-HBA (LSI SAS3081E SAS Controller -- HBA) (8 ports)
visi-92:  DskExp: VIT7156 (Vitesse 7156 Disk Expander)
visi-92: PwrSupl: PS1 (type=Emerson, fw=v.01.00.00)
visi-92: PwrSupl: PS2 (type=Emerson, fw=v.01.00.00)
visi-92: ChasCnt: 1 (Single-Chassis System)
visi-92:   NetIF: em*4,ib*2 (4x Intel, 2x IB)
visi-92:  IBType: ISI25208 (Isilon MT25208-based IB Card)
visi-92:  LCDver: NoriVFD (Noritake VFD)
visi-92:     IMB: Board Version 0x20f
visi-92: Power Supplies OK
visi-92: CPU Operation (raw 0x882E0000)  = Normal
visi-92: CPU Speed Limit                 = 100.00%
visi-92: Chassis Fan1 (ISI F1)           = 7268
visi-92: Chassis Fan2 (ISI F2)           = 7307
visi-92: Chassis Fan3 (ISI F3)           = 7190
visi-92: Power Supply 1 Fan              = 5376
visi-92: Power Supply 2 Fan              = 5696
visi-92: Temp Until CPU Throttle (CPU 0) = 46.0
visi-92: Temp Until CPU Throttle (CPU 1) = 46.0
visi-92: Temp Front Panel                = 17.2
visi-92: Temp Chassis 2 (ISI T2)         = 24.0
visi-92: Temp System                     = 27.0
visi-93:   SerNo: S987654321P
visi-93:  Config: 912-111-231
visi-93:   Class: storage
visi-93:  Series: x_series
visi-93: Product: IQ 32000x-ssd
visi-93: Comps:  Chassis CPU     Motherboard     NVRam   FlashDrive      Disk_Controller Disk_Expander   Power_Supplies 
visi-93:   HWGen: MaunaLoa (MaunaLoa Hardware)
visi-93: Chassis: ISI36 (Isilon (CIDesigns) 36-Bay Chassis)
visi-93:     CPU: GenuineIntel (2.33GHz, stepping 0x0001067a)
visi-93:    PROC: Dual-proc, Quad-core
visi-93:     RAM: 17150812160 Bytes
visi-93:    Mobo: SupMicX7DBU (SuperMicro X7DBU Motherboard)
visi-93:   NVRam: MT25208 (Mellanox 25208 based Card) (492MB card) (size 515899392B)
visi-93: FlshDrv: None (No physical dongle supported) ((null))
visi-93:  DskCtl: LSI3081E-HBA (LSI SAS3081E SAS Controller -- HBA) (8 ports)
visi-93:  DskExp: VIT7156 (Vitesse 7156 Disk Expander)
visi-93: PwrSupl: PS1 (type=Emerson, fw=v.03.05.00)
visi-93: PwrSupl: PS2 (type=Emerson, fw=v.03.05.00)
visi-93: ChasCnt: 1 (Single-Chassis System)
visi-93:   NetIF: em*4,ib*2 (4x Intel, 2x IB)
visi-93:  IBType: ISI25208 (Isilon MT25208-based IB Card)
visi-93:  LCDver: NoriVFD (Noritake VFD)
visi-93:     IMB: Board Version 0x20f
visi-93: Power Supplies OK
visi-93: CPU Operation (raw 0x88250000)  = Normal
visi-93: CPU Speed Limit                 = 100.00%
visi-93: Chassis Fan1 (ISI F1)           = 7387
visi-93: Chassis Fan2 (ISI F2)           = 7152
visi-93: Chassis Fan3 (ISI F3)           = 7152
visi-93: Power Supply 1 Fan              = 8512
visi-93: Power Supply 2 Fan              = 8512
visi-93: Temp Until CPU Throttle (CPU 0) = 37.0
visi-93: Temp Until CPU Throttle (CPU 1) = 38.0
visi-93: Temp Front Panel                = 17.4
visi-93: Temp Chassis 2 (ISI T2)         = 25.0
visi-93: Temp System                     = 28.0

我目前的代码：

^{pr2}$

目前为止我的代码输出：

^{3}$

所需输出-以第一个单词作为主机名的字典键/值对。 每个主机名的单独词典：

{'hostname':'visi-91', 'SerNo':'S123456789P', 'Config':'912-111-231', 'Class':'storage', 'Series':'x_series', 'PwrSupl':'PS1(type=Emerson,fw=v.01.00.00)', 'Chassis':'ISI36(Isilon(CIDesigns)36-BayChassis)', 'NetIF':'em*4,ib*2(4xIntel,2xIB)', 'IBType':'ISI25208(IsilonMT25208-basedIBCard)'} //etc...

{'hostname':'visi-92', 'SerNo':'S123456785K', 'Config':'912-111-231', 'Class':'storage', 'Series':'x_series', 'PwrSupl':'PS1(type=Emerson,fw=v.01.00.00)' 'Chassis':'ISI36(Isilon(CIDesigns)36-BayChassis)', 'NetIF':'em*4,ib*2(4xIntel,2xIB)', 'IBType':'ISI25208(IsilonMT25208-basedIBCard)'}  //etc...

Answer 1

我认为最好的方法是使用^{}：

#!/usr/bin/env python2.7

import re

results = {}

with open('in.txt', 'r') as f:
    for line in [ l.rstrip() for l in f.readlines() ]:
        match = re.match('^(?P<hostname>[^:]+): +(?P<key>[^:=]+?)(?: *[:=] *(?P<value>.+?) *)?$', line)

        if match is None:
            # skip failed matches
            #print(line)
            continue

        hostname, key, value = match.groups()

        if key == 'Comps':
            value = [ x.strip() for x in re.sub('  +',' ',value).split(' ') ]
        else:
            try:
                value = float(value)
            except:
                pass

        if hostname not in results:
            results[hostname] = {}

        if key in results[hostname]:
            old_value = results[hostname][key]
            if isinstance(old_value, list):
                old_value.append(value)
            else:
                results[hostname][key] = [ old_value, value ]
        else:
            results[hostname][key] = value

from pprint import pprint
pprint(results)

我也有一些自由：

• 提出了另一种结果格式results[hostname][key]
• 将'Comps'数据转换为数组
• 将任何要转换为float的内容转换为数字

输出如下：

^{pr2}$

编辑：现在通过形成一个数组（例如：PwrSupl）来处理给定“key”的多次出现。在

Answer 2

你需要把每一条线分开。 将第一个作为“hostname”放入字典，然后成对处理其余的。比如：

fields = line.split(':')
my_dict["hostname"] = field[0]
for pos in range(1, len(fields), 2):
    my_dict[field(pos)] = field(pos+1)

这是基本策略；我将把实现细节留给您。：-）