在python中使用readlines（）读取日志文件

Question

我有一个示例日志文件，其中每个msisdn值（例如“msisdn”：“22969000034”）都有3个不同的状态值，如（例如“状态”：“已完成”、“状态”：“正在进行”和“状态”：“已输入”）。我必须创建一个包含以下列的报告：“msisdn（包含msisdn字段值）、requestID（包含requestID字段值）、state1（包含输入的状态字段值）、state2（包含正在进行的状态字段值）、state3（包含完成的状态字段值）、text”（仅当msisdn的状态值已完成时才包含文本字段值，否则，如果状态值正在进行/输入，则将为“NA”）

日志文件：

{"methodName":"offlineNotifQueueListener()","className":"com.ericsson.ms.business.jms.listener.impl.ActiveMqListener","text":"Processing Offline request","errorText":"NA","status":"NA","requestID":"OFFLINE_Notification_10.10.46.95_e6444dbc-b7ef-41d6-9111-eaa384717b27","orgTransactionID":"895139242296900003","msisdn":"22969000034","action":"Notification","input":"pre_Adhoc","sourceChannel":"OFFLINE","product":"1399","transactionDateTime":"2020-03-24T22:59:59+0100","state":"ENTERED"} {"methodName":"addNotificationText()","className":"com.ericsson.ms.business.service.notification.impl.MSProductNotificationImpl","text":"Evaluating notification for product : 1399","errorText":"NA","status":"NA","requestID":"OFFLINE_Notification_10.10.46.95_e6444dbc-b7ef-41d6-9111-eaa384717b27","orgTransactionID":"895139242296900003","msisdn":"22969000034","action":"Notification","input":"pre_Adhoc","sourceChannel":"OFFLINE","product":"1399","transactionDateTime":"2020-03-24T22:59:59+0100","state":"IN_PROGRESS"} {"methodName":"executeConstraint()","className":"com.ericsson.ms.business.service.constraint.impl.MSConstraintsImpl","text":"Executing constraint : OfferCHeck","errorText":"NA","status":"NA","requestID":"OFFLINE_Notification_10.10.46.95_e6444dbc-b7ef-41d6-9111-eaa384717b27","orgTransactionID":"895139242296900003","msisdn":"22969000034","action":"Notification","input":"pre_Adhoc","sourceChannel":"OFFLINE","product":"NA","transactionDateTime":"2020-03-24T22:59:59+0100","state":"IN_PROGRESS"} {"methodName":"getValueFromAIR()","className":"com.ericsson.ms.business.service.templateparam.impl.TemplateParamServiceImpl","text":"Evaluating parameter --\u003e ${OFFER_123089_VALUE}","errorText":"NA","status":"NA","requestID":"OFFLINE_Notification_10.10.46.95_e6444dbc-b7ef-41d6-9111-eaa384717b27","orgTransactionID":"895139242296900003","msisdn":"22969000034","action":"Notification","input":"pre_Adhoc","sourceChannel":"OFFLINE","product":"1399","transactionDateTime":"2020-03-24T22:59:59+0100","state":"IN_PROGRESS"} {"methodName":"executeCommand","className":"com.ericsson.ms.business.service.templateparam.impl.TemplateParamServiceImpl","text":"Executing AIR command ---\u003e GetOffers","errorText":"NA","status":"NA","requestID":"OFFLINE_Notification_10.10.46.95_e6444dbc-b7ef-41d6-9111-eaa384717b27","orgTransactionID":"895139242296900003","msisdn":"22969000034","action":"Notification","input":"pre_Adhoc","sourceChannel":"OFFLINE","product":"1399","transactionDateTime":"2020-03-24T22:59:59+0100","state":"COMPLETED"}

我正在使用的代码仅从日志文件的第一行提供输出。如何读取完整的文件以获得所需的输出

代码：

with open('home/msLog.log', 'r') as f:
    data = '['+f.readlines()[0].replace('} {','}, {')+']'
    print(data)
    data1 = json.loads(data)
#    print(data1)
    #i += 1
    for x in data1:
    #print(f"msisdn: {x['msisdn']}, status: {x['state']}, requestID: {x['requestID']}")
        print(x['msisdn'],x['state'],x['product'])
    if x['state'] == 'ENTERED':
        msd = x['msisdn']
        RID = x['requestID']
        state1 = x['state']
        value = msd, RID, state1
        list1.append(value)
    elif x['state'] == 'IN_PROGRESS' and x['msisdn'] == msd:
        state2 = x['state']
        list1.append(state2)
    elif x['state'] == 'COMPLETED' and x['msisdn'] == msd:
        state3 = x['state']
        text = x['text']
        value = state3, text
        list1.append(value)
print(list1)

Answer 1

看起来文件是jsonlines格式的——每一行都是一个单独的json对象。因此，您可以逐行反序列化，如下所示：

with open('home/msLog.log', 'r') as f:
    for line in f:
        x = json.loads(line)

        # Process the line (code unchanged from the question)
        if x['state'] == 'ENTERED':
            msd = x['msisdn']
            RID = x['requestID']
            state1 = x['state']
            value = msd, RID, state1
            list1.append(value)
        elif x['state'] == 'IN_PROGRESS' and x['msisdn'] == msd:
            state2 = x['state']
            list1.append(state2)
        elif x['state'] == 'COMPLETED' and x['msisdn'] == msd:
            state3 = x['state']
            text = x['text']
            value = state3, text
            list1.append(value)
print(list1)