解析日志行并将唯一IP另存为JSON blob

Question

我试图解析包含SSH日志的文本文件。示例日志行如下所示：

6T08:07:19.052699Z [SSHService b'ssh-userauth' on HoneyPotSSHTransport,666,5.188.86.172] login attempt [b'root'/b'admin'] succeeded
6T08:07:19.052699Z [SSHService b'ssh-userauth' on HoneyPotSSHTransport,666,5.188.86.173] login attempt [b'root'/b'admin'] succeeded
6T08:07:19.052699Z [SSHService b'ssh-userauth' on HoneyPotSSHTransport,666,5.188.86.172] login attempt [b'root'/b'admin'] succeeded
6T08:07:19.052699Z [SSHService b'ssh-userauth' on HoneyPotSSHTransport,666,5.188.86.172] login attempt [b'root'/b'admin'] succeeded

日志行中会有重复的IP

我的用例是解析每个日志行，获取唯一的IP地址，并生成一个JSON blob作为输出。最终的JSON文件应该只有唯一IP地址的JSON结果

使用Python实现这一点的可能方法有哪些

Answer 1

我认为使用ipaddresspython标准库回答这个问题会很好

import sys
import json
import re

from ipaddress import IPv4Address, AddressValueError

ipre = re.compile(r'\d+\.\d+\.\d+\.\d+')

with open(sys.argv[1]) as fin:
    data = fin.read()

ips = []

for ip in ipre.findall(data):

    # Validate found IP addresses
    try:
        ips.append(str(IPv4Address(ip)))
    except AddressValueError as e:
        print(f"IP address '{ip}' is invalid: {e}")

print(json.dumps(list(set(ips))))

这将为您提供一个唯一的已验证IP地址列表，其中包含一个JSON格式的列表

输入文件的第一行显示无效的IP地址

输入

6T08:07:19.052699Z [SSHService b'ssh-userauth' on HoneyPotSSHTransport,666,500.188.86.172] login attempt [b'root'/b'admin'] succeeded
6T08:07:19.052699Z [SSHService b'ssh-userauth' on HoneyPotSSHTransport,666,5.188.86.173] login attempt [b'root'/b'admin'] succeeded
6T08:07:19.052699Z [SSHService b'ssh-userauth' on HoneyPotSSHTransport,666,5.188.86.172] login attempt [b'root'/b'admin'] succeeded
6T08:07:19.052699Z [SSHService b'ssh-userauth' on HoneyPotSSHTransport,666,5.188.86.172] login attempt [b'root'/b'admin'] succeeded

输出

IP address '500.188.86.172' is invalid: Octet 500 (> 255) not permitted in '500.188.86.172'
["5.188.86.172", "5.188.86.173"]

Answer 2

您是否尝试读取IP并创建一组？ 而不是基于它制作json相关的东西？ 根据定义，集合具有唯一的值

Answer 3

要提取文本并获取IP地址，可以使用^{}方法或使用regular expression。然后，将IP地址添加到Set()。当您使用^{}时，将不会有重复项。最后，将Set()转换为列表变量并另存为JSON File