<p>更新代码以匹配完整日志格式,如下所示:</p>
<pre><code>2018-11-06 16:54:43.350| on thread[140447603222272 c23]| IP[192.168.0.214:5000]| master| 192.168.0.244| sqream| (stmt : 30) | sqream| Connection id - 23
</code></pre>
<p>代码:</p>
<pre><code>import re
from datetime import datetime
class Event:
__slots__ = ('start', 'statement', 'end', 'success', 'stmt')
# This limits what attribute class can have. Originally class use Dictionary to save attributes,
# but using __slots__ uses Tuple instead, and saves memory if there's lots of class instances.
"""
Class isn't necessary, and longer than other solutions.
But class gives you more control when expansion / changes are needed.
"""
def __init__(self, start, statement, end, success):
self.start = start.split('- ')[-1]
self.statement = statement.split('Statement ')[-1].strip(';')
self.end = end.split('- ')[-1]
self.success = success.split()[-1]
self.stmt = re.search(r"(?<=stmt : )[^)]*", statement).group(0)
def __str__(self):
"""
When str() or print() is called on this class instances - this will be output.
"""
return f"Event starting at {self.start}, Took {self.delta_time} sec."
def __repr__(self):
"""
repr should returns string with data that is enough to recreate class instance.
"""
output = [f"stmt : {self.stmt}",
f"Took : {self.delta_time} sec",
f"Statement: {self.statement}",
f"Status : {self.success}",
f"Start : {self.start}",
f"End : {self.end}"]
return '\n'.join(output)
@property
def delta_time(self):
"""
Converting string to datetime object to perform delta time calculation.
"""
date_format = "%Y-%m-%d %H:%M:%S"
start = datetime.strptime(self.start, date_format)
end = datetime.strptime(self.end, date_format)
return (end - start).total_seconds()
def generate_events(file):
def line_yield():
"""
Generates line inside file without need to load whole file in memory.
As generator is one-shot, using this to simplify pause / continue of
line iteration.
"""
for line_ in file:
yield line_.strip("\n")
find_list = ("Start Time", "Statement", "End Time")
generator = line_yield()
while True:
group = []
for target in find_list:
for line in generator: # our generator keeps state after the loop.
if target in line: # 'in' finds faster than regex.
group.append(line)
break
for line in generator: # now find either statement was Successful or not.
if "Success" in line or "Failed" in line:
group.append(line)
break
try:
yield Event(*group)
except TypeError:
return
def find_slowest(log_file):
formed = list(generate_events(log_file))
sorted_output = sorted(formed, key=lambda event_: event_.delta_time)
print("Recorded Events:")
for output in sorted_output:
print(output)
late_runner = sorted_output[-1]
print('\n< Slowest >')
print(repr(late_runner))
with open("logfile.log", 'r') as log:
find_slowest(log)
</code></pre>
<p>包含完整日志文件的结果:</p>
<pre><code>Recorded Events:
Event starting at 2018-11-06 16:52:01, Took 0.0 sec.
Event starting at 2018-11-06 16:52:19, Took 0.0 sec.
Event starting at 2018-11-06 16:52:27, Took 0.0 sec.
Event starting at 2018-11-06 16:52:28, Took 0.0 sec.
Event starting at 2018-11-06 16:52:30, Took 0.0 sec.
Event starting at 2018-11-06 16:52:33, Took 0.0 sec.
Event starting at 2018-11-06 16:52:38, Took 0.0 sec.
Event starting at 2018-11-06 16:52:54, Took 0.0 sec.
Event starting at 2018-11-06 16:53:04, Took 0.0 sec.
Event starting at 2018-11-06 16:53:05, Took 0.0 sec.
Event starting at 2018-11-06 16:53:18, Took 0.0 sec.
Event starting at 2018-11-06 16:53:32, Took 0.0 sec.
Event starting at 2018-11-06 16:53:36, Took 0.0 sec.
Event starting at 2018-11-06 16:53:51, Took 0.0 sec.
Event starting at 2018-11-06 16:53:55, Took 0.0 sec.
Event starting at 2018-11-06 16:53:56, Took 0.0 sec.
Event starting at 2018-11-06 16:54:03, Took 0.0 sec.
Event starting at 2018-11-06 16:54:07, Took 0.0 sec.
Event starting at 2018-11-06 16:54:27, Took 0.0 sec.
Event starting at 2018-11-06 16:54:36, Took 0.0 sec.
Event starting at 2018-11-06 16:52:14, Took 1.0 sec.
Event starting at 2018-11-06 16:53:25, Took 1.0 sec.
Event starting at 2018-11-06 16:53:40, Took 1.0 sec.
Event starting at 2018-11-06 16:54:21, Took 1.0 sec.
< Slowest >
stmt : 27
Took : 1.0 sec
Statement: drop table tati
Status : Success
Start : 2018-11-06 16:54:21
End : 2018-11-06 16:54:22
Process finished with exit code 0
</code></pre>