<p>我建议您首先使用一个脚本将数据转换成合适的CSV格式,然后可以使用Pandas毫无问题地加载。你知道吗</p>
<p>大多数数据都是固定宽度的格式,因此列位置可以硬编码,并应用<code>strip()</code>删除任何空格。可以从时间戳行中提取时间和日期,并将其添加到其中每个条目的开头。你知道吗</p>
<p><a href="https://docs.python.org/3/library/itertools.html?highlight=groupby#itertools.groupby" rel="nofollow noreferrer">^{<cd2>}</a>用于以块的形式读取文件,这还具有跳过段之间不需要的行的效果。它的工作原理是一次读取一行文件,然后将每一行传递给<code>key</code>函数(这是使用<code>lambda</code>函数内联提供的,以节省空间)。如果函数的结果更改,groupby函数将返回具有相同返回的行的列表。在本例中,它正在测试行是否以<code>A=<</code>字符开头。因此您得到一个返回,其中<code>key</code>是<code>True</code>包含以时间戳开始的行列表。然后是以<code>A=<</code>开头的行列表(在本例中只有一行)。这使得处理整个片段更容易,而不必担心它在哪里完成。第一个条目是时间戳,后跟所有条目。你知道吗</p>
<p>列表理解用于从行中提取每个单独的值,并创建值的<code>row</code>。<a href="https://docs.python.org/3.6/library/itertools.html#itertools-recipes" rel="nofollow noreferrer">^{<cd10>}</a>配方用于从<code>cols</code>读取固定列位置,以获得滑动<code>start</code>和<code>end</code>列位置。它们用作字符串片段,用于提取每列的字符。然后,该字符串应用<code>.strip()</code>来删除任何周围的空格。你知道吗</p>
<pre><code>from itertools import groupby, tee
import csv
def pairwise(iterable):
"s -> (s0,s1), (s1,s2), (s2, s3), ..."
a, b = tee(iterable)
next(b, None)
return zip(a, b)
cols = [0, 5, 12, 15, 19, 24, 28, 32, 37, 41, 45, 50, 54, 58, 63, 68, 71, 76] # + rest
header = None
with open('data.txt') as f_input, open('output.csv', 'w', newline='') as f_output:
csv_output = csv.writer(f_output)
for key, group in groupby(f_input, lambda x: not x.startswith('A=<')):
if key:
# Extract the timestamp from the first returned line in the segment
# Split the line on spaces and take the first 3 elements
timestamp = next(group).split()[:3]
block = []
for line in group:
row = [line[start:end].strip() for start, end in pairwise(cols)]
row.append(line[cols[-1]:].strip()) # Add GHI column
block.append(timestamp + row)
if not header:
header = block[0][3:]
header[2] = 'Unknown'
csv_output.writerow(['Day', 'Date', 'Time'] + header)
csv_output.writerows(block[1:])
</code></pre>
<p>对于你提供的数据,这将给你</p>
<pre class="lang-none prettyprint-override"><code>Day,Date,Time,IFJ,SC/LY,Unknown,AB,CD?,EF,GH,IJK,LM,NO,PQR,ST,UV,WXY,ZA,BC,DEF,GHI
Tuesday,19-April-2010,00:01,1234,SC 122,,A,20?,31,1,4?,10,3,2?,19,9,5?,-,,-?,30
Tuesday,19-April-2010,00:01,1234,SC 123,',B,60?,11,2,3?,10,5,6?,19,9,4?,17,4,1?,30
Tuesday,19-April-2010,00:01,1234,SC 124,,C,20?,21,2,2?,10,2,1?,9,9,0?,-,,-?,34
Tuesday,19-April-2010,00:01,1234,SC 125,^,1,100?,-,,-?,0,3,3?,-,,-?,-,,-?,10
Tuesday,19-April-2010,00:01,1234,SC 226,*,5,60?,-,,-?,14,4,3?,9,9,2?,7,3,3?,18
Tuesday,19-April-2010,00:03,1234,SC 122,,A,21?,31,5,4?,17,3,2?,19,9,1?,-,,-?,31
Tuesday,19-April-2010,00:03,1234,SC 123,',B,61?,11,2,3?,19,5,6?,19,9,4?,17,4,3?,32
Tuesday,19-April-2010,00:03,1234,SC 124,,C,21?,21,5,2?,10,2,1?,9,9,0?,-,,-?,33
Tuesday,19-April-2010,00:03,1234,SC 125,^,1,101?,-,,-?,7,3,3?,-,,-?,-,,-?,14
Tuesday,19-April-2010,00:03,0000,SC 226,*,5,61?,-,,-?,14,4,3?,9,9,2?,7,3,3?,18
</code></pre>
<p>为了更好地理解脚本的实际工作方式,我建议您添加一个小的测试文件(带有两个段),然后还添加一些print语句。你知道吗</p>