回答此问题可获得 20 贡献值,回答如果被采纳可获得 50 分。
<p>我试图优化我的代码,以便在ASC光栅文件中循环。该函数的输入是来自ASC文件的数据数组,其形状为1.000 x 1.000(1mio数据点)、ASC文件信息和一个列跳过值。在这种情况下,跳过值并不重要</p>
<p>我的函数带有for循环代码,如果data==nodata\u值,则执行得很好并跳过一个数组单元格。以下是函数:</p>
<pre><code>def asc_process_single(self, asc_array, asc_info, skip=1):
# ncols = asc_info['ncols']
nrows = asc_info['nrows']
xllcornor = asc_info['xllcornor']
yllcornor = asc_info['yllcornor']
cellsize = asc_info['cellsize']
nodata_value = asc_info['nodata_value']
raster_size_y = cellsize*nrows
# raster_size_x = cellsize*ncols
# Looping over array rows and cols with skipping
xyz = []
for row in range(asc_array.shape[0])[::skip]:
for col in range(asc_array.shape[1])[::skip]:
val_z = asc_array[row, col] # Z value of datapoint
# The no data value is not processed
if val_z == nodata_value:
pass
else:
# Xcoordinate for current Z value
val_x = xllcornor + (col * cellsize)
# Ycoordinate for current Z value
val_y = yllcornor + raster_size_y - (row * cellsize)
# x, y, z to LIST
xyz.append([val_x, val_y, val_z])
return xyz
</code></pre>
<p>在存在nodata_值的ASC文件上重复7次的计时为:</p>
<pre><code>593 ms ± 34.4 ms per loop (mean ± std. dev. of 10 runs, 1 loop each)
</code></pre>
<p>我想通过列表理解我可以做得更好:</p>
<pre><code>def asc_process_single_listcomprehension(self, asc_array, asc_info, skip=1):
# ncols = asc_info['ncols']
nrows = asc_info['nrows']
xllcornor = asc_info['xllcornor']
yllcornor = asc_info['yllcornor']
cellsize = asc_info['cellsize']
nodata_value = asc_info['nodata_value']
raster_size_y = cellsize*nrows
# raster_size_x = cellsize*ncols
# Looping over array rows and cols with skipping
rows = range(asc_array.shape[0])[::skip]
cols = range(asc_array.shape[1])[::skip]
xyz = [[xllcornor + (col * cellsize),
yllcornor + raster_size_y - (row * cellsize),
asc_array[row, col]]
for row in rows for col in cols
if asc_array[row, col] != nodata_value]
return xyz
</code></pre>
<p>然而,这比我的for循环执行得慢,我想知道为什么</p>
<pre><code>757 ms ± 58.4 ms per loop (mean ± std. dev. of 10 runs, 1 loop each)
</code></pre>
<p>是因为列表理解查找asc_数组[row,col]两次吗?光是这项手术就要花很多钱</p>
<pre><code>193 ns ± 11.4 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
</code></pre>
<p>而不是仅使用我的for循环中数组中已存在的查找值中的z值进行赋值</p>
<pre><code>51.2 ns ± 1.18 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
</code></pre>
<p>这样做100万次,就可以把列表理解所花费的时间加起来。
如何进一步优化我的列表理解,使其比for循环性能更好?还有其他提高性能的方法吗</p>
<p>编辑:
解决方案:
我尝试了给出的两个建议</p>
<ol>
<li>在我的列表理解中引用我的Z值,不要这样做
在数组中查找两次,这需要更长的时间</李>
<li>重新编写函数以处理numpy阵列的问题</li>
</ol>
<p>我将列表改写为:</p>
<pre><code>xyz = [[xllcornor + (col * cellsize),
yllcornor + raster_size_y - (row * cellsize),
val_z]
for row in rows for col in cols for val_z in
[asc_array[row, col]]
if val_z != nodata_value]
</code></pre>
<p>numpy函数变成了这样:</p>
<pre><code>def asc_process_numpy_single(self, asc_array, asc_info, skip):
# ncols = asc_info['ncols']
nrows = asc_info['nrows']
xllcornor = asc_info['xllcornor']
yllcornor = asc_info['yllcornor']
cellsize = asc_info['cellsize']
nodata_value = asc_info['nodata_value']
raster_size_y = cellsize*nrows
# raster_size_x = cellsize*ncols
rows = np.arange(0,asc_array.shape[0],skip)[:,np.newaxis]
cols = np.arange(0,asc_array.shape[1],skip)
x = np.zeros((len(rows),len(cols))) + xllcornor + (cols * cellsize)
y = np.zeros((len(rows),len(cols))) + yllcornor + raster_size_y - (rows *
cellsize)
z = asc_array[::skip,::skip]
xyz = np.asarray([x,y,z]).T.transpose((1,0,2)).reshape(
(int(len(rows)*len(cols)), 3) )
mask = (xyz[:,2] != nodata_value)
xyz = xyz[mask]
return xyz
</code></pre>
<p>我在numpy函数的最后两行添加了掩码,因为我不想要nodata_值。
演出顺序如下:;对于循环、列表理解、列表理解建议和numpy函数建议:</p>
<pre><code>609 ms ± 44.8 ms per loop (mean ± std. dev. of 10 runs, 1 loop each)
706 ms ± 22 ms per loop (mean ± std. dev. of 10 runs, 1 loop each)
604 ms ± 21.5 ms per loop (mean ± std. dev. of 10 runs, 1 loop each)
70.4 ms ± 1.26 ms per loop (mean ± std. dev. of 10 runs, 1 loop each)
</code></pre>
<p>列表理解在优化时与for循环相比,但numpy函数以9倍的速度加快了参与方的速度</p>
<p>非常感谢您的评论和建议。我今天学到了很多</p>