从HDF5迁移到NetCDF来存储pyomo模型结果

Question

当从HDF5迁移到NetCDF数据格式以存储pandas数据帧字典时，我遇到了一些问题，其中包含pyomo模型的数据和结果。在

当前的HDF5保存脚本运行正常，如下所示：

import pandas as pd
def save(prob, filename):
    with pd.HDFStore(filename, mode='w') as store:
        for name in prob._data.keys():
            store['data/'+name] = prob._data[name]
        for name in prob._result.keys():
            store['result/'+name] = prob._result[name]

其中prob是已解决的pyomo模型实例。在

由于我们在项目中迁移到PyPy是出于运行时的原因，目前还不支持h5py，因此我们还希望转移到NetCDF而不是HDF5来存储模型实例。在

{cdf}与以下cdf}格式兼容：

^{pr2}$

尽管看起来与前面的HDF5脚本非常相似，但我在这里得到以下错误：

  urbs.save(prob, os.path.join(result_dir, '{}.nc'.format(sce)))
File "/home/scandas/nas/pypy_for_asinus/urbs_pypy/urbs/saveload.py", line 63, in save
  ds['data/'+name] = prob._data[name]
File "/home/scandas/nas/pypy_for_asinus/pypy3-v6.0.0-linux64/site-packages/xarray/core/dataset.py", line 899, in __setitem__
  self.update({key: value})
File "/home/scandas/nas/pypy_for_asinus/pypy3-v6.0.0-linux64/site-packages/xarray/core/dataset.py", line 2305, in update 
  variables, coord_names, dims = dataset_update_method(self, other)
File "/home/scandas/nas/pypy_for_asinus/pypy3-v6.0.0-linux64/site-packages/xarray/core/merge.py", line 580, in dataset_update_method
  indexes=dataset.indexes)
File "/home/scandas/nas/pypy_for_asinus/pypy3-v6.0.0-linux64/site-packages/xarray/core/merge.py", line 434, in merge_core
  aligned = deep_align(coerced, join=join, copy=False, indexes=indexes)
File "/home/scandas/nas/pypy_for_asinus/pypy3-v6.0.0-linux64/site-packages/xarray/core/alignment.py", line 213, in deep_align
  exclude=exclude)
File "/home/scandas/nas/pypy_for_asinus/pypy3-v6.0.0-linux64/site-packages/xarray/core/alignment.py", line 164, in align
  new_obj = obj.reindex(copy=copy, **valid_indexers)
File "/home/scandas/nas/pypy_for_asinus/pypy3-v6.0.0-linux64/site-packages/xarray/core/dataarray.py", line 906, in reindex
  indexers=indexers, method=method, tolerance=tolerance, copy=copy)
File "/home/scandas/nas/pypy_for_asinus/pypy3-v6.0.0-linux64/site-packages/xarray/core/dataset.py", line 1812, in reindex
  tolerance, copy=copy)
File "/home/scandas/nas/pypy_for_asinus/pypy3-v6.0.0-linux64/site-packages/xarray/core/alignment.py", line 324, in reindex_variables
  int_indexer = get_indexer_nd(index, target, method, tolerance)
File "/home/scandas/nas/pypy_for_asinus/pypy3-v6.0.0-linux64/site-packages/xarray/core/indexing.py", line 117, in get_indexer_nd
  flat_indexer = index.get_indexer(flat_labels, **kwargs)
File "/home/scandas/nas/pypy_for_asinus/pypy3-v6.0.0-linux64/site-packages/pandas/core/indexes/multi.py", line 2042, in get_indexer
  indexer = self._engine.get_indexer(target)
File "pandas/_libs/index.pyx", line 654, in pandas._libs.index.BaseMultiIndexCodesEngine.get_indexer
ValueError: operands could not be broadcast together with shapes (244,3) (4,) (244,3)

对于prob的_data中的某些键，形状不匹配（在连接过程中？），在分配xarray数据集的元素时出错。然而，具有类似分配的HDF5存储过程工作平稳，没有任何错误。在

编辑：prob._data字典如下所示：

{'global_prop':                value                                        description
Property                                                               
CO2 limit  150000000  Limits the sum of all created (as calculated b..., 'site':              area
Name             
Mid     280000000
South  5000000000, 
'commodity':                         price  max  maxperhour
Site  Commodity Type                          
Mid   Biomass   Stock     6.0  inf         inf
      CO2       Env       0.0  inf         inf
      Coal      Stock     7.0  inf         inf
      Elec      Demand    NaN  NaN         NaN
      Gas       Stock    27.0  inf         inf
      Hydro     SupIm     NaN  NaN         NaN
      Lignite   Stock     4.0  inf         inf
      Slack     Stock   999.0  inf         inf
      Solar     SupIm     NaN  NaN         NaN
      Wind      SupIm     NaN  NaN         NaN
South Biomass   Stock     6.0  inf         inf
      CO2       Env       0.0  inf         inf
      Coal      Stock     7.0  inf         inf
      Elec      Demand    NaN  NaN         NaN
      Elec buy  Buy       1.0  inf         inf
      Elec sell Sell      3.0  inf         inf
      Gas       Stock    27.0  inf         inf
      Hydro     SupIm     NaN  NaN         NaN
      Lignite   Stock     4.0  inf         inf
      Slack     Stock   999.0  inf         inf
      Solar     SupIm     NaN  NaN         NaN
      Wind      SupIm     NaN  NaN         NaN, 
'process':              inst-cap  cap-lo  cap-up  max-grad  min-fraction  inv-cost  fix-cost  var-cost  wacc  depreciation  area-per-cap  annuity-factor
Site  Process                                                                                                                                           
Mid   Biomass plant            0       0    5000  1.200000          0.00    875000     28000      1.40  0.07            25           NaN        0.085811
      Gas plant                0       0   80000  4.800000          0.25    450000      6000      1.62  0.07            30           NaN        0.080586
      Hydro plant              0       0    1400       inf          0.00   1600000     20000      0.00  0.07            50           NaN        0.072460
      Lignite plant            0       0   60000  0.900000          0.65    600000     18000      0.60  0.07            40           NaN        0.075009
      Photovoltaics            0   15000  160000       inf          0.00    600000     12000      0.00  0.07            25       14000.0        0.085811
      Slack powerplant    999999  999999  999999       inf          0.00         0         0    100.00  0.07             1           NaN        1.070000
      Wind park                0       0   13000       inf          0.00   1500000     30000      0.00  0.07            25           NaN        0.085811
South Biomass plant            0       0    2000  1.200000          0.00    875000     28000      1.40  0.07            25           NaN        0.085811
      Coal plant               0       0  100000  0.600000          0.50    600000     18000      0.60  0.07            40           NaN        0.075009
      Feed-in                  0       0    1500       inf          0.00         0         0      0.00  0.07             1           NaN        1.070000
      Gas plant                0       0  100000  4.800000          0.25    450000      6000      1.62  0.07            30           NaN        0.080586
      Hydro plant              0       0       0       inf          0.00   1600000     20000      0.00  0.07            50           NaN        0.072460
      Photovoltaics            0   20000  600000       inf          0.00    600000     12000      0.00  0.07            25       14000.0        0.085811
      Purchase                 0       0    1500       inf          0.00         0        80      0.00  0.07             1           NaN        1.070000
      Slack powerplant    999999  999999  999999       inf          0.00         0         0    999.00  0.07             1           NaN        1.070000
      Wind park                0       0  200000       inf          0.00   1500000     30000      0.00  0.07            25           NaN        0.085811, 
'process_commodity':                   ratio  ratio-min
Process          Commodity Direction                   
Biomass plant    Biomass   In         1.0000        NaN
                 CO2       Out        0.0000        NaN
                 Elec      Out        0.3500        NaN
Coal plant       CO2       Out        0.3000        NaN
                 Coal      In         1.0000        1.4
                 Elec      Out        0.4000        NaN
Feed-in          Elec      In         1.0000        NaN
                 Elec sell Out        1.0000        NaN
Gas plant        CO2       Out        0.2000        NaN
                 Elec      Out        0.6000        NaN
                 Gas       In         1.0000        1.2
Hydro plant      Elec      Out        1.0000        NaN
                 Hydro     In         1.0000        NaN
Lignite plant    CO2       Out        0.4000        NaN
                 Elec      Out        0.4000        NaN
                 Lignite   In         1.0000        2.0
Photovoltaics    Elec      Out        1.0000        NaN
                 Solar     In         1.0000        NaN
Purchase         CO2       Out        0.0005        NaN
                 Elec      Out        1.0000        NaN
                 Elec buy  In         1.0000        NaN
Slack powerplant CO2       Out        0.0000        NaN
                 Elec      Out        1.0000        NaN
                 Slack     In         1.0000        NaN
Wind park        Elec      Out        1.0000        NaN
                 Wind      In         1.0000        NaN, 
'transmission':                          eff  inv-cost  fix-cost  var-cost  inst-cap  cap-lo  cap-up  wacc  depreciation  annuity-factor
Site In Site Out Transmission Commodity                                                                                                 
Mid     South    hvac         Elec       0.9   1650000     16500         0         0       0     inf  0.07            40        0.075009
South   Mid      hvac         Elec       0.9   1650000     16500         0         0       0     inf  0.07            40        0.075009, 
'storage':                    inst-cap-c  cap-lo-c  cap-up-c  inst-cap-p  cap-lo-p  cap-up-p  eff-in  eff-out  inv-cost-p  inv-cost-c  fix-cost-p  fix-cost-c  var-cost-p  var-cost-c  wacc  depreciation  init  discharge  annuity-factor
Site  Storage      Commodity                                                                                                                                                                                                              
Mid   Hydrogen     Elec                0         0       inf           0         0       inf    0.64     0.64       42000        6.54           0       0.327        0.02           0  0.07            50   0.5   0.000003         0.07246
      Pump storage Elec                0     60000       inf           0      8000       inf    0.94     0.94      100000        0.00       20000       0.000        0.02           0  0.07            50   0.5   0.000000         0.07246
South Hydrogen     Elec                0         0       inf           0         0       inf    0.64     0.64       42000        6.54           0       0.327        0.02           0  0.07            50   0.5   0.000003         0.07246
      Pump storage Elec                0    163000       inf           0       500       inf    0.94     0.94      100000        0.00       20000       0.000        0.02           0  0.07            50   0.5   0.000000         0.07246, 
'demand':   Mid       South        North
           Elec        Elec         Elec
t                                       
0      0.000000     0.00000      0.00000
1  43102.490062  4877.39981  11001.19176,
'supim':                Mid                     South                     North                
       Wind Solar     Hydro      Wind Solar     Hydro      Wind Solar     Hydro
t                                                                              
0  0.000000     0  0.000000  0.000000     0  0.000000  0.000000     0  0.000000
1  0.935265     0  0.416194  0.457772     0  0.353497  0.602583     0  0.651799, 
'buy_sell_price':   Elec buy Elec sell
t                   
0                       0.00   0.00000
1                       0.08  -0.02106
'dsm': Empty DataFrame
Columns: [delay, eff, recov, cap-max-do, cap-max-up]
Index: []}

其中list['global_prop', 'commodity,', 'process', 'process_commodity', 'transmission', 'storage', 'demand', 'supim', 'buy_sell_price', 'dsm']是dict键的列表，通过它我迭代构建xarray数据集（然后将其保存到NetCDF文件中）。具体地说，我在步骤name='transmission'处得到了提到的错误。在