fast-tabnet 0.2.0

fastai的TabNet

This is an adaptation of TabNet (Attention-based network for tabular data) for fastai (>=2.0) library. The original paper https://arxiv.org/pdf/1908.07442.pdf.

安装

pip install fast_tabnet

如何使用

model = TabNetModel(emb_szs, n_cont, out_sz, embed_p=0., y_range=None, n_d=8, n_a=8, n_steps=3, gamma=1.5, n_independent=2, n_shared=2, epsilon=1e-15, virtual_batch_size=128, momentum=0.02)

参数emb_szs, n_cont, out_sz, embed_p, y_range与fastai TabularModel相同。在

• 注意：内景 预测层的维数（通常在4到64之间）
• 内景：内景 注意层的尺寸（通常在4到64之间）
• n_步骤：int 新作业中成功的步骤数（通常在3到10之间）
• gamma:浮动 浮动在1以上，注意力更新的比例因子（通常在1.0到2.0之间）
• 动量：浮动 介于0和1之间的浮点值，将用于所有批次定额中的动量
• n_独立：int 每个GLU块中独立GLU层的数量（默认2）
• n_共享：int 每个GLU块中独立GLU层的数量（默认2）
• epsilon:浮动 避免使用log（0），这个值应该保持很低

示例

下面是fastai库中的一个示例，但使用的模型是TabNet

fromfastai.basicsimport*fromfastai.tabular.allimport*fromfast_tabnet.coreimport*

.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}

cat_names=['workclass','education','marital-status','occupation','relationship','race','native-country','sex']cont_names=['age','fnlwgt','education-num']procs=[Categorify,FillMissing,Normalize]splits=RandomSplitter()(range_of(df_main))

to=TabularPandas(df_main,procs,cat_names,cont_names,y_names="salary",y_block=CategoryBlock(),splits=splits)

dls=to.dataloaders(bs=32)

dls.valid.show_batch()

to_tst=to.new(df_test)to_tst.process()to_tst.all_cols.head()

.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}

emb_szs=get_emb_sz(to)

这就是模型的用途

model=TabNetModel(emb_szs,len(to.cont_names),dls.c,n_d=8,n_a=8,n_steps=5,mask_type='entmax');

learn=Learner(dls,model,CrossEntropyLossFlat(),opt_func=Adam,lr=3e-2,metrics=[accuracy])

learn.lr_find()

SuggestedLRs(lr_min=0.2754228591918945, lr_steep=1.9054607491852948e-06)

learn.fit_one_cycle(5)

Tabnet可解释性

# feature importance for 2k rowsdl=learn.dls.test_dl(df.iloc[:2000],bs=256)feature_importances=tabnet_feature_importances(learn.model,dl)

# per sample interpretationdl=learn.dls.test_dl(df.iloc[:20],bs=20)res_explain,res_masks=tabnet_explain(learn.model,dl)

plt.xticks(rotation='vertical')plt.bar(dl.x_names,feature_importances,color='g')plt.show()

defplot_explain(masks,lbls,figsize=(12,12)):"Plots masks with `lbls` (`dls.x_names`)"fig=plt.figure(figsize=figsize)ax=fig.add_axes([0.1,0.1,0.8,0.8])plt.yticks(np.arange(0,len(masks),1.0))plt.xticks(np.arange(0,len(masks[0]),1.0))ax.set_xticklabels(lbls,rotation=90)plt.ylabel('Sample Number')plt.xlabel('Variable')plt.imshow(masks)

plot_explain(res_explain,dl.x_names)

基于贝叶斯优化的超参数搜索

如果你的数据集不是很大，你可以用贝叶斯优化为表格模型调整超参数。如果度量足够敏感，可以使用这种方法直接优化度量（在我们的示例中不是这样，而是使用验证损失）。另外，您应该创建第二个验证集，因为您将使用第一个验证集作为贝叶斯优化的训练集。在

您可能需要安装优化器pip install bayesian-optimization

# The function we'll optimize@lru_cache(1000)defget_accuracy(n_d:Int,n_a:Int,n_steps:Int):model=TabNetModel(emb_szs,len(to.cont_names),dls.c,n_d=n_d,n_a=n_a,n_steps=n_steps,gamma=1.5)learn=Learner(dls,model,CrossEntropyLossFlat(),opt_func=opt_func,lr=3e-2,metrics=[accuracy])learn.fit_one_cycle(5)returnfloat(learn.validate(dl=learn.dls.valid)[1])

这种贝叶斯优化的实现不能自然地使用descreet值。这就是为什么我们将包装与lru_cache一起使用。在

deffit_accuracy(pow_n_d,pow_n_a,pow_n_steps):n_d,n_a,n_steps=map(lambdax:2**int(x),(pow_n_d,pow_n_a,pow_n_steps))returnget_accuracy(n_d,n_a,n_steps)

frombayes_optimportBayesianOptimization# Bounded region of parameter spacepbounds={'pow_n_d':(0,8),'pow_n_a':(0,8),'pow_n_steps':(0,4)}optimizer=BayesianOptimization(f=fit_accuracy,pbounds=pbounds,)

optimizer.maximize(init_points=15,n_iter=100,)

|   iter    |  target   |  pow_n_a  |  pow_n_d  | pow_n_... |
-------------------------------------------------------------

| ?[0m 1       ?[0m | ?[0m 0.8235  ?[0m | ?[0m 0.9408  ?[0m | ?[0m 1.898   ?[0m | ?[0m 1.652   ?[0m |

| ?[95m 2       ?[0m | ?[95m 0.8273  ?[0m | ?[95m 4.262   ?[0m | ?[95m 5.604   ?[0m | ?[95m 0.2437  ?[0m |

| ?[0m 3       ?[0m | ?[0m 0.8172  ?[0m | ?[0m 7.233   ?[0m | ?[0m 6.471   ?[0m | ?[0m 2.508   ?[0m |

| ?[95m 4       ?[0m | ?[95m 0.8311  ?[0m | ?[95m 5.935   ?[0m | ?[95m 1.241   ?[0m | ?[95m 0.3809  ?[0m |

| ?[0m 6       ?[0m | ?[0m 0.8189  ?[0m | ?[0m 4.592   ?[0m | ?[0m 2.138   ?[0m | ?[0m 2.824   ?[0m |

| ?[0m 7       ?[0m | ?[0m 0.8058  ?[0m | ?[0m 6.186   ?[0m | ?[0m 7.016   ?[0m | ?[0m 3.316   ?[0m |

| ?[0m 8       ?[0m | ?[0m 0.7616  ?[0m | ?[0m 2.018   ?[0m | ?[0m 1.316   ?[0m | ?[0m 3.675   ?[0m |

| ?[0m 10      ?[0m | ?[0m 0.7983  ?[0m | ?[0m 5.203   ?[0m | ?[0m 7.719   ?[0m | ?[0m 3.407   ?[0m |

| ?[0m 11      ?[0m | ?[0m 0.8308  ?[0m | ?[0m 6.048   ?[0m | ?[0m 4.376   ?[0m | ?[0m 0.08141 ?[0m |

| ?[0m 12      ?[0m | ?[0m 0.8161  ?[0m | ?[0m 7.083   ?[0m | ?[0m 1.385   ?[0m | ?[0m 2.806   ?[0m |

| ?[0m 13      ?[0m | ?[0m 0.8238  ?[0m | ?[0m 4.812   ?[0m | ?[0m 3.785   ?[0m | ?[0m 1.396   ?[0m |
| ?[0m 14      ?[0m | ?[0m 0.8172  ?[0m | ?[0m 7.672   ?[0m | ?[0m 6.719   ?[0m | ?[0m 2.72    ?[0m |

| ?[0m 15      ?[0m | ?[0m 0.8215  ?[0m | ?[0m 6.464   ?[0m | ?[0m 7.954   ?[0m | ?[0m 2.647   ?[0m |

| ?[0m 16      ?[0m | ?[0m 0.8259  ?[0m | ?[0m 0.1229  ?[0m | ?[0m 7.83    ?[0m | ?[0m 0.3708  ?[0m |

| ?[0m 17      ?[0m | ?[0m 0.8259  ?[0m | ?[0m 0.03098 ?[0m | ?[0m 3.326   ?[0m | ?[0m 0.007025?[0m |

| ?[0m 18      ?[0m | ?[0m 0.8294  ?[0m | ?[0m 7.81    ?[0m | ?[0m 7.976   ?[0m | ?[0m 0.0194  ?[0m |

| ?[0m 19      ?[0m | ?[0m 0.2489  ?[0m | ?[0m 0.4499  ?[0m | ?[0m 0.138   ?[0m | ?[0m 0.001101?[0m |

| ?[0m 20      ?[0m | ?[0m 0.8221  ?[0m | ?[0m 0.0     ?[0m | ?[0m 6.575   ?[0m | ?[0m 4.0     ?[0m |

| ?[0m 21      ?[0m | ?[0m 0.2489  ?[0m | ?[0m 8.0     ?[0m | ?[0m 0.0     ?[0m | ?[0m 0.0     ?[0m |

| ?[0m 22      ?[0m | ?[0m 0.8251  ?[0m | ?[0m 0.0     ?[0m | ?[0m 4.502   ?[0m | ?[0m 2.193   ?[0m |

| ?[0m 23      ?[0m | ?[0m 0.789   ?[0m | ?[0m 8.0     ?[0m | ?[0m 3.702   ?[0m | ?[0m 4.0     ?[0m |

| ?[0m 24      ?[0m | ?[0m 0.7549  ?[0m | ?[0m 6.009   ?[0m | ?[0m 0.0     ?[0m | ?[0m 4.0     ?[0m |

| ?[0m 25      ?[0m | ?[0m 0.8292  ?[0m | ?[0m 3.522   ?[0m | ?[0m 8.0     ?[0m | ?[0m 0.0     ?[0m |

| ?[0m 27      ?[0m | ?[0m 0.7983  ?[0m | ?[0m 0.0     ?[0m | ?[0m 0.0     ?[0m | ?[0m 4.0     ?[0m |

| ?[0m 31      ?[0m | ?[0m 0.8016  ?[0m | ?[0m 8.0     ?[0m | ?[0m 8.0     ?[0m | ?[0m 4.0     ?[0m |

| ?[0m 32      ?[0m | ?[0m 0.8294  ?[0m | ?[0m 5.864   ?[0m | ?[0m 8.0     ?[0m | ?[0m 0.0     ?[0m |

| ?[0m 33      ?[0m | ?[0m 0.8208  ?[0m | ?[0m 1.776   ?[0m | ?[0m 8.0     ?[0m | ?[0m 2.212   ?[0m |

| ?[0m 34      ?[0m | ?[0m 0.8289  ?[0m | ?[0m 5.777   ?[0m | ?[0m 2.2     ?[0m | ?[0m 1.31    ?[0m |

| ?[0m 35      ?[0m | ?[0m 0.8016  ?[0m | ?[0m 2.748   ?[0m | ?[0m 5.915   ?[0m | ?[0m 4.0     ?[0m |

| ?[0m 37      ?[0m | ?[0m 0.821   ?[0m | ?[0m 5.093   ?[0m | ?[0m 0.172   ?[0m | ?[0m 1.64    ?[0m |

| ?[0m 38      ?[0m | ?[0m 0.8243  ?[0m | ?[0m 8.0     ?[0m | ?[0m 5.799   ?[0m | ?[0m 0.0     ?[0m |

| ?[0m 39      ?[0m | ?[0m 0.8278  ?[0m | ?[0m 1.62    ?[0m | ?[0m 3.832   ?[0m | ?[0m 1.151   ?[0m |

| ?[0m 40      ?[0m | ?[0m 0.7966  ?[0m | ?[0m 2.198   ?[0m | ?[0m 8.0     ?[0m | ?[0m 4.0     ?[0m |

| ?[0m 41      ?[0m | ?[0m 0.7641  ?[0m | ?[0m 8.0     ?[0m | ?[0m 1.03    ?[0m | ?[0m 4.0     ?[0m |

| ?[0m 42      ?[0m | ?[0m 0.8232  ?[0m | ?[0m 0.0     ?[0m | ?[0m 2.504   ?[0m | ?[0m 2.135   ?[0m |

| ?[0m 43      ?[0m | ?[0m 0.8294  ?[0m | ?[0m 0.0     ?[0m | ?[0m 5.441   ?[0m | ?[0m 0.0     ?[0m |

| ?[0m 44      ?[0m | ?[0m 0.8276  ?[0m | ?[0m 4.636   ?[0m | ?[0m 1.476   ?[0m | ?[0m 0.0     ?[0m |

| ?[0m 45      ?[0m | ?[0m 0.8262  ?[0m | ?[0m 0.0     ?[0m | ?[0m 7.071   ?[0m | ?[0m 2.071   ?[0m |

| ?[0m 47      ?[0m | ?[0m 0.8279  ?[0m | ?[0m 6.579   ?[0m | ?[0m 6.485   ?[0m | ?[0m 0.0     ?[0m |

| ?[0m 48      ?[0m | ?[0m 0.8265  ?[0m | ?[0m 8.0     ?[0m | ?[0m 4.293   ?[0m | ?[0m 1.74    ?[0m |

| ?[0m 49      ?[0m | ?[0m 0.8308  ?[0m | ?[0m 7.909   ?[0m | ?[0m 7.827   ?[0m | ?[0m 1.323   ?[0m |

| ?[0m 50      ?[0m | ?[0m 0.8303  ?[0m | ?[0m 4.946   ?[0m | ?[0m 1.246   ?[0m | ?[0m 1.589   ?[0m |

| ?[95m 51      ?[0m | ?[95m 0.8314  ?[0m | ?[95m 5.664   ?[0m | ?[95m 2.626   ?[0m | ?[95m 0.003048?[0m |

| ?[0m 54      ?[0m | ?[0m 0.82    ?[0m | ?[0m 4.579   ?[0m | ?[0m 5.017   ?[0m | ?[0m 2.928   ?[0m |
| ?[0m 55      ?[0m | ?[0m 0.8259  ?[0m | ?[0m 0.02565 ?[0m | ?[0m 3.699   ?[0m | ?[0m 0.9808  ?[0m |

| ?[0m 56      ?[0m | ?[0m 0.8015  ?[0m | ?[0m 1.927   ?[0m | ?[0m 5.92    ?[0m | ?[0m 2.53    ?[0m |

| ?[0m 57      ?[0m | ?[0m 0.8197  ?[0m | ?[0m 0.7796  ?[0m | ?[0m 4.576   ?[0m | ?[0m 3.952   ?[0m |

| ?[0m 58      ?[0m | ?[0m 0.8297  ?[0m | ?[0m 3.525   ?[0m | ?[0m 4.198   ?[0m | ?[0m 0.02314 ?[0m |

| ?[0m 59      ?[0m | ?[0m 0.8194  ?[0m | ?[0m 6.711   ?[0m | ?[0m 3.848   ?[0m | ?[0m 2.395   ?[0m |

| ?[0m 61      ?[0m | ?[0m 0.8279  ?[0m | ?[0m 7.962   ?[0m | ?[0m 6.151   ?[0m | ?[0m 1.119   ?[0m |

| ?[0m 66      ?[0m | ?[0m 0.7836  ?[0m | ?[0m 3.68    ?[0m | ?[0m 3.977   ?[0m | ?[0m 3.919   ?[0m |

| ?[0m 67      ?[0m | ?[0m 0.8131  ?[0m | ?[0m 5.907   ?[0m | ?[0m 0.9452  ?[0m | ?[0m 2.168   ?[0m |

| ?[0m 70      ?[0m | ?[0m 0.8254  ?[0m | ?[0m 0.009375?[0m | ?[0m 5.081   ?[0m | ?[0m 3.79    ?[0m |

| ?[0m 79      ?[0m | ?[0m 0.8264  ?[0m | ?[0m 3.438   ?[0m | ?[0m 7.982   ?[0m | ?[0m 1.829   ?[0m |

| ?[0m 81      ?[0m | ?[0m 0.8238  ?[0m | ?[0m 0.03221 ?[0m | ?[0m 1.306   ?[0m | ?[0m 3.909   ?[0m |

| ?[0m 82      ?[0m | ?[0m 0.8306  ?[0m | ?[0m 1.575   ?[0m | ?[0m 2.689   ?[0m | ?[0m 0.8684  ?[0m |

| ?[0m 83      ?[0m | ?[0m 0.831   ?[0m | ?[0m 2.765   ?[0m | ?[0m 5.439   ?[0m | ?[0m 0.04047 ?[0m |

| ?[0m 84      ?[0m | ?[0m 0.8253  ?[0m | ?[0m 0.1961  ?[0m | ?[0m 4.123   ?[0m | ?[0m 0.02039 ?[0m |

| ?[0m 85      ?[0m | ?[0m 0.8213  ?[0m | ?[0m 7.937   ?[0m | ?[0m 7.939   ?[0m | ?[0m 2.895   ?[0m |

| ?[0m 86      ?[0m | ?[0m 0.8235  ?[0m | ?[0m 0.06921 ?[0m | ?[0m 5.7     ?[0m | ?[0m 2.778   ?[0m |

| ?[0m 87      ?[0m | ?[0m 0.8184  ?[0m | ?[0m 7.965   ?[0m | ?[0m 5.261   ?[0m | ?[0m 2.661   ?[0m |

内存不足的数据集

如果你的数据集太大以至于无法放入内存中，你可以在每个历元中加载一个数据块。在

标签：

• 项目

欢迎加入QQ群-->： 979659372

推荐PyPI第三方库

• odoo11-addon-account-check-printing-report-dlt103

  帐核打印报告DLT103

• travest

  未知

• glcs

  一个将gitlab集成到cs课程中的实用程序

• tornado

  tornado是一个python web框架和异步网络库，最初在friendfeed开发。

• django-smb

  一个简单的django应用程序来管理smb位置（windows共享目录）。

• alauda_test

  命令行

• drift-python

  一个简单的漂移api包装器。

• Substra

  一个小的示例包

• PythonFormat

  pyformat-boost：：格式化python模块

• karel

  用于程序综合和程序归纳的karel数据集

• pythonnest

  pypi模拟器和克隆工具。

• mframe

  一个轻量级的单文件数据帧实现，可在较旧的python发行版（如jython）上工作。

• pypinindia

  印度pincodes及其相关信息

• python-process-tests

  测试过程的工具

• odoo9-addon-connector-prestashop

  Prestashop Odoo连接器