MemoryError:MicrosoftML服务与Python

Question

下面的脚本在小示例中运行良好

EXECUTE sp_execute_external_script
@language = N'Python',
@script = N'
print(df_training["flResp"].value_counts())',
@input_data_1 = N'SELECT * FROM tb_training_teste',
@input_data_1_name = N'df_training';

我测试了8419条记录，结果还可以，如下所示：

Mensagem(ns) STDOUT do script externo:
0 4964
1 3452
9 3
Name: flResp, dtype: int64

但是，我的原始表有超过500000条记录，由于以下错误，我无法运行。有人能帮我找出问题所在吗？如何修复它

Error in execution. Check the output for more information.
MemoryError

SqlSatelliteCall error: Error in execution. Check the output for more information.
Mensagem(ns) STDOUT do script externo:
SqlSatelliteCall function failed. Please see the console output for more information.
Traceback (most recent call last):
File "C:\Program Files\Microsoft SQL Server\MSSQL14.MSSQLSERVER\PYTHON_SERVICES\lib\site-packages\revoscalepy\computecontext\RxInSqlServer.py", line 587, in rx_sql_satellite_call
rx_native_call("SqlSatelliteCall", params)
File "C:\Program Files\Microsoft SQL Server\MSSQL14.MSSQLSERVER\PYTHON_SERVICES\lib\site-packages\revoscalepy\RxSerializable.py", line 358, in rx_native_call
ret = px_call(functionname, params)
RuntimeError: revoscalepy function failed.

Answer 1

我刚刚遇到了这个问题，所以我将添加我的发现，以防将来对某人有所帮助

SQL server似乎使用资源池来限制外部进程（如Python和R）的可用资源

SELECT * FROM sys.resource_governor_external_resource_pools

默认情况下，将有一个池（称为default），其中max_memory_percent为20。可以使用ALTER EXTERNAL RESOURCE POOL命令增加，例如：

ALTER EXTERNAL RESOURCE POOL [default]
WITH (
    MAX_MEMORY_PERCENT = 95
)
GO

ALTER RESOURCE GOVERNOR RECONFIGURE
GO

或者，可以在SQL Server Management Studio中对其进行更改：