Python中文网 - 问答频道, 解决您学习工作中的Python难题和Bug
Python常见问题
我一直在尝试在我的HPC上安装带有Cuda（理想情况下为11）的PyTorch 1.9，但我做不到
集群表示：
Package typing-extensions conflicts for:
typing-extensions
torchvision -> pytorch==1.8.1 -> typing-extensionsThe following specifications were found to be incompatible with your system:
- feature:/linux-64::__glibc==2.17=0
- feature:|@/linux-64::__glibc==2.17=0
- cffi -> libgcc-ng[version='>=7.5.0'] -> __glibc[version='>=2.17']
- cudatoolkit=11.0 -> __glibc[version='>=2.17,<3.0.a0']
- cudatoolkit=11.0 -> libgcc-ng[version='>=7.3.0'] -> __glibc[version='>=2.17']
- freetype -> libgcc-ng[version='>=7.3.0'] -> __glibc[version='>=2.17']
- jpeg -> libgcc-ng[version='>=7.5.0'] -> __glibc[version='>=2.17']
- lcms2 -> libgcc-ng[version='>=7.3.0'] -> __glibc[version='>=2.17']
- libffi -> libgcc-ng[version='>=7.3.0'] -> __glibc[version='>=2.17']
- libgcc-ng -> __glibc[version='>=2.17']
- libmklml -> libgcc-ng -> __glibc[version='>=2.17']
- libpng -> libgcc-ng[version='>=7.3.0'] -> __glibc[version='>=2.17']
- libstdcxx-ng -> __glibc[version='>=2.17']
- libtiff -> libgcc-ng[version='>=7.3.0'] -> __glibc[version='>=2.17']
- libwebp-base -> libgcc-ng[version='>=7.3.0'] -> __glibc[version='>=2.17']
- lz4-c -> libgcc-ng[version='>=7.5.0'] -> __glibc[version='>=2.17']
- mkl-service -> libgcc-ng[version='>=7.5.0'] -> __glibc[version='>=2.17']
- mkl_fft -> libgcc-ng[version='>=7.3.0'] -> __glibc[version='>=2.17']
- mkl_random -> libgcc-ng[version='>=7.5.0'] -> __glibc[version='>=2.17']
- ncurses -> libgcc-ng[version='>=7.3.0'] -> __glibc[version='>=2.17']
- ninja -> libgcc-ng[version='>=7.3.0'] -> __glibc[version='>=2.17']
- numpy -> libgcc-ng[version='>=7.5.0'] -> __glibc[version='>=2.17']
- numpy-base -> libgcc-ng[version='>=7.5.0'] -> __glibc[version='>=2.17']
- openjpeg -> libgcc-ng[version='>=7.5.0'] -> __glibc[version='>=2.17']
- openssl -> libgcc-ng[version='>=7.5.0'] -> __glibc[version='>=2.17']
- pillow -> libgcc-ng[version='>=7.5.0'] -> __glibc[version='>=2.17']
- python=3.9 -> libgcc-ng[version='>=7.5.0'] -> __glibc[version='>=2.17']
- pytorch==1.9 -> cudatoolkit[version='>=11.1,<11.2'] -> __glibc[version='>=2.17,<3.0.a0']
- readline -> libgcc-ng[version='>=7.3.0'] -> __glibc[version='>=2.17']
- sqlite -> libgcc-ng[version='>=7.5.0'] -> __glibc[version='>=2.17']
- tk -> libgcc-ng[version='>=7.3.0'] -> __glibc[version='>=2.17']
- torchvision -> cudatoolkit[version='>=11.1,<11.2'] -> __glibc[version='>=2.17|>=2.17,<3.0.a0']
- xz -> libgcc-ng[version='>=7.3.0'] -> __glibc[version='>=2.17']
- zlib -> libgcc-ng[version='>=7.3.0'] -> __glibc[version='>=2.17']
- zstd -> libgcc-ng[version='>=7.3.0'] -> __glibc[version='>=2.17']
Your installed version is: 2.17
但我不明白如何使用这些信息来安装它。我能为系统管理员做些什么吗
当我尝试用conda安装它时，我收到一条消息，告诉我它已经安装好了。但是，conda列表greps显示版本仅为CPU，而不是GPU：
(metalearning_gpu) miranda9~/automl-meta-learning $ conda install pytorch torchvision torchaudio cudatoolkit=11.1 -c pytorch -c nvidia
Collecting package metadata (current_repodata.json): done
Solving environment: done
# All requested packages already installed.
(metalearning_gpu) miranda9~/automl-meta-learning $
(metalearning_gpu) miranda9~/automl-meta-learning $ conda list | grep torch
cpuonly 1.0 0 pytorch
ffmpeg 4.3 hf484d3e_0 pytorch
pytorch 1.9.0 py3.9_cpu_0 [cpuonly] pytorch
torch 1.9.0+cpu pypi_0 pypi
torchaudio 0.9.0 pypi_0 pypi
torchmeta 1.7.0 pypi_0 pypi
torchvision 0.10.0+cpu pypi_0 pypi
尝试使用pip安装它完全失败：
(metalearning_gpu) miranda9~/automl-meta-learning $ pip3 install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html
Looking in links: https://download.pytorch.org/whl/torch_stable.html
Collecting torch==1.9.0+cu111
ERROR: Exception:
Traceback (most recent call last):
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 173, in _main
status = self.run(options, args)
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 203, in wrapper
return func(self, options, args)
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/commands/install.py", line 315, in run
requirement_set = resolver.resolve(
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 94, in resolve
result = self._result = resolver.resolve(
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 472, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 341, in resolve
self._add_to_criteria(self.state.criteria, r, parent=None)
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 172, in _add_to_criteria
if not criterion.candidates:
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_vendor/resolvelib/structs.py", line 151, in __bool__
return bool(self._sequence)
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 140, in __bool__
return any(self)
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 128, in <genexpr>
return (c for c in iterator if id(c) not in self._incompatible_ids)
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 32, in _iter_built
candidate = func()
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 204, in _make_candidate_from_link
self._link_candidate_cache[link] = LinkCandidate(
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 295, in __init__
super().__init__(
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__
self.dist = self._prepare()
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 227, in _prepare
dist = self._prepare_distribution()
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 305, in _prepare_distribution
return self._factory.preparer.prepare_linked_requirement(
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 508, in prepare_linked_requirement
return self._prepare_linked_requirement(req, parallel_builds)
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 550, in _prepare_linked_requirement
local_file = unpack_url(
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 239, in unpack_url
file = get_http_url(
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 102, in get_http_url
from_path, content_type = download(link, temp_dir.path)
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/network/download.py", line 132, in __call__
resp = _http_get_download(self._session, link)
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/network/download.py", line 115, in _http_get_download
resp = session.get(target_url, headers=HEADERS, stream=True)
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_vendor/requests/sessions.py", line 555, in get
return self.request('GET', url, **kwargs)
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/network/session.py", line 454, in request
return super().request(method, url, *args, **kwargs)
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_vendor/requests/sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_vendor/requests/sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_vendor/cachecontrol/adapter.py", line 44, in send
cached_response = self.controller.cached_request(request)
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_vendor/cachecontrol/controller.py", line 139, in cached_request
cache_data = self.cache.get(cache_url)
File "/home/miranda9/miniconda3/envs/metalearning_gpu/lib/python3.9/site-packages/pip/_internal/network/cache.py", line 54, in get
return f.read()
MemoryError
当前安装脚本：
## Installation script
# to install do: bash ~/automl-meta-learning/install.sh
#conda update conda
#conda create -y -n metalearning_gpu python=3.9
#conda activate metalearning_gpu
#conda remove --name metalearning_gpu --all
module load cuda-toolkit/11.1
module load gcc/9.2.0
# A40, needs cuda at least 11.0, but 1.9 requires 11
conda activate metalearning_gpu
conda install pytorch torchvision torchaudio cudatoolkit=11.1 -c pytorch -c nvidia
pip3 install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html
#conda activate metalearning_cpu
#conda install pytorch torchvision torchaudio cpuonly -c pytorch
#pip3 install torch==1.9.0+cpu torchvision==0.10.0+cpu torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html
# uutils installs
conda install -y dill
conda install -y networkx>=2.5
conda install -y scipy
conda install -y scikit-learn
conda install -y lark-parser -c conda-forge
# due to compatibility with torch=1.7.x, https://stackoverflow.com/questions/65575871/torchtext-importerror-in-colab
#conda install -y torchtext==0.8.0 -c pytorch
conda install -y tensorboard
conda install -y pandas
conda install -y progressbar2
conda install -y transformers
conda install -y requests
conda install -y aiohttp
conda install -y numpy
conda install -y plotly
conda install -y matplotlib
pip install wandb
# for automl
conda install -y pyyml
conda install -y torchviz
#conda install -y graphviz
#pip install tensorflow
#pip install learn2learn
#pip install -U git+https://github.com/brando90/pytorch-meta.git
#pip install --no-deps torchmeta==1.6.1
pip install --no-deps torchmeta==1.7.0
# 'torch>=1.4.0,<1.9.0',
# 'torchvision>=0.5.0,<0.10.0',
#pip install -y numpy
pip install Pillow
pip install h5py
#pip install requests
pip install ordered-set
pip install higher
# 'torch'
#pip install -U git+https://github.com/moskomule/anatome
pip install --no-deps -U git+https://github.com/moskomule/anatome
# 'torch>=1.9.0',
# 'torchvision>=0.10.0',
pip install tqdm
# - using conda develop rather than pip because uutils installs incompatible versions with the vision cluster
## python -c "import sys; [print(p) for p in sys.path]"
conda install conda-build
# conda develop ~/ultimate-utils/ultimate-utils-proj-src
# conda develop ~/automl-meta-learning/automl-proj-src
# pip install ultimate-utils
# -- extra notes
# local editable installs
# HAL installs, make sure to clone from wmlce 1.7.0 that has h5py ~= 2.9.0 and torch 1.3.1 and torchvision 0.4.2
# pip install torchmeta==1.3.1
Tags： installpipinhomegpuversionpackagesng
热门问题
- 如何用强化学习更新函数逼近中的权重？
- 如何用归并排序计算倒数？
- 如何用当前数据拟合正弦波？
- 如何用当前页面的值填充表单？
- 如何用彩色地图在2dpython直方图中勾勒出箱子的轮廓？
- 如何用彩色地图给等高线标签上色？
- 如何用彩色打印到控制台？
- 如何用彩色条绘制2d直方图，在Python中显示第三个变量的平均值？
- 如何用彩色条绘制直方图，其中的颜色应与xaxis中的值一致？
- 如何用彩色贴图填充曲线和原始边之间的区域？
- 如何用循环分割数组并对其应用操作？
- 如何用循环创建多个子集的数据帧
- 如何用循环和递归实现求和
- 如何用循环填充数组？
- 如何用循环当前引用的位置更新变量？
- 如何用循环求这个级数的和
- 如何用循环解这个方程？
- 如何用循环语句逐个读取数据帧中的数据？
- 如何用循环除法？（Python）
- 如何用感知一致性对HSV空间进行采样
热门文章
- Python覆盖写入文件
- 怎样创建一个 Python 列表？
- Python3 List append()方法使用
- 派森语言
- Python List pop()方法
- Python Django Web典型模块开发实战
- Python input() 函数
- Python3 列表(list) clear()方法
- Python游戏编程入门
- 如何创建一个空的set?
- python如何定义(创建)一个字符串
- Python标准库 [The Python Standard Library by Ex
- Python网络数据爬取及分析从入门到精通（分析篇）
- Python3 for 循环语句
- Python List insert() 方法
- Python 字典(Dictionary) update()方法
- Python编程无师自通 专业程序员的养成
- Python3 List count()方法
- Python 网络爬虫实战 [Web Crawler With Python]
- Python Cookbook（第2版）中文版
给出的答案是正确的。不幸的是，您可能不得不尝试使用conda和pip安装它。其中一个最终为我工作。由于某些原因，vision cluster/HPC拒绝下载conda版本，无法测试是否是由于HPC造成的……此外，如果您的HPC没有得到很好的管理，那么安装东西可能需要很多时间（没有办法解决），而且似乎通过gpu获得交互式作业有助于安装过程
我使用gpu和大量CPU（16）在交互式作业中运行此命令：
而且，这似乎与我也需要的torchtext兼容（使用
0.10.0）。见：https://github.com/pytorch/text/issues/1394#issuecomment-927484153首先，正如@Francois所建议的，尝试卸载pytorch的CPU版本。在安装脚本中，还应使用
conda或pip3然后，您可能需要尝试以下尝试：
conda：将conda-forge通道添加到命令（conda install pytorch torchvision torchaudio cudatoolkit=11.1 -c pytorch -c nvidia -c conda-forge）。并确保conda已更新pip：将no-cache-dir插入到命令（pip3 no-cache-dir install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html）中以避免MemoryError相关问题 更多 >
编程相关推荐