- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi
I've been an user of intel devcloud since long. I was able to submit and run my scripts without any problem. For some reason this is not working anymore (last time I used and working was 1 or 2 months ago).
What I see is that default python is now Python2 (?) and packages such as sklearn is no longer available. Can you please let me know how to proceed?
Thanks.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
[Edited]
I made some progress since my last reply. To summarize I have a work-around hack that allows the "Intel DevCloud for oneAPI Projects" (what @couto__carlos terms connecting via ssh) environment to be used. Found this by working backwards from the jupyterlab notebook environment
As we saw before after logging in via ssh you see the wrong version of python
> python
python 2.7.17 (default, Nov 7 2019, 10:07:09)
[GCC 7.4.0] on linux2
Here is a work around - You can force it to use the same python executable as jupyterlab by adding this line to the end of your ~/.bashrc
export PATH='/glob/development-tools/versions/oneapi/2024.1/oneapi/intelpython/envs/pytorch/bin/:'${PATH}
reload your shell
source ~/.bashrc
Now you will get the same python version used by jupyterlab
> python
Python 3.9.18 (tags/v3.9.18-26-g6b320c3b2f6-dirty:6b320c3b2f6, Sep 28 2023, 00:35:27)
[GCC 13.2.0] :: Intel Corporation on linux
(null)Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution
>>>
pip now work as expected
>> pip list | grep -i torch
intel-extension-for-pytorch 2.2.0+cpu
pytorchvideo 0.1.5
torch 2.2.0+cpu
torchaudio 2.2.0+cpu
torchvision 0.17.1+4fd856b
Note that to really be able to use the environment you need to log into an interactive node using something like (see https://devcloud.intel.com/oneapi/documentation/job-submission)
qsub -I -l nodes=1:gpu:ppn=2 -d .
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Does anyone have a solution to the question? I am experiencing the same issue. The last time I used devcloud was in January 2024. when python was working fine. Now I also see python version is 2.7.17
Simple commands now also give errors for example
pip list
-bash: /glob/intel-python/python2/bin/pip: /glob/intel-python/versions/2018u2/intelpython2/bin/python: bad interpreter: No such file or directory
conda --help
-bash: /glob/intel-python/versions/2018u2/intelpython2/bin/conda: /glob/intel-python/versions/2018u2/intelpython2/bin/python: bad interpreter: No such file or directory
Does anyone know what may have gone wrong with the environment
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Which DevCloud environment are you experiencing this issue within? (ex. Edge, oneAPI)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks for your observation. I am using Intel DevCloud for oneAPI Projects (It is a command line interface)
I also verified that these issues do not occur using the jupyterLab environment. The nice thing there is that one can pick and change the python kernel
I forced the oneApi for projects to use one of the python versions I used in jupyterLab (for example pytorch 2024.1)
It sort of works until I try import all the packages I need (opencv, torch, etc) where I start getting errors, for example
>>> import torch
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/glob/development-tools/versions/oneapi/2023.2.0.1/inteloneapi/pytorch/latest/lib/python3.9/site-packages/torch/__init__.py", line 229, in <module>
from torch._C import * # noqa: F403
ImportError: libtorch_cpu.so: failed to map segment from shared object
I think this generally means a mismatch between versions of the packages used when building torch
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
[Edited]
I made some progress since my last reply. To summarize I have a work-around hack that allows the "Intel DevCloud for oneAPI Projects" (what @couto__carlos terms connecting via ssh) environment to be used. Found this by working backwards from the jupyterlab notebook environment
As we saw before after logging in via ssh you see the wrong version of python
> python
python 2.7.17 (default, Nov 7 2019, 10:07:09)
[GCC 7.4.0] on linux2
Here is a work around - You can force it to use the same python executable as jupyterlab by adding this line to the end of your ~/.bashrc
export PATH='/glob/development-tools/versions/oneapi/2024.1/oneapi/intelpython/envs/pytorch/bin/:'${PATH}
reload your shell
source ~/.bashrc
Now you will get the same python version used by jupyterlab
> python
Python 3.9.18 (tags/v3.9.18-26-g6b320c3b2f6-dirty:6b320c3b2f6, Sep 28 2023, 00:35:27)
[GCC 13.2.0] :: Intel Corporation on linux
(null)Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution
>>>
pip now work as expected
>> pip list | grep -i torch
intel-extension-for-pytorch 2.2.0+cpu
pytorchvideo 0.1.5
torch 2.2.0+cpu
torchaudio 2.2.0+cpu
torchvision 0.17.1+4fd856b
Note that to really be able to use the environment you need to log into an interactive node using something like (see https://devcloud.intel.com/oneapi/documentation/job-submission)
qsub -I -l nodes=1:gpu:ppn=2 -d .
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@couto__carlos I was able to reproduce your issue within the environment you specified and I was also able to test @mike02's solution, it works perfectly. While it would be nice to just have Intel fix the issue instead of using such a hacky solution whatever works works. If you test his solution and it works for you I suggest accepting his solution so that others see it in the future.
-Mark
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Many thanks to @mike02 and @Mark_Rosenbaum for your replies. This is working following the solution @mike02 . I am marking this as solved. It is likely that some problems still exist as mentioned using pytorch in the previous replies.
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page