Python 3?

[couto__carlos]

Hi

I've been an user of intel devcloud since long. I was able to submit and run my scripts without any problem. For some reason this is not working anymore (last time I used and working was 1 or 2 months ago).

What I see is that default python is now Python2 (?)  and packages such as sklearn is no longer available. Can you please let me know how to proceed?

Thanks.

[mike02]

Does anyone have a solution to the question? I am experiencing the same issue. The last time I used devcloud was in January 2024. when python was working fine. Now I also see python version is 2.7.17

Simple commands now also give errors for example

pip list

-bash: /glob/intel-python/python2/bin/pip: /glob/intel-python/versions/2018u2/intelpython2/bin/python: bad interpreter: No such file or directory

conda --help

-bash: /glob/intel-python/versions/2018u2/intelpython2/bin/conda: /glob/intel-python/versions/2018u2/intelpython2/bin/python: bad interpreter: No such file or directory

Does anyone know what may have gone wrong with the environment

[Mark_Rosenbaum]

Which DevCloud environment are you experiencing this issue within? (ex. Edge, oneAPI)

[Mark_Rosenbaum]

Within the Edge JupyterLab environment I'm unable to reproduce the issue.(see attached screenshot)

[mike02]

Thanks for your observation. I am using Intel DevCloud for oneAPI Projects (It is a command line interface)

I also verified that these issues do not occur using the jupyterLab environment. The nice thing there is that one can pick and change the python kernel

I forced the oneApi for projects to use one of the python versions I used in jupyterLab (for example pytorch 2024.1) 

It sort of works until I try import all the packages I need (opencv, torch, etc) where I start getting errors, for example

>>> import torch
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/glob/development-tools/versions/oneapi/2023.2.0.1/inteloneapi/pytorch/latest/lib/python3.9/site-packages/torch/__init__.py", line 229, in <module>
from torch._C import * # noqa: F403
ImportError: libtorch_cpu.so: failed to map segment from shared object

I think this generally means a mismatch between versions of the packages used when building torch

[couto__carlos]

It is still not working. Attached is the "Terminal" launched from JupyterLab as well as connecting via ssh. Notice that via ssh this was working until recently when I was requested to extend my access.

*edit: some corrections to original reply.

[mike02 · Accepted solution]

[Edited]

I made some progress since my last reply.  To summarize I have a work-around hack  that allows the  "Intel DevCloud for oneAPI Projects"  (what @couto__carlos terms connecting via ssh) environment to be used. Found this by working backwards from the  jupyterlab notebook environment

As we saw before after logging in via ssh  you see the wrong version of python 

> python

python 2.7.17 (default, Nov 7 2019, 10:07:09)
[GCC 7.4.0] on linux2

Here is a work around - You can force it to use the same python executable as jupyterlab by adding this line to the end of your ~/.bashrc

export PATH='/glob/development-tools/versions/oneapi/2024.1/oneapi/intelpython/envs/pytorch/bin/:'${PATH}

reload your shell

source ~/.bashrc

Now you will get the same python version used by jupyterlab

> python 

Python 3.9.18 (tags/v3.9.18-26-g6b320c3b2f6-dirty:6b320c3b2f6, Sep 28 2023, 00:35:27)
[GCC 13.2.0] :: Intel Corporation on linux
(null)Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution
>>> 

pip now work as expected

>> pip list | grep -i torch
intel-extension-for-pytorch 2.2.0+cpu
pytorchvideo                0.1.5
torch                       2.2.0+cpu
torchaudio                  2.2.0+cpu
torchvision                 0.17.1+4fd856b

Note that to really be able to use the environment you need to log into an interactive node using something like (see https://devcloud.intel.com/oneapi/documentation/job-submission) 

qsub -I -l nodes=1:gpu:ppn=2 -d .

[Mark_Rosenbaum]

@couto__carlos I was able to reproduce your issue within the environment you specified and I was also able to test @mike02's solution, it works perfectly.  While it would be nice to just have Intel fix the issue instead of using such a hacky solution whatever works works.  If you test his solution and it works for you I suggest accepting his solution so that others see it in the future.

-Mark

[couto__carlos]

Many thanks to @mike02 and @Mark_Rosenbaum for your replies. This is working following the solution @mike02 . I am marking this as solved. It is likely that some problems still exist as mentioned using pytorch in the previous replies.