I am having trouble with my vscode and ssh connection.
I have installed miniconda on my local machine at home/sam/miniconda3, and I have installed anaconda on a remote HPC machine on home/sam/anaconda3.
I have replicated conda envirenments in both using a yml file. Recently, the I keep getting errors while connecting to ssh that "python was not found". After digging for a while, I realize that when running a jupter nb, it looks for python under the local machine path (home/sam/miniconda3). Despite the case that I have selected the kernel from the remote machine path (/home/sam/anaconda3/envs/myenv/bin/python3.9).
How can I get the notebook in vscode to look at the remote path when connected to the remote machine?
Thanks heaps!
Related
I am running jupyter on a remote cluster.
How can I write files to my local machine?
I am running python 3.7.1 and jupyter core 4.6.2
Is there something like the following two
numpy.save(localhost:local/path, DATA)
Or
pyplot.savefig(localhost:local/path)
I have anaconda, hence spyder, installed on a local machine. What I am trying to do is to use my local spyder installation to open a .py script saved on a remote cluster (in my office) via ssh. The issues that I am encountering are the following:
I cannot run spyder from the cluster - there is no graphical device whatsoever. For example, we have actually anaconda installed on the cluster, but when I ran spyder from the command line, I get the following error message: Could not connect to any X display
I cannot mount the (remote) drivers, where the .py scripts are located, onto my local machine when I am working from home (which is the case when I am at work, connected to the internet via cable). If this was the case, I could simply launch spyder on my local machine, then open the scripts. I can only access the files on some drivers mounted onto the cluster via ssh.
As, however, I can access the .py scripts saved on the cluster via ssh (I can open then with programs installed locally e.g. vim, jpico etc), I was wondering whether it is possible to use the command line to open a script saved on a remote cluster using my local spyder installation, something like $ spyder /path/to/myScript/savedOnTheRemoteCluster.py
(Spyder maintainer here) As of May 2019 our editor is not capable of working with files on remote locations. So your best option right now is to mount your remote server with sshfs to make it appear as a local directory and then open any file present there in Spyder.
In my local machine with macOS Mojave, I installed a virtual environment with Python 3.6, opencv package and some additional unrelated packages.
I started a very simple Jupyter Notebook that is executed with no problem, here it's the code:
import cv2
print(cv2.__version__)
my_img = cv2.imread("colibri_763_460.jpeg",1)
cv2.imshow("Original", my_img)
cv2.waitKey(0)
cv2.destroyAllWindows()
Now, I'm trying to execute the same code in a Jupyter Notebook in my remote Ubuntu machine but I can't make it work. These are the steps that I took:
I copied the image file and the Notebook ypnb file to the remote ubuntu machine.
I opened an ssh session, selected a virtualenv with the same packages than my local machine and executed the "jupyter notebook"
I opened another terminal with ssh -L 8000:localhost:8888 mpastorg#mpgubu18 for the ssh tunneling
I opened my local browser in the localhost:8000 to execute the Notebooks that I have in my remote Ubuntu machine, it works with any Notebook not involving opencv.
I tried to solve my problem using two different ways:
Installing in the ubuntu remote machine the package opencv-contrib-python-headless: when I try to use cv2.imshow, I got the error method not found
Installing in the ubuntu remote machine the package opencv-contrib-python: I got the error that kernel is dead.
Does anyone know if there's any way to execute the remote jupyter notebook and see the picture locally? it maybe would be possible a workaround to embed the picture in the browser window?
Thank you very much
Marcos Pastor
I wonder if the image is being rendered using X - which you have not explicitly enabled, in both your host machine, nor in your ssh tunnel.
On the Host machine. (Where you want to see the Image).
Enter this comment
xhost+
This now allows remote X-Hosts to send you data. As you are on a Mac - you will need to have the XQuartz package installed.
Now modify your ssh command. - so that you allow X - this is typically done using a -X flag (Or it can be placed in your .ssh/config file like this
Host *
ForwardX11 yes
ForwardX11Trusted no
XAuthLocation /opt/X11/bin/xauth
ForwardAgent yes
Compression yes
KeepAlive yes
Compression yes
If you are not using a config try the -X to start with.
Hope that helps
I would really appreciate some help here, basically I'm learning to use tensorflow, I've decided that the easiest way to go about this would be to install ubuntu on either VMware and/or Virtualbox and then access the ipython notebook (came with anaconda) through the browser on the host computer.
I have successfully installed both vmware and virtualbox, I downloaded a ubuntu image and also successfully installed anaconda on both, I get it to work without a problem on both VMs and even installed tensorflow.
Some research online on how to expose the ipython to the host machine suggested port forwarding or ssh tunneling, none of these have worked (very likely I'm doing it wrong). Can someone please help? think of me as a newbie.
Generally you must edit the jupyter configuration file to allow network access to the notebook server. See this link: http://jupyter-notebook.readthedocs.org/en/latest/public_server.html for details. (Even if it is not a "public" server, you still intend to access the notebook server living in the VM from the host machine via a network connection...)
Here is a quotation from the linked documentation that indicates by default, you can only access the notebook via the localhost.
By default, a notebook server runs locally at 127.0.0.1:8888 and is accessible only from localhost. You may access the notebook server from the browser using http://127.0.0.1:8888.
I have iPython running from a secured server on an Ubuntu server VM running on my laptop.
Command line ipython works on the server vm from the virtualenv. I can also start the notebook server on the server vm from the virtualenv without errors.
I can access notebooks from the host laptop and execute code in cells, but if I start the notebook server after activating a virtualenv I can't import any of the Python modules I've installed in the virtualenv.
It looks like the notebook server process is running the system Python but not the version in my virtualenv. Is there a way to tell the notebook server process which virtualenv to use?
Because virtualenv on activation adds its own ways to the begining of the PATH environment variable, you have two options:
a) create correct virtualenv on the notebook server and install everything from there
b) modify PYTHONPATH variable in order to get access to your libraries.