We Keep Coding

How to add to the pythonpath in jupyter lab

I am trying to work with jupyterlab on a remote server that I don't manage, and I want to add my custom libraries to the path so that I can import and use them. Normally, I would go into .bashrc and add to PYTHONPATH there using
export PYTHONPATH="/home/username/path/to/module:$PYTHONPATH"
but this hasn't worked. I have tried this in .bashrc and .bash_profile to no fortune. I have also tried
export JUPYTER_PATH="/home/username/path/to/module:$JUPYTER_PATH"
as I read that somewhere else, and tried it in both the files named above.
What else can I try?
Ideally I'd like to put in some line in jupyterlab that returns the file it is using to add to the path, is that possible?
Or perhaps there is some command I can type directly into a terminal that I can access through jupyterlab that would allow me to add things to my path perminantley. I know that I can use os.path.insert (or similar) at the start of a notebook but as there are certain things I will want to use in every notebook this is a less than ideal solution for me.
Thanks

In a Specific Notebook
Manually append the path to sys.path in the first cell of the notebook
import sys
extra_path = ... # whatever it is
if extra_path not in sys.path:
sys.path.append(extra_path)
As a System Configuration
Modify ~/.ipython/profile_default/ipython_config.py using the shell functionality so that the path gets modified for every notebook.
If that file does not exist, create it by using ipython profile create.
Then insert the modification to sys.path into it by modifying the c.InteractiveShellApp.exec_lines variable, e.g.
c.InteractiveShellApp.exec_lines = [
'import sys; sys.path.append(<path to append>)'
]
Partially stolen from this answer, which has a different enough context to warrant being a different question.

Error : Failed to create temp directory "C:\Users\user\AppData\Local\Temp\conda-<RANDOM>\"

When I try to Activate "conda activate tensorflow_cpu"
conda activate tensorflow_cpu
Error : Failed to create temp directory "C:\Users\user\AppData\Local\Temp\conda-\"

It is due to a bug from conda developers. The bug is the temp path is having names with spaces, so to overcome please reassign the Env Variables TEMP, TMP.
(for windows)
go to environment variables
In "User Variables for " section look for TEMP, TMP
double click on TMP and in "variable value", type "C:\conda_tmp"
similarly do it for TEMP
close env variables section
Restart the anaconda prompt, the error should vanish

Making a little tempory workaround by editing "C:\ProgramData\Anaconda3\Scripts\activate.bat" to add the following just before the first "#if" :
#set TEMP=C:/temp
#set TMP=C:/temp
This is avoid modifying the whole thing by changing Env variable as mentioned by some of the users.
Thanks to Hepson for this suggestion

I came across this error as well. I was following the instructions to update Spyder. When I opened the Anaconda Prompt (Anaconda3) I got the following error:
Failed to create temp directory "C:\Users\username with spaces\AppData\Local\Temp\conda<RANDOM>\"
Which this led to the following error coming up when I tried 'conda update anaconda':
'conda' is not recognized as an internal or external command, operable program or batch file.
The above solution may work for Anaconda, but I was hesitant about it's effect on other applications that use the TEMP and TMP folders. So after further research I came across a series of issues on GitHub https://github.com/conda/conda/issues/9757 which points to the root cause being that it's unable to handle Windows user names with spaces in them, at least for me, not sure about the OP, my assumption is #Hassan masked his name?
My workaround was to open the Anaconda Powershell Prompt, it seems to be updating just fine within this prompt.

looking #Kurian Benoy's answer seems to work best, but his file path was different than the one I needed.
Essentially look for the 'programdat' or 'programdata' in 'C' and look for 'condabin'
the file i found was '_conda_activate.bat'
and basically add the same lines above the first '#if'
#set TEMP=C:/temp
#set TMP=C:/temp
but also to check, open a anaconda prompt and activate your environment using
conda activate <environment name>

Jupyter Notebook error. No such file or directory C:.../HOME

I have installed Anaconda2, and today decided to update to Anaconda3. I installed Anaconda3 at the same time as I uninstalled Anaconda2.
Now, Jupyter Notebook shows this error and quits every time I try to start it.
The path in the error message is: "C:\Users\Hongyi\Documents\%HOME%"
What is causing Jupyter Notebook to quit?

Okay it looks like I found a solution.
Jupyter started running after I created the appropriate folder in the appropriate path.
In my case, it is creating a folder named %HOME% (with percent signs), in the path shown in the error message.
Now Jupyter is up and running.
Thank you guys for your thoughts and suggestions.

After readed this solution I tried another one (a comfortable one for me):
Open the location of the "Jupyter Notebook" shortcut (right click > More > File location) and right click again > Properties. In the Destination field (Sorry if the label is another ... the thing is that my Windows is in Spanish and I don't remember how it's labeled this field in English) just change the %HOME% at the end for a path to your "Work Folder" (if you have not created one then just create it). And this way It works also.

How to set env variable in Jupyter notebook

I've a problem that Jupyter can't see env variable in bashrc file, is there a way to load these variables in jupyter or add custome variable to it?

To set an env variable in a jupyter notebook, just use a % magic commands, either %env or %set_env, e.g., %env MY_VAR=MY_VALUE or %env MY_VAR MY_VALUE. (Use %env by itself to print out current environmental variables.)
See: http://ipython.readthedocs.io/en/stable/interactive/magics.html

You can also set the variables in your kernel.json file:
My solution is useful if you need the same environment variables every time you start a jupyter kernel, especially if you have multiple sets of environment variables for different tasks.
To create a new ipython kernel with your environment variables, do the following:
Read the documentation at https://jupyter-client.readthedocs.io/en/stable/kernels.html#kernel-specs
Run jupyter kernelspec list to see a list with installed kernels and where the files are stored.
Copy the directory that contains the kernel.json (e.g. named python2) to a new directory (e.g. python2_myENV).
Change the display_name in the new kernel.json file.
Add a env dictionary defining the environment variables.
Your kernel json could look like this (I did not modify anything from the installed kernel.json except display_name and env):
{
"display_name": "Python 2 with environment",
"language": "python",
"argv": [
"/usr/bin/python2",
"-m",
"ipykernel_launcher",
"-f",
"{connection_file}"
],
"env": {"LD_LIBRARY_PATH":""}
}
Use cases and advantages of this approach
In my use-case, I wanted to set the variable LD_LIBRARY_PATH which effects how compiled modules (e.g. written in C) are loaded. Setting this variable using %set_env did not work.
I can have multiple python kernels with different environments.
To change the environment, I only have to switch/ restart the kernel, but I do not have to restart the jupyter instance (useful, if I do not want to loose the variables in another notebook). See -however - https://github.com/jupyter/notebook/issues/2647

If you're using Python, you can define your environment variables in a .env file and load them from within a Jupyter notebook using python-dotenv.
Install python-dotenv:
pip install python-dotenv
Load the .env file in a Jupyter notebook:
%load_ext dotenv
%dotenv

You can setup environment variables in your code as follows:
import sys,os,os.path
sys.path.append(os.path.expanduser('~/code/eol_hsrl_python'))
os.environ['HSRL_INSTRUMENT']='gvhsrl'
os.environ['HSRL_CONFIG']=os.path.expanduser('~/hsrl_config')
This if of course a temporary fix, to get a permanent one, you probably need to export the variables into your ~.profile, more information can be found here

A gotcha I ran into: The following two commands are equivalent. Note the first cannot use quotes. Somewhat counterintuitively, quoting the string when using %env VAR ... will result in the quotes being included as part of the variable's value, which is probably not what you want.
%env MYPATH=C:/Folder Name/file.txt
and
import os
os.environ['MYPATH'] = "C:/Folder Name/file.txt"

If you need the variable set before you're starting the notebook, the only solution which worked for me was env VARIABLE=$VARIABLE jupyter notebook with export VARIABLE=value in .bashrc.
In my case tensorflow needs the exported variable for successful importing it in a notebook.

A related (short-term) solution is to store your environment variables in a single file, with a predictable format, that can be sourced when starting a terminal and/or read into the notebook. For example, I have a file, .env, that has my environment variable definitions in the format VARIABLE_NAME=VARIABLE_VALUE (no blank lines or extra spaces). You can source this file in the .bashrc or .bash_profile files when beginning a new terminal session and you can read this into a notebook with something like,
import os
env_vars = !cat ../script/.env
for var in env_vars:
key, value = var.split('=')
os.environ[key] = value
I used a relative path to show that this .env file can live anywhere and be referenced relative to the directory containing the notebook file. This also has the advantage of not displaying the variable values within your code anywhere.

If your notebook is being spawned by a Jupyter Hub, you might need to configure (in jupyterhub_config.py) the list of environment variables that are allowed to be carried over from the JupyterHub process environment to the Notebook environment by setting
c.Spawner.env_keep = [VAR1, VAR2, ...]
(https://jupyterhub.readthedocs.io/en/stable/api/spawner.html#jupyterhub.spawner.Spawner.env_keep)
See also: Spawner.environment

If you are using systemd I just found out that you seem to have to add them to the systemd unit file. This on Ubuntu 16. Putting them into the .profile and .bashrc (even the /etc/profile) resulted in the ENV Vars not being available in the juypter notebooks.
I had to edit:
/lib/systemd/system/jupyer-notebook.service
and put in the variable i wanted to read in the unit file like:
Environment=MYOWN_VAR=theVar
and only then could I read it from within juypter notebook.

you can run jupyter notebook with docker and don(t have to manage dependancy leaks.
docker run -p 8888:8888 -v /home/mee/myfolder:/home/jovyan --name notebook1 jupyter/notebook
docker exec -it notebook1 /bin/bash
then kindly ask jupyter about the opened notebooks,
jupyter notebook list
http:// 0.0.0.0:8888/?token=012456788997977a6eb11e45fffff
Url can be copypasted, verify port if you have changed it.
Create a notebook and paste the following,
into the notebook
!pip install python-dotenv
import dotenv
%load_ext dotenv
%dotenv

PyCharm cannot find library

I am using PyCharm 5 to run a Python 2.7 (Anaconda) script in Ubuntu. My script imports a module with import tensorflow, but this causes the error ImportError: libcudart.so.7.0: cannot open shared object file: No such file or directory. So, it seems that the library libcudart.so.7.0 is needed by this module, but it cannot be found.
Now, I have seen that this library is on my machine in /usr/local/cuda-7.0/targets/x86_64-linux/lib. So, in PyCharm, I went to Settings->Project Interpreters->Interpreter Paths. This had a list of paths, such as /home/karnivaurus/Libraries/Anaconda/python2.7. I then added to this list, the path mentioned above which contains the required library.
However, this did not fix the problem. I still get an error telling me that libcudart.so.7.0 cannot be found. If I run my script from the shell though (python myfile.py), then it runs fine.
How can I tell PyCharm where to find this library?
I have noticed that if I have print sys.path in my script, the paths it prints out are entirely different to those in Settings->Project Interpreters->Interpreter Paths... should they be the same?

I came across this problem just recently using a remote debugger, however I believe it's still the same solution. I just added the following to the Environment Variables section in the Run/Debug Configuration options found in Run > Edit Configurations... dialog: LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH

The path to your cuda library seems strange to me. I would expect it to be /usr/local/cuda-7.0/lib64 or /usr/local/cuda-7.0/lib.
Did you follow all of the cuda installation procedure?
If you type env on the command line, do you see a path to cuda in your LD_LIBRARY_PATH?
Update from comments below:
The issue is that PyCharm was invoked from the desktop, and wasn't getting the right environment variables. Solution is to either:
invoke from the command line,
create a script to set environment and then invoke, and make a link to that script on the desktop,
or set environment variables on the desktop item

While some of these answers are correct, and could work, I haven't seen what the OP specifically asked for, and that is where to set environments for the python console. This can be accomplished inside pycharm at:
File > Settings > Build,Execution,Deployment > Console > Python Console
In the options there, you'll find a place to define Environment Variables. Set LD_LIBRARY_PATH there.

Edit your pycharm.desktop, specify the environment variable in exec, like below:
[Desktop Entry]
Version=1.0
Type=Application
Name=Pycharm
Exec=env LD_LIBRARY_PATH=:/usr/local/cuda/lib64:/usr/local/cuda/lib64 /home/cwh/software/pycharm-2016.1.4/bin/pycharm.sh
Icon=/home/cwh/software/pycharm-2016.1.4/bin/pycharm.png
Name[zh_CN]=Pycharm
so pycharm will find cuda

Have you selected the right python interpreter in your project's settings?
See here.
I had a similar issue and changing the interpreter solved it without having to create a new icon.

The following works for me on Community edition 2019.3
To set globally for a project:
Open File/Settings/Project/Project Interpreter
click on the cog icon next to the interpreter
choose show all
click on the little folder with tree icon bottom right
add the path to "Interpreter Paths"