We Keep Coding

How to ensure environment variables sourced while in venv context are not acessible out of it?

I know it's possible to create a venv
python3 -m venv env
source env/bin/activate
Than create a .env file and fill it with with some variable:
export PASSWORD="123"
Than set the variables to you enviroment:
source .env
And create a script.py file that accesses your variable:
import os
print(os.environ['PASSWORD'])
execute it with env activated
python script.py
It successfully prints 123.
However, if you exit the venv with:
deactivate
And execute the script again withing the same terminal, it still works, printing 123.
However, when trying to execute script from a new terminal (non sourced into venv and .env), finally an error is thrown:
Traceback (most recent call last):
File "script.py", line 2, in <module>
print(os.environ['PASSCODE'])
File "/usr/lib/python3.8/os.py", line 675, in __getitem__
raise KeyError(key) from None
KeyError: 'PASSWORD'
It seems to me that the variable isn't in fact setted "in the virtual environment" but in the terminal instance. As it's possible to access it even after deactivating the venv.
How exactly does setting environment variables in a venv works, and is it possible to truthfully make them only accessible while inside the venv?

configure jupyter notebook config file for a virtual environment to save .py files

Im trying to run python on virtaual env, my virtual environment is activated, I want to modify the jypter notebook file config to want to save the script in form of .py evrytime that i created an ipynb.
Let's notice that on my venv i tried this command to generate the config file :
jupyter notebook --generate-config
the file is created in the main of my machine
/Users/paulbergan/.jupyter/jupyter_notebook_config.py
which is not what i want .
So i need to access the config file crreated in the venv and add these rows below to do the approriate save (ipynb and py):
c.NotebookApp.contents_manager_class="jupytext.TextFileContentsManager"
c.ContentsManager.formats = "ipynb,py"

Source Python Virtual Environment for Hive UDF

I'm using a Python virtual environment to load modules that aren't available on our cluster for use in a Hive UDF. I'm unable to source the venv, so when the Python UDF is called in the shell script, the script errors since the modules cannot be found.
When calling ls from the shell script, the venv appears in the list.
DELETE FILE /temp/venv;
ADD FILE /temp/venv;
DELETE FILE udf.sh;
ADD FILE udf.sh;
SOURCE venv/bin/activate;
SELECT TRANSFORM(1)
USING 'bash udf.sh'
AS (test_result)
Results in File: venv/bin/activate is not a file.
SOURCE ../venv/bin/activate;
Results in FAILED: ParseException line 1:2 cannot recognize input near 'This' 'file' 'must'
Within the shell script, If I try to use:
. venv/bin/activate
It returns an exit code 1.
Any thoughts?
Thanks,
Dave
Solved using this: https://stackoverflow.com/a/23069201/10542262
Within the shell script, instead of doing:
python [path]/[script].py
You can call Python from the venv and no longer need to activate the venv.
[path/to/venv/]/bin/python [path]/[script].py

Python doesn't detect Heroku config

From my Windows 10 command line:
heroku config
yields the output:
=== test-bot Config Vars
GROUPME_BOT_ID: [111111111111111111]
My Python 3 code:
data = {
'bot_id' : os.getenv('GROUPME_BOT_ID'),
'text' : "hi",
}
print(data)
yields:
{'bot_id': None, 'text': 'hi'}
The bot_id is None instead of 111111111111111111. Why?
For testing, I execute my code from the command line of my Windows computer via:
python app.py

heroku:config only sets environment variables on Heroku. To run your code locally you will have to set your environment variables locally. (They can be different in development and production. That's the whole idea—they're environment-specific.)
There are several ways to do this. Here are two:
On your local machine you can use a .env file containing data in the format
VAR1=value
VAR2=value2
and then run your application using Foreman, which will automatically read the .env file and set appropriate environment variables:
foreman start
You can set environment variables using something like direnv, which will read an .envrc file for your project whenever you cd into its directory and set environment variables accordingly. In this case you don't need to use Foreman; python app.py should work fine.
There are editor and IDE plugins for direnv that can automate this in your editor as well.

How do I set environment variables in pipenv?

I need to set some access token environment variables for my python project that I am running in a pipenv. I will want to set these environment variables every time I start the pipenv.
How do I do this?

If you want to load automatically some environment variables each time you start the project, you can set a .env file at the root folder of the project, next to the Pipfile. See Automatic Loading of .env.
You can run the following command from the right folder to create this .env file :
echo MY_TOKEN=SuperToKen >.env # create the file and write into
echo MY_VAR=SuperVar >>.env # append to the file
or just create it manually to obtain:
MY_TOKEN=SuperToKen
MY_VAR=SuperVar
This file will be loaded automatically with pipenv shell or pipenv run your_command and the environment variables will be available.
You can access/check them in your code with :
print(os.getenv('MY_TOKEN', 'Token Not found'))
Not sure about other IDE, but within Pycharm you need the plugin Env File to load it automatically (access Env File tab from the Run/Debug configurations).
You can add comments in this file with a leading #
# My test token
MY_TOKEN=SuperToKen
Note : Of course you must exclude this file from your version control (like git).