Running python tests in VSCode. I have a .env file that is loading (I can add new vars, and see the change reflected in a print(... of os.environ.
However, variable expansion within that file is doing something odd. It seems to expand ${workspaceFolder} in some cases (specifically, PYTHONPATH=) but not others. ??
Here's my .env file. Again, it is loading, as I can see changes if i add vars, or change the contents of the (fully working) PYTHONPATH var:
FOOBAR1=${workspaceFolder}/foobar1.txt
PYTHONPATH=${workspaceFolder}/backend/src:/tmp/foo-2:${env:PYTHONPATH}
however print('ing all the os.environ's shows:
...
?
v
FOOBAR1=/foobar1.txt
PYTHONPATH=/home/-omitted-/backend/src:/tmp/foo-2:
...
${workspaceFolder} is expanded for one var, but not the other?? I've tried re-ordering. Don't see anything here or on google.
Help !? Thank you!
There is no variable expansion in vscode .env files at all. Any ${workspaceFolder} or similar variables are replaced by empty strings.
Note: Variable substitution is only supported in VS Code settings files, it will not work in .env environment files.
https://code.visualstudio.com/docs/python/environments#_environment-variable-definitions-file
Something else is probably prepending the /home/-omitted- to PYTHONPATH.
Related
I am trying to create a pipeline and have defined some variable in my gitlab job which will be used by python script to populate results.
Is there any way I can call/define those variable in my python script.
For example:
.test-report:
extends: .test_report_setup
tags: *def_runners_tags
variables:
value-path: ${CI_PROJECT_DIR}/value
Now in my python script I want to call the variable 'value-path' to fetch or read files located in that directory. (Please note that variable is a custom variable on gitlab)
file_path = '<value-path>' <---- To get the gitlab job variable here
file_in = open(file_path + "id.txt")
Please help me how I can get it in my python script as I am a bit stuck on it.
Any suggestion/help on this will be appreciated. Thanks in advance.
You can access environment variables with os.environ or os.getenv. os.environ is a dict with the environment variables and will fail if you attempt to retrieve a key that doesn't exist. os.getenv is basically os.environ.get, which allows you to set a default value if there is no environment variable with that name.
I have been searching online for a guide to python coding. In this website, https://learnpythonthehardway.org
The author mentions this code if python is not detected in Windows powershell.
-> [ENVIRONMENT]::SETENVIRONMENTVARIABLE("PATH", "$ENV:PATH;C:\PYTHON27", "USER")
So what does this code do exactly? I will like to understand this and not just copy and paste into the powershell.
[ENVIRONMENT]::SETENVIRONMENTVARIABLE("PATH", "$ENV:PATH;C:\PYTHON27", "USER")
It sets the environment variable PATH to the value composed of the previous value of PATH augmented with C:\PYTHON27
The environment variable is set for the current user i.e. at HKEY_CURRENT_USER\Environment
in contrast with "MACHINE", which would set it system-wide i.e in the registry at HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Environment
or "PROCESS" which would set it for the current process.
It does exactly what the function name (Setenvironmentvariable()) says it does. It sets the value of the environment variable PATH to the current value of the PATH environment variable, followed by the path to your Python directory. The last argument is optional but dictates whether the scope of the variable.
See also https://msdn.microsoft.com/en-us/library/system.environment.setenvironmentvariable(v=vs.110).aspx (the first result of a search for [ENVIRONMENT]::SETENVIRONMENTVARIABLE)
i am using SCONS Construction tool.
i am unable to use the environment variable which is initialized in python script.
In My project USER can change some variables to work with the compiler.
For that we have 2 files.
Config.py
Sconstruct
Config.py is having all the variables which are like Include directories, CFLAGS , CPPDEFINES etc. So, Here we can set some variables. Those variables i need to use in Sconstruct file. In config.py i set a variable like below
SCONS_INC = "Include files"
os.environ["SCONS_INC"] = SCONS_INC
I need to use those variables in Sconstruct File. The code is
env["CPPPATH"] = os.environ["SCONS_INC"]
But I am getting an error like Undefined variable SCONS_INC.
How to do this?
SCons by default does not use the invoked environment, this is to make sure that you can reproduce the build no matter which configurations your environment have.
The environment variables are stored within the scons environment under the key ENV so you access the general environment variables like this:
env = Environment()
variable = env['ENV']['SomeVariable']
env['ENV']['SomeVariable'] = SomeValue
I understand your question like you need to use variables set in the python script within SCons. To do this you need to transfer them using the two method you describe in combination.
env = Enviroment()
python_variable = os.environ['SomeVariable']
env['ENV']['SomeVariable'] = python_variable
I would however perhaps recommend other ways of controlling the build, so you do not have to go with the hassle of transferring environment variable. IMHO using arguments are simpler. The arguments are simply a dict that are generated by the invocation of scons, so when you say:
scons -D some_argument=blob
You can get that argument by simply:
some_variable = ARGUMENTS["some_argument"]
Of course I do not know why you need the environment variables, so this might be completely irrelevant for you.
I once had a similar need, where the compiler was looking for a certain Env variable that hadnt been set. I was able to solve this problem as follows:
env = Environment()
env['ENV']['THE_VARIABLE'] = 'SomeValue'
I am using the pydev plugin to debug a remote application.
This (remote) application has a structure of files that differs from the structure where my Eclipse is running. This leads to problems when I set the breakpoints from the Eclipse IDE because the pydev debugger server cannot match the absolute path of the file with the file on the remote application and hence the breakpoint isn´t hit.
I dont want to hardcode the pydevd_file_utils.PATHS_FROM_ECLIPSE_TO_PYTHON constant to enable filepath translations.
Do you know some way to modify this value without changing the file?
Thanks!
There are 2 ways of setting the path translation:
Use an environment variable such as PATHS_FROM_ECLIPSE_TO_PYTHON that maps the paths from the client to the server side.
The value is a json string with a list(list(str, str)) such that:
PATHS_FROM_ECLIPSE_TO_PYTHON=[["c:/local/path", "/path/in/server"]]
Note that you may set the environment variable in any place you'd like (such as the Environment tab in the Python interpreter preferences page, in the OS itself, in the launch config, etc).
Use the pydevd API to set the tracing at runtime from the python process:
from pydevd_file_utils import setup_client_server_paths
MY_PATHS_FROM_ECLIPSE_TO_PYTHON = [
('/home/user/local-project', '/remote/path/to/project'),
]
setup_client_server_paths(MY_PATHS_FROM_ECLIPSE_TO_PYTHON)
# At this point we could connect to the remote debugger client with:
import pydevd
pydevd.settrace("10.0.0.12")
See: https://www.pydev.org/manual_adv_remote_debugger.html for more info on the Remote Debugging.
Note: the mapping set in Window > Preferences select PyDev > Debug > Source Locator doesn't really map to that environment variable nor the actual debugger mapping (that's a separate translation that only translates paths which are found on Eclipse locally and it's not really passed on to the debugger to hit breakpoints remotely).
You can do that by setting a new environment variable like this:
PATHS_FROM_ECLIPSE_TO_PYTHON='[["client_src_fullpath", "remote_src_fullpath"]]'
In linux simply run that before starting the program from the command line, or set is as a global variable.
In windows you will need to set it as a global system variable.
Variable name: PATHS_FROM_ECLIPSE_TO_PYTHON
Variable value: [["client_src_path", "remote_src_path"]]
As an alternative, you can also do this in code, BUT you need to do it BEFORE you import pydevd:
import os
os.environ.setdefault("PATHS_FROM_ECLIPSE_TO_PYTHON",'[["client_src_path","remote_src_path"]]')
import pydevd
pydevd.settrace("10.0.2.2", port=5678,stdoutToServer=True, stderrToServer=True, suspend=False,patch_multiprocessing=True)
(I'm aware this is a very old question, but none of the answers were updated to the current code)
Unfortunately there is no good way to do that.
As a workaround I explicitly replaced function NormFileToServer by adding the following code at the beginning of my source file.
def SrcPathMapping(file):
eclipse_src_path = 'C:\\tmp\\workspace\\test\\Scripts\\'
server_src_path = '/home/tester/test/Scripts/'
return file.replace(eclipse_src_path, server_src_path)
import pysrc.pydevd as pydevd
pydevd.NormFileToServer = SrcPathMapping
This simplistic mapping is sufficient when all source files are located in one directory. For proper implementation of the mapping function check NormFileToServer in the pydevd_file_utils.
In relation to another question, how do you account for paths that may change? For example, if a program is calling a file in the same directory as the program, you can simply use the path ".\foo.py" in *nix. However, apparently Windows likes to have the path hard-coded, e.g. "C:\Python_project\foo.py".
What happens if the path changes? For example, the file may not be on the C: drive but on a thumb drive or external drive that can change the drive letter. The file may still be in the same directory as the program but it won't match the drive letter in the code.
I want the program to be cross-platform, but I expect I may have to use os.name or something to determine which path code block to use.
Simple answer: You work out the absolute path based on the environment.
What you really need is a few pointers. There are various bits of runtime and environment information that you can glean from various places in the standard library (and they certainly help me when I want to deploy an application on windows).
So, first some general things:
os.path - standard library module with lots of cross-platform path manipulation. Your best friend. "Follow the os.path" I once read in a book.
__file__ - The location of the current module.
sys.executable - The location of the running Python.
Now you can fairly much glean anything you want from these three sources. The functions from os.path will help you get around the tree:
os.path.join('path1', 'path2') - join path segments in a cross-platform way
os.path.expanduser('a_path') - find the path a_path in the user's home directory
os.path.abspath('a_path') - convert a relative path to an absolute path
os.path.dirname('a_path') - get the directory that a path is in
many many more...
So combining this, for example:
# script1.py
# Get the path to the script2.py in the same directory
import os
this_script_path = os.path.abspath(__file__)
this_dir_path = os.path.dirname(this_script_path)
script2_path = os.path.join(this_dir_path, 'script2.py')
print script2_path
And running it:
ali#work:~/tmp$ python script1.py
/home/ali/tmp/script2.py
Now for your specific case, it seems you are slightly confused between the concept of a "working directory" and the "directory that a script is in". These can be the same, but they can also be different. For example the "working directory" can be changed, and so functions that use it might be able to find what they are looking for sometimes but not others. subprocess.Popen is an example of this.
If you always pass paths absolutely, you will never get into working directory issues.
If your file is always in the same directory as your program then:
def _isInProductionMode():
""" returns True when running the exe,
False when running from a script, ie development mode.
"""
return (hasattr(sys, "frozen") or # new py2exe
hasattr(sys, "importers") # old py2exe
or imp.is_frozen("__main__")) #tools/freeze
def _getAppDir():
""" returns the directory name of the script or the directory
name of the exe
"""
if _isInProductionMode():
return os.path.dirname(sys.executable)
return os.path.dirname(__file__)
should work. Also, I've used py2exe for my own application, and haven't tested it with other exe conversion apps.
What -- specifically -- do you mean by "calling a file...foo.py"?
Import? If so, the path is totally outside of your program. Set the PYTHONPATH environment variable with . or c:\ or whatever at the shell level. You can, for example, write 2-line shell scripts to set an environment variable and run Python.
Windows
SET PYTHONPATH=C:\path\to\library
python myapp.py
Linux
export PYTHONPATH=./relative/path
python myapp.py
Execfile? Consider using import.
Read and Eval? Consider using import.
If the PYTHONPATH is too complicated, then put your module in the Python lib/site-packages directory, where it's put onto the PYTHONPATH by default for you.
I figured out by using os.getcwd(). I also learned about using os.path.join to automatically determine the correct path format based on the OS. Here's the code:
def openNewRecord(self, event): # wxGlade: CharSheet.<event_handler>
"""Create a new, blank record sheet."""
path = os.getcwd()
subprocess.Popen(os.path.join(path, "TW2K_char_rec_sheet.py"), shell=True).stdout
It appears to be working. Thanks for the ideas.