I'm writing a Python 3 script meant to be run from Jenkins; However, I'd like it to print several debug messages only when it runs locally on a developer's PC.
I know a possible solution would be creating an Environment variable in the developer's IDE to be passed to the Interpreter and then check for it on start-up:
debug_mode = False
if 'DEBUGMODE' in os.environ:
debug_mode = bool(os.environ.get('DEBUGMODE'))
print('Script is starting up')
(...) # Do stuff
if debug_mode:
print('So many things to do...')
(...) # Do other stuff
Actually, I don't like to force the developer to define DEBUGMODE in his/her environment, so I'm
wondering if there's any other way for my script to automatically know it's running in a Jenkins job and not in a Debugger.
Thanks in advance!
Max
When a Jenkins job executes, it always sets some default environment variables.
In your python code you can just check to see if one (or more) of these variables exists.
You can go for the JENKINS_URL environment variable as it is quite unique and probably wont be used for any other purpose beside what you want to achieve.
So your code can look like:
debug_mode = 'JENKINS_URL' not in os.environ
print('Script is starting up')
(...) # Do stuff
if debug_mode:
print('So many things to do...')
(...) # Do other stuff
Related
I have this simple environment set script in Python:
import os
os.environ['API_USER'] = 'SuperUser'
os.environ['API_PASSWORD'] = 'SuperPass'
USER = os.environ.get('API_USER')
PASSWORD = os.environ.get('API_PASSWORD')
print(USER)
print(PASSWORD)
When I run it I get this:
SuperUser
SuperPass
So far OK, but when I comment out two lines where I set user and pass and run it again, I get this:
None
None
Why those two environment variable are removed and are not saved? I was wondering, that once I set some environment variable, I can use it again later without setting it again (until I delete them), right?
What is wrong?
Thank you!
You are only setting the environment variable for that specific running Python process. Once the process exits, the variable ceases to exist. It appears you cannot set a persistent environment variable from Python, as is described here:
Why can't environmental variables set in python persist?
It looks like you'll need to shell execute whatever variable you want to persist, as is described in the second answer to the above questions, like so:
import pipes
import random
r = random.randint(1,100)
print("export BLAHBLAH=%s" % (pipes.quote(str(r))))
Edit:
As per #SuperStormer's comment below--they are correct--this solution is Unix/Mac specifc. If you need to accomplish the same thing on Windows you would utilize setx like so (this would set the variable for only current user, you need to add a /m switch on the end to set for the local machine):
import pipes
import random
r = random.randint(1,100)
print("setx BLAHBLAH %s" % (pipes.quote(str(r))))
Can someone help me with this please?
I am trying to compile a program in this case programmed in python that I can run in win9Xdos, that I can call/start from a 9xDos batchfile, that will find the Current working Dir & by that I mean identify the cwd (current working directory) from where the python program and batchfile are executed. The python program will be called getcwd.py which I am hoping someone will outline what I need to do to convert to EXE/COM file. there is a program called Py2EXE but not sure if this will compile for Ms-dos file. Anyways heres my simple code thus far. Can someone tell me if I am on the right track please? Oh by the way what I am trying to do is find the CWD & inject the resultant path into a variable that can be read from 9Xdos. The current Var would be %cwd%
# importing os module
import os
# some websites say use: del cwd (to remove variable if it exists)
cwd = none
cwd = os.getcwd()
print(cwd)
The print line may need interchanging with code below, not sure help needed:
print(type(path))
# <class 'str'>
would the above code work, say in the root e.g. C:\ with & work in obtaining the CWD variable & if correct how would I proceed to compile it to an exe/com file? do I need to take into account LFN's & Spaces between possible paths e.g C:\Program Files & possible backslashes etc?
Your code isn't perfect but it is on the right track. All you need is this:
import os
if __name__ == '__main__':
print(os.getcwd())
There is no need for an auxiliary variable, and I don't know what websites are recommending that you delete the variable before creating it. Trying to delete a nonexistent Python variable is a runtime error. So I would stay away from those websites.
But your question is about setting an environment variable. Calling print() won't do that. All it will do is echo the current working directory to the console. There is no way to change the environment of a running process that will affect the parent process. This is not a Python restriction nor a Windows restriction: it is quite general. The OS sets up the environment of the process when it creates the process. You can make changes to the environment (using os.environ[env-var]) but those changes will only be visible inside that Python process and will not be visible to the environment of the batch file that runs the Python program. To do that, you need to pass the value back to the calling process.
One way to do that is like this:
In Python:
import os
if __name__ == '__main__':
print(f"set CWDIR={os.getcwd()}", file=open("mycd.bat","w"))
I haven't had a Python 1.5.2 environment for 15 years, so I can't test this, but I think the equivalent would have been
if __name__ == '__main__':
print >> open("mycd.bat","w"), "set CWDIR=%s" % (os.getcwd(),)
In a cmd.exe console:
call mycd.bat
Though if your Win9XDos doesn't provide %cd% (which, as far as I recall, was available in MS-DOS 5, or maybe further back still) there is no way of telling if it supports call either. (Are you maybe running command.com instead of cmd.exe? That would explain why things that should be there are missing).
I used pyinstaller to create a 64-bit .exe and that resulted in a file of about 6MB. Now, 32-bit executables are smaller, but it might be that the resulting executable is still too big to load.
So I think the Python route may turn out to be more trouble than it is worth.
I am using the pydev plugin to debug a remote application.
This (remote) application has a structure of files that differs from the structure where my Eclipse is running. This leads to problems when I set the breakpoints from the Eclipse IDE because the pydev debugger server cannot match the absolute path of the file with the file on the remote application and hence the breakpoint isn´t hit.
I dont want to hardcode the pydevd_file_utils.PATHS_FROM_ECLIPSE_TO_PYTHON constant to enable filepath translations.
Do you know some way to modify this value without changing the file?
Thanks!
There are 2 ways of setting the path translation:
Use an environment variable such as PATHS_FROM_ECLIPSE_TO_PYTHON that maps the paths from the client to the server side.
The value is a json string with a list(list(str, str)) such that:
PATHS_FROM_ECLIPSE_TO_PYTHON=[["c:/local/path", "/path/in/server"]]
Note that you may set the environment variable in any place you'd like (such as the Environment tab in the Python interpreter preferences page, in the OS itself, in the launch config, etc).
Use the pydevd API to set the tracing at runtime from the python process:
from pydevd_file_utils import setup_client_server_paths
MY_PATHS_FROM_ECLIPSE_TO_PYTHON = [
('/home/user/local-project', '/remote/path/to/project'),
]
setup_client_server_paths(MY_PATHS_FROM_ECLIPSE_TO_PYTHON)
# At this point we could connect to the remote debugger client with:
import pydevd
pydevd.settrace("10.0.0.12")
See: https://www.pydev.org/manual_adv_remote_debugger.html for more info on the Remote Debugging.
Note: the mapping set in Window > Preferences select PyDev > Debug > Source Locator doesn't really map to that environment variable nor the actual debugger mapping (that's a separate translation that only translates paths which are found on Eclipse locally and it's not really passed on to the debugger to hit breakpoints remotely).
You can do that by setting a new environment variable like this:
PATHS_FROM_ECLIPSE_TO_PYTHON='[["client_src_fullpath", "remote_src_fullpath"]]'
In linux simply run that before starting the program from the command line, or set is as a global variable.
In windows you will need to set it as a global system variable.
Variable name: PATHS_FROM_ECLIPSE_TO_PYTHON
Variable value: [["client_src_path", "remote_src_path"]]
As an alternative, you can also do this in code, BUT you need to do it BEFORE you import pydevd:
import os
os.environ.setdefault("PATHS_FROM_ECLIPSE_TO_PYTHON",'[["client_src_path","remote_src_path"]]')
import pydevd
pydevd.settrace("10.0.2.2", port=5678,stdoutToServer=True, stderrToServer=True, suspend=False,patch_multiprocessing=True)
(I'm aware this is a very old question, but none of the answers were updated to the current code)
Unfortunately there is no good way to do that.
As a workaround I explicitly replaced function NormFileToServer by adding the following code at the beginning of my source file.
def SrcPathMapping(file):
eclipse_src_path = 'C:\\tmp\\workspace\\test\\Scripts\\'
server_src_path = '/home/tester/test/Scripts/'
return file.replace(eclipse_src_path, server_src_path)
import pysrc.pydevd as pydevd
pydevd.NormFileToServer = SrcPathMapping
This simplistic mapping is sufficient when all source files are located in one directory. For proper implementation of the mapping function check NormFileToServer in the pydevd_file_utils.
Is it possible to change environment variables of current process?
More specifically in a python script I want to change LD_LIBRARY_PATH so that on import of a module 'x' which depends on some xyz.so, xyz.so is taken from my given path in LD_LIBRARY_PATH
is there any other way to dynamically change path from where library is loaded?
Edit: I think I need to mention that I have already tried thing like
os.environ["LD_LIBRARY_PATH"] = mypath
os.putenv('LD_LIBRARY_PATH', mypath)
but these modify the env. for spawned sub-process, not the current process, and module loading doesn't consider the new LD_LIBRARY_PATH
Edit2, so question is can we change environment or something so the library loader sees it and loads from there?
The reason
os.environ["LD_LIBRARY_PATH"] = ...
doesn't work is simple: this environment variable controls behavior of the dynamic loader (ld-linux.so.2 on Linux, ld.so.1 on Solaris), but the loader only looks at LD_LIBRARY_PATH once at process startup. Changing the value of LD_LIBRARY_PATH in the current process after that point has no effect (just as the answer to this question says).
You do have some options:
A. If you know that you are going to need xyz.so from /some/path, and control the execution of python script from the start, then simply set LD_LIBRARY_PATH to your liking (after checking that it is not already so set), and re-execute yourself. This is what Java does.
B. You can import /some/path/xyz.so via its absolute path before importing x.so. When you then import x.so, the loader will discover that it has already loaded xyz.so, and will use the already loaded module instead of searching for it again.
C. If you build x.so yourself, you can add -Wl,-rpath=/some/path to its link line, and then importing x.so will cause the loader to look for dependent modules in /some/path.
Based on the answer from Employed Russian, this is what works for me
oracle_libs = os.environ['ORACLE_HOME']+"/lib/"
rerun = True
if not 'LD_LIBRARY_PATH' in os.environ:
os.environ['LD_LIBRARY_PATH'] = ":"+oracle_libs
elif not oracle_libs in os.environ.get('LD_LIBRARY_PATH'):
os.environ['LD_LIBRARY_PATH'] += ":"+oracle_libs
else:
rerun = False
if rerun:
os.execve(os.path.realpath(__file__), sys.argv, os.environ)
In my experience trying to change the way the loader works for a running Python is very tricky; probably OS/version dependent; may not work. One work-around that might help in some circumstances is to launch a sub-process that changes the environment parameter using a shell script and then launch a new Python using the shell.
The below code is to set the LD_LIBRARY_PATH or any other environment variable paths that is required by the import modules.
if os.getenv('LD_LIBRARY_PATH')==None:
os.environ['LD_LIBRARY_PATH']='<PATH>'
try:
sys.stdout.flush()
os.execl(sys.executable,sys.executable, *sys.argv)
except OSError as e:
print(e)
elif <path> not in os.getenv('LD_LIBRARY_PATH'):
os.environ['LD_LIBRARY_PATH'] = ':'.join([os.getenv('LD_LIBRARY_PATH'),'<PATH>'])
try:
sys.stdout.flush()
os.execl(sys.executable,sys.executable, *sys.argv)
except OSError as e:
print(e)
# import X
The function os.execl will replace the current process. In UNIX a new executable will be loaded into the current process.
By having this code before the import of the 'X' module, now it will be looking for the files in the new path that was set.
More on execl
well, the environment variables are stored in the dictionary os.environ, so if you want to change , you can do
os.environ["PATH"] = "/usr/bin"
Hey I was wondering... I am using the pydev with eclipse and I'm really enjoying the powerful debugging features, but I was wondering:
Is it possible to set a breakpoint in eclipse and jump into the interactive python interpreter during execution?
I think that would be pretty handy ;)
edit: I want to emphasize that my goal is not to jump into a debugger. pydev/eclipse have a great debugger, and I can just look at the traceback and set break points.
What I want is to execute a script and jump into an interactive python interpreter during execution so I can do things like...
poke around
check the values of things
manipulate variables
figure out some code before I add it to the app
I know you can do this all with a debugger, but I can do it faster in the interactive interpreter because I can try something, see that it didn't work, and try something else without having get the app back to the point of executing that code again.
So roughly a year on from the OP's question, PyDev has this capability built in. I am not sure when this feature was introduced, but all I know is I've spent the last ~2hrs Googling... configuring iPython and whatever (which was looking like it would have done the job), but only to realise Eclipse/PyDev has what I want ootb.
As soon as you hit a breakpoint in debug mode, the console is right there ready and waiting!
I only didn't notice this as there is no prompt or blinking cursor; I had wrongly assumed it was a standard, output-only, console... but it's not. It even has code-completion.
Great stuff, see http://pydev.org/manual_adv_debug_console.html for more details.
This is from an old project, and I didn't write it, but it does something similar to what you want using ipython.
'''Start an IPython shell (for debugging) with current environment.
Runs Call db() to start a shell, e.g.
def foo(bar):
for x in bar:
if baz(x):
import ipydb; ipydb.db() # <-- start IPython here, with current value of x (ipydb is the name of this module).
.
'''
import inspect,IPython
def db():
'''Start IPython shell with callers environment.'''
# find callers
__up_frame = inspect.currentframe().f_back
eval('IPython.Shell.IPShellEmbed([])()', # Empty list arg is
# ipythons argv later args to dict take precedence, so
# f_globals() shadows globals(). Need globals() for IPython
# module.
dict(globals().items() + __up_frame.f_globals.items()),
__up_frame.f_locals)
edit by Jim Robert (question owner): If you place the above code into a file called my_debug.py for the sake of this example. Then place that file in your python path, and you can insert the following lines anywhere in your code to jump into a debugger (as long as you execute from a shell):
import my_debug
my_debug.db()
I've long been using this code in my sitecustomize.py to start a debugger on an exception. This can also be triggered by Ctrl+C. It works beautifully in the shell, don't know about eclipse.
import sys
def info(exception_type, value, tb):
if hasattr(sys, 'ps1') or not sys.stderr.isatty() or not sys.stdin.isatty() or not sys.stdout.isatty() or type==SyntaxError:
# we are in interactive mode or we don't have a tty-like
# device, so we call the default hook
sys.__excepthook__(exception_type, value, tb)
else:
import traceback
import pdb
if exception_type != KeyboardInterrupt:
try:
import growlnotify
growlnotify.growlNotify("Script crashed", sticky = False)
except ImportError:
pass
# we are NOT in interactive mode, print the exception...
traceback.print_exception(exception_type, value, tb)
print
# ...then start the debugger in post-mortem mode.
pdb.pm()
sys.excepthook = info
Here's the source and more discussion on StackOverflow.
You can jump into an interactive session using code.InteractiveConsole as described here; however I don't know how to trigger this from Eclipse.
A solution might be to intercept Ctrl+C to jump into this interactive console (using the signal module: signal.signal(signal.SIGINT, my_handler)), but it would probably change the execution context and you probably don't want this.
If you are already running in debug mode you can set an additional breakpoint if the program execution is currently paused (e.g. because you are already at a breakpoint). I just tried it out now with the latest Pydev - it works just fine.
If you are running normally (i.e. not in debug mode) all breakpoints will be ignored. No changes to breakpoints will alter the way a non-debug run works.