I wrote a module that, if it is imported, automatically changes the error output of my program. It is quite handy to have it in almost any python code I write.
Thus I don't want to add the line import my_errorhook to every code I write but want to have this line added automatically.
I found this answer, stating that it should be avoided to change the behavior of python directly. So I thought about changing the command line, something like
python --importModule my_errorhook main.py
and defining an alias in the bashrc to overwrite the python command to automatically add the parameter. Is there any way I could achieve such a behavior?
There is no such thing like --importModule in python command line. The only way you can incept the code without explicitly importing is by putting your functions in builtins module. However, this is a practice that is discouraged because it makes your code hard to maintain without proper design.
Let's assume that your python file main.py is the entry point of the whole program. Now you can create another file bootstrap.py, and put below codes into the new file.
import main
__builtins__.func = lambda x: x>=0
main.main()
Then the function func() can be called from all modules without being imported. For example in main.py
def main():
...
print(func(1))
...
I have a main file that imports a class from another file as such:
from pybrain.rl.environments.HoldemTask import HoldemTask.
When I change HoldemTask.py, the changes are not reflected in the main file. The only workaround I have found is to run Pybrain's
python setup.py install
Can I reload the module or something? Reload() doesn't seem to work.
First off: python setup.py install generally makes a copy of the code it is installing, so if you're finding that you need to run that before changes take effect, chances are that for development you should be adjusting your PYTHONPATH or sys.path so that your relevant imports come directly from the source tree rather than from the Python site-packages library. You can quickly check which file your code is importing by putting this on the top of the main file when you run it:
from pybrain.rl.environments import HoldemTask # module object, not class
print(HoldemTask.__file__)
Secondly, in general it is far better to restart a Python process when making code changes to ensure that they come into effect. If you really need to get changes to show up without a restart, read on.
Reloading a module in Python only affects future imports. For a reload to work in-process, you have to replace the imported class object after the reload. For example, in the context of the "main file" performing the import you listed (inside a class method or function is fine):
# we need a module object to reload(), not the class inside it
from import pybrain.rl.environments import HoldemTask as HoldemTask_module
reload(HoldemTask_module)
# we then need to replace the old class object with the reloaded one
# in the main file's module-wide (aka "global") namespace
global HoldemTask
HoldemTask = HoldemTask_module.HoldemTask
One final caveat here is that any existing HoldemTask objects will continue to use the old code, because they embed in themselves a reference to the pre-reload class object. The only way for an in-process reload to be complete is if the code is specifically written to throw away every instance of anything it made based on the original module.
Im trying to figure out how to include a .pyc file in a python script.
For example my script is called:
myscript.py
and the script I would like to include is called:
included_script.pyc
So, do I just use:
import included_script
And will that automatically execute the included_script.pyc ? Or is there something further I need to do, to get my included_script.pyc to run inside the myscript.py?
Do I need to pass the variables used in included_script.pyc also? If so, how might this be achieved?
Unfortunately, no, this cannot be done automatically. You can, of course, do it manually in a gritty ugly way.
Setup:
For demonstration purposes, I'll first generate a .pyc file. In order to do that, we first need a .py file for it. Our sample test.py file will look like:
def foo():
print("In foo")
if __name__ == "__main__":
print("Hello World")
Super simple. Generating the .pyc file can done with the py_compile module found in the standard library. We simply pass in the name of the .py file and the name for our .pyc file in the following way:
py_compile.compile('test.py', 'mypyc.pyc')
This will place mypyc.pyc in our current working directory.
Getting the code from .pyc files:
Now, .pyc files contain bytes that are structured in the following way:
First 4 bytes signalling a 'magic number'
Next 4 bytes holding a modification timestamp
Rest of the contents are a marshalled code object.
What we're after is that marshalled code object, so we need to import marshal to un-marshall it and execute it. Additionally, we really don't care/need the 8 first bytes, and un-marshalling the .pyc file with them is disallowed, so we'll ignore them (seek past them):
import marshal
s = open('mypyc.pyc', 'rb')
s.seek(8) # go past first eight bytes
code_obj = marshal.load(s)
So, now we have our fancy code object for test.py which is valid and ready to be executed as we wish. We have two options here:
Execute it in the current global namespace. This will bind all definitions inside our .pyc file in the current namespace and will act as a sort of: from file import * statement.
Create a new module object and execute the code inside the module. This will be like the import file statement.
Emulating from file import * like behaviour:
Performing this is pretty simple, just do:
exec(code_obj)
This will execute the code contained inside code_obj in the current namespace and bind everything there. After the call we can call foo like any other funtion:
foo()
# prints: In foo!
Note: exec() is a built-in.
Emulating import file like behaviour:
This includes another requirement, the types module. This contains the type for ModuleType which we can use to create a new module object. It takes two arguments, the name for the module (mandatory) and the documentation for it (optional):
m = types.ModuleType("Fancy Name", "Fancy Documentation")
print(m)
<module 'Fancy Name' (built-in)>
Now that we have our module object, we can again use exec to execute the code contained in code_obj inside the module namespace (namely, m.__dict__):
exec(code_obj, m.__dict__)
Now, our module m has everything defined in code_obj, you can verify this by running:
m.foo()
# prints: In foo
These are the ways you can 'include' a .pyc file in your module. At least, the ways I can think of. I don't really see the practicality in this but hey, I'm not here to judge.
I need to know how to run a python script from a python script present in other directory like the following algorithm:
if option==true
run /path/to/the/directory/PYTHON SCRIPT
else
ch3ka points out that you can use exec to do this. There are other ways like subprocess or os.system as well.
But Python works well with itself by design - this is the entire concept behind creating and importing modules. I think for most cases you'd be better off just encapsulating the script in a class, and moving the code that was previously in the if __name__ == '__main__' section of the script into the __init__ section of the class:
class PYTHON_SCRIPT:
def __init__(self):
# put your logic here
Then you could just import the class:
import PYTHON_SCRIPT
# no need to say if a boolean is true, just say if boolean
if option:
PYTHON_SCRIPT()
This would additionally give you the benefit of being able to use properties within your script as you saw fit.
use execfile.
execfile(...)
execfile(filename[, globals[, locals]])
Read and execute a Python script from a file.
The globals and locals are dictionaries, defaulting to the current
globals and locals. If only globals is given, locals defaults to it.
In pyton3, execfile is gone. You can use exec(open('/path/to/file.py').read()) instead.
Already answered here
How do I execute a program from python? os.system fails due to spaces in path
use subprocess module
import subprocess
subprocess.call(['C:\\Temp\\a b c\\Notepad.exe', 'C:\\test.txt'])
other methods include making system calls using os library or execfile in the other post
if the script is well designed it probably just launch a main function (often called main), so the most proper way to do this is to import this main function in your code and call it, this is the pythonic way. You just need to add the directory of the script into your python path.
if it's possible, always try to avoid exec, subprocess, os.system, Popen etc ..
example :
import sys
sys.path.insert(0, 'path/to/the/directory')
import python_script
sys.path.pop(0)
if option:
python_script.main()
I'm trying to get the name of the Python script that is currently running.
I have a script called foo.py and I'd like to do something like this in order to get the script name:
print(Scriptname)
You can use __file__ to get the name of the current file. When used in the main module, this is the name of the script that was originally invoked.
If you want to omit the directory part (which might be present), you can use os.path.basename(__file__).
import sys
print(sys.argv[0])
This will print foo.py for python foo.py, dir/foo.py for python dir/foo.py, etc. It's the first argument to python. (Note that after py2exe it would be foo.exe.)
For completeness' sake, I thought it would be worthwhile summarizing the various possible outcomes and supplying references for the exact behaviour of each.
The answer is composed of four sections:
A list of different approaches that return the full path to the currently executing script.
A caveat regarding handling of relative paths.
A recommendation regarding handling of symbolic links.
An account of a few methods that could be used to extract the actual file name, with or without its suffix, from the full file path.
Extracting the full file path
__file__ is the currently executing file, as detailed in the official documentation:
__file__ is the pathname of the file from which the module was loaded, if it was loaded from a file. The __file__ attribute may be missing for certain types of modules, such as C modules that are statically linked into the interpreter; for extension modules loaded dynamically from a shared library, it is the pathname of the shared library file.
From Python3.4 onwards, per issue 18416, __file__ is always an absolute path, unless the currently executing file is a script that has been executed directly (not via the interpreter with the -m command line option) using a relative path.
__main__.__file__ (requires importing __main__) simply accesses the aforementioned __file__ attribute of the main module, e.g. of the script that was invoked from the command line.
From Python3.9 onwards, per issue 20443, the __file__ attribute of the __main__ module became an absolute path, rather than a relative path.
sys.argv[0] (requires importing sys) is the script name that was invoked from the command line, and might be an absolute path, as detailed in the official documentation:
argv[0] is the script name (it is operating system dependent whether this is a full pathname or not). If the command was executed using the -c command line option to the interpreter, argv[0] is set to the string '-c'. If no script name was passed to the Python interpreter, argv[0] is the empty string.
As mentioned in another answer to this question, Python scripts that were converted into stand-alone executable programs via tools such as py2exe or PyInstaller might not display the desired result when using this approach (i.e. sys.argv[0] would hold the name of the executable rather than the name of the main Python file within that executable).
If none of the aforementioned options seem to work, probably due to an atypical execution process or an irregular import operation, the inspect module might prove useful, as suggested in another answer to this question:
import inspect
source_file_path = inspect.getfile(inspect.currentframe())
However, inspect.currentframe() would raise an exception when running in an implementation without Python stack frame.
Note that inspect.getfile(...) is preferred over inspect.getsourcefile(...) because the latter raises a TypeError exception when it can determine only a binary file, not the corresponding source file (see also this answer to another question).
From Python3.6 onwards, and as detailed in another answer to this question, it's possible to install an external open source library, lib_programname, which is tailored to provide a complete solution to this problem.
This library iterates through all of the approaches listed above until a valid path is returned. If all of them fail, it raises an exception. It also tries to address various pitfalls, such as invocations via the pytest framework or the pydoc module.
import lib_programname
# this returns the fully resolved path to the launched python program
path_to_program = lib_programname.get_path_executed_script() # type: pathlib.Path
Handling relative paths
When dealing with an approach that happens to return a relative path, it might be tempting to invoke various path manipulation functions, such as os.path.abspath(...) or os.path.realpath(...) in order to extract the full or real path.
However, these methods rely on the current path in order to derive the full path. Thus, if a program first changes the current working directory, for example via os.chdir(...), and only then invokes these methods, they would return an incorrect path.
Handling symbolic links
If the current script is a symbolic link, then all of the above would return the path of the symbolic link rather than the path of the real file and os.path.realpath(...) should be invoked in order to extract the latter.
Further manipulations that extract the actual file name
os.path.basename(...) may be invoked on any of the above in order to extract the actual file name and os.path.splitext(...) may be invoked on the actual file name in order to truncate its suffix, as in os.path.splitext(os.path.basename(...)).
From Python 3.4 onwards, per PEP 428, the PurePath class of the pathlib module may be used as well on any of the above. Specifically, pathlib.PurePath(...).name extracts the actual file name and pathlib.PurePath(...).stem extracts the actual file name without its suffix.
Note that __file__ will give the file where this code resides, which can be imported and different from the main file being interpreted. To get the main file, the special __main__ module can be used:
import __main__ as main
print(main.__file__)
Note that __main__.__file__ works in Python 2.7 but not in 3.2, so use the import-as syntax as above to make it portable.
The Above answers are good . But I found this method more efficient using above results.
This results in actual script file name not a path.
import sys
import os
file_name = os.path.basename(sys.argv[0])
For modern Python versions (3.4+), Path(__file__).name should be more idiomatic. Also, Path(__file__).stem gives you the script name without the .py extension.
Try this:
print __file__
If you're doing an unusual import (e.g., it's an options file), try:
import inspect
print (inspect.getfile(inspect.currentframe()))
Note that this will return the absolute path to the file.
As of Python 3.5 you can simply do:
from pathlib import Path
Path(__file__).stem
See more here: https://docs.python.org/3.5/library/pathlib.html#pathlib.PurePath.stem
For example, I have a file under my user directory named test.py with this inside:
from pathlib import Path
print(Path(__file__).stem)
print(__file__)
running this outputs:
>>> python3.6 test.py
test
test.py
we can try this to get current script name without extension.
import os
script_name = os.path.splitext(os.path.basename(__file__))[0]
You can do this without importing os or other libs.
If you want to get the path of current python script, use: __file__
If you want to get only the filename without .py extension, use this:
__file__.rsplit("/", 1)[1].split('.')[0]
Since the OP asked for the name of the current script file I would prefer
import os
os.path.split(sys.argv[0])[1]
all that answers are great, but have some problems You might not see at the first glance.
lets define what we want - we want the name of the script that was executed, not the name of the current module - so __file__ will only work if it is used in the executed script, not in an imported module.
sys.argv is also questionable - what if your program was called by pytest ? or pydoc runner ? or if it was called by uwsgi ?
and - there is a third method of getting the script name, I havent seen in the answers - You can inspect the stack.
Another problem is, that You (or some other program) can tamper around with sys.argv and __main__.__file__ - it might be present, it might be not. It might be valid, or not. At least You can check if the script (the desired result) exists !
the library lib_programname does exactly that :
check if __main__ is present
check if __main__.__file__ is present
does give __main__.__file__ a valid result (does that script exist ?)
if not: check sys.argv:
is there pytest, docrunner, etc in the sys.argv ? --> if yes, ignore that
can we get a valid result here ?
if not: inspect the stack and get the result from there possibly
if also the stack does not give a valid result, then throw an Exception.
by that way, my solution is working so far with setup.py test, uwsgi, pytest, pycharm pytest , pycharm docrunner (doctest), dreampie, eclipse
there is also a nice blog article about that problem from Dough Hellman, "Determining the Name of a Process from Python"
BTW, it will change again in python 3.9 : the file attribute of the main module became an absolute path, rather than a relative path. These paths now remain valid after the current directory is changed by os.chdir()
So I rather want to take care of one small module, instead of skimming my codebase if it should be changed somewere ...
Disclaimer: I'm the author of the lib_programname library.
if you get script path in base class, use this code, subclass will get script path correctly.
sys.modules[self.__module__].__file__