In general I'm curious, does using a utility that converts a non-exe to an exe increase speed? It's my understanding that they just package the interpreter inside the exe.
In specific, if you have a python script and use py2exe on it, does the resulting executable run faster than the .py? My boss seems to have the assumption it does but I'm not so sure.
Especially when dealing with multiple modules. For example say you have modules first.py and second.py. You compile them all to executables. When they were .py they second.py could be called as
from second import main
main()
Now that they're executables you have to start a new process, which surely is slower?
subproccess.call(["second.exe"], shell=True)
Do I understand this correctly? Or does importing from another python module actually start a new instance of the python interpreter or something?
In our case the target platform is always Windows.
Your boss is misinformed. All py2exe does is package your program into a self-contained package capable of running without dependencies. It is still the same bytecode running on the same interpreter (well, whatever one is packaged).
See this other answer for about all of the "optimization" you can get out of using -o flags.
Also, yes, definitely run some benchmarks to confirm for yourself.
Related
Excuse the awkward question wording.
I've made a script. I would like for others to download it from github, and run it by typing programName argument1 argument2, similar to any other popular app used through the terminal such as Jupyter or even opening Atom/Sublime/etc. (ex:jupyter notebook, atom .). However, unlike Jupyter or sublime, my script isn't launching another app, it's a small app meant to be used in the shell.
Currently, to use my script, one must type into the command line python programName.py arg1 etc from within the file's directory.
How do I allow others to dl it and use it from anywhere (not having to be within the directory), without having to type out the whole python programName.py part, and only having to type programName arg1?
This blog post explains step by step how to create a distribution that you can install and it would turn into an executable.
You can refer to this github repo for a sample application.
The full documentation of setuptools is available here.
In general, you should configure your setup.py in order to use the command in the entry-point option:
setup(
name = "your_app_name",
packages = ["package_name"],
entry_points = {
"console_scripts": ['cmd_name = package_name.package_name:main']
},
....
)
This solution would work on every OS where you have installed python.
Your script may need to have an interpreter, "shebang", besides being "reachable" by the $PATH
#!interpreter [optional-arg]
For example, you could have something like
#!/usr/bin/env python
or to force a specific version
#!/usr/local/bin/python2.7
Next, your script needs to be available within the $PATH, check this answer that covers that part: https://unix.stackexchange.com/q/29608/53084
You can simply add your script to PATH variable in order to launch it from anywhere.
In Linux distros, you can simply do it by using a bash command PATH=$PATH:/path/to/your/script.
Make sure you don't have the space around the "=" operator.
Now, the second thing is you don't want your script to be named as pythonProgram.py.You can simply remove the extension .py from PythonProgram.py by adding a single line to the starting of your script.
Open up your script and at the very begining type #!/usr/bin/python.This should be the first line of your code.This line is called shebang and is used to tell the bash which interpreter to be used for compiling the script.
If everything went right, you will be able to run your script as pythonProgram arg1.
In addition to mabe02: Python is a scripting language and usually not compiled, which means you will need an interpreter to run your program.
Programms made in C f.e. can be run on its own because they were compiled into machine code by a compiler.
An interpreter is similar to a compiler as it reads your script and interprets it at runntime. That is why you need to write python before your programm, to tell your computer to interpret your script on runntime, using python. This doesn't mean that there are no possibilities to compile python as can be seen in the other answer and in this link Can a python program be run on a computer without Python? What about C/C++? (py2exe and py2app).
I think I already answered my own question in my mind but in case there are other reasons I don't see, I am wondering what's the benefit/difference between running a python script with
python script.py
and using the shebang
#!/usr/local/bin/python
I usually run my scripts as self executing scripts so I use the shebang. I just came across a team member who runs his programs using
python script.py
My question is which way is better or is this a personal preference thing?
If I run with shebang, I can specify which version I want/need to use. The only problem is I need to make sure that version is installed at that location.
If I run it the other way (not sure what to call it, non-shebang way?), I need to make sure the version I want to use is in my path or defined correctly in my path. I could also type in the full path of python but that can get rather tiring if the path is very long.
Are there other benefits/drawbacks that I don't see? Thanks.
If you want to run your script with shebang, you'll have to make sure that the user running the script has execution rights on the script in question (chmod u+x script.py). That's not the case if you call python with the script as an argument.
Another issue with the shebang solution is that you're forcing the location of the python executable. If I try to run your script and my version of python is in /usr/bin/python instead of /usr/local/bin/python, I'll have to edit your script. On other platforms, such as Windows, I'll have to edit it too.
With the shebang the script must be located on a volume that permits execution, must be allowed to execute by any security modules, and the interpreter specified in the shebang must be present.
Invoking the interpreter directly only requires the script to be readable.
In general environment they are the same.
However, shebang gives you an extra benefit that you can replace your executable very easily without changing any other files just by substituting the file. The second(better) implementation can potentially be anything. For a script based environment, I always use shebang, with a fixture of languages.
for the shebang line, the argument that you need to be concerned about the location of python binary is not valid. you can always write:
#!/usr/bin/env python
to have local environment to decide for you.
If you are treating python as scripting language that one might execute from a POSIX compliant shell then it really doesn't matter.
That said, given that Python is more or less becoming standardized with its package management and installation, I would argue none of the above. What should be done is that all scripts be turned into callables (either functions or callable class instances) that live within modules (preferably defined a namespace package). That callable is the entry point to your program/script. Then a setuptools compliant setup.py should be placed in the root directory of the project for this module and define that as the entry point to your program.
An example:
In example.package/example/package/script.py:
def main():
print('hello world')
The setup.py:
from setuptools import setup, find_packages
setup(
name='example.package',
description='An example package with an entry point',
...
entry_points="""
# -*- Entry points: -*-
[console_scripts]
script = example.package.script:main
""",
)
Note that the script is defined to be example.package.script:main, where the element before the : corresponds to the package and the second element is the callable. In this case, it corresponds to the main function in example/package/script.py (standard python module layout).
Installing the package (preferably in a virtualenv) will generate the correct "executable" that will be immediately accessible from the shell, across all operating systems (i.e. on Windows, a script.exe will be created that basically will start Python with the script).
There was a simple task - well, in Python it took around one hundred lines of code (the task was only to ask users for input files, process them and write results on disk). But the requirement was that it must be executable on PCs without Python interpreter. I used py2exe to make an executable file (it has increased the size from 3Kb to ~12 Mb but doesn't really matter).
The problem is - when one tries to run this *.exe, it shows black command line window for half a minute and only after that - file-selecting dialogue. Is there a possibility to get rid of that half-minute delay? Of maybe there are other .py to .exe converters which would suit better in this situation?
In py2exe
console = ['yourscript.py']
will generate a command window, use the following instead
windows = ['yourscript.py']
http://www.py2exe.org/index.cgi/ListOfOptions
This is perfectly normal when making exe files with Python code. When you make an executable file, Python itself is bundled into the .exe. This is so the user does not have to install Python on their machine to make it work. Python is an interpreted language and requires the interpreter to be bundled.
You could always try using alternatives to see if the compression rate is smaller, but chances are, its not a big deal.
If it is the code that is taking a long time, you may consider posting your code on Stack Exchanges' Code Review to see if there is anything that could be improved.
Further, if you are using Python 2.7, you should consider checking out PyInstaller. It is surprisingly easy, however, it has a couple of problems - especially with the PySide Framework - works great for plain PyQt though.
pyinstaller myscript.py --name="MyApp" --onefile
However, for a full list of optional parameters you should really check out the documentation.
I'm currently working on a small python script, for controlling my home PC (really just a hobby project - nothing serious).
Inside the script, there is two threads running at the same time using thread (might start using threading instead) like this:
thread.start_new_thread( Function, (Args) )
Its works as intended when testing the script... but after compiling the code using Pyinstaller there are two processes (One for each thread - I think).
How do I fix this?
Just kill the loader from the main program if it really bothers you. Here's one way to do it.
import os
import win32com.client
proc_name = 'MyProgram.exe'
my_pid = os.getpid()
wmi = win32com.client.GetObject('winmgmts:')
all_procs = wmi.InstancesOf('Win32_Process')
for proc in all_procs:
if proc.Properties_("Name").Value == proc_name:
proc_pid = proc.Properties_("ProcessID").Value
if proc_pid != my_pid:
print "killed my loader %s\n" % (proc_pid)
os.kill(proc_pid, 9)
Python code does not need to be "compiled with pyinstaller"
Products like "Pyinstaller" or "py2exe" are usefull to create a single executable file that you can distribute to third parties, or relocate inside your computer without worrying about the Python instalation - however, they don add "speed" nor is the resulting binary file any more "finished" than your original .py (or .pyw on Windows) file.
What these products do is to create another copy of the Python itnrepreter, alogn with all the modules your porgram use, and pack them inside a single file. It is likely the Pyinstaller keep a second process running to check things on the main script (like launching it, maybe there are options on it to keep the script running and so on). This is not part of a standard Python program.
It is not likely Pyinstaller splits the threads into 2 separate proccess as that would cause compatibility problems - thread run on the same process and can transparently access the same data structures.
How a "canonical" Python program runs: the main process, seen by the O.S. is the Python binary (Python.exe on Windows) - it finds the Python script it was called for - if there is a ".pyc" file for it, that is loaded - else, it loads your ".py" file and compiles that to Python byte code (not to windwos executable). This compilation is authomatic and transparent to people running the program. It is analogous to a Java compile from a .java file to a .class - but there is no explicit step needed by the programmer or user - it is made in place - and other factors control wether Python will store the resulting bytecode as .pyc file or not.
To sum up: there is no performance impact in running the ".py" script directly instead of generating an .exe file with Pyinstaller or other product. You have a disk-space usage inpact if you do, though, as you will have one copy of the Python interpreter and libraries for each of your scripts.
The URL pointeded by Janne Karila on the comment nails it - its even worse than I thought:
in order to run yioru script, pyinstaller unpacks Python DLLs and modules in a temporary directory. The time and system resources needed todo that, compared with a single script run is non-trivial.
http://www.pyinstaller.org/export/v2.0/project/doc/Manual.html?format=raw#how-one-file-mode-works
I would like to distribute an application written in python as a .exe file. I have already been able to do this using py2exe, but now I have incorporated the veusz library into my code. Ideally my program should open up a veusz plot (as it does on my computer, which has python, numpy, etc. all installed). However, I want to distribute an executable that does this without having to install python.
When I try running my setup.py with py2exe, everything goes fine and the exe is built. However, once the application runs and gets to the point where it is to display the graph, it sends up:
Runtime error: Unable to find veusz executable on system path.
Can I fix this without having to install a bunch of stuff on my clients' computers? Is this possible? And if I must install something, what is the minimum amount of software I need to install?
Veusz runs its user interface in a separate python process so that it does not block python. If you look at veusz/embed.py, it tries to start up python or a veusz executable. You'd need to modify embed.py to start your .exe (sys.executable) if frozen instead of veusz and pass some special parameter which your program would interpret to start running veusz.embed_remote.runremote.
The python multiprocessing module has to do something similar - you need to call a multiprocess function which checks whether the program was starting by multiprocessing - to get around the fact that Windows doesn't have a working fork.