Background
I am working on face recognition following this link and I would like to build a standalone application using Python. My main.py script looks like the following.
# main.py
# Import packages and other scripts
import tkinter as tk
...
# Some functions
def initialization():
# OpenCV and sklearn are used here
...
def registration():
# OpenCV, and sklearn are used here
...
def face_recognition():
# OpenCV and sklearn are used here
...
# Start the Tkinter GUI
window = tk.Tk()
# Initialize the face recognition model
initialization()
# Input name for registration
tk.Label(window, text = "Name").grid(...)
entry1 = tk.Entry(window)
entry1.grid(row=0, column=1)
# When the button is clicked, different command is executed
tk.Button(window, text='Registration', command=registeration).grid(...)
tk.Button(window, text='Face Recognition', command=face_recognition).grid(...)
window.mainloop()
Using python interpret to run the script (python main.py), everything works fine.
Problem
I use Pyinstaller to convert the scripts to a single exe with this command:
pyinstaller --onefile \
--windowed \
--hidden-import sklearn.neighbors.typedefs \
main.py
Then, I have two exe generated. The first one is in dist/main and the second one is in dist/main.app/Contents/MacOS/main
When running the exe, the exe keeps duplicate itself. I display the running process and found this result
$ ps
PID TTY TIME CMD
... ... ... /path/to/main -B -s -S -E -c from multiprocessing.semaphore_tracker import main:main(8)
... ... ... /path/to/main -B -s -S -E -c from multiprocessing.semaphore_tracker import main:main(8)
... ... ... /path/to/main -B -s -S -E -c from multiprocessing.semaphore_tracker import main:main(7)
... ... ... /path/to/main -B -s -S -E -c from multiprocessing.semaphore_tracker import main:main(7)
... ... ... /path/to/main -B -s -S -E -c from multiprocessing.semaphore_tracker import main:main(8)
I have no idea what happens to the exe since I do not import multiprocessing packages in my scripts. Any idea how to fix this problem? Thanks
Update 1
I added a --log-level ERROR when using Pyinstaller and gives this result. Not sure if it is related to my problem.
Traceback (most recent call last):
File "<string>", line 2, in <module>
ModuleNotFoundError: No module named 'Crypto.Math'
174598 INFO: MKL libraries found when importing numpy. Adding MKL to binaries
176282 ERROR: Can not find path ./libtbb.dylib (needed by /Users/user/anaconda3/lib/libmkl_tbb_thread.dylib)
Update 2
I found out that one of the packages that I am using -- imultis VideoStream involves threading. I guess it is the reason for observing from multiprocessing.semaphore_tracker as shown above even though I do not import multiprocessing explicitly.
Then I come across with this post, suggesting to add multiprocessing.freeze_support(). It can suppress the GUI window from keep duplicating, but the background processes as shown using $ ps still keep duplicating. The problem has not fixed yet.
Update 3 (Issue located but not fix)
After debugging the code for quite a while, it turns out that the causes of this infinite looping is NOT due to the threading of imultis VideoStream (I write another script to test imultis and it is completely OK). But the problem comes from importing sklearn!!! This problem has been reported in GitHub in this link.
This GitHub link also suggest to include multiprocessing.freeze_support(). Moreover one of the responses suggests to import sklearn after multiprocessing.freeze_support() is called. I try it with a simple script but the problem is still here.
Conclusion: sklearn causes the executable keeps opening. But I have no idea why it is the case, and I do not know how to solve it. Any help would be appreciated. Thanks.
This answer does a good job explaining why this happens: https://stackoverflow.com/a/55382641/109525
First try setting this before starting your program:
export JOBLIB_MULTIPROCESSING=0
If it works, you can add a runtime hook to your program to set the environment variable automatically. See: https://pythonhosted.org/PyInstaller/when-things-go-wrong.html#changing-runtime-behavior
Starting my main entry point with:
from multiprocessing import freeze_support
freeze_support()
Worked for me!
I met the same issue. Scikit-learn provides a workaround.
https://scikit-learn.org/stable/faq.html#why-do-i-sometime-get-a-crash-freeze-with-n-jobs-1-under-osx-or-linux
I added below codes at the beginning of my script and fixed my issue:
import multiprocessing
multiprocessing.set_start_method('forkserver', force=True)
multiprocessing.freeze_support()
Hope this can also fix yours.
Related
I created a project that follows the example in MutPy's README file.
However, when I run mut.py --target calculator --unit-test test_calculator -m, it just opens mut.py instead of actually running MutPy.
The file has the following content:
#!c:\users\admin\appdata\local\programs\python\python39\python.exe
import sys
from mutpy import commandline
if __name__ == '__main__':
commandline.main(sys.argv)
The README doesn't mention any other configurations that need to be done and says that it should be compatible with Python 3.3+ (I'm on Python 3.9.6).
Is there something I'm doing wrong?
Are you runing this from cmd.exe?
If so try:
python.exe mut.py --target calculator --unit-test test_calculator -m
Note that you need python in your PATH Variable.
It will probably also work if you set python as the default application for .py files.
I've tried to make a PyQt5 project an executable file.
I used a PyInstaller module, and I got a half success.
pyinstaller --clean -w -F --specpath=spec -n=project_name -i="..\resource\logo.ico" src\main.py
The executable file generated by this command did not run successfully.
The error message was like this.
pyinstaller --clean -c -F --specpath=spec -n=project_name -i="..\resource\logo.ico" src\main.py
The executable file generated by this command ran successfully.
But it has a terminal even though it is a GUI project.
The difference is just -c and -w. But one can be executed and can not one.
How should I do it?
The problem was subprocess.Popen with Python v3.7.5.
I didn't set the stdin. I set only stdout, stderr.
After I set the stdin=subprocess.PIPE, it works well.
And I want to add 1 more thing.
I've imported the win32api module with Python v3.8.0. This raises issues.
So, I've added the module pywintypes, and now the issue is resolved.
Before
import win32api
After
import pywintypes
import win32api
I created a python script that I would like to execute at each startup. I modified the etc/rc.local, but I don't get the script to run.
etc/rc.local addition (I added the sleep thinking it may help):
(sleep 10; /usr/bin/python3 /home/pi/mower-gps-tracking/app/gps_logger.py)&
imports in the different python scripts (I don't know if it matters):
from ftplib import FTP
import os
import serial
import time
import threading
from gpiozero import LED, Button
When I start the etc/rc.local manually via a ssh command, it runs fine.
Any idea what I'm missing ?
Check if the script (rc.local) is executable ie. has the 'x' attribute set.
And you will probably need a
#!/bin/bash
as line #1 of your script.
Hope this helps.
And as per Barmar above, probably more approp in the Unix forums.
I tried this on my linux box no issues .. As a test create a script that will wrap your script, have it write to a tmp file so you can see if it actually runs. Use nohup and & in your scripts.
/etc/rc.local:
nohup /root/script.sh &
/root/script.sh:
#!/bin/bash
echo "I'm starting something .." > /tmp/startup_thing.log
/root/gps_logger.py
note: all scripts need to be chmod a+x file.ext. if it's not running check for the file /tmp/startup_thing.log . If it's there then you have some other issue. If it isn't then rc.local isn't working.
How can I run a python script with my own command line name like myscript without having to do python myscript.py in the terminal?
Add a shebang line to the top of the script:
#!/usr/bin/env python
Mark the script as executable:
chmod +x myscript.py
Add the dir containing it to your PATH variable. (If you want it to stick, you'll have to do this in .bashrc or .bash_profile in your home dir.)
export PATH=/path/to/script:$PATH
The best way, which is cross-platform, is to create setup.py, define an entry point in it and install with pip.
Say you have the following contents of myscript.py:
def run():
print('Hello world')
Then you add setup.py with the following:
from setuptools import setup
setup(
name='myscript',
version='0.0.1',
entry_points={
'console_scripts': [
'myscript=myscript:run'
]
}
)
Entry point format is terminal_command_name=python_script_name:main_method_name
Finally install with the following command.
pip install -e /path/to/script/folder
-e stands for editable, meaning you'll be able to work on the script and invoke the latest version without need to reinstall
After that you can run myscript from any directory.
I usually do in the script:
#!/usr/bin/python
... code ...
And in terminal:
$: chmod 755 yourfile.py
$: ./yourfile.py
Another related solution which some people may be interested in. One can also directly embed the contents of myscript.py into your .bashrc file on Linux (should also work for MacOS I think)
For example, I have the following function defined in my .bashrc for dumping Python pickles to the terminal, note that the ${1} is the first argument following the function name:
depickle() {
python << EOPYTHON
import pickle
f = open('${1}', 'rb')
while True:
try:
print(pickle.load(f))
except EOFError:
break
EOPYTHON
}
With this in place (and after reloading .bashrc), I can now run depickle a.pickle from any terminal or directory on my computer.
The simplest way that comes to my mind is to use "pyinstaller".
create an environment that contains all the lib you have used in your code.
activate the environment and in the command window write pip install pyinstaller
Use the command window to open the main directory that codes maincode.py is located.
remember to keep the environment active and write pyinstaller maincode.py
Check the folder named "build" and you will find the executable file.
I hope that this solution helps you.
GL
I've struggled for a few days with the problem of not finding the command py -3 or any other related to pylauncher command if script was running by service created using Nssm tool.
But same commands worked when run directly from cmd.
What was the solution? Just to re-run Python installer and at the very end click the option to disable path length limit.
I'll just leave it here, so that anyone can use this answer and find it helpful.
I'm using the trick "python -c 'import myscript.py'" to perform a syntax check on a script which uses 'import gtk'.
I get the following error when performing the syntax check, which implies that the gtk module is executing a check for the X display, even though all that's being done at this point is to import the module.
Traceback (most recent call last):
File "<stdin>", line 15, in ?
File "myscript.py", line 21, in ?
import gtk
File "/usr/src/build/463937-i386/install/usr/lib/python2.3/site-packages/gtk-2.0/gtk/__init__.py", line 37, in ?
RuntimeError: could not open display
Is there a way to avoid this error when performing the syntax check?
Before you ask - I'm not able to set $DISPLAY before the syntax check is run. The check is being run on remote servers as part of a distributed build system. These servers do not have an X display available.
Importing modules in Python executes their code!
Well-behaved modules use the if __name__ == '__main__' trick to avoid side effects, but they can still fail - as happened to you.
[BTW, getting to ImportError means the whole file already has correct syntax.]
If you just want to check syntax, without running at all:
python -m py_compile my_script.py
will check one file (and produce a .pyc as a side effect).
python -m compileall ./
will check a whole dir recursively.
python -c 'compile(open("myscript.py").read(), "myscript.py", "exec")'
avoids creating a .pyc.
But note that merely checking the syntax in Python catches very few bugs! Importing does catch more, e.g. mispelled names. For even better checks, use tools like Pychecker / Pyflakes.
What exactly do you mean by 'syntax checking'?
Can't you use a tool like pylint to check for syntax errors?
Otherwise: a very ugly (but probably possible hack):
In your python script detect whether X is present.
If it's not => use GTK on DirectFramebuffer (no X needed then). You'll need to compile GTK on DirectFB (and/or pygtk) from source (some pointers here).
If the remote machine has vncserver installed, you can have a dummy server running and connect to that. Sample instructions:
remotemachine $ vncserver -depth 16 -geometry 800x600 :7
New 'X' desktop is remotemachine:7
Starting applications specified in /home/user/.vnc/xstartup
Log file is /home/user/.vnc/userve:7.log
remotemachine $ DISPLAY=:7 python -c 'import myscript.py'
…
remotemachine $ vncserver -kill :7
Killing Xtightvnc process ID 32058
In your myscript.py, you could do like this
if __name__=="__main__":
import gtk
That will not execute gtk's __init__.py when you do "python -c 'import myscript.py'"
If you are editing with IDLE, Alt+X will check syntax of current file without running it.