I'm trying to freeze a Python application and Celery, packaged up with esky for update capabilities. I need Celery to be frozen so that it has visibility into the modules from the first executable that will also be present in the library.zip file that py2exe creates.
The problem is that I cannot get Celery to freeze with esky. Here's a bare minimum, stripped down setup.py file that I'm using to try and find a solution around the problem:
from esky import bdist_esky
from distutils.core import setup
setup(name='ColdCelery',
scripts=['C:\\Python27\\Lib\\site-packages\\celery\\bin\\celery.py'],
options = {
'bdist_esky':{
'freezer_module': 'py2exe',
}
}
)
When I run the following command:
python setup.py bdist_esky
I get the following error:
running bdist_esky
running build_scripts
*** searching for required modules ***
error: c:\temp\tmpz5146o\scripts\celery.py: The process cannot access the file
because it is being used by another process
There is no running Python process on the machine that could be using Celery. I assume this is a conflict between py2exe and esky, but don't know how to overcome it.
I can freeze Celery using py2exe without referencing esky without a problem, but I will require being able to update this project in the future, so esky support is a must.
I had problems running py2exe and esky in the tutorial
I use cx freeze in my project and i haven't had any problems.
It also supports python3, linux and mac so i can recommend giving it a go.
http://cx-freeze.readthedocs.org/en/latest/
Ultimately, we did not find the reason for the failures that were occurring, but we did pick up on an important point and we changed our environments to get around the issue.
Our build machine did double duty for development. It's not the best idea of course and it was probably one of the many, many packages installed for that development that was causing trouble.
So we started with a bare bones Windows installation and installed only the minimum number of packages needed to actually perform the build. That seems to have done the trick.
I rediscovered our fix for this issue. The error message is misleading - after debugging I found that the actual error was caused by esky adding some lines to the top of celery.py during the packaging operation. These lines were going above a from __future__ import absolute_import, unicode_literals causing the following error:
SyntaxError: from __future__ imports must occur at the beginning of the file
To fix:
Edit C:\Python27\Lib\site-packages\celery\bin\celery.py and strip all comments and empty lines from the top of the file.
Related
I am trying to get setup with a easily distributable python program which happens to use scipy. I have two windows 10 64 bit virtual machines. On one I have installed the dependencies for my program, and it runs correctly. Additionally I have installed pyinstaller and performed a freeze. The program runs correctly as the building user on the first VM, as well as a new user on the first VM. The issue is that I get a error loading dll on the second VM, which I just spun up as a test.
I attempted to find all imports from scipy and include them as hidden imports. I also included the scipy "extra dll".
pyinstaller --windowed --icon="%SCRIPT_FOLDER%\res\icon.ico" --hidden-import=numpy --hidden-import=scipy._lib._util --hidden-import=scipy.special._ufuncs --hidden-import=scipy --hidden-import=scipy.stats --hidden-import=scipy.interpolate --hidden-import=scipy.special --hidden-import=scipy.cluster.hierarchy --paths C:\Users\User\AppData\Local\Programs\Python\Python37\Lib\site-packages\scipy\extra-dll --workpath "%WORK_FOLDER%" --distpath "%OUTPUT_FOLDER%" "%SCRIPT_FOLDER%\..\something.py" -n something
Also, I can see the referenced dll files in the correct spot in the frozen output folder under scipy/special there are _ufuncs.cp37-win_amd64.pyd and also _ufuncs_cxx.cp37-win_amd64.pyd
The error is generated specifically on the line "from scipy import special".
Also of note, though I guess it does not help much, is that I have followed a similar procedure for OSX and Linux platforms with no such issue, and no need for any manual hidden import specification...
Would appreciate any help that you can give to help figure out what is going wrong.
I have a project (https://github.com/jfkominsky/PyHab) which is essentially an extension of PsychoPy (http://psychopy.org/). I want to use Sphinx and ReadTheDocs for code documentation. I use the autoclass features from Sphinx for my documentation and have most of it in the code itself, which works great on local builds. On ReadTheDocs, this means that I needed to add a requirements file with psychopy as a requirement. This causes the build to fail in the following way:
/home/docs/checkouts/readthedocs.org/user_builds/pyhab/envs/latest/bin/pip install --exists-action=w --cache-dir /home/docs/checkouts/readthedocs.org/user_builds/pyhab/.cache/pip -rdocs/requirements.txt
Command killed due to excessive memory consumption
I'm very amateur at this. Is there a way to use a setup.py virtual environment to get psychopy to work? Is readthedocs just being too stingy with memory? I've noticed psychopy itself has some issues with RTD builds right now, unsure if that's related...
EDIT: I should add, I was using Cpython2 as my environment in RTD. When I use CPython3 instead I get a different failure:
I tried again with Py3 and got a different error:
ERROR: failed building wxWidgets
Traceback (most recent call last):
File "build.py", line 1321, in cmd_build_wx
wxbuild.main(wxDir(), build_options)
File "/tmp/pip-build-7a1lqn7v/wxPython/buildtools/build_wxwidgets.py", line 374, in main
"Error running configure")
File "/tmp/pip-build-7a1lqn7v/wxPython/buildtools/build_wxwidgets.py", line 85, in exitIfError
raise builder.BuildError(msg)
buildtools.builder.BuildError: Error running configure
Finished command: build_wx (0m10.842s)
Finished command: build (0m10.842s)
Command '"/home/docs/checkouts/readthedocs.org/user_builds/pyhab/envs/latest/bin/python" -u build.py build' failed with exit code 1.
I have devised a solution, at least for CPython2.x. The long and short of it, which RTD needs to advertise MUCH better, is that you can slip a bunch of 'mocks' into conf.py in order to bypass modules you don't want to actually import when compiling your docs. You need mock in your requirements file in 2.7 (presumably unittest.mock in 3.x)
The idea came from here: http://blog.rtwilson.com/how-to-make-your-sphinx-documentation-compile-with-readthedocs-when-youre-using-numpy-and-scipy/
Which connects back to this in the FAQ: http://read-the-docs.readthedocs.io/en/latest/faq.html#i-get-import-errors-on-libraries-that-depend-on-c-modules
I realized that you could create mock modules for literally anything, and if you use MagicMock you can add attributes to them as needed. It took me several tries to get all the right mock modules (mostly because the RTD environment is hard to emulate locally and it would only show you the first module it failed to import in the error message), but eventually, I was able to convince RTD that it had everything it needed without exceeding its memory limits.
This solution is not psychopy-specific. This will work for anything that you have trouble pip installing on RTD.
I have created a desktop gui application which i want to package. I was directed to use pyinstaller to package it so i did. I have a mainwindow.py file where i import a resource file (converted to python code) as well as other imports going on in there. However when i run the output file,it gives me an error. I suspected it was due to my resource file, from the error so i tried pyinstaller on one of the modules and it worked perfectly. however i keep getting an error with the MainWindow.py file. I have the image of the error here for your consideration. How do i go around this error?
I have managed to solve the problem. After reading this answer, I tried pyrcc4 with the '-py3' flag and it worked. I think without the py3 flag,the resource file is converted into a python 2.x code. And I was trying to interpret py2 code with py3 interpreter. That's my understanding of the problem tho. But in case you have a similar problem,try using the -'py3' flag for python3 interpreter.Good Luck
Just working my way through a (very good) book call Test Driven Development using Python.
This makes use of Python3.4 by the way. By the way, I am running in a Windows 7 OS.
I've got all the stuff working using a simple text editor and running from the command line... in the course of which in particular I used "pip install" to install Django and Selenium, as per book's instructions.
This created folders "selenium" and "django" under ...\Python34\Lib\site-packages\ ... so I added these to the PythonPath for my Eclipse/PyDev project.
With the correct interpreter selected I then tried to run a file which runs fine on the command line: "> python3 functional_tests.py"... but I get
File "D:\apps\Python34\lib\site-packages\django\http\__init__.py", line 1, in <module>
from django.http.cookie import SimpleCookie, parse_cookie
File "D:\apps\Python34\lib\site-packages\django\http\cookie.py", line 5, in <module>
from django.utils.six.moves import http_cookies
ImportError: cannot import name 'http_cookies'
... to me this looks like a dependency thing... as though "pip install" handles dependency matters in a way just including a single folder doesn't.
Question boils down to this: what's the "proper" way to install a python module using PyDev?
several days later
wow... nothing? Nothing! I suppose this must mean that you either have to add dependencies manually or use something like Ant, Maven or Gradle within Eclipse itself. These latter are not my strong areas, even outside an IDE. Would still be nice to have an answer from a PyDev expert!
Well, pip install should work for PyDev (it should automatically recognize the dependency)...
I.e.: in your use case, the only folder that should be in the PYTHONPATH is D:\apps\Python34\lib\site-packages (and pip should install packages to that folder -- make sure you don't add extra folders for "D:\apps\Python34\lib\site-packages\django" nor anything else inside the site-packages to the PYTHONPATH).
If it's still not working, please check if the module django.utils.six.moves.http_cookies is indeed where you expect it to be. Also, you can print the PYTHONPATH being used in runtime with:
import sys
print('\n'.join(sorted(sys.path)))
To check if that's really what you expect.
I have written a sudoku solver using especially, but not only, tkinter and multiprocessing. I want to make it a standalone app, which I'm using py2app for.
I've run py2applet-3.3 --make-setup sudoku_app.py (that is what it's called when installed through macports), and it succeeded at that. However when building with python3.3 (python3.3 setup.py py2app), it returns a SyntaxError with an almost endless stack trace that ends like this:
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/modulegraph/modulegraph.py", line 954, in load_module
co = compile(contents, pathname, 'exec', 0, True)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/sympy/mpmath/libmp/exec_py2.py", line 2
exec string in globals, locals
^
SyntaxError: invalid syntax
When running the same setup file with python2.7, it does succeed to build, but there's a bunch of errors when trying to run the created .app. I've written my code in python3, although trying to maintain python2 compatibility through various try ... except - blocks and from future imports. At some point though, it got too complicated and I decided to just go with python3.
Why does py2app not work with python3 there?
Your application doesn’t work with py2app (yet) because I didn’t consider the way you do py3k support. In particular, modulegraph assumes that all modules in the dependency graph contain valid python code for the current python version.
Could you file a bug at my bitbucket page (https://bitbucket.org/ronaldoussoren/py2app) to ensure that I don’t forget to fix this?