We Keep Coding

Compiling python project with Cx-Freeze

I'm trying to make and executable from a project I'm working on, to test it on a different computer, without requiring python installed.
I want to use cx freeze to build and exe file, and this is my project tree:
- Project
- objects
blocks.py
enemy.py
item.py
player.py
__init__.py
- scripts
camera.py
chunk_manager.py
__init__.py
- textures
- items
- player
- terrain
- UI
- void
index.py
__init__.py
main.py
settings.py
setup.py
map.json
As you can probably tell, this is a game.
Anyways, what I have done is executing just cxfreeze main.py --target-dir=./ dist and it generated a build directory with a lot of stuff in it.
It generated a linux executable, but thats fine, I want to test if I can make it python-independent first, and I'll solve the .exe part later.
So, I executed the file, and nothing happened. Then I tried running from the terminal and it said that it was missing the camera.py file. I went into the generated directory and I found out that cxfreeze had not copied that file. I tried moving in in there myself, but it did not work.
I started checking for other files, and I found out that only the textures had been copied.
I thought this was just because of the one line command I used, so I am now trying to make a setup.py file (as you can see in the file tree) but I am lost in how to import my custom packages (objects, scripts, and textures) and the files in the same directory as main.py
This is how my setup.py file looks like:
import sys
from cx_Freeze import setup, Executable
options = {
'build_exe': {
'includes': ['camera', 'chunk_manager'], 'path': sys.path + ['scripts']
}
}
setup(
name = "Void Boats",
version = "alpha0.1",
options = options,
executables = [Executable("main.py")])
I am not sure how would I put all the packages in the 'include' section, and I can't find anything on the internet that uses more than a simple file like helloworld.py, and the samples of cxfreeze on their github only show how to import files from one directory. Do I have to move everything into one single package and put all the files in the same 'include'? Or can I have one include for each package I have? Any help would be pretty much appreciated.
Edit: I'm running on ParrotSec 4.9 (Debian based)

Use "python -m setup.py build" on cmd, that will build exe.
Example setup with extra folders:
from cx_Freeze import setup, Executable
import sys
version = "0.0.6"
build_options = {
"packages": [],
"excludes": [],
"build_exe": "X:\\builds\\",
"include_files": ["Sequence_Sample/", "icons/"],
}
base = "Win32GUI" if sys.platform == "win32" else None
executables = [Executable("MainUIController.py", base=base, targetName="pym")]
setup(
name="Python Image Macro Project",
version=version,
description="Image Based macro project.",
options={"build_exe": build_options},
executables=executables,
)
Python 3.8 and later has problem with cx_freeze 6.1 - not copying python dll.
cx_freeze 6.2 is strongly recommended if that's the case.
You'll have to clone cx_freeze and build it, then install it to use 6.2.

Why current working directory affects install path of setup.py? How to prevent that?

I have created a custom python package following this guide, so I have the following structure:
mypackage/ <-- VCS root
mypackage/
submodule1/
submodule2/
setup.py
And setup.py contains exactly the same information as in the guide:
from setuptools import setup, find_packages
setup(name='mypackage',
version='0.1',
description='desc',
url='vcs_url',
author='Hodossy, Szabolcs',
author_email='myemail#example.com',
license='MIT',
packages=find_packages(),
install_requires=[
# deps
],
zip_safe=False)
I have noticed if I go into the folder where setup.py is, and then call python setup.py install in a virtual environment, in site-packages the following structure is installed:
.../site-packages/mypackage-0.1-py3.6.egg/mypackage/
submodule1/
submodule2/
but if I call it from one folder up like python mypackage/setup.py install, then the structure is the following:
.../site-packages/mypackage-0.1-py3.6.egg/mypackage/
mypackage/
submodule1/
submodule2/
This later one ruins all imports from my module, as the path is different for the submodules.
Could you explain what is happening here and how to prevent that kind of behaviour?
This is experienced with Python 3.6 on both Windows and Linux.

Your setup.py does not contain any paths, but seems to only find the files via find_packages. So of course it depends from where you run it. The setup.py isn't strictly tied to its location. Of course you could do things like chdir to the basename of the setup file path in sys.argv[0], but that's rather ugly.
The question is, WHY do you want to build it that way? It looks more like you would want a structure like
mypackage-source
mypackage
submodule1
submodule2
setup.py
And then execute setup.py from the work directory. If you want to be able to run it from anywhere, the better workaround would be to put a shellscript next to it, like
#!/bin/sh
cd ``basename $0``
python setup.py $#
which separates the task of changing to the right directory (here I assume the directory with setup.py in the workdir) from running setup.py

How to detect if module is installed in "editable mode"?

I'm pip-installing my module like so:
cd my_working_dir
pip install -e .
When I later import the module from within Python, can I somehow detect if the module is installed in this editable mode?
Right now, I'm just checking if there's a .git folder in os.path.dirname(mymodule.__file__)) which, well, only works if there's actually a .git folder there. Is there a more reliable way?

Another workaround:
Place an "not to install" file into your package. This can be a README.md, or a not_to_install.txt file. Use any non-pythonic extension, to prevent that file installation. Then check if that file exists in your package.
The suggested source structure:
my_repo
|-- setup.py
`-- awesome_package
|-- __init__.py
|-- not_to_install.txt
`-- awesome_module.py
setup.py:
# setup.py
from setuptools import setup, find_packages
setup(
name='awesome_package',
version='1.0.0',
# find_packages() will ignore non-python files.
packages=find_packages(),
)
The __init__.py or the awesome_module.py:
import os
# The current directory
__here__ = os.path.dirname(os.path.realpath(__file__))
# Place the following function into __init__.py or into awesome_module.py
def check_editable_installation():
'''
Returns true if the package was installed with the editable flag.
'''
not_to_install_exists = os.path.isfile(os.path.join(__here__, 'not_to_install.txt'))
return not_to_install_exists

Run pip list, and there is a column called "Editable project location". If that column has a value, specifically the directory from which you installed it, then the package is pip installed in editable mode.

I don't know of a way to detect this directly (e.g. ask setuptools).
You could try to detect that you package can not be reached through the paths in sys.path. But that's tedious. It's also not bullet proof -- what if it can be reached through sys.path but it's also installed as en editable package?
The best option is to look at the artifacts an editable install leaves in your site-packages folder. There's a file called my_package.egg-link there.
from pathlib import Path
# get site packages folder through some other magic
# assuming this current file is located in the root of your package
current_package_root = str(Path(__file__).parent.parent)
installed_as_editable = False
egg_link_file = Path(site_packages_folder) / "my_package.egg-link"
try:
linked_folder = egg_link_file.read_text()
installed_as_editable = current_package_root in linked_folder
except FileNotFoundError:
installed_as_editable = False
Note: to make this a bit more bullet-proof, read only the first line of the egg-link file and parse it using Path() as well to account for proper slashes etc.

Recently I had to test if various packages were installed in editable mode across different machines. Running pip show <package name> reveals not only the version, but other information about it, which includes the location of the source code. If the package was not installed in editable mode, this location will point to site-packages, so for my case it was sufficient with checking the output of such command:
import subprocess
def check_if_editable(name_of_the_package:str) -> bool:
out = subprocess.check_output(["pip", "show", f"{name_of_the_package}"]).decode()
return "site-packages" in out

Haven't found a definite source for this, but if the output of
$ pip show my-package -f
is something like this:
..
..
Files:
__editable__.my-package-0.0.5.pth
my-package-0.0.5.dist-info/INSTALLER
my-package-0.0.5.dist-info/METADATA
my-package-0.0.5.dist-info/RECORD
my-package-0.0.5.dist-info/REQUESTED
my-package-0.0.5.dist-info/WHEEL
my-package-0.0.5.dist-info/direct_url.json
my-package-0.0.5.dist-info/top_level.txt
then it's probably editable.

How to run a python script installed from distutils

I have a medium sized python command line program that runns well from my source code, and I've created a source distribution file and installed it into the virtual environment using "python setup.py install"
Since this is a pure Python program, and provided that the end users have installed Python, and the required packages, my idea is that i can distribute it through PyPi for all available platforms as a source distribution.
Upon install, I get an 'appname' directory within the virtualenv site-packages directory, and it also runs correctly when I write "python 'pathtovirtualenv'/Lib/sitepackages/'myappname'
But is this the way the end user is supposed to run distutils-distributed programs from the command line.
I fnd a lot of information on how to distribute a program using distutils, but not on how the end user is supposed to launch it after installing it.

Since you already created a setup.py, I would recommend looking at the entry_points:
entry_points={
'console_scripts': [
'scriptname=yourpackage.module:function',
],
},
Here, you have a package named yourpackage, and a module named module in it, and you refer to the function function. This function will wrapped by the script called scriptname, which will be installed in the users bin folder, which is normally in the $PATH, so the user can simply type scriptname after he installed your package via pip install.
To sum up: a user will install the package via pip install yourpackage and finally be able to call the function in module via script name.
Here are some docs on this topic:
https://pythonhosted.org/setuptools/setuptools.html#automatic-script-creation
http://www.scotttorborg.com/python-packaging/command-line-scripts.html

Well, I eventually figured it out.
Initially, I wanted to just use distutils, I like it when the end user can install it with minimum of extra dependencies. But I have now discovered that setuptools is the better option in my case.
My directory structure looks like this (Subversion):
trunk
|-- appname
| |-- __init__.py # an empty file
| |-- __main__.py # calls appname.main()
| |-- appname.py # contains a main() and imports moduleN
| |-- module1.py
| |-- module2.py
| |-- ...
|-- docs
| |-- README
| |-- LICENSE
| |-- ...
|-- setup.py
And my setyp.py basically looks like this:
# This setup file is to be used with setuptools source distribution
# Run "python setup sdist to deploy
from setuptools import setup, find_packages
setup( name = "appname",
...
include_package_data = True,
packages = find_packages(),
zip_safe = True,
entry_points = {
'console_scripts' : 'appname=appname.appname:main'
}
)
The next step now is to figure out how to install the contents of the docs directory on the users computer.
But right now, I'm thinking about adding --readme, --license, --changes, --sample (and so forth) options to the main script, to display them at run time.

pip installing data files to the wrong place

The source for the package is here
I'm installing the package from the index via:
easy_install hackertray
pip install hackertray
easy_install installs images/hacker-tray.png to the following folder:
/usr/local/lib/python2.7/dist-packages/hackertray-1.8-py2.7.egg/images/
While, pip installs it to:
/usr/local/images/
My setup.py is as follows:
from setuptools import setup
setup(name='hackertray',
version='1.8',
description='Hacker News app that sits in your System Tray',
packages=['hackertray'],
data_files=[('images', ['images/hacker-tray.png'])])
My MANIFEST file is:
include images/hacker-tray.png

Don't use data_files with relative paths. Actually, don't use data_files at all, unless you make sure the target paths are absolute ones properly generated in a cross-platform way insted of hard coded values.
Use package_data instead:
setup(
# (...)
package_data={
"hackertray.data": [
"hacker-tray.png",
],
},
)
where hackertray.data is a proper python package (i.e. is a directory that contains a file named __init__.py) and hacker-tray.png is right next to __init__.py.
Here's how it should look:
.
|-- hackertray
| |-- __init__.py
| `-- data
| |-- __init__.py
| `-- hacker-tray.png
`-- setup.py
You can get the full path to the image file using:
from pkg_resources import resource_filename
print os.path.abspath(resource_filename('hackertray.data', 'hacker-tray.png'))
I hope that helps.
PS: Python<2.7 seems to have a bug regarding packaging of the files listed in package_data. Always make sure to have a manifest file if you're using something older than Python 2.7 for packaging. See here for more info: https://groups.google.com/d/msg/python-virtualenv/v5KJ78LP9Mo/OiBqMcYVFYAJ