How can you install pygame to your yocto project? Can only seem to find old solutions that doesn't seem to work. I mange to install python3 without any issues, but the meta-python layer doesn't contain pygame as standard. How can you create your own layer/recipe to get pygame?
Tried to create my own recipe from the script here
but when I use bitbake add-layer <PATH>
I get an error saying that:
Specified layer directory <PATH> doesn't contain a conf/layer.conf file
SUMMARY = "Python Game Development"
HOMEPAGE = "https://www.pygame.org"
AUTHOR = "A community project. <pygame#pygame.org>"
LICENSE = "LGPL-2.0"
LIC_FILES_CHKSUM = "file://setup.py;md5=59f9a291287e87473bbb98f2c9291c5c"
SRC_URI = "https://files.pythonhosted.org/packages/9a/50/67767e5586a45e7e7b02e6f0e07853f8fcb81b54c66db6278f1a1344491f/pygame-2.1.2.tar.gz"
SRC_URI[md5sum] = "8467a125e9075fa701f082806d228366"
SRC_URI[sha256sum] = "d6d0eca28f886f0477cd0721ac688189155a587f2bb8eae740e52ca56c3ad23c"
S = "${WORKDIR}/pygame-2.1.2"
RDEPENDS_${PN} = ""
inherit setuptools3
How my other files look.
local.conf:
MACHINE ??= "raspberrypi4-64"
RASPBERRYPI_DISPLAY = " 1"
IMAGE_INSTALL:append = " python3"
IMAGE_FSTYPES = "wic wic.bmap"
bblayers.conf:
<PATH>/poky/meta \
<PATH>/poky/meta-poky \
<PATH>/poky/meta-yocto-bsp \
<PATH>/meta-openembedded/meta-oe \
<PATH>/meta-openembedded/meta-python \
<PATH>/meta-raspberrypi \
Have you managed to add pygame to Yocto? I'm trying to achieve the same thing.
I've also tried generating a recipe with pipoe. (Without creating a custom layer. I've added the recipe to ../sources/poky/meta/recipes-devtools/python.) The recipe is parsed successfully, however the compilation fails.
This is my python3-pygame_2.1.3.dev8.bb:
SUMMARY = "Python Game Development"
HOMEPAGE = "https://www.pygame.org"
AUTHOR = "A community project. <pygame#pygame.org>"
LICENSE = "LGPL-2.0"
LIC_FILES_CHKSUM = "file://setup.py;md5=f28719c8b1d7e19071bd25e2f3a54c06"
SRC_URI = "https://files.pythonhosted.org/packages/f0/34/fc13b596b637cc59961ed606447a27a96d31bfda3ca965b73ebd16a02750/pygame-2.1.3.dev8.tar.gz"
SRC_URI[md5sum] = "6cf32b59a453b9824f421065a0205b52"
SRC_URI[sha256sum] = "94d98328e73d56be35cc3ae8e31f9d95fbce4f3dca6a74f05c58f4c5ef18c33d"
S = "${WORKDIR}/pygame-2.1.3.dev8"
RDEPENDS_${PN} = "libsdl2"
inherit setuptools3
The solution that worked for me:
Add meta-python and meta-netoworking via meta-openembedded.
In local.conf add IMAGE_INSTALL:append = " python3-pip".
While up and running use the command pip3 install pygame
I have created the SConstruct to install the systemd user services but when I try to scons uninstall the temporary services files are creates which should not happen.
import os
PATH_WD = os.path.abspath(os.curdir)
env = Environment(
SUBSTFILESUFFIX = '.service',
SUBST_DICT = { '{{PATH_ROOT}}' : os.path.dirname(PATH_WD) },
ENV = {
'DBUS_SESSION_BUS_ADDRESS' : os.environ['DBUS_SESSION_BUS_ADDRESS'],
'XDG_RUNTIME_DIR' : os.environ['XDG_RUNTIME_DIR']
}
)
INSTALLED = env.Install(
target = os.path.expanduser('~/.config/systemd/user/'),
source = [
env.Substfile('service1'),
env.Substfile('service2'),
]
)
env.AddPostAction(INSTALLED, env.Action('systemctl --user daemon-reload'))
Alias('install', INSTALLED)
NoClean(INSTALLED)
Command('uninstall', INSTALLED, Delete(INSTALLED))
Default('install')
Second try..
Here's a trivial example which should work for you..
env=Environment()
prog=env.Program('main.c')
Default(prog)
installed_prog = env.Install('install_dir', prog)
Alias('install', installed_prog)
NoClean(installed_prog)
# You don't have to specify targets to Alias.. so it won't
# try to build those before executing the Action
Alias('uninstall', action=Delete(installed_prog))
AlwaysBuild('uninstall')
SCons builder calls are statements of relationships between nodes. You've associated the target "uninstall" with the source INSTALLED by calling the Command builder. So in order to "build" this target, you need the source, and the source is the list of nodes returned by calling the Install builder. So the Install has to happen before the uninstall can take place. Is there a reason you don't want SCons' clean functionality to be used here? To see this, try: scons --tree=all,linedraw -n uninstall
I want to encode the git hash of the lastest commit at the time of installing as the output of a function in my library.
I am using a structure
.git/
setup.py
|- mylib/
| - __init__.py
| - git.py
whereby git.py contains a dummy, unconfigured, function
def git():
return None
At the time of installing I want to overwrite git.py such that it returns the hash of the latest git commit, e.g.:
def git():
return "2d04d1a10f81d24183df7622c95398d60106dfff"
(whereby the hash is the output of git rev-parse HEAD at the time of installing).
So my question:
How can I overwrite the to-be-installed git.py, using setuptools, without overwriting the file in the source directory?
Current work-around setup.py
import subprocess
git_hash = subprocess.check_output(["git", "rev-parse", "HEAD"]).strip().decode('UTF-8')
cmd = '''def git():
return "{0:s}"
'''.format(git_hash)
os.rename('mylib/git.py', 'mylib/git.py.bak')
with open('mylib/git.py', 'w') as f:
f.write(cmd)
setup(
name = 'mylib',
version = __version__,
# ...
packages = find_packages(),
)
os.rename('mylib/git.py.bak', 'mylib/git.py')
Which has the down-sight of temporarily changing git.py in the source directory. I'd rather leave that file completely untouched.
With the help of #webknjaz on setuptools' git repository (see issue) I found a solution using setuptools_scm that works:
File structure
setup.py
|- mylib/
| - __init__.py
| - _version.py
setup.py
setup(
name = 'mylib',
packages = find_packages(),
use_scm_version = {'write_to': 'mylib/_version.py'},
setup_requires = ['setuptools_scm'],
)
init.py
from ._version import *
_version.py (dummy, overwritten at install-time)
version = "None"
version_tuple = (0, 0, 0, "None", "None")
My Python application uses Dropbox and I need to include this library in my Yocto image. I managed to generate a recipe for it with pipoe (pipoe -p dropbox). Here it is:
SUMMARY = "Official Dropbox API Client"
HOMEPAGE = "http://www.dropbox.com/developers"
AUTHOR = "Dropbox <dev-platform#dropbox.com>"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://LICENSE;md5=ef3219362ea5e452a18031af12f35eb6"
SRC_URI = "https://files.pythonhosted.org/packages/85/33/bffd4a7596f3280f8bd2796b4f09c5c701b99d5c8e68715351cc2b3eeca8/dropbox-10.3.0.tar.gz"
SRC_URI[md5sum] = "f6d1a30af7e202237d660213d22e48a7"
SRC_URI[sha256sum] = "5f296f13ee7c358ab41779a73c4922ed81199447566c1a4c8fbf23dbcda25e20"
S = "${WORKDIR}/dropbox-10.3.0"
RDEPENDS_${PN} = "python-requests python-six "
inherit setuptools
When I run it, it says:
| ERROR: Do not try to fetch `pytest-runner' for building. Please add its native recipe to DEPENDS.
and then
| raise DistributionNotFound(req, requirers)
| pkg_resources.DistributionNotFound: The 'pytest-runner' distribution was not found and is required by the application
so I added the following line to the generated recipe:
DEPENDS = "python3-pytest-runner-native "
But I still get the same error. Any idea?
Probably an issue with inconsistent Python version.
You inherit setuptools which is for python2. The RDEPENDS are for python2 dependencies. But you add python3-pytest-runner-native in DEPENDS.
I'd suggest finding out which python version to use for this software and use the same for all RDEPENDS, inherited classes (setuptools3 for python3), and DEPENDS.
We use py2app extensively at our facility to produce self contained .app packages for easy internal deployment without dependency issues. Something I noticed recently, and have no idea how it began, is that when building an .app, py2app started including the .git directory of our main library.
commonLib, for instance, is our root python library package, which is a git repo. Under this package are the various subpackages such as database, utility, etc.
commonLib/
|- .git/ # because commonLib is a git repo
|- __init__.py
|- database/
|- __init__.py
|- utility/
|- __init__.py
# ... etc
In a given project, say Foo, we will do imports like from commonLib import xyz to use our common packages. Building via py2app looks something like: python setup.py py2app
So the recent issue I am seeing is that when building an app for project Foo, I will see it include everything in commonLib/.git/ into the app, which is extra bloat. py2app has an excludes option but that only seems to be for python modules. I cant quite figure out what it would take to exclude the .git subdir, or in fact, what is causing it to be included in the first place.
Has anyone experienced this when using a python package import that is a git repo?
Nothing has changed in our setup.py files for each project, and commonLib has always been a git repo. So the only thing I can think of being a variable is the version of py2app and its deps which have obviously been upgraded over time.
Edit
I'm using the latest py2app 0.6.4 as of right now. Also, my setup.py was first generated from py2applet a while back, but has been hand configured since and copied over as a template for every new project. I am using PyQt4/sip for every single one of these projects, so it also makes me wonder if its an issue with one of the recipes?
Update
From the first answer, I tried to fix this using various combinations of exclude_package_data settings. Nothing seems to force the .git directory to become excluded. Here is a sample of what my setup.py files generally look like:
from setuptools import setup
from myApp import VERSION
appname = 'MyApp'
APP = ['myApp.py']
DATA_FILES = []
OPTIONS = {
'includes': 'atexit, sip, PyQt4.QtCore, PyQt4.QtGui',
'strip': True,
'iconfile':'ui/myApp.icns',
'resources':['src/myApp.png'],
'plist':{
'CFBundleIconFile':'ui/myApp.icns',
'CFBundleIdentifier':'com.company.myApp',
'CFBundleGetInfoString': appname,
'CFBundleVersion' : VERSION,
'CFBundleShortVersionString' : VERSION
}
}
setup(
app=APP,
data_files=DATA_FILES,
options={'py2app': OPTIONS},
setup_requires=['py2app'],
)
I have tried things like:
setup(
...
exclude_package_data = { 'commonLib': ['.git'] },
#exclude_package_data = { '': ['.git'] },
#exclude_package_data = { 'commonLib/.git/': ['*'] },
#exclude_package_data = { '.git': ['*'] },
...
)
Update #2
I have posted my own answer which does a monkeypatch on distutils. Its ugly and not preferred, but until someone can offer me a better solution, I guess this is what I have.
I am adding an answer to my own question, to document the only thing I have found to work thus far. My approach was to monkeypatch distutils to ignore certain patterns when creating a directory or copying a file. This is really not what I wanted to do, but like I said, its the only thing that works so far.
## setup.py ##
import re
# file_util has to come first because dir_util uses it
from distutils import file_util, dir_util
def wrapper(fn):
def wrapped(src, *args, **kwargs):
if not re.search(r'/\.git/?', src):
fn(src, *args, **kwargs)
return wrapped
file_util.copy_file = wrapper(file_util.copy_file)
dir_util.mkpath = wrapper(dir_util.mkpath)
# now import setuptools so it uses the monkeypatched methods
from setuptools import setup
Hopefully someone will comment on this and tell me a higher level approach to avoid doing this. But as of now, I will probably wrap this into a utility method like exclude_data_patterns(re_pattern) to be reused in my projects.
I can see two options for excluding the .git directory.
Build the application from a 'clean' checkout of the code. When deploying a new version, we always build from a fresh svn export based on a tag to ensure we don't pick up spurious changes/files. You could try the equivalent here - although the git equivalent seems somewhat more involved.
Modify the setup.py file to massage the files included in the application. This might be done using the exclude_package_data functionality as described in the docs, or build the list of data_files and pass it to setup.
As for why it has suddenly started happening, knowing the version of py2app you are using might help, as will knowing the contents of your setup.py and perhaps how this was made (by hand or using py2applet).
I have a similar experience with Pyinstaller, so I'm not sure it applies directly.
Pyinstaller creates a "manifest" of all files to be included in the distribution, before running the export process. You could "massage" this manifest, as per Mark's second suggestion, to exclude any files you want. Including anything within .git or .git itself.
In the end, I stuck with checking out my code before producing a binary as there was more than just .git being bloat (such as UML documents and raw resource files for Qt). A checkout guaranteed a clean result and I experienced no issues automating that process along with the process of creating the installer for the binary.
There is a good answer to this, but I have a more elaborate answer to solve the problem mentioned here with a white-list approach. To have the monkey patch also work for packages outside site-packages.zip I had to monkey patch also copy_tree (because it imports copy_file inside its function), this helps in making a standalone application.
In addition, I create a white-list recipe to mark certain packages zip-unsafe. The approach makes it easy to add filters other than white-list.
import pkgutil
from os.path import join, dirname, realpath
from distutils import log
# file_util has to come first because dir_util uses it
from distutils import file_util, dir_util
# noinspection PyUnresolvedReferences
from py2app import util
def keep_only_filter(base_mod, sub_mods):
prefix = join(realpath(dirname(base_mod.filename)), '')
all_prefix = [join(prefix, sm) for sm in sub_mods]
log.info("Set filter for prefix %s" % prefix)
def wrapped(mod):
name = getattr(mod, 'filename', None)
if name is None:
# ignore anything that does not have file name
return True
name = join(realpath(dirname(name)), '')
if not name.startswith(prefix):
# ignore those that are not in this prefix
return True
for p in all_prefix:
if name.startswith(p):
return True
# log.info('ignoring %s' % name)
return False
return wrapped
# define all the filters we need
all_filts = {
'mypackage': (keep_only_filter, [
'subpackage1', 'subpackage2',
]),
}
def keep_only_wrapper(fn, is_dir=False):
filts = [(f, k[1]) for (f, k) in all_filts.iteritems()
if k[0] == keep_only_filter]
prefixes = {}
for f, sms in filts:
pkg = pkgutil.get_loader(f)
assert pkg, '{f} package not found'.format(f=f)
p = join(pkg.filename, '')
sp = [join(p, sm, '') for sm in sms]
prefixes[p] = sp
def wrapped(src, *args, **kwargs):
name = src
if not is_dir:
name = dirname(src)
name = join(realpath(name), '')
keep = True
for prefix, sub_prefixes in prefixes.iteritems():
if name == prefix:
# let the root pass
continue
# if it is a package we have a filter for
if name.startswith(prefix):
keep = False
for sub_prefix in sub_prefixes:
if name.startswith(sub_prefix):
keep = True
break
if keep:
return fn(src, *args, **kwargs)
return []
return wrapped
file_util.copy_file = keep_only_wrapper(file_util.copy_file)
dir_util.mkpath = keep_only_wrapper(dir_util.mkpath, is_dir=True)
util.copy_tree = keep_only_wrapper(util.copy_tree, is_dir=True)
class ZipUnsafe(object):
def __init__(self, _module, _filt):
self.module = _module
self.filt = _filt
def check(self, dist, mf):
m = mf.findNode(self.module)
if m is None:
return None
# Do not put this package in site-packages.zip
if self.filt:
return dict(
packages=[self.module],
filters=[self.filt[0](m, self.filt[1])],
)
return dict(
packages=[self.module]
)
# Any package that is zip-unsafe (uses __file__ ,... ) should be added here
# noinspection PyUnresolvedReferences
import py2app.recipes
for module in [
'sklearn', 'mypackage',
]:
filt = all_filts.get(module)
setattr(py2app.recipes, module, ZipUnsafe(module, filt))