We Keep Coding

Can't use .so on Mac (has bad ELF magic)

I'm using buildozer to convert a python program to a phone app on Mac connected to an Android phone with the command line:
buildozer android debug deploy run
The previous command line runs the converted app on the connected phone.But the app crashes as soon as playsound is used. As for the methods before playsound work just fine.
When I run:
adb logcat | grep python
I get the error:
ImportError: dlopen failed: "/data/data/org.test.myapp/files/app/_python_bundle/site-packages/gi/_gi.so" has bad ELF magic
When I looked it up I found that Mac cannot use .so files.
Does anybody know how I can solve this?

Okay, so I ended up fixing this when I got a similar error. (dawg.so has bad ELF magic)
Basically, the reason I got this error was because the library ("gi" in your case) was not been read properly by the android phone when deployed and hence, was "corrupted".
The bottomline reason (for me) was that it was a C/C++ library under the hood and used Cython to be converted to a Python library. Hence, this error usually means that your library needs a custom recipe.
Steps to solve it:
In the root directory (where .buildozer folder is found), I added a folder named dawg (the library name), and then, inside dawg, I git cloned the source files of dawg. To get the source files, you can just go to the PyPi page for that library and go to the Project links -> Homepage of their GitHub site. Once cloned, you can also remove .git, .gitignore, from the source files.
Once that's done, run python3 setup.py install in the dawg directory to install and hence, "cythonize" the source files
In .buildozer/android/platform/python-for android/pythonforandroid/recipes, add a new folder named "dawg" (your library name) and then, inside /dawg, make _init_.py where you will add your custom recipe.
In _init_.py, you can add your recipe and the path to the source files. Here is a template that worked for me, but you can customize it for you as per your requirements.
from pythonforandroid.recipe import IncludedFilesBehaviour, CppCompiledComponentsPythonRecipe
import os
import sys
class DAWGRecipe(IncludedFilesBehaviour, CppCompiledComponentsPythonRecipe):
version = '0.8.0'
src_filename = "../../../../../../../dawg"
name = 'dawg'
# Libraries it depends on
depends = ['setuptools']
call_hostpython_via_targetpython = False
install_in_hostpython = True
def get_recipe_env(self, arch):
env = super().get_recipe_env(arch)
env['LDFLAGS'] += ' -lc++_shared'
return env
recipe = DAWGRecipe()
Don't forget to alter the buildozer.spec. To its p4a.local_recipes, add the local path to /.buildozer/android/platform/python-for-android/pythonforandroid/recipes as that's where we add our recipes.
Clean the previous build by running buildozer android clean
Lastly, run buildozer -v android debug deploy run to build the app on android phone again.
Hope this helps :)

Python poetry - how to install optional dependencies?

Python's poetry dependency manager allows specifying optional dependencies via command:
$ poetry add --optional redis
Which results in this configuration:
[tool.poetry.dependencies]
python = "^3.8"
redis = {version="^3.4.1", optional=true}
However how do you actually install them? Docs seem to hint to:
$ poetry install -E redis
but that just throws and error:
Installing dependencies from lock file
[ValueError]
Extra [redis] is not specified.

You need to add a tool.poetry.extras group to your pyproject.toml if you want to use the -E flag during install, as described in this section of the docs:
[tool.poetry.extras]
caching = ["redis"]
The key refers to the word that you use with poetry install -E, and the value is a list of packages that were marked as --optional when they were added. There currently is no support for making optional packages part of a specific group during their addition, so you have to maintain this section in your pyproject.toml file by hand.
The reason behind this additional layer of abstraction is that extra-installs usually refer to some optional functionality (in this case caching) that is enabled through the installation of one or more dependencies (in this case just redis). poetry simply mimics setuptools' definition of extra-installs here, which might explain why it's so sparingly documented.

I will add that not only you have to have this extras section added by hand, as well your optional dependencies cannot be in dev section.
Example of code that won't work:
[tool.poetry]
name = "yolo"
version = "1.0.0"
description = ""
authors = []
[tool.poetry.dependencies]
python = "2.7"
Django = "*"
[tool.poetry.dev-dependencies]
pytest = "*"
ipdb = {version = "*", optional = true}
[tool.poetry.extras]
dev_tools = ["ipdb"]
But this WILL work:
[tool.poetry]
name = "yolo"
version = "1.0.0"
description = ""
authors = []
[tool.poetry.dependencies]
python = "2.7"
Django = "*"
ipdb = {version = "*", optional = true}
[tool.poetry.dev-dependencies]
pytest = "*"
[tool.poetry.extras]
dev_tools = ["ipdb"]

Up-voted Drachenfels's answer.
Dev dependency could not be optional, otherwise, no matter how you tweak it with extras or retry with poetry install -E, it will just never get installed.
This sounds like a bug but somehow by design,
...this is not something I want to add. Extras will be referenced in the distributions metadata when packaging the project but development dependencies do not which will lead to a broken extras.
— concluded in Poetry PR#606 comment by one maintainer. See here for detailed context: https://github.com/python-poetry/poetry/pull/606#issuecomment-437943927
I would say that I can accept the fact that optional dev-dependency cannot be implemented. However, at least Poetry should warn me when I have such a config. If so, I wouldn't have been confused for a long time, reading each corner of the help manual and found nothing helpful.
I found some people did get trap in this problem (Is poetry ignoring extras or pyproject.toml is misconfigured?) but their questions are closed, marked duplicated and re-linked to this question. Thus I decided to answer here and give more details about this problem.

This is now possible (with Poetry version 1.2; perhaps even an earlier version), using the "extras" group:
poetry add redis --group=extras
It will appear in the section
[tool.poetry.group.extras.dependencies]
which is also newer style (compared to [tool.poetry.extras] or [tool.poetry.extras.dependencies]
See the documentation. Interestingely, this still follows the older style, [tool.poetry.extras], and doesn't show the use of poetry add, but the above result is what I get.

What am I doing wrong pypi missing "Links for"

I'm trying to tryout pypi to publish some libraries. So I started with a simple project.
I have the following setup.py:
import os
from distutils.core import setup
setup(
name='event_amd',
packages = ["event_amd"],
description='Port for EventEmitter from nodejs',
version='1.0.7',
author="Borrey Kim",
author_email="borrey#gmail.com",
url="https://bitbucket.org/borreykim/event_amd",
download_url="https://bitbucket.org/borreykim/event_amd/downloads/event_amd-1.0.6.tar.gz",
keywords=['events'],
long_description = """\
This is an initial step to port over EventEmitter of nodejs. This is done with the goal of having libraries that are cross platform so that cross communication is easier, and collected together.
"""
)
I've registered it but: sudo pip install event_amd gives me an error:
DistributionNotFound: No distributions at all found for event-amd
(I'm not sure how event_amd turns to event-amd?)
Also there is no links under (which other projects seem to have ):
https://pypi.python.org/simple/event_amd/
I was wondering if I am doing something wrong in the setup.py or what may be causing this.
Thanks in advance.

You need to upload a source archive after registering the release: python setup.py register sdist upload

How would you install a python module with chef?

We're using EngineYard which has Python installed by default. But when we enabled SSL we received the following error message from our logentries chef recipe.
"WARNING: The "ssl" module is not present. Using unreliable workaround, host identity cannot be verified. Please install "ssl" module or newer version of Python (2.6) if possible."
I'm looking for a way to install the SSL module with chef recipe but I simply don't have enough experience. Could someone point me in the right direction?
Resources:
Logentries chef recipe: https://github.com/logentries/le_chef
Logentries EY docs: https://logentries.com/doc/engineyard/
SSL Module: http://pypi.python.org/pypi/ssl/

There now appears to be a solution with better community support (based on the fact that it is documented on the opscode website).
You might try:
include_recipe 'python'
python_pip 'ssl'
As documented: here or here

I just wrote a recipe for this, and now am able to run the latest Logentries client on EngineYard. Here you go:
file_dir = "/mnt/src/python-ssl"
file_name = "ssl-1.15.tar.gz"
file_path = File.join(file_dir,file_name)
uncompressed_file_dir = File.join(file_dir, file_name.split(".tar.gz").first)
directory file_dir do
owner "deploy"
group "deploy"
mode "0755"
recursive true
action :create
end
remote_file file_path do
source "http://pypi.python.org/packages/source/s/ssl/ssl-1.15.tar.gz"
mode "0644"
not_if { File.exists?(file_path) }
end
execute "gunzip ssl" do
command "gunzip -c #{file_name} | tar xf -"
cwd file_dir
not_if { File.exists?(uncompressed_file_dir) }
end
installed_file_path = File.join(uncompressed_file_dir, "installed")
execute "install python ssl module" do
command "python setup.py install"
cwd uncompressed_file_dir
not_if { File.exists?(installed_file_path) }
end
execute "touch #{installed_file_path}" do
action :run
end

You could install a new Python using PythonBrew: https://github.com/utahta/pythonbrew. Just make you install libssl before you build, or it still won't be able to use SSL. However, based on the warning, it seems that SSL might work, but it won't be able to verify host. Of course, that is one major purposes of SSL, so that is likely a non-starter.
HTH

python path problem: ImportError when calling zodbconvert (FreeBSD 8.1)

I guess this is a python path problem (on FreeBSD 8.1).
Im trying to convert a Data.fs to Postgresql using zodbconvert. Ive downloaded RelStorage-1.5.0b2 and is running:
/usr/local/Plone/Python-2.6/bin/python zodbconvert.py fstodb.conf
, to use the version that Plone is running with.
The error I get:
Traceback (most recent call last):
File "zodbconvert.py", line 22, in <module>
from persistent.TimeStamp import TimeStamp
ImportError: No module named persistent.TimeStamp
Versions:
Plone 4.0.5 (python 2.6)
Postgresql 9.0.3
FreeBSD 8.1
python26-2.6.6_1
python27-2.7.1_1
PS by default "python --version" is 2.7.1
Thanks.
Nikolaj G.

If you are using buildout (I do hope you are) the easiest way to get all the zodbconvert dependencies properly included in the python path is to have buildout create the script for you:
[buildout]
...
parts =
...
zodbconvert
[zodbconvert]
recipe = zc.recipe.egg
eggs = ${buildout:eggs}
scripts = zodbconvert
Buildout then will create a new bin/zodbconvert script for you that includes all the buildout eggs in sys.path.
Alternatively, you can create a generic python script runner that includes all eggs in your buildout and can run arbitrary scripts; you can use this instead of the bare-bones python interpreter to run arbitrary python scripts with all the buildout eggs in sys.path:
[buildout]
...
parts =
...
zopepy
[zopepy]
recipe = zc.recipe.egg
eggs = ${buildout:eggs}
interpreter = zopepy
scripts = zopepy
The bin/zopepy script can then be use to run arbitrary python scripts with all your buildout eggs already in sys.path, so bin/zopepy zodbconvert.py fstodb.conf should work.
Note that the Plone unified installer already comes with the zopepy part included, and my choice of partname for this script was deliberately using the same name.
If you are not using buildout (and with Plone 4, that's not a good idea on the whole), you can also list the required packages (ZODB3, zope.interface, RelStorage, psycopg2) in your PYTHONPATH environment variable.

quick fix..
locate persistent
export PYTHONPATH=$PYTHONPATH:/path/to/your/python_persistent_dir

You have not included the ZODB package with your Python installation. Either adjust the PYTHONPATH to include the ZODB package or just easy_install ZODB - depending on what you are trying to do.