We Keep Coding

How to use Python Dbus bindings in Anaconda

I am trying to install dbus on Anaconda python environment and I am struggling.
Here is the error message I am getting:
e#gateway:~$ python
Python 3.5.4 |Anaconda custom (64-bit)| (default, Oct 13 2017, 11:22:58)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import dbus
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/e/anaconda3/lib/python3.5/site-packages/dbus/__init__.py", line 77, in <module>
import dbus.types as types
File "/home/e/anaconda3/lib/python3.5/site-packages/dbus/types.py", line 6, in <module>
from _dbus_bindings import (
ImportError: /home/e/anaconda3/lib/python3.5/site-packages/_dbus_bindings.so: undefined symbol: _Py_ZeroStruct
>>>
Here are some of the outputs I think may be asked:
e#gateway:~$ conda install dbus
Fetching package metadata ...........
Solving package specifications: .
# All requested packages already installed.
# packages in environment at /home/e/anaconda3:
#
dbus 1.10.22 h3b5a359_0
e#gateway:~$ sudo apt-get install libdbus-glib-1-dev libdbus-1-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
libdbus-glib-1-dev is already the newest version (0.106-1).
libdbus-1-dev is already the newest version (1.10.6-1ubuntu3.3).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
e#gateway:~$ sudo apt-get install dbus
Reading package lists... Done
Building dependency tree
Reading state information... Done
dbus is already the newest version (1.10.6-1ubuntu3.3).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
e#gateway:~$ which python
/home/e/anaconda3/bin/python
e#gateway:~$ conda --version
conda 4.3.31
e#gateway:~$ sudo /home/e/anaconda3/bin/python -m pip install dbus-python
The directory '/home/e/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/e/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied: dbus-python in ./anaconda3/lib/python3.5/site-packages
DBus is working fine on the system python, however not working on Anaconda Python.
Python 2.7:
e#gateway:~$ which python
/usr/bin/python
e#gateway:~$ python
Python 2.7.12 (default, Nov 20 2017, 18:23:56)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import dbus
>>>
Python 3.5:
e#gateway:~$ which python3
/usr/bin/python3
e#gateway:~$ python3
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import dbus
>>>
Can anyone help me? Am I missing something blatantly obvious here?
Thanks in advance.

I had similar issues, there are few cases where dbus and python don't work well out-of-the-box. The consensus appears to be that you need a system-level install (i.e. apt-get) to get dbus to work. I believe the /home/e/anaconda3/lib/python3.5/site-packages/_dbus_bindings.so: undefined symbol: _Py_ZeroStruct error you're seeing is directly related to that.
conda install dbus does not add anything to ~/anaconda3/lib/python3.6/site-packages, but instead appears to install some executables in ~/anaconda3/bin/ like dbus-run-session, dbus-daemon, etc. This makes some sense when you analyze the contents of the dbus tarball https://anaconda.org/conda-forge/dbus, as it's all C files and executables. I'm not sure it's supposed to be the dbus python module, but I could be wrong.
EDIT:
I searched the conda repositories and found a few individuals that uploaded a version of dbus-python, presumably that they compiled and installed. I tried this one out in a py3.6 conda environment via:
conda install -c scottwales dbus-python
I was then able to import dbus. This is a hacky approach and should not be used in production, I'd recommend listening to
Carlos Cordoba's post below. But if you need a solution now, search through some user conda packages or try to compile the library yourself.

Can anyone help me? Am I missing something blatantly obvious here?
Yes, you are. There's one thing people still don't understand about conda: conda is not a pip replacement. It is a general package manager, in the same vein as apt-get, yum, brew, emerge, etc, but cross-platform and based on Python.
In this case, that means that conda install dbus does not install the Python Dbus bindings, as you would expect with pip . It installs the Dbus C package itself, which is needed by Qt 5 (again, the C++ library, not the Python bindings to it).
Unfortunately, there are no Conda packages for dbus-python. To make matters worse, it seems there's no easy way to create packages for it, as pointed out here.
Finally, you said
Here is the error message I am getting
The (most probable) cause of that error is because you added your system Python dist-packages path to the PYTHONPATH of Anaconda or because you blindly copied the dbus module from system Python to Anaconda. Please don't do that ever again. System Python and Anaconda packages are compiled with different compilers and under different conditions. So mixing them is the cause of incomprehensible errors, just like the one you reported.

After installation of vtk with brew (macosx): "import vtk" aborts python console

I tried to install vtk with homebrew but, now when I import vtk on Python, my python session is aborted...
I work on MacOSX and I ran this line:
brew install vtk --with-qt --with-python --with-pyqt
It returns:
....
[ 29%] Built target vtkFiltersCore
make: *** [all] Error 2
READ THIS: https://git.io/brew-troubleshooting
If reporting this issue please do so at (not Homebrew/brew):
https://github.com/Homebrew/homebrew-science/issues
....
After that I read that I should use this line:
HOMEBREW_MAKE_JOBS=1 VERBOSE=1 brew install vtk --qt --python --pyqt
After a long time, it returns:
....
==> Caveats
Even without the --with-qt option, you can display native VTK render windows
from python. Alternatively, you can integrate the RenderWindowInteractor
in PyQt, PySide, Tk or Wx at runtime. Read more:
import vtk.qt4; help(vtk.qt4) or import vtk.wx; help(vtk.wx)
Python modules have been installed and Homebrew's site-packages is not
in your Python sys.path, so you will not be able to import the modules
this formula installed. If you plan to develop with these modules,
please run:
mkdir -p /Users/TheUser/Library/Python/2.7/lib/python/site-packages
echo 'import site; site.addsitedir("/usr/local/lib/python2.7/site-packages")' >> /Users/TheUser/Library/Python/2.7/lib/python/site-packages/homebrew.pth
==> Summary
🍺 /usr/local/Cellar/vtk/7.0.0_3: 3,203 files, 107.7M
So I ran the mkdir & echo lines
And when I tried import vtk in the Python console, it returns:
Fatal Python error: PyThreadState_Get: no current thread
Abort trap: 6
I retried to reinstall it later:
==> Installing vtk from homebrew/science
Error: vtk-7.0.0_3 already installed
To install this version, first `brew unlink vtk`
So I did the unlink, and I reran HOMEBREW_MAKE_JOBS=1 VERBOSE=1 brew install vtk --qt --python --pyqt .
The result:
==> Patching VTK to use system Python 2
==> Caveats
Even without the --with-qt option, you can display native VTK render windows from python. Alternatively, you can integrate the RenderWindowInteractor in PyQt, PySide, Tk or Wx at runtime. Read more:
import vtk.qt4; help(vtk.qt4) or import vtk.wx; help(vtk.wx)
VTK was linked against your system copy of Python.
If you later decide to change Python installations, relink VTK with:
brew postinstall vtk
==> Summary
🍺 /usr/local/Cellar/vtk/7.0.0_5: 3,204 files, 107.7M
And when I import vtk on the python console: abortion!
How can I fix that?

I am currently updating to vtk/7.0.0_5 (used 7.0.0_3 before), the line I used to install it was :
brew install vtk --with-qt --with-qt5 --with-python
And I did not get any issue using it in python (2.7).
Ok - So just finished to do the upgrade to 7.0.0_5 and I got the same message tan you. But no issue at all to launch python and launch :
> $ python
Python 2.7.12 (default, Oct 11 2016, 05:20:59)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import vtk
>>> import vtk.qt4
Hope that help,
Stephane

Some Python Versions not Recognizing Installed Packages

I have installed python on my macbook with the python installer from Python.org. Subsequently, I went and installed pip, ipython, and numpy. Everything seemed fine. However, now I am getting the following problem. I can import numpy when I run ipython, but not when I run regular python.
E.g.
Logister-MacBook-Pro:~ Logister$ ipython
Python 2.7.6 (default, Sep 9 2014, 15:04:36)
Type "copyright", "credits" or "license" for more information.
IPython 3.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: import numpy
In [2]: import site; site.getsitepackages()
Out[2]:
['/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python',
'/Library/Python/2.7/site-packages']
However, when I try to do the same thing in python 2.7.9:
Logister-MacBook-Pro:~ Logister$ python
Python 2.7.9 (v2.7.9:648dcafa7e5f, Dec 10 2014, 10:10:46)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named numpy
>>> import site; site.getsitepackages()
['/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages',
'/Library/Frameworks/Python.framework/Versions/2.7/lib/site-python',
'/Library/Python/2.7/site-packages']
When I try to install numpy via pip it gives me the following response:
Logister-MacBook-Pro:~ Logister$ sudo -H pip install numpy
Requirement already satisfied (use --upgrade to upgrade):
numpy in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
It seems like python 2.7.6 understands where to find numpy, but 2.7.9 does not. Either, how can I point 2.7.9 to the right place, or how can I install numpy so 2.7.9 sees it as well?
Edit: I can run:
site.addsitedir('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python')
In 2.7.9 and then I am able to import numpy. But I dont want to do this every time I launch 2.7.9. Is there a permanent fix? Also, how do I get ipython to run 2.7.9 instead of 2.7.6?

I had a similar problem.
There is two differences versions of python installed on your computer.
Then you´ve installed numpy on python 2.7.6. but the folder of python 2.7.9 haven't numpy. Do you understand?
Other thing your macOS have a groups of variables called "Environment Variables" there it's identifying the command python with python 2.7.9.
Then if you want to use numpy with the interpreter of python do you need change the Enviroment Variables for the command "python" take the version 2.7.6 or install numpy at python 2.7.9 and work with both versions.
Good luck!

With 2 different version you must have 2 python interpreters on your machine. They are installed on different locations with different libraries. So its just right that you cannot find the module installed on interpreter A while using the interpreter B. The which command can be useful to you to figure out where those interpreter are. You can make a symbolic link manually from a folder to another but it would mess up your mind, and you probably gonna get lost later.
I recommed you to install python and ipython via Homebrew, so it would automatically do all the hard work for you.
Install homebrew
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
Since you said numpy, you are probably looking for scientific stuff so:
# set up some taps and update brew
brew tap homebrew/science # a lot of cool formulae for scientific tools
brew tap homebrew/python # numpy, scipy, matplotlib, ...
brew update && brew upgrade
# install a brewed python
brew install python
Later you can run, but I recommend you to follow this tutorial:
brew install zmq
pip install ipython[all]

I was able to solve the problem by adding the following line to my .bash_profile:
export PYTHONPATH=${PYTHONPATH}:/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
I then changed the Ipython version by following this stackoverflow question.

Too many different Python versions on my system and causing problems

During the past years, I have installed many Python libraries with various Python versions. To make them ready to work immediately, I installed them blindly without control. Currently they're causing problems when I tried to install pynest which invokes numpy, scipy and matplotlib. After struggling, I am going to clean and reinstall Python and the libraries.
After investigation, I found Python 2.5/2.6/2.7/3.2 on my system, and each of them has some copies or other things at: (my OS == Mac OS X 10.7.5 Lion)
/Library/Frameworks/
/opt/local/Library/Frameworks/
/opt/local/bin/
/Applications/
/usr/local/bin/
/usr/bin/
/System/Library/Frameworks/
I know I'm crazy to have these. Now I have removed all these except the things in /System/Libarary/Frameworks (I never remove any thing from /System/Library/). After the clean work, which python now gives /usr/bin/python which links to /System/Library/Frameworks.
Now, is it a clear environment for me to reinstall python? How to double check that there's no other versions existing? How should I reinstall them to guarantee that they and their libraries won't be everywhere and have many copies again?
I want to install a clean Python 2.7 onto a proper location, and make my system know exactly where it is and never install any libraries somewhere else. Please give me some advice that how to manage it like in a professional way.
For your information, here is my current $PATH, I think it should be modified:
/opt/local/bin:/opt/local/sbin:/opt/nest/lib/python2.7/site-packages:/usr/local/lib/python2.7/site-packages:/Library/Frameworks/Python.framework/Versions/2.7/bin:/usr/texbin:/Library/Frameworks/Python.framework/Versions/3.2/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/texbin:/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/django/bin:/usr/X11/bin:/opt/local/bin:/opt/local/sbin:/usr/local/lib/python2.7/site-packages:/Library/Frameworks/Python.framework/Versions/2.7/bin:/Library/Frameworks/Python.framework/Versions/3.2/bin
Please let me know If you need more information. Thank you!
UPDATE:
I'm rethinking profoudly why it becomes so crazy. I believe it's because I installed things via:
easy_install / macports / homebrew / fink / pip sometimes;
.dmg sometimes;
.pkg sometimes;
compile source code sometimes;
and they made things at different locations. I wonder what's the mechanism behind these ways? How do they choose target location? How to prevent them from messing things up?

Why did it get messed up?
There're a couples of different way to install Python, as the update of OP says, and they locate files in different locations. For example, macports puts things into /opt/local/, while homebrew puts things into /usr/local/. Also, Mac OS X brings a few python versions with itself. So, if you install python many times via different ways, you will get many python versions existing independently on your system.
What problem does it cause?
I don't know exactly. I guess the problem is that if you have many versions of python, then which one to use and where to find packages will be determined by the path order in your system PATH and the PYTHONPATH respectively. So you may lose control of where to install python modules. Consider that if you run sudo python setup.py install to install a module (it finds python by the root's PATH) and then try to import the module by python -c "import it" (this time it finds python by your PATH), maybe something will go wrong. This is my guess, I didn't validate it. But in my own case, something did go wrong.
How to avoid this?
I think the principle would be that be aware of that different ways and tools install things independently to different locations, so use them mindfully.
Unless you intend to, don't install the same thing twice via different
ways. (If you intend to do it for python, you might want to check out virtualenv)
Keep an eye on the path order in your PATH and consider if it's
correct.
When installing modules, be clear which python (or pip) is
running and where the module is installed.
So, how did I solve my own case?
Since it had been messing up already and seemed to be very hard to cure, so finally I solved this question by a full OS re-installation, and started to follow the DOs-and-DONTs above. For the installation of the scientific environment with python (numpy/scipy/matplotlib, which had shown problems to make me ask this question), I found this tutorial was extremely helpful. So, problem solved finally.

Here is what was confusing me and how I solved it.
$ which python
/usr/bin/python
$ which python3
/usr/local/bin/python3
$ ls /usr/local/bin/python
ls: /usr/local/bin/python: No such file or directory
So notice I didn't have a HomeBrew installation of python2.7, but did have the python3 installation. The version under /usr/bin/python is using the system default. You can tell based on the module search path:
$ /usr/bin/python
Python 2.7.10 (default, Feb 7 2017, 00:08:15)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
`enter code here`Type "help", "copyright", "credits" or "license" for
more information.
>>> import sys
>>> sys.path
['', '/Library/Python/2.7/...
Notice the '/Library/Python'... that's Mac OS's version of python. But I want to stay strictly on a user installed version (i.e. HomeBrew).
So here's what I did to fix this:
$ brew install python
...
Warning: python 2.7.13 is already installed, it's just not linked.
You can use `brew link python` to link this version.
$ brew link --overwrite python
$ which python
/usr/local/bin/python
$ python
Python 2.7.10 (default, Feb 7 2017, 00:08:15)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/local/Cellar/python/2.7.13...
Its no longer /Library/.. but /usr/local.
Now its finding all of my pip installed modules! Problem solved!
UPDATE:
After updating brew to version 1.5.4, it seems the symbolic links were removed. And now you have to add this to your path:
export PATH="/usr/local/opt/python/libexec/bin:$PATH"
Read the Caveats section in 'brew info python':
==> Caveats
This formula installs a python2 executable to /usr/local/bin.
If you wish to have this formula's python executable in your PATH then add
the following to ~/.bash_profile:
export PATH="/usr/local/opt/python/libexec/bin:$PATH"
Pip and setuptools have been installed. To update them
pip2 install --upgrade pip setuptools
You can install Python packages with
pip2 install <package>
They will install into the site-package directory
/usr/local/lib/python2.7/site-packages
See: https://docs.brew.sh/Homebrew-and-Python.html

In order to install a python distributions into specific folder, you can use the --prefix scheme during python installation. Using the prefix scheme, you can for example install Python 2.7 into the folder /opt/py27. Now, in order to use the new installed Python distribution you have to: cleanup you PATH and LD_LIBRARY_PATH:
Remove all 'old' Python paths and
configure (according to my example) the environment variables like this:
PATH: Add /opt/py27/bin
LD_LIBRARY_PATH: Add /opt/py27/lib
That's it.
(In case you need multiple environments of Python installed at the same time, I'd suggest to have a look at virtualenv)

tl;dr
brew install python
Symptoms
I had similar issues with python programs not finding dependencies.
My python3 version was a broken symlink.
My pip was pointing to a python 3.8
And my pip3 was pointing to 3.9
python -V was outputting some python 2.7 version
python3 -V was outputting some python3.8 version
Solution
I ran brew install python and it fixed all my problems.

Python and Yum issue CentOS 5.4

I'm new to Linux/CentOS and I'm not having the most fun using it. I was developing something locally on my Mac using python 2.7 when I ran it on the server I ran into issues which I put down to being the outdated version of Python (2.4.3) I then updated it, which wasn't a very good idea. I built it from source and then ran into issues, Yum and other modules disappeared.
I'm not back using Python 2.4.3 but I still don't have Yum back.
[root#issues ~]# yum Could not find platform independent libraries
Could not find platform dependent libraries
Consider setting $PYTHONHOME to [:]
'import site' failed; use -v for traceback There was a problem importing one of the
Python modules required to run yum. The error leading to this problem
was:
No module named yum
Please install a package which provides this module, or verify that
the module is installed correctly.
It's possible that the above module doesn't match the current version
of Python, which is:
2.4.3 (#1, Jun 18 2012, 08:55:31) [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)]
If you cannot solve this problem yourself, please go to the yum faq
at: http://wiki.linux.duke.edu/YumFaq
Searching loads of forums everyone says that it's because of a different version of Python, this is no longer the case for me, I think it's something to do with Yum being mapped to a different directory.
I found this command somewhere else and ran this:
[root#issues ~]# rpm -V python
missing /usr/bin/python2
missing /usr/bin/python2.4
Does anyone know what to do or where to go for help? I really need to get this fixed as soon as possible!
Thanks
Actually just to add, If I run python I get this output.
[root#issues ~]# python
Could not find platform independent libraries
Could not find platform dependent libraries
Consider setting $PYTHONHOME to [:]
'import site' failed; use -v for traceback
Python 2.4.3 (#1, Jun 18 2012, 08:55:31)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>

Yes on Linux systems, it is not a good idea to upgrade default python which is 2.4.x version and Linux package management and many other tools require it to function correctly. It is always better to install other python versions separately and use it directly (like: /usr/bin/python2.7)
Python executable itself is not working correctly as it is not able to load default module 'site' during start-up.
The python installation is broken.
Since you can not use "Yum" or "RPM" to bootstrap your packages, I suggest that you use your "OS" installation CD or ISO image to install the rpm/yum packages directly. It will fix the python installation as dependency.
I had gone through this issue and after various trial, this was only way I could fix it.
[Edit:]
See: Python 2.x multiple version issues regarding PYTHONPATH
Also the error is that it is not able to find the package directory.
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
See the paths that I have. Check out the path of packages and can you set "PYTHONHOME".
# installing zipimport hook
import zipimport # builtin
# installed zipimport hook
# /usr/lib64/python2.4/site.pyc matches /usr/lib64/python2.4/site.py
import site # precompiled from /usr/lib64/python2.4/site.pyc
# /usr/lib64/python2.4/os.pyc matches /usr/lib64/python2.4/os.py
import os # precompiled from /usr/lib64/python2.4/os.pyc
If you are not able to locate similar package location, can you use RPM to directly install python2.4 again
rpm --force -hUv url_for_python_rpm