We Keep Coding

How to use Python Dbus bindings in Anaconda

I am trying to install dbus on Anaconda python environment and I am struggling.
Here is the error message I am getting:
e#gateway:~$ python
Python 3.5.4 |Anaconda custom (64-bit)| (default, Oct 13 2017, 11:22:58)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import dbus
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/e/anaconda3/lib/python3.5/site-packages/dbus/__init__.py", line 77, in <module>
import dbus.types as types
File "/home/e/anaconda3/lib/python3.5/site-packages/dbus/types.py", line 6, in <module>
from _dbus_bindings import (
ImportError: /home/e/anaconda3/lib/python3.5/site-packages/_dbus_bindings.so: undefined symbol: _Py_ZeroStruct
>>>
Here are some of the outputs I think may be asked:
e#gateway:~$ conda install dbus
Fetching package metadata ...........
Solving package specifications: .
# All requested packages already installed.
# packages in environment at /home/e/anaconda3:
#
dbus 1.10.22 h3b5a359_0
e#gateway:~$ sudo apt-get install libdbus-glib-1-dev libdbus-1-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
libdbus-glib-1-dev is already the newest version (0.106-1).
libdbus-1-dev is already the newest version (1.10.6-1ubuntu3.3).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
e#gateway:~$ sudo apt-get install dbus
Reading package lists... Done
Building dependency tree
Reading state information... Done
dbus is already the newest version (1.10.6-1ubuntu3.3).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
e#gateway:~$ which python
/home/e/anaconda3/bin/python
e#gateway:~$ conda --version
conda 4.3.31
e#gateway:~$ sudo /home/e/anaconda3/bin/python -m pip install dbus-python
The directory '/home/e/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/e/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied: dbus-python in ./anaconda3/lib/python3.5/site-packages
DBus is working fine on the system python, however not working on Anaconda Python.
Python 2.7:
e#gateway:~$ which python
/usr/bin/python
e#gateway:~$ python
Python 2.7.12 (default, Nov 20 2017, 18:23:56)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import dbus
>>>
Python 3.5:
e#gateway:~$ which python3
/usr/bin/python3
e#gateway:~$ python3
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import dbus
>>>
Can anyone help me? Am I missing something blatantly obvious here?
Thanks in advance.

I had similar issues, there are few cases where dbus and python don't work well out-of-the-box. The consensus appears to be that you need a system-level install (i.e. apt-get) to get dbus to work. I believe the /home/e/anaconda3/lib/python3.5/site-packages/_dbus_bindings.so: undefined symbol: _Py_ZeroStruct error you're seeing is directly related to that.
conda install dbus does not add anything to ~/anaconda3/lib/python3.6/site-packages, but instead appears to install some executables in ~/anaconda3/bin/ like dbus-run-session, dbus-daemon, etc. This makes some sense when you analyze the contents of the dbus tarball https://anaconda.org/conda-forge/dbus, as it's all C files and executables. I'm not sure it's supposed to be the dbus python module, but I could be wrong.
EDIT:
I searched the conda repositories and found a few individuals that uploaded a version of dbus-python, presumably that they compiled and installed. I tried this one out in a py3.6 conda environment via:
conda install -c scottwales dbus-python
I was then able to import dbus. This is a hacky approach and should not be used in production, I'd recommend listening to
Carlos Cordoba's post below. But if you need a solution now, search through some user conda packages or try to compile the library yourself.

Can anyone help me? Am I missing something blatantly obvious here?
Yes, you are. There's one thing people still don't understand about conda: conda is not a pip replacement. It is a general package manager, in the same vein as apt-get, yum, brew, emerge, etc, but cross-platform and based on Python.
In this case, that means that conda install dbus does not install the Python Dbus bindings, as you would expect with pip . It installs the Dbus C package itself, which is needed by Qt 5 (again, the C++ library, not the Python bindings to it).
Unfortunately, there are no Conda packages for dbus-python. To make matters worse, it seems there's no easy way to create packages for it, as pointed out here.
Finally, you said
Here is the error message I am getting
The (most probable) cause of that error is because you added your system Python dist-packages path to the PYTHONPATH of Anaconda or because you blindly copied the dbus module from system Python to Anaconda. Please don't do that ever again. System Python and Anaconda packages are compiled with different compilers and under different conditions. So mixing them is the cause of incomprehensible errors, just like the one you reported.

Astropy isn't working with Python

I've been having difficulty with Python recently, mainly since I think I had several versions and conflicts (due to Anaconda installs, canopy installs etc.). So I cleaned those out.
I reinstalled python (2.7) via brew.
I reinstalled numpy and matplotlib via pip.
I also reinstalled astropy and h5py via pip.
However, I get a clean import of numpy and matplotlib, but not of astropy and h5py:
~ > python
Python 2.7.10 (default, Oct 23 2015, 19:19:21)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> import matplotlib
>>> import astropy
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named astropy
>>> import h5py
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named h5py
>>>

My suspicion is that your pip executable is not linked to your python executable, which means when you run pip install astropy it is installing it in the site-packages for a different python.
One way to make sure you're using the correct pip is to not use
$ pip install astropy
but instead use
$ python -m pip install astropy
If this fails, it probably means that you don't have pip installed for the python instance you're using, and you need to install it (note that for Python 2 version 2.7.9 or later, or Python 3 version 3.4 or later, pip comes bundled with Python).
If this still doesn't work, then something stranger is going on. It may be due to having $PYTHONPATH or $LD_LIBRARY_PATH/$DYLD_LIBRARY_PATH set in a way that interferes with your python imports. In this case, you could clear these variables and try again. Otherwise, I'd consider using a package bundle such as conda or canopy. It makes these kinds of installation issues much smoother.
Edit: I see now that you've used conda and canopy, and you suspect these were causing your problems. Conda and canopy, by design, both sandbox their python installations so that they shouldn't get interference from other installs in your system, unless you force such interference by setting the environment variables I mentioned above. I'd suggest reinstalling conda and wiping those environment variables from your bash/csh startup script.

I can't call pymongo in the python shell

I'm trying to use pymongo in the IDLE shell on MAC OS X 10.9, but I can only do it in terminal.
when I call import pymongo in IDLE I get the following error:
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
import pymongo
ImportError: No module named 'pymongo'
but I use python in terminal I get this:
$ python
Python 2.7.5 (default, Sep 12 2013, 21:33:34)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pymongo
>>>
I can then use it with MongoDB.
Have I made some sort of install error? I'm doing this to learn mongoDB so I'm relatively new to this. Any help is much appreciated.

From the information you supplied in the comments, it appears you have installed a version of Python 3.3.4 on your system but you have installed the PyMongo distribution to an instance of Python 2.7, probably the Apple-supplied system Python 2.7 shipped with OS X 10.9. When you install a third-party package (or "distribution"), it is normally associated only with the Python instance that you used to install it. There are several common ways to install such packages. One way is to use the easy_install command, as is suggested on the PyMongo page. However, the easy_install command is also associated with a particular Python instance. On recent OS X releases, Apple supplies easy_install commands that are associated with and install into the system Pythons. So it's a common pitfall on OS X to install a newer version of Python alongside the system Python but then use the default easy_install command with the result that the package you want ends up installed in the wrong Python version. One solution is to install a separate version of easy_install for each Python version you install. The easy_install command is provided by the setuptools package. However, these days the recommended installer tool for Python is pip which provides more features than easy_install, including the ability to uninstall packages, and is actively supported in the community.
curl -O https://raw.github.com/pypa/pip/master/contrib/get-pip.py
python3.3 get-pip.py
python3.3 -m pip install pymongo
python3.3 -c "import pymongo; print(pymongo.version)"
-> 2.6.3
You also need to install a version of pip for each Python instance that you use. There are other ways to invoke pip but, by using the way shown above, you know which version of Python you are using and you are less likely to end up with the situation you have now. There are other tools you can use in addition, like virtualenv, but, particularly on OS X, pip should be sufficient to handle most beginning use cases.

WARNING: IPython History requires SQLite, your history will not be saved

Hi I'm using Ubuntu release 12.10 (quantal) 32-bit with Linux Kernel 3.5.0-21-generic. I'm trying to get IPython's History to work. I've set it up using pythonbrew and a virtual environment. In there I use pip to install IPython. Currently, when I start up IPython in a terminal I get:
WARNING: IPython History requires SQLite, your history will not be saved
Python 2.7.3 (default, Nov 8 2012, 18:25:10)
Type "copyright", "credits" or "license" for more information.
IPython 0.13.1 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
Searching on the warning in the first line, I found this issue report, so I went back and installed the following:
sudo apt-get install libsqlite0 libsqlite0-dev libsqlite3-0 libsqlite3-dev
and then removed and reinstalled pysqlite using pip
pip uninstall pysqlite
pip install pysqlite
After that I thought I would check the installation by importing the module:
Python 2.7.3 (default, Nov 8 2012, 18:25:10)
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/me/.pythonbrew/pythons/Python-2.7.3/lib/python2.7/sqlite3/__init__.py", line 24, in <module>
from dbapi2 import *
File "/home/me/.pythonbrew/pythons/Python-2.7.3/lib/python2.7/sqlite3/dbapi2.py", line 27, in <module>
from _sqlite3 import *
ImportError: No module named _sqlite3
So now it seems the file _sqlite3.so can't be found. That's when I found this SO question. Either it doesn't exist or it's not in my PYTHONPATH environment variable. Searching for the file, I get:
$ locate _sqlite3.so
/home/me/Desktop/.dropbox-dist/_sqlite3.so
/home/me/epd/lib/python2.7/lib-dynload/_sqlite3.so
/usr/lib/python2.7/lib-dynload/_sqlite3.so
So the file is there, but when I looked in my python path:
import sys
for p in sys.path:
print p
none of the above paths that contain _sqlite3.so were contained in my PYTHONPATH. For giggles, I added the path /usr/lib/python2.7/lib-dynload to my PYTHONPATH in a terminal and then tried to import sqlite3 again:
Python 2.7.3 (default, Nov 8 2012, 18:25:10)
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.append("/usr/lib/python2.7/lib-dynload")
>>> import sqlite3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/me/.pythonbrew/pythons/Python-2.7.3/lib/python2.7/sqlite3/__init__.py", line 24, in <module>
from dbapi2 import *
File "/home/me/.pythonbrew/pythons/Python-2.7.3/lib/python2.7/sqlite3/dbapi2.py", line 27, in <module>
from _sqlite3 import *
ImportError: /usr/lib/python2.7/lib-dynload/_sqlite3.so: undefined symbol: PyUnicodeUCS4_DecodeUTF8
Uh oh. Now I'm completely stuck. Can anyone help me out? I've also read in a few places that I may have to rebuild Python. I have no idea how to do this in pythonbrew. Can anyone point me in the right direction?

I've also read in a few places that I may have to rebuild Python.
This is correct. SQLite is part of the standard library,
and is built when you compile Python. There are a few 'optional' parts
of the standard library, which Python will simply skip (with minimal warning, unfortunately)
if the dependencies are missing at build time, and sqlite is one of these.
You should be able to just install libsqlite3-dev,
then rebuild Python and you should be set.
Keep an eye on the build messages,
as they do report which modules they are skipping due to missing dependencies.

Thanks to minrk for pointing me in the right direction. All I had to do was rebuild python. I've outlined the steps below for those that are using pythonbrew. Notice that I already installed the libsqlite3-dev package up in the question section.
First, with the proper version of python and virtual environment loaded up run the command:
$ pip freeze -l > requirements.txt
This gives us a text file list of all of the pip packages that have been installed in the virtual environment for this particular python release in pythonbrew. Then we remove the version of python from pythonbrew and reinstall it (this is the "rebuild python" step):
$ pythonbrew uninstall 2.7.3
$ pythonbrew install 2.7.3
After that, we switch over to the newly installed python version 2.7.3 and create a new virtual environment (which I've called "sci"):
$ pythonbrew switch 2.7.3
$ pythonbrew venv create sci
$ pythonbrew venv use sci
Ideally you should be able to run the command:
$ pip install -r requirements.txt
and according to this pip should reinstall all the modules that you had in the virtual environment before we clobbered that version of python (2.7.3). It didn't work for me for whatever reason so I manually installed all of the modules using pip individuality.
$ ipython --pylab
Python 2.7.3 (default, Jan 5 2013, 18:48:27)
Type "copyright", "credits" or "license" for more information.
IPython 0.13.1 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
and IPython history works!

What worked for me (using osx + homebrew + brewed python):
# Reinstall Python 2.7 with sqlite
brew remove python
brew install readline sqlite gdbm --universal
brew install python --universal --framework
# Reinstall iPython with correct bindings
pip uninstall ipython
pip install ipython
And you should be good to go.

You should rebuild your python with sqlite support
sudo apt-get install libsqlite3-dev
wget https://www.python.org/ftp/python/2.7.15/Python-2.7.15.tgz
tar -xvf Python-2.7.15.tgz
cd Python-2.7.15
./configure
make
sudo make install
Recreate your virtual environment and you should be good to go
rmvirtualenv venv
mkvirtualenv -p python2 venv
workon venv
pip install -r requirements.txt
# or
pip install ipython

This warning appears on macOS when python is installed with pyenv. By default it installs python without sqlite. These commands reinstall python with sqlite support:
pyenv uninstall 3.7
CFLAGS="-I$(xcrun --show-sdk-path)/usr/include" pyenv install 3.7

Already installed pyzmq but still getting "ImportError: No module named zmq"

I am trying to get ipython notebook run.
I already installed pyzmq.
Do you know why it's still giving this error?

The only reason you would be seeing that error (having manually installed pyzmq) is if pyzmq was installed to a different pythonpath than where ipython is running from.
If you followed the install instructions, you should have no problems:
easy_install ipython pyzmq tornado
# or
pip install ipython pyzmq tornado
It is also possible that the installer saw the dependency already satisfied in another location and didn't do anything, and then ipython might be running with a different pythonpath that isn't seeing it. You can force an upgrade of everything:
easy_install -U ipython pyzmq tornado

pip3 uninstall pyzmq
pip3 install pyzmq
Worked for me

Try the following:
pip3 install --upgrade pip
pip3 install jupyter
The above assumes you have aliased pip to pip3.

check what version on python you installed it with
python --version
When you run the application make sure you are using that same version you used to install it with.
The problem for me was that I installed it for python 3.6 but ran it under python 2.7

I have all the packages installed & it also shows up in the path but does throw the same error
jabira#ubuntu:~$ python
Python 2.7.3 (default, Feb 27 2014, 19:58:35)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/share/pyshared/zmq', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-linux2', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', '/usr/lib/pymodules/python2.7']
>>>
jabira#ubuntu:~$ ls /usr/share/pyshared/zmq/
core devices eventloop green __init__.py log ssh tests utils web
jabira#ubuntu:~$ less /usr/share/pyshared/zmq/__init__.py
jabira#ubuntu:~$ python
Python 2.7.3 (default, Feb 27 2014, 19:58:35)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import zmq
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named zmq
>>>
jabira#ubuntu:~$ dpkg -l|grep zmq
ii libzmq3 3.2.2+dfsg-1lucid lightweight messaging kernel (shared library)
ii python-zmq 2.2.0.1-1lucid3 0MQ is a fast messaging library
jabira#ubuntu:~$

you should add Phython path to windows local variable PATH before install zmq

For me the solution was to rename my script from zmq.py to anything else.
This happens because using the name zmq.py crates a name conflict with the package as python tries to include the script itself rather than the library as scripts have priority in the include hierarchy.

I had a file that was trying to import zmq that was throwing:
ModuleNotFoundError: No module named 'zmq'
But pip install zmq or pip install pyzmq kept telling me I had a version already installed, and thus Requirement already satisfied at Library/blahblah/Python/3.6 ...
I've already aliased pip3 to pip, but trying pip3 didn't work either
I wound up having to reset my $PYTHONPATH and deleted that Python 3.6 install I didn't actually need.
After that pip install pyzmq STILL didn't work though.
What finally worked was pip3 install zmq ... Weird since I already had that aliased, but something w/the paths might still be screwy.

On Ubuntu 20.04 and Python 3.9, this did the trick for me:
sudo apt-get install python3-zmq
Source: https://askubuntu.com/a/100543