We Keep Coding

How to use Python Dbus bindings in Anaconda

I am trying to install dbus on Anaconda python environment and I am struggling.
Here is the error message I am getting:
e#gateway:~$ python
Python 3.5.4 |Anaconda custom (64-bit)| (default, Oct 13 2017, 11:22:58)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import dbus
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/e/anaconda3/lib/python3.5/site-packages/dbus/__init__.py", line 77, in <module>
import dbus.types as types
File "/home/e/anaconda3/lib/python3.5/site-packages/dbus/types.py", line 6, in <module>
from _dbus_bindings import (
ImportError: /home/e/anaconda3/lib/python3.5/site-packages/_dbus_bindings.so: undefined symbol: _Py_ZeroStruct
>>>
Here are some of the outputs I think may be asked:
e#gateway:~$ conda install dbus
Fetching package metadata ...........
Solving package specifications: .
# All requested packages already installed.
# packages in environment at /home/e/anaconda3:
#
dbus 1.10.22 h3b5a359_0
e#gateway:~$ sudo apt-get install libdbus-glib-1-dev libdbus-1-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
libdbus-glib-1-dev is already the newest version (0.106-1).
libdbus-1-dev is already the newest version (1.10.6-1ubuntu3.3).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
e#gateway:~$ sudo apt-get install dbus
Reading package lists... Done
Building dependency tree
Reading state information... Done
dbus is already the newest version (1.10.6-1ubuntu3.3).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
e#gateway:~$ which python
/home/e/anaconda3/bin/python
e#gateway:~$ conda --version
conda 4.3.31
e#gateway:~$ sudo /home/e/anaconda3/bin/python -m pip install dbus-python
The directory '/home/e/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/e/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied: dbus-python in ./anaconda3/lib/python3.5/site-packages
DBus is working fine on the system python, however not working on Anaconda Python.
Python 2.7:
e#gateway:~$ which python
/usr/bin/python
e#gateway:~$ python
Python 2.7.12 (default, Nov 20 2017, 18:23:56)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import dbus
>>>
Python 3.5:
e#gateway:~$ which python3
/usr/bin/python3
e#gateway:~$ python3
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import dbus
>>>
Can anyone help me? Am I missing something blatantly obvious here?
Thanks in advance.

I had similar issues, there are few cases where dbus and python don't work well out-of-the-box. The consensus appears to be that you need a system-level install (i.e. apt-get) to get dbus to work. I believe the /home/e/anaconda3/lib/python3.5/site-packages/_dbus_bindings.so: undefined symbol: _Py_ZeroStruct error you're seeing is directly related to that.
conda install dbus does not add anything to ~/anaconda3/lib/python3.6/site-packages, but instead appears to install some executables in ~/anaconda3/bin/ like dbus-run-session, dbus-daemon, etc. This makes some sense when you analyze the contents of the dbus tarball https://anaconda.org/conda-forge/dbus, as it's all C files and executables. I'm not sure it's supposed to be the dbus python module, but I could be wrong.
EDIT:
I searched the conda repositories and found a few individuals that uploaded a version of dbus-python, presumably that they compiled and installed. I tried this one out in a py3.6 conda environment via:
conda install -c scottwales dbus-python
I was then able to import dbus. This is a hacky approach and should not be used in production, I'd recommend listening to
Carlos Cordoba's post below. But if you need a solution now, search through some user conda packages or try to compile the library yourself.

Can anyone help me? Am I missing something blatantly obvious here?
Yes, you are. There's one thing people still don't understand about conda: conda is not a pip replacement. It is a general package manager, in the same vein as apt-get, yum, brew, emerge, etc, but cross-platform and based on Python.
In this case, that means that conda install dbus does not install the Python Dbus bindings, as you would expect with pip . It installs the Dbus C package itself, which is needed by Qt 5 (again, the C++ library, not the Python bindings to it).
Unfortunately, there are no Conda packages for dbus-python. To make matters worse, it seems there's no easy way to create packages for it, as pointed out here.
Finally, you said
Here is the error message I am getting
The (most probable) cause of that error is because you added your system Python dist-packages path to the PYTHONPATH of Anaconda or because you blindly copied the dbus module from system Python to Anaconda. Please don't do that ever again. System Python and Anaconda packages are compiled with different compilers and under different conditions. So mixing them is the cause of incomprehensible errors, just like the one you reported.

Do I need to install django separately for different python version?

I had python version 2.7.3 and i wanted to learn django so i installed django version 1.8.2 on my ubuntu 12.0.4 .
invivtus#invictus:~/bin$ python
Python 2.7.3 (default, Sep 26 2013, 20:08:41)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import django
>>> django.VERSION
(1, 8, 2, 'final', 0)
Then i read that best way yo work with django is to work on python version 3.3 so i installed python version 3.3.6 on my system where py is symbolic link pointing to /opt/python3.3/bin/python3.3
invictus#invictus:~/bin$ py
Python 3.3.6 (default, Jun 21 2015, 16:13:35)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
when I try and import django here i get error
>>> import django
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'django
I see django got installed my python 2.7 directory.
>>> import django
>>> django
<module 'django' from '/usr/local/lib/python2.7/dist-packages/django/__init__.pyc'>
How can i use this django with my 3.3.6 version. My default python version is 2.7.3
What are possible workout here?

Yes, each Python version has its own folder with installed packages. You'll have to install Django separately for Python 3.3. The same is true for every package that is not available by default.
(If you're using Python 3, why not go for the latest and greatest, 3.4?)

As some of the comments said, you should be using a virtualenv to isolate your environments. You would do it like this:
1) Ensure you have virtualenv installed. On Ubuntu for instance, that would be package virtualenv.
2) Create a new, empty environment. You choose which python version it will be like this:
virtualenv -p /usr/bin/python3.4 env
3) That created an env folder. Activate the newly created environment:
. env/bin/activate
This updates your paths so now, when you run python or pip from this shell, they will execute in the context of your virtualenv.
4) Update the virtualenv (optional)
pip install -U pip
5) Install whatever packages you need. The recommended way is to have a requirements.txt file at the root of your project. You would pull them this way:
pip install -r myproject/requirements.txt
That's it. Use the pip command as usual. As long as you're working with the virtualenv active, your python command will only see the modules you explicitly install in it.
6) Don't forget to re-run . env/bin/activate in every new shell. If you think you'll probably forget, you can add this to your manage.py:
import sys
if __name__ == "__main__":
if not hasattr(sys, 'real_prefix'):
sys.stderr.write('Running outside of any virtualenv - did you forget to activate one?\n')
What are the benefits?
You have an isolated environment for every project (no conflicts).
You may use different versions of the same module in different projects.
System updates will not break your project.
You are not polluting your system with unmanaged files.
You never run stuff as root, which means both added isolation, and the possibility of running your project without having root access to the system.
As long as you keep your requirements.txt up to date (using pip freeze), you can rebuild the virtualenv on another system and it will work.
[edit: using requirements.txt]
That's just a file that has pip install specifications, one by line. It allows to rebuild the virtualenv from scratch easily. You can generate it from your current virtualenv using:
pip freeze > requirements.txt
So the idea is just to remember to re-run this command everytime you change your environment (installing, removing or upgrading some package).

Django, Pyenv and Git file structure on Ubuntu

I am working to set up a django project on ec2 with an Ubuntu 14.4 LTS instance. I want to write my code using python 3 and django. I've been advised that the best way to do this is to use a virtualenv. Following
https://robinwinslow.co.uk/2013/12/26/python-3-4-virtual-environment/
I tried:
~$ pyvenv-3.4 djenv
Which appears to create a virtualenv (please see screenshot). Now I have 2 questions:
1) What folder should I place my django project. - I'm thinking within the djenv folder. In other words I'd run:
/home/ubuntu/djenv$ django-admin.py startproject testproject.
2) init a git repository. I'm assuming I'd to it it in the same location, i.e.
/home/ubuntu/djenv$ git init
from within
Does this seem correct or is there a better way to do this?

Your project source code should be entirely separate from your virtual env in the file system. If they are in the same place, as you suggest, then you will end up checking libraries into your git repository needlessly and that will take up extra space end up causing problems.
Once you have activated a virtualenv you can run Python and use all the libraries in it. You don't need any connection in the file system.
You should store a PIP file in your git repo somewhere that describes how to install the relevant dependencies into your virtualenv so you can re-create it on another machine.
On my machine my projects are in /home/me/projects/«project» and my virtualenvs are in /home/me/envs/«envname». I use virtualenvwrapper which makes things easy.
Create an environment
$ mkvirtualenv test
New python executable in test/bin/python
Installing Setuptools......done.
Installing Pip.........done.
Activate it
$ workon test
Python now refers to the one in my environment. It has its own site-packages etc.
$ which python
/Users/joe/Envs/test/bin/python
If we run it and look at the paths, they point to the virtualenv. This is where it looks for packages (lots removed from my path for simplicity).
$ python
Python 2.7.5 (default, Mar 9 2014, 22:15:05)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/Users/joe/Envs/test/lib/python27.zip', '/Users/joe/Envs/test/lib/python2.7', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', '/Users/joe/Envs/test/lib/python2.7/site-packages']
>>>

I can't call pymongo in the python shell

I'm trying to use pymongo in the IDLE shell on MAC OS X 10.9, but I can only do it in terminal.
when I call import pymongo in IDLE I get the following error:
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
import pymongo
ImportError: No module named 'pymongo'
but I use python in terminal I get this:
$ python
Python 2.7.5 (default, Sep 12 2013, 21:33:34)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pymongo
>>>
I can then use it with MongoDB.
Have I made some sort of install error? I'm doing this to learn mongoDB so I'm relatively new to this. Any help is much appreciated.

From the information you supplied in the comments, it appears you have installed a version of Python 3.3.4 on your system but you have installed the PyMongo distribution to an instance of Python 2.7, probably the Apple-supplied system Python 2.7 shipped with OS X 10.9. When you install a third-party package (or "distribution"), it is normally associated only with the Python instance that you used to install it. There are several common ways to install such packages. One way is to use the easy_install command, as is suggested on the PyMongo page. However, the easy_install command is also associated with a particular Python instance. On recent OS X releases, Apple supplies easy_install commands that are associated with and install into the system Pythons. So it's a common pitfall on OS X to install a newer version of Python alongside the system Python but then use the default easy_install command with the result that the package you want ends up installed in the wrong Python version. One solution is to install a separate version of easy_install for each Python version you install. The easy_install command is provided by the setuptools package. However, these days the recommended installer tool for Python is pip which provides more features than easy_install, including the ability to uninstall packages, and is actively supported in the community.
curl -O https://raw.github.com/pypa/pip/master/contrib/get-pip.py
python3.3 get-pip.py
python3.3 -m pip install pymongo
python3.3 -c "import pymongo; print(pymongo.version)"
-> 2.6.3
You also need to install a version of pip for each Python instance that you use. There are other ways to invoke pip but, by using the way shown above, you know which version of Python you are using and you are less likely to end up with the situation you have now. There are other tools you can use in addition, like virtualenv, but, particularly on OS X, pip should be sufficient to handle most beginning use cases.

Python package installed globally, but not in a virtualenv (PyGTK)

I'm having some strange issues with PyGTK in "virtualenv". gtk does not import in my virtualenv, while it does import in my global python install. (I wasn't having this particular issue last week, guessing some software update upset something.)
Is there a good way to resolve this behavior?
Shown here: importing gtk globally,
tom#zeppelin:~$ python
Python 2.7.1+ (r271:86832, Sep 27 2012, 21:12:17)
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import gtk
>>> gtk
<module 'gtk' from '/usr/lib/pymodules/python2.7/gtk-2.0/gtk/__init__.pyc'>
and then failing to import gtk,
tom#zeppelin:~$ workon py27
(py27)tom#zeppelin:~$ python
Python 2.7.1+ (r271:86832, Sep 27 2012, 21:12:17)
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import gtk
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named gtk
Unfortunately, this has broken my ipython --pylab environment: http://pastebin.com/mM0ur7Hc
UPDATE:
I was able to fix this by adding symbolic links as suggested by grepic / this thread: Python: virtualenv - gtk-2.0
with a minor difference, namely that my "cairo" package was located in /usr/lib/pymodules/python2.7/cairo/ rather than in /usr/lib/python2.7/dist-packages/cairo.
SECOND UPDATE:
I also found it useful to add the following lines to my venv/bin/activate:
export PYTHONPATH=$PYTHONPATH:/home/tom/.virtualenvs/py27/lib/python2.7/dist-packages
export PYTHONPATH=$PYTHONPATH:/home/tom/.virtualenvs/py27/lib/python2.7/dist-packages/gtk-2.0
export PYTHONPATH=$PYTHONPATH:/usr/lib/pymodules/python2.7/gtk-2.0
(I suspect that one or more of these is unneccessary, but I've been fiddling around with this for too long and have decided to stop investigating -- my setup now works and so I'm satisfied.)
Problem solved! Thanks everyone.

Try creating your virtual environment with the --system-site-packages flag.

So gtk normally lives in a place like /usr/lib/python2.7/dist-packages which is in your Python path in your global environment, but not in your virtual environment.
You may wish to just add the path to gtk manually with something like
import sys
sys.path.append("/usr/lib/python2.7/dist-packages/gtk")
You could also change the path when you activate the virtual environment. Open up venv/bin/activate. Its a scary looking file, but at the end you can just put:
export PATH=$PATH:/my/custom/path
Save that and the next time you activate the virtual environment with:
source venv/bin/activate
your custom path will be in the path. You can verify this with
echo $PATH
An alternative approach suggested Python: virtualenv - gtk-2.0 is to go into your virtualenv directory and add a 'dist-packages' directory and create symbolic links to the gtk package you were using previously:
mkdir -p venv/lib/python2.7/dist-packages/
cd venv/lib/python2.7/dist-packages/
For GTK2:
ln -s /usr/lib/python2.7/dist-packages/glib/ glib
ln -s /usr/lib/python2.7/dist-packages/gobject/ gobject
ln -s /usr/lib/python2.7/dist-packages/gtk-2.0* gtk-2.0
ln -s /usr/lib/python2.7/dist-packages/pygtk.pth pygtk.pth
ln -s /usr/lib/python2.7/dist-packages/cairo cairo
For GTK3:
ln -s /usr/lib/python2.7/dist-packages/gi gi
Full disclosure: I feel that both these solutions are somewhat hackish, which is ok given that you say the question is urgent. There is probably a 'proper' way to extend a virtual environment so let us know if you eventually discover the better solution. You may have some luck with http://www.virtualenv.org/en/latest/index.html#creating-your-own-bootstrap-scripts

Another way to do this is to create a .pth file in your virtualenv's site-packages dir
eg
(in <virtualenv>/lib/python2.7/site-packages/dist-packages.pth)
/usr/lib/python2.7/dist-packages/
This fixed the issue I was having with apt-get installed version of pycairo

If you want to include the links to the relevant system's python gtk-2.0 in the virtualenv, you can just use pip to install ruamel.venvgtk:
pip install ruamel.venvgtk
You don't have import anything, the links are setup during installation.
This is especially handy if you are using tox, in that case you only need to include the dependency (for tox):
deps:
pytest
ruamel.venvgtk
and a newly setup python2.7 environment will have the relevant links included before the tests are run.

It is now possible to resolve this using vext. Vext allows you to install packages in a virtualenv that individually access your system packages. To access PyGTK, do the following:
pip install vext
pip install vext.pygtk