We Keep Coding

py.test: ImportError - cannot import ----

I am using someone else's code, available on GitHub. To run their code I created a virtualenv and installed all the dependencies listed - both python libraries and clones of other repositories. When I proceed to run the included tests, I get an ImportError:
Namespace(all=False, regr=False, sci=False, unit=True)
[localhost] local: py.test -x -v engine/test
==================================================================================== test session starts =====================================================================================
platform linux2 -- Python 2.7.6, pytest-2.8.2, py-1.4.31, pluggy-0.3.1 -- /home/compomics/local/METASPACE/SM_distributed/SM_engine/bin/python
cachedir: engine/test/.cache
rootdir: /home/compomics/local/METASPACE/SM_distributed/engine/test, inifile:
collecting 6 items / 1 errors
=========================================================================================== ERRORS ===========================================================================================
_______________________________________________________________________ ERROR collecting test_formula_img_validator.py _______________________________________________________________________
engine/test/test_formula_img_validator.py:7: in <module>
from engine.formula_img_validator import filter_sf_images,get_compute_img_measures, ImgMeasures
engine/formula_img_validator.py:7: in <module>
from pyIMS.image_measures import measure_of_chaos, isotope_image_correlation, isotope_pattern_match
E ImportError: cannot import name measure_of_chaos
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================================================== 1 error in 0.99 seconds ===================================================================================
Fatal error: local() encountered an error (return code 2) while executing 'py.test -x -v engine/test'
However, if I open the python interpreter and try to do the exact same imports, it does it just fine without any error. Similar questions suggested:
adding an empty __init__.py to the test directory
making sure pytest is installed in the virtualenv
I did both these things, and the error persists.
I added to the beginning of the test script:
import os
print(os.environ["PYTHONPATH"].split(os.pathsep))
print(os.listdir("."))
and confirmed that the folder from where I'm trying to import is indeed in the resulting list.
Not sure how to proceed. Would appreciate any help I can get :)

In file formula_img_validator.py change
from pyIMS.image_measures import measure_of_chaos,isotope_image_correlation, isotope_pattern_match
to
from engine.pyIMS.image_measures import measure_of_chaos, isotope_image_correlation, isotope_pattern_match
That'll solve the problem. For complete solution go to GitHub for new updated code.

there was a conflict with other library
EDIT - this was my own stupidity for not remembering I had cloned a previous version of the dependent repos, which was also on my path, and that did not include the function this code was trying to load. Sorry for not having deleted the question when I noticed, I couldn't for the life of me find the delete button :)

Permission denied on dl.open() with ipython but not with python

My initial goal is to open a dll file on Cygwin using ctypes. However I found some issues with it. I dug up to sys.dl which returns an unknown Permission denied only on IPython.
With python everything looks fine:
$ ls
my.dll
$ python
Python 2.7.8 (default, Jul 28 2014, 01:34:03)
[GCC 4.8.3] on cygwin
>>> import dl
>>> dl.open('my.dll')
<dl.dl object at 0xfffaa0c0>
With ipython I get the error:
$ ipython
Python 2.7.8 (default, Jul 28 2014, 01:34:03)
In [1]: import dl
In [2]: dl.open('my.dll')
---------------------------------------------------------------------------
error Traceback (most recent call last)
<ipython-input-2-c681630fa713> in <module>()
----> 1 dl.open('my.dll')
error: Permission denied
I investigated on this using strace. The output log for `IPython is huge, more than 4MB. Fortunately, I identified some weird things:
symlink.check(C:\Users\user\Home\projects\foo\my.dll, 0x28AB88) (0x4022)
35 2705178 [main] python2.7 16924 path_conv::check: this->path(C:\Users\user\Home\projects\foo\my.dll), has_acls(1)
37 2705215 [main] python2.7 16924 cwdstuff::get: posix /cygdrive/c/Users/user/Home/projects/foo
32 2705247 [main] python2.7 16924 cwdstuff::get: (C:\Users\user\Home\projects\foo) = cwdstuff::get (0x8006ECF0, 32768, 0, 0), errno 11
--- Process 14376, exception c0000138 at 7726163E
3286 2708533 [main] python2.7 16924 seterrno_from_win_error: /home/corinna/src/cygwin/cygwin-1.7.35/cygwin-1.7.35-1.i686/src/src/winsup/cygwin/dlfcn.cc:174 windows error 182
42 2708575 [main] python2.7 16924 geterrno_from_win_error: unknown windows error 182, setting errno to 13
36 2708611 [main] python2.7 16924 dlopen: ret 0x0
Who is /home/corinna? I have no corinna user in my installation, neither on my Windows. Corinna does not come from my installation. Is it some hard-coded stuff?
Now, here is what I get from strace for python:
symlink.check(C:\Users\user\Home\projects\foo\my.dll, 0x28B728) (0x4022)
26 10440048 [main] python 12604 path_conv::check: this->path(C:\Users\user\Home\projects\foo\my.dll), has_acls(1)
23 10440071 [main] python 12604 cwdstuff::get: posix /cygdrive/c/Users/user/Home/projects/foo
25 10440096 [main] python 12604 cwdstuff::get: (C:\Users\user\Home\projects\foo) = cwdstuff::get (0x8006ECF0, 32768, 0, 0), errno 0
3405 10443501 [main] python 12604 dlopen: ret 0x5B9C0000
dlopen is returning 0x0 in IPython while it is returning 0x5B9C0000 for python. I notice that cwdstuff::get is raising an error before dlopen is called.
EDIT
I sent a message to Cygwin's mailing list and the answer of Corinna regarding this issue is:
This is not Cygwin's fault, AFAICS. Cygwin never loads functions by
ordinal. This is also a bit on the lean side as far as information is
concerned. One can't see how the process calls dlopen, for instance.
Corinna
How to solve this issue?
My earlier tests using ctypes
Initially when I asked my question I was just playing with ctypes.
I am working on Cygwin 32-bit and Windows 7.
With IPython I got an OSError when I tried to load a dll using cdll.LoadLibrary.

Two ideas:
1) in the next cell, type %pdb, and then interactively "print self._name" to see what it is.
2) Use a full path to cdll.LoadLibrary("foo.dll") to see if that works.
Once you know what the issue is, then you can decide whose bug it is, and report it (could be a ctypes issue, but probably ipython)

More ideas:
Check that the user accessing the DLL is the same. You do that like this:
import getpass
print(getpass.getuser())
Check what's the current process actually doing. I haven't used cygwin but in the linux shell the executable strace should show you this.
Usage: get the PID of your current process: import os; os.getpid()
After this, you can use (from outside the python/ipython console the command strace -p <the pid> -e file. After this setup, you can try to load your DLL.
Remarks: the -e file flag should be written exactly like that. The word file tells strace to report all file operations that the process makes. If no differences occur when running on python/ ipython, you can try dropping the -e file flag. Then you will see all system calls that the process makes. I haven't worked on windows like that, so this might not work at all there, but on linux at least, this should report to you everything that the process did. You could see there at least all the files that got opened, but more interesting things could be found there as well. If the outputs are identical, then the problem can be debugged further in python/ ipython. This would require basically what #Doug Blank suggested, but I'd also recommend investigating every name (variable) that gets touched. The self, _dlopen and mode names also sound like they might contain useful information.
Otherwise, do a dir(self) and dir(_dlopen) to see what other properties you could find there that might have gotten modified by IPYthon.
Try these first, and after that we can help you dig further.

Perhaps Python executable and IPython kernel use different manifest files which define loading policy?
Try appending the DLL path to sys.path in both cases.
Check admin rights (UAC) in both cases.
Use dependency walker to figure out dependencies of this DLL. Maybe the problem comes from dependencies?
Possibly your machine has multiple copies of this DLL?
Finally you can use Process Explorer to see the list of DLLs loaded in both cases and see any differences.
I'm working on a very similar issue:
ipython notebook & script difference - loading DLLs

Astroquery: AttributeError("'bytes' object has no attribute 'encode'")

I would like to run the NED query of the astroquery package.
from astroquery.ned import Ned
result_table = Ned.query_object("NGC 224")
this is given by the example in the Documentary.
I got the following issue:
TableParseError: Failed to parse NED result! The raw response can be found in
self.response, and the error in self.table_parse_error.
In self.table_parse_error I find:
AttributeError("'bytes' object has no attribute 'encode'")
I have no idea what has gone wrong.
Here are my versions:
Python 3.4.1 (default, May 19 2014, 17:23:49) [GCC 4.9.0 20140507 (prerelease)]
SciPy 0.14.0
Cython 0.20.1
OS posix [linux]
Numpy 1.8.1
IPython 2.1.0

This was raised as an issue on astroquery and is a confirmed bug:
https://github.com/astropy/astroquery/pull/343
it should be fixed shortly

Import error with ctypes in python27, segmentation fault

I am running python 2.7 on MacOSX, and started to get a segmentation fault when running anything related to ctypes. E.g.
>>> import ctypes
Segmentation fault
The error message being: "Python quit unexpectedly while using the _ctypes.so plugin." I use modules with ctype components on a daily basis, and haven't made any major changes to my python installation recently. The actually _ctypes.so file looks normal:
$ ls -l _ctypes.so
-rwxr-xr-x 1 root wheel 148720 Nov 12 22:12 _ctypes.so

This is a known bug in read only filesystems, in this thread you can find a patch along with more information.

pylint complains on py.test: "Module 'pytest' has no 'raises' member"

With the following code:
import pytest
def test_a():
with pytest.raises(Exception):
1/0
If I run pylint on it, it will make a complain that "raises" is not a member of module pytest:
E: 3,9:test_a: Module 'pytest' has no 'raises' member
Which is obviously not true. Any idea why pylint is making such a mistake? Is this a known bug?
py.test version:
> py.test --version
This is py.test version 2.2.3, imported from C:\Python27\lib\site-packages\pytest.pyc
PyLint version:
> pylint --version
No config file found, using default configuration
pylint 0.25.1,
astng 0.23.1, common 0.57.1
Python 2.7.2 (default, Jun 24 2011, 12:22:14) [MSC v.1500 64 bit (AMD64)]

You can silence this in a pylintrc file with:
ignored-classes=pytest

Last time I looked pylib does some heavy dynamic in low level python stuff, such as completely redefining the import code. It is very likely that this completely baffles pylint/astng, and prevents it from getting what is inside the pytest module: pylint/astng does not import the code it analyzes, it parses it, meaning that stuff which is dynamically initialized at import time will usually go unnoticed, which in turn generates false positives such as the one you report.
From there, you face the following choices:
use another unittest framework, less dynamic than py.test
silence the warnings / errors on your test code manually
use another linter which is happier than pylint on py.test (I'm interested to know how pychecker / pyflakes fare on that code)
write the astng plugin which will help astng grok the pylib tricks and submit it as a patch to the astng maintainers (and get extra credit from that)