Why do the Theano tests fail with many "KnownFailureTest"s? - python
Theano is failing it's tests when I do:
python -c "import theano; theano.test();"
If these are known failures, shouldn't it still pass? IE when I test other libraries, KnownFailures sometimes trigger, but the overall test still passes with "OK" (but will still note the KnownFails and Skipped tests).
My guess is this is ok, and the test really is "passing", but since I'm doing a fresh install following the deeplearning.net tutorials, and I'm getting this error, I assume others might have this question as well, and a search on Google, and SO, isn't really helpful.
Forgive the error-code-dump, I am sure no one will need to read all through this, but it's here for reference if someone else has this question. Here are the errors at the end of the tests:
======================================================================
ERROR: test_none (theano.compile.tests.test_function_module.T_function)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/theano/compile/tests/test_function_module.py", line 42, in test_none
raise KnownFailureTest('See #254: Using None as function output leads to [] return value')
KnownFailureTest: See #254: Using None as function output leads to [] return value
======================================================================
ERROR: test002_generator_one_scalar_output (theano.sandbox.scan_module.tests.test_scan.TestScan)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/theano/sandbox/scan_module/tests/test_scan.py", line 474, in test002_generator_one_scalar_output
raise KnownFailureTest('Work-in-progress sandbox ScanOp is not fully '
KnownFailureTest: Work-in-progress sandbox ScanOp is not fully functional yet
======================================================================
ERROR: test003_one_sequence_one_output_and_weights (theano.sandbox.scan_module.tests.test_scan.TestScan)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/theano/sandbox/scan_module/tests/test_scan.py", line 512, in test003_one_sequence_one_output_and_weights
raise KnownFailureTest('Work-in-progress sandbox ScanOp is not fully '
KnownFailureTest: Work-in-progress sandbox ScanOp is not fully functional yet
======================================================================
ERROR: test_alloc_inputs2 (theano.scan_module.tests.test_scan.T_Scan)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/theano/scan_module/tests/test_scan.py", line 2844, in test_alloc_inputs2
"This tests depends on an optimization for scan "
KnownFailureTest: This tests depends on an optimization for scan that has not been implemented yet.
======================================================================
ERROR: test_infershape_seq_shorter_nsteps (theano.scan_module.tests.test_scan.T_Scan)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/theano/scan_module/tests/test_scan.py", line 3040, in test_infershape_seq_shorter_nsteps
raise KnownFailureTest('This is a generic problem with infershape'
KnownFailureTest: This is a generic problem with infershape that has to be discussed and figured out
======================================================================
ERROR: test_outputs_info_not_typed (theano.scan_module.tests.test_scan.T_Scan)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/numpy/testing/decorators.py", line 213, in knownfailer
raise KnownFailureTest(msg)
KnownFailureTest: This test fails because not typed outputs_info are always gived the smallest dtype. There is no upcast of outputs_info in scan for now.
======================================================================
ERROR: test_arithmetic_cast (theano.tensor.tests.test_basic.test_arithmetic_cast)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/theano/tensor/tests/test_basic.py", line 5583, in test_arithmetic_cast
raise KnownFailureTest('Known issue with '
KnownFailureTest: Known issue with numpy >= 1.6.x see #761
======================================================================
ERROR: test_abs_grad (theano.tensor.tests.test_complex.TestRealImag)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/numpy/testing/decorators.py", line 213, in knownfailer
raise KnownFailureTest(msg)
KnownFailureTest: Complex grads not enabled, see #178
======================================================================
ERROR: test_complex_grads (theano.tensor.tests.test_complex.TestRealImag)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/numpy/testing/decorators.py", line 213, in knownfailer
raise KnownFailureTest(msg)
KnownFailureTest: Complex grads not enabled, see #178
======================================================================
ERROR: test_mul_mixed (theano.tensor.tests.test_complex.TestRealImag)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/numpy/testing/decorators.py", line 213, in knownfailer
raise KnownFailureTest(msg)
KnownFailureTest: Complex grads not enabled, see #178
======================================================================
ERROR: test_mul_mixed0 (theano.tensor.tests.test_complex.TestRealImag)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/numpy/testing/decorators.py", line 213, in knownfailer
raise KnownFailureTest(msg)
KnownFailureTest: Complex grads not enabled, see #178
======================================================================
ERROR: test_mul_mixed1 (theano.tensor.tests.test_complex.TestRealImag)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/numpy/testing/decorators.py", line 213, in knownfailer
raise KnownFailureTest(msg)
KnownFailureTest: Complex grads not enabled, see #178
======================================================================
ERROR: test_polar_grads (theano.tensor.tests.test_complex.TestRealImag)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/numpy/testing/decorators.py", line 213, in knownfailer
raise KnownFailureTest(msg)
KnownFailureTest: Complex grads not enabled, see #178
======================================================================
ERROR: test_gradient (theano.tensor.tests.test_fourier.TestFourier)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/numpy/testing/decorators.py", line 213, in knownfailer
raise KnownFailureTest(msg)
KnownFailureTest: Complex grads not enabled, see #178
======================================================================
ERROR: theano.tensor.tests.test_opt.test_log_add
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/local/lib/python2.7/dist-packages/theano/tensor/tests/test_opt.py", line 1508, in test_log_add
raise KnownFailureTest(('log(add(exp)) is not stabilized when adding '
KnownFailureTest: log(add(exp)) is not stabilized when adding more than 2 elements, see #623
======================================================================
ERROR: Currently Theano enable the constant_folding optimization before stabilization optimization.
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/local/lib/python2.7/dist-packages/theano/tensor/tests/test_opt.py", line 3068, in test_constant_get_stabilized
"Theano optimizes constant before stabilization. "
KnownFailureTest: Theano optimizes constant before stabilization. This breaks stabilization optimization in some cases. See #504.
======================================================================
ERROR: test_dot (theano.tests.test_rop.test_RopLop)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/theano/tests/test_rop.py", line 277, in test_dot
self.check_rop_lop(tensor.dot(self.x, W), self.in_shape)
File "/usr/local/lib/python2.7/dist-packages/theano/tests/test_rop.py", line 191, in check_rop_lop
raise KnownFailureTest("Rop doesn't handle non-differentiable "
KnownFailureTest: Rop doesn't handle non-differentiable inputs correctly. Bug exposed by fixing Add.grad method.
======================================================================
ERROR: test_elemwise0 (theano.tests.test_rop.test_RopLop)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/theano/tests/test_rop.py", line 280, in test_elemwise0
self.check_rop_lop((self.x + 1) ** 2, self.in_shape)
File "/usr/local/lib/python2.7/dist-packages/theano/tests/test_rop.py", line 191, in check_rop_lop
raise KnownFailureTest("Rop doesn't handle non-differentiable "
KnownFailureTest: Rop doesn't handle non-differentiable inputs correctly. Bug exposed by fixing Add.grad method.
----------------------------------------------------------------------
Ran 2441 tests in 807.791s
FAILED (errors=18)
Thanks!
KnownFailureTest are a valid return value for nosetests. When Theano started, we where creating tests for features to implement and raised KnownFailureTest in them until we implement them. We do not do that anymore as we end up with to much questions from people about this. So this cause too much distraction. But we didn't changed the old tests that did that.
I just created an issue to change that: https://github.com/Theano/Theano/issues/2375
I do not know when it will be changed.
Related
SoxNotInstalled exception raised when importing pyaudioconvert
enter image description hereAs can be seen in the traceback the exception occurs when pyaudioconvert is imported. How can this be fixed? Traceback below: Traceback (most recent call last): File "C:\Users\320052863\PycharmProjects\Test\venv\lib\site-packages\pyaudioconvert\__init__.py", line 43, in <module> assert(bool_which('sox')) AssertionError During handling of the above exception, another exception occurred: Traceback (most recent call last): File "C:/Users/320052863/PycharmProjects/Test/hello.py", line 1, in <module> import pyaudioconvert File "C:\Users\320052863\PycharmProjects\Test\venv\lib\site-packages\pyaudioconvert\__init__.py", line 45, in <module> raise SoxNotInstalled() pyaudioconvert.SoxNotInstalled
As the error message says, you need to install SoX. It is also listed as a requirement for pyaudioconvert.
Gensim - AttributeError: Can't get attribute 'FixedCorpusWeight'
I've just installed Gensim and have run a test. 484 tests run successfully however I hit an error: ====================================================================== ERROR: testAddMorphemesToEmbeddings (gensim.test.test_varembed_wrapper.TestVarembed) Test add morphemes to Embeddings ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/franciskim/Downloads/gensim-2.0.0/gensim/test/test_varembed_wrapper.py", line 56, in testAddMorphemesToEmbeddings vectors=varembed_model_vector_file, morfessor_model=varembed_model_morfessor_file) File "/Users/franciskim/Downloads/gensim-2.0.0/gensim/models/wrappers/varembed.py", line 69, in load_varembed_format morfessor_model = morfessor.MorfessorIO().read_binary_model_file(morfessor_model) File "/Users/franciskim/anaconda/lib/python3.6/site-packages/morfessor/io.py", line 179, in read_binary_model_file model = pickle.load(fobj) AttributeError: Can't get attribute 'FixedCorpusWeight' on <module 'morfessor.baseline' from '/Users/franciskim/anaconda/lib/python3.6/site-packages/morfessor/baseline.py'> ---------------------------------------------------------------------- I can only find 1 other instance of this happening on the internet, but there are no answers to this yet.
rpy2.tests, no item called "None" on the search list
I am using Rpy2 and running the tests using $python -m "rpy2.tests". I got the following errors. I am using ubuntu 14.03, python 2.7 and trying to calling R in python using this package. rpy2 version: 2.6.0 - built against R version: 3-2.1--68531 - running linked to R version: R version 3.2.1 (2015-06-18) ....../tmp/tmpRWL4lU.py:17: UserWarning: ri.baseenv['eval'](ri.parse(rcode)) ..................................................................................................................................................................................x.........................................................................................................E Stderr: /usr/local/lib/python2.7/dist-packages/rpy2-2.6.0-py2.7-linux-x86_64.egg/rpy2/robjects/methods.py:80: UserWarning: Error in as.environment(where) : no item called "None" on the search list StrSexpVector((cls_packagename, ))) E...................................sssssssss................................E.s...s..s..... ====================================================================== ERROR: testRS4Auto_Type (robjects.tests.testMethods.MethodsTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/rpy2-2.6.0-py2.7-linux-x86_64.egg/rpy2/robjects/tests/testMethods.py", line 115, in testRS4Auto_Type robjects.methods.RS4)): File "/usr/lib/python2.7/dist-packages/six.py", line 617, in with_metaclass return meta("NewBase", bases, {}) File "/usr/local/lib/python2.7/dist-packages/rpy2-2.6.0-py2.7-linux-x86_64.egg/rpy2/robjects/methods.py", line 154, in __new__ cls_def = getclassdef(cls_rname, cls_rpackagename) File "/usr/local/lib/python2.7/dist-packages/rpy2-2.6.0-py2.7-linux-x86_64.egg/rpy2/robjects/methods.py", line 80, in getclassdef StrSexpVector((cls_packagename, ))) RRuntimeError: Error in as.environment(where) : no item called "None" on the search list Stderr: /usr/local/lib/python2.7/dist-packages/rpy2-2.6.0-py2.7-linux-x86_64.egg/rpy2/robjects/methods.py:80: UserWarning: Error in as.environment(where) : no item called "None" on the search list StrSexpVector((cls_packagename, ))) ====================================================================== ERROR: testRS4Auto_Type_nopackname (robjects.tests.testMethods.MethodsTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/rpy2-2.6.0-py2.7-linux-x86_64.egg/rpy2/robjects/tests/testMethods.py", line 122, in testRS4Auto_Type_nopackname robjects.methods.RS4)): File "/usr/lib/python2.7/dist-packages/six.py", line 617, in with_metaclass return meta("NewBase", bases, {}) File "/usr/local/lib/python2.7/dist-packages/rpy2-2.6.0-py2.7-linux-x86_64.egg/rpy2/robjects/methods.py", line 154, in __new__ cls_def = getclassdef(cls_rname, cls_rpackagename) File "/usr/local/lib/python2.7/dist-packages/rpy2-2.6.0-py2.7-linux-x86_64.egg/rpy2/robjects/methods.py", line 80, in getclassdef StrSexpVector((cls_packagename, ))) RRuntimeError: Error in as.environment(where) : no item called "None" on the search list ====================================================================== ERROR: test_Rconverter (ipython.tests.test_rmagic.TestRmagic) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/rpy2-2.6.0-py2.7-linux-x86_64.egg/rpy2/ipython/tests/test_rmagic.py", line 127, in test_Rconverter tuple(fromr_dataf_np.ix[col_i].values)) File "/usr/lib/python2.7/dist-packages/numpy/core/records.py", line 418, in __getattribute__ raise AttributeError("record array has no attribute %s" % attr) AttributeError: record array has no attribute ix ---------------------------------------------------------------------- Ran 383 tests in 3.719s FAILED (errors=3, skipped=12, expected failures=1) Any ideas to fix these three errors? Thanks,
Python tests pass on cmd and fail on VS2012
I'm setting up Python (CPython 3.4, 64-bit) on a new machine (Windows 10). I installed numpy & nose, and ran numpy.test() through the interpreter prompt to make sure everything's working as expected: Running unit tests for numpy NumPy version 1.9.0 NumPy is installed in C:\Python34\lib\site-packages\numpy Python version 3.4.2 (v3.4.2:ab2c023a9432, Oct 6 2014, 22:16:31) [MSC v.1600 64 bit (AMD64)] nose version 1.3.4 ---------------------------------------------------------------------- Ran 5162 tests in 36.783s OK (KNOWNFAIL=10, SKIP=20) So far so good, but when I do the same thing through PTVS on VS2012 (my team uses TFS for source control), there are errors and test failures (below): Running unit tests for numpy NumPy version 1.9.0 NumPy is installed in C:\Python34\lib\site-packages\numpy Python version 3.4.2 (v3.4.2:ab2c023a9432, Oct 6 2014, 22:16:31) [MSC v.1600 64 bit (AMD64)] nose version 1.3.4 ====================================================================== ERROR: test_basic (test_multiarray.TestResize) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python34\lib\site-packages\numpy\core\tests\test_multiarray.py", line 2850, in test_basic x.resize((5, 5)) ValueError: cannot resize an array that references or is referenced by another array in this way. Use the resize function ====================================================================== ERROR: test_freeform_shape (test_multiarray.TestResize) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python34\lib\site-packages\numpy\core\tests\test_multiarray.py", line 2880, in test_freeform_shape x.resize(3, 2, 1) ValueError: cannot resize an array that references or is referenced by another array in this way. Use the resize function ====================================================================== ERROR: test_int_shape (test_multiarray.TestResize) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python34\lib\site-packages\numpy\core\tests\test_multiarray.py", line 2862, in test_int_shape x.resize(3) ValueError: cannot resize an array that references or is referenced by another array in this way. Use the resize function ====================================================================== ERROR: test_obj_obj (test_multiarray.TestResize) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python34\lib\site-packages\numpy\core\tests\test_multiarray.py", line 2892, in test_obj_obj a.resize(15,) ValueError: cannot resize an array that references or is referenced by another array in this way. Use the resize function ====================================================================== ERROR: test_zeros_appended (test_multiarray.TestResize) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python34\lib\site-packages\numpy\core\tests\test_multiarray.py", line 2885, in test_zeros_appended x.resize(2, 3, 3) ValueError: cannot resize an array that references or is referenced by another array in this way. Use the resize function ====================================================================== ERROR: Ticket #950 ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python34\lib\site-packages\numpy\core\tests\test_regression.py", line 1272, in test_blasdot_uninitialized_memory x.resize((m, 0)) ValueError: cannot resize an array that references or is referenced by another array in this way. Use the resize function ====================================================================== FAIL: test_blasdot.test_dot_3args ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python34\lib\site-packages\nose\case.py", line 198, in runTest self.test(*self.arg) File "C:\Python34\lib\site-packages\numpy\core\tests\test_blasdot.py", line 54, in test_dot_3args assert_equal(sys.getrefcount(r), 2) File "C:\Python34\lib\site-packages\numpy\testing\utils.py", line 334, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: ACTUAL: 3 DESIRED: 2 ====================================================================== FAIL: test_1d (test_indexing.TestMultiIndexingAutomated) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python34\lib\site-packages\numpy\core\tests\test_indexing.py", line 940, in test_1d self._check_single_index(a, index) File "C:\Python34\lib\site-packages\numpy\core\tests\test_indexing.py", line 859, in _check_single_index self._compare_index_result(arr, index, mimic_get, no_copy) File "C:\Python34\lib\site-packages\numpy\core\tests\test_indexing.py", line 875, in _compare_index_result assert_equal(sys.getrefcount(arr), 3) File "C:\Python34\lib\site-packages\numpy\testing\utils.py", line 334, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: ACTUAL: 4 DESIRED: 3 ====================================================================== FAIL: test_multidim (test_indexing.TestMultiIndexingAutomated) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python34\lib\site-packages\numpy\core\tests\test_indexing.py", line 922, in test_multidim self._check_multi_index(self.a, index) File "C:\Python34\lib\site-packages\numpy\core\tests\test_indexing.py", line 836, in _check_multi_index self._compare_index_result(arr, index, mimic_get, no_copy) File "C:\Python34\lib\site-packages\numpy\core\tests\test_indexing.py", line 875, in _compare_index_result assert_equal(sys.getrefcount(arr), 3) File "C:\Python34\lib\site-packages\numpy\testing\utils.py", line 334, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: ACTUAL: 4 DESIRED: 3 ====================================================================== FAIL: test_dot_3args (test_multiarray.TestDot) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Python34\lib\site-packages\numpy\core\tests\test_multiarray.py", line 3285, in test_dot_3args assert_equal(sys.getrefcount(r), 2) File "C:\Python34\lib\site-packages\numpy\testing\utils.py", line 334, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: ACTUAL: 3 DESIRED: 2 ---------------------------------------------------------------------- Ran 5162 tests in 181.506s FAILED (KNOWNFAIL=10, SKIP=20, errors=6, failures=4) Press any key to continue . . . I ran the script that I wrote in VS through the command line, and the results were the same as running the test through the interpreter directly, so I'm confident there's something amiss in my VS/Python setup rather than the script itself. What could the problem be?
There's nothing amiss in your setup. It's just that numpy tests seem to be fragile in a sense that they don't tolerate debuggers. I'm not sure what's going on there, but it seems that tests involving sys.getrefcount, and semantics that depend on it being something specifically (usually just having a single reference to an array) are what's failing. You can repro this when running the interpreter directly by registering your own trace function with sys.settrace, e.g.: import sys def trace_func(f, e, a): return trace_func sys.settrace(trace_func) import numpy numpy.test() Note that you can run the script from VS without debugging it via Debug -> Start Without Debugging. This will give you results identical to running the interpreter directly.
Memory error in iGraph Python when adding large number of edges
Regarding this thread : http://goo.gl/uEyFua I am experiencing a quite similar issue with the following code, trying to import a large Graph : for (edge_id) in cursorSQL: L.add((edge_id[2], str(edge_id[1]))) g = igraph.Graph.TupleList(L) I get the following errors : Traceback (most recent call last): File "C.py", line 707, in __getitem__ return self._ids[item] KeyError: '184840900' During handling of the above exception, another exception occurred: Traceback (most recent call last): g = igraph.Graph.TupleList(L) File "C:\Python34\lib\site-packages\igraph\__init__.py", line 2473, in TupleList edge_list.append((idgen[item[0]], idgen[item[1]])) File "C:\Python34\lib\site-packages\igraph\datatypes.py", line 709, in __getitem__ self._ids[item] = next(self._generator) MemoryError Just to make it clear. This code works perfectly until number of edges is too large (~4millions.) Thanks.
Switched to python-64bits, and problem is now solved. Problem was the 2GB-per-process limit in win7 for python-32bits.