We Keep Coding

SageMath: running `make` ends with ImportError in dochtml phase

I am trying to install sagemath package in my macbook. I entered
./configure followed by make. Here are the last few
lines of output. Couldn't debug the cause.
[sagelib-8.7] Finished cleaning, time: 0.40 seconds.
[sagelib-8.7] if [ "$UNAME" = "CYGWIN" ]; then \
[sagelib-8.7] sage-rebase.sh "$SAGE_LOCAL" 2>/dev/null; \
[sagelib-8.7] fi
[sagelib-8.7]
[sagelib-8.7] real 19m37.851s
[sagelib-8.7] user 59m16.416s
[sagelib-8.7] sys 2m58.759s
cp /Users/satya/Downloads/SageMath/src/bin/sage-env-config /Users/satya/Downloads/SageMath/local/bin/sage-env-config
cd ../.. && sage-logger -p './sage --docbuild --no-pdf-links all html ' logs/dochtml.log
[dochtml] Traceback (most recent call last):
[dochtml] File "/Users/satya/Downloads/SageMath/local/lib/python2.7/runpy.py", line 163, in _run_module_as_main
[dochtml] mod_name, _Error)
[dochtml] File "/Users/satya/Downloads/SageMath/local/lib/python2.7/runpy.py", line 111, in _get_module_details
[dochtml] __import__(mod_name) # Do not catch exceptions initializing package
[dochtml] File "/Users/satya/Downloads/SageMath/local/lib/python2.7/site-packages/sage_setup/docbuild/__init__.py", line 60, in <module>
[dochtml] import sage.all
[dochtml] File "/Users/satya/Downloads/SageMath/local/lib/python2.7/site-packages/sage/all.py", line 98, in <module>
[dochtml] from sage.symbolic.all import *
[dochtml] File "/Users/satya/Downloads/SageMath/local/lib/python2.7/site-packages/sage/symbolic/all.py", line 3, in <module>
[dochtml] from sage.libs.pynac.pynac import I
[dochtml] File "sage/symbolic/expression.pxd", line 4, in init sage.libs.pynac.pynac (build/cythonized/sage/libs/pynac/pynac.cpp:30147)
[dochtml] File "sage/symbolic/expression.pyx", line 161, in init sage.symbolic.expression (build/cythonized/sage/symbolic/expression.cpp:74075)
[dochtml] ImportError: dlopen(/Users/satya/Downloads/SageMath/local/lib/python2.7/site-packages/sage/symbolic/ring.so, 2): Symbol not found: __ZNSt3__16vectorIN5GiNaC2exENS_9allocatorIS2_EEE11__vallocateEm
[dochtml] Referenced from: /Users/satya/Downloads/SageMath/local/lib/python2.7/site-packages/sage/symbolic/ring.so
[dochtml] Expected in: flat namespace
[dochtml] in /Users/satya/Downloads/SageMath/local/lib/python2.7/site-packages/sage/symbolic/ring.so
make[3]: *** [doc-html] Error 1
make[2]: *** [all-start] Error 2
real 19m41.468s
user 59m18.862s
sys 3m0.256s
***************************************************************
Error building Sage.
The following package(s) may have failed to build (not necessarily
during this run of 'make all-start'):
The build directory may contain configuration files and other potentially
helpful information. WARNING: if you now run 'make' again, the build
directory will, by default, be deleted. Set the environment variable
SAGE_KEEP_BUILT_SPKGS to 'yes' to prevent this.
make[1]: *** [all-start] Error 1
make: *** [all] Error 2
Any help would be appreciated.

Running make for SageMath proceeds in two steps: make build
followed by make doc.
In your case, the make build phase seems to have completed,
and there was an issue only in the make doc phase.
This means you probably already have a functional SageMath,
with the drawback that the documentation failed to build.
So you might decide to try and use it as is.
Following #John Palmieri, I would recommend posting to sage-devel
or sage-support to get help debugging the make failure. It will
be useful to specify the operating system on which it occurred
(what version of macOS), if the buid was started from a directory
obtained by git clone or from a source tarball, what exact
sequence of operations were run.
Also, on your next attempt, you might want to move the SageMath
folder out of the Downloads folder before you run make, since
a Sage installation cannot be moved after it has been started once
(if Sage is moved, one has to run make again before it can be used).

Building off Samuel's answer, a different thing you might want to try is simply to run make doc-clean and then make doc again. I have found that (on Mac) there is sometimes a weird burp I can never track down, and this usually gets past it.

How to install Bob on Python3?

I have installed all necessary dependencies on CentOS and Python levels and have thougt I'm near the finish but this is what I have got at the end. And now I have no idea how to make it work.
Installed /tmp/easy_install-3f76uq1w/bob.blitz-2.0.13/.eggs/bob.extension-2.4.5-py3.4.egg
In file included from /usr/include/python3.4m/pyatomic.h:8:0,
from /usr/include/python3.4m/Python.h:53,
from /tmp/easy_install-3f76uq1w/bob.blitz-2.0.13/bob/blitz/include/bob.blitz/capi.h:24,
from /tmp/easy_install-3f76uq1w/bob.blitz-2.0.13/bob/blitz/include/bob.blitz/cppapi.h:12,
from bob/blitz/api.cpp:9:
/usr/include/python3.4m/dynamic_annotations.h:472:3: error: template with C linkage
template <class T>
^
Traceback (most recent call last):
File "/usr/lib64/python3.4/distutils/unixccompiler.py", line 126, in _compile
extra_postargs)
File "/usr/lib64/python3.4/distutils/ccompiler.py", line 909, in spawn
spawn(cmd, dry_run=self.dry_run)
File "/usr/lib64/python3.4/distutils/spawn.py", line 36, in spawn
_spawn_posix(cmd, search_path, dry_run=dry_run)
File "/usr/lib64/python3.4/distutils/spawn.py", line 162, in _spawn_posix
% (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'gcc' failed with exit status 1
It seems this error somewhere inside Python or its libs. And unfortunately I'm not so familiar with C, Linux and Python to fix it by myself. Also I haven't found any workaround in google.
Does workaround for this really exist?
I know about Anaconda and using Bob via its environment. But I need to cythonize my python app for calling it from C. And I'm not sure that it will work fine.

This might be too late to answer.
The version of gcc you are using might be old and does not support new C++ features.
Also about your concern on not using conda, you can use cython with a conda installation as well.
Please take a look at https://www.idiap.ch/software/bob/install for the latest instructions on how to install Bob.

Is it possible to use the python3 bindings for VirtualBox?

I am trying to use the python 3 bindings to VirtualBox but there appears to be broken dependencies. It seems odd to me that this hasn't been fixed over the ~4 years that people have been having this issue. Perhaps I'm missing something obvious. It's been known to happen.
I have installed the virtualbox host modules, sdk, and extensions through my OS's pacakage manager. Then, through pip:
pip install pyvbox
The imports work:
from virtualbox import VirtualBox, Session, Manager, WebServiceManager
But then any attempt to instantiate anything results in an exception complaining about a missing vboxapi.
box = VirtualBox()
Traceback:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/usr/lib/python3.6/site-packages/virtualbox/library_ext/vbox.py", line 22, in __init__
manager = virtualbox.Manager()
File "/usr/lib/python3.6/site-packages/virtualbox/__init__.py", line 130, in __init__
with import_vboxapi() as vboxapi:
File "/usr/lib/python3.6/contextlib.py", line 82, in __enter__
return next(self.gen)
File "/usr/lib/python3.6/site-packages/virtualbox/__init__.py", line 45, in import_vboxapi
import vboxapi
File "/home/$USER/.eclipse/org.eclipse.platform_4.6.3_155965261_linux_gtk_x86_64/plugins/org.python.pydev_5.7.0.201704111357/pysrc/_pydev_bundle/pydev_import_hook.py", line 20, in do_import
module = self._system_import(name, *args, **kwargs)
ModuleNotFoundError: No module named 'vboxapi'
There is a vboxapi on PyPi, but it won't install as there is no code associated with it, nor any useful information on the PyPi page:
https://pypi.python.org/pypi/vboxapi
Here are a couple links to the valiant efforts of braver souls than I. It is not immediately clear to me which is the correct solution or if either are still relevant, given that they are from 3 and 4 years ago, respectively.
https://github.com/GreatFruitOmsk/vboxapi-py3
https://github.com/jbuergel/vboxapi-py3
Also from 3 years ago, word of a vboxapi.diff and intergration into vboxapi:
https://www.virtualbox.org/pipermail/vbox-dev/2014-April/012231.html

I'm the current maintainer of the pyvbox package.
The VirtualBox SDK already supports Python 3, I use Python 3.5 to develop the library. I recommend uninstalling and reinstalling the latest version of the SDK (which at the time of writing this is 5.1.22).
You can find the SDK on the VirtualBox downloads page. Unzip the archive and run the vboxapisetup.py file using your system Python with the following command:
python vboxapisetup.py install
You don't need to install this in any virtualenv, as pyvbox will search your system libraries in addition to virtualenv installations for better ease of use.
If you have problems using the pyvbox package after running these steps, please open an issue and include as much information as possible including the steps you took, OS, where your system Python is located, which version of VirtualBox & SDK you're using, and I'll help you as best I can.

Yes you can, it is possible, very tricky to setup but it work fine for me now (Ubuntu 18.04 / python3.6 / virtualbox 6.0) .
The error:
ModuleNotFoundError: No module named 'vboxapi'
mean that python3 does not find vboxapi module, now there is two methods to "force-install" the vboxapi package to python3:
First Method [easy]: Assuming pyvbox is already installed and work fine with python2.7, in that case you can simply copy the package from python2.7 dist-packages to python3 dist-package with:
sudo cp -r /usr/lib/python2.7/dist-packages/vboxapi /usr/lib/python3/dist-packages
Second method [more tricky]: Go to VirtualBox, then download the last Software Developer Kit (SDK), actually the 6.0.4
Unzip the archive and run the vboxapisetup.py file using Python3 with the following command:
sudo python3 vboxapisetup.py install
You will get this issue:
Traceback (most recent call last):
File "vboxapisetup.py", line 90, in <module>
main(sys.argv)
File "vboxapisetup.py", line 63, in main
raise Exception("No VBOX_INSTALL_PATH defined, exiting")
Exception: No VBOX_INSTALL_PATH defined, exiting
You may directly edit the current file vboxapisetup.py and replace line 57, from vboxDest = os.environ.get("VBOX_MSI_INSTALL_PATH", None) to vboxDest = "/usr/lib/virtualbox"
Then run agin:
sudo python3 vboxapisetup.py install
And now you will get something like that:
running install
running build
running build_py
copying vboxapi/__init__.py -> build/lib/vboxapi
running install_lib
creating /usr/local/lib/python3.6/dist-packages/vboxapi
copying build/lib/vboxapi/__init__.py -> /usr/local/lib/python3.6/dist-packages/vboxapi
copying build/lib/vboxapi/VirtualBox_constants.py -> /usr/local/lib/python3.6/dist-packages/vboxapi
byte-compiling /usr/local/lib/python3.6/dist-packages/vboxapi/__init__.py to __init__.cpython-36.pyc
byte-compiling /usr/local/lib/python3.6/dist-packages/vboxapi/VirtualBox_constants.py to VirtualBox_constants.cpython-36.pyc
running install_egg_info
Removing /usr/local/lib/python3.6/dist-packages/vboxapi-1.0.egg-info
Writing /usr/local/lib/python3.6/dist-packages/vboxapi-1.0.egg-info
which mean that we are ok with vboxapi package installation !
Now, let's try again to load virtualbox() inside python3:
from virtualbox import VirtualBox, Session, Manager, WebServiceManager
box = VirtualBox()
this probably will raise this new issue:
Traceback (most recent call last):
File "virtualbox_python3_test.py", line XX, in <module>
vbox = virtualbox.VirtualBox()
File "/usr/local/lib/python3.6/dist-packages/virtualbox/library_ext/vbox.py", line 22, in __init__
manager = virtualbox.Manager()
File "/usr/local/lib/python3.6/dist-packages/virtualbox/__init__.py", line 143, in __init__
self.manager = vboxapi.VirtualBoxManager(mtype, mparams)
File "/usr/local/lib/python3.6/dist-packages/vboxapi/__init__.py", line 989, in __init__
self.platform = PlatformXPCOM(dPlatformParams)
File "/usr/local/lib/python3.6/dist-packages/vboxapi/__init__.py", line 750, in __init__
import xpcom.vboxxpcom
File "/usr/lib/virtualbox/sdk/bindings/xpcom/python/xpcom/vboxxpcom.py", line 78, in <module>
raise Exception('Cannot find VBoxPython module (tried: %s)' % (', '.join(_asVBoxPythons),))
Exception: Cannot find VBoxPython module (tried: VBoxPython3_6m, VBoxPython3m, VBoxPython)
If you dig you will find a lot of questions (question 1,question 2,question 3, question 4 etc...) relative to this issue on the web ...
But according to my dig & research, if you are lucky (and have a Virtualbox built with python3 native support) you can try:
cd /usr/lib/virtualbox/
sudo cp VBoxPython3_5m.so VBoxPython3_6m.so
But if you got the following error:
cp: cannot stat 'VBoxPython3_5m.so': No such file or directory
It mean that you don't have native python3 support in Virtualbox...
This could be solved like this:
Go here and download the python3-virtualbox-5.2.16 binary package (we don't care about the VirtualBox version...)
Now open python3-virtualbox-5.2.16-lp150.4.11.1.x86_64.rpm archive, browse it to /./usr/lib/virtualbox/, then extract the file VBoxPython3_6m.so, then drop this file in your current working directory, after that from this directory you have to do:
sudo cp VBoxPython3_6m.so /usr/lib/virtualbox/
And now, you can use python3 binding for virtualbox !

Error running fully_connected_feed.py

When I run the fully_connected_feed.py code:
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/tutorials/mnist/fully_connected_feed.py
I get an error:
Traceback (most recent call last):
File "C:/Users/AppData/Local/Continuum/Anaconda3/Lib/site-packages/tensorflow/examples/tutorials/mnist/fully_connected_feed.py", line 277, in <module>
tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
File "C:\Users\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\platform\app.py", line 43, in run
sys.exit(main(sys.argv[:1] + flags_passthrough))
File "C:/Users/AppData/Local/Continuum/Anaconda3/Lib/site-packages/tensorflow/examples/tutorials/mnist/fully_connected_feed.py", line 222, in main
run_training()
File "C:/Users/AppData/Local/Continuum/Anaconda3/Lib/site-packages/tensorflow/examples/tutorials/mnist/fully_connected_feed.py", line 120, in run_training
data_sets = input_data.read_data_sets(FLAGS.input_data_dir, FLAGS.fake_data)
File "C:\Users\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\contrib\learn\python\learn\datasets\mnist.py", line 211, in read_data_sets
SOURCE_URL + TRAIN_IMAGES)
File "C:\Users\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\contrib\learn\python\learn\datasets\base.py", line 142, in maybe_download
gfile.Copy(temp_file_name, filepath)
File "C:\Users\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\lib\io\file_io.py", line 316, in copy
compat.as_bytes(oldpath), compat.as_bytes(newpath), overwrite, status)
File "C:\Users\AppData\Local\Continuum\Anaconda3\lib\contextlib.py", line 66, in __exit__
next(self.gen)
File "C:\Users\AppData\Local\Continuum\Anaconda3\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 469, in raise_exception_on_not_ok_status
pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.OutOfRangeError: Read fewer bytes than requested
How do I resolve this issue?

After doing the following, I was able to run the script without errors. The key for getting it to work for me, was the version of tensorflow installed has to match the tutorial code, otherwise there were exceptions. Although, I got a different exception than you, at first.
After installing tensorflow, check version. Details of this step may be different if you installed it pip or some other method:
$ conda list tensorflow
# packages in environment at /Users/agr/miniconda3/envs/tensorflow:
#
tensorflow 0.11.0 py35_0 conda-forge
Clone the git repo
$ git clone https://github.com/tensorflow/tensorflow.git
Inspect the tags available and checkout the release matching your install:
$ cd tensorflow
$ git tag -l -n1
...
$ git checkout v0.11.0
Run script!
$ cd examples/tutorials/mnist/
$ python fully_connected_feed.py
The key point being, run the script from here, not from the link you posted in the original question.

TL; DR
Something else is altering your files as you create them. Find the process and stop it.
Research
I've just run the demo with Windows 10, Python 3.5, tensorflow 0.12.0 with no errors. It is therefore something about your environment.
Looking at the actual line of the error, you are failing to read the required number of bytes from the open file. Going further up the stack you can see that CopyFile is actually trying to read all the bytes of a file into a string in this function. This starts by finding out the current file size and then trying to read all the bytes.
The problem is that the file size at the start of this process doesn't match the size by the end of the copy. In other words, something else has altered your file.
What next?
Your best bet is to try to find out what else is accessing your file. I suggest you use the techniques explained here to see what else has the file open as you are running the copy.

I encountered the same problem on Windows 2012 Server.
As suggested in the previous post, I downloaded and launched Process Monitor, then set the filter: "Path contains mnist" see the image. The datasets were downloaded and unpacked correctly, while running code both from Spyder and Jupyter.
I suspect that there is a race condition in the library code, i.e. missing synchronization between downloading and unpacking operations. As Process Monitor introduced additional delays, the datasets were sucessfully downloaded before the next operation started, hence the hazardous behavior was not observed.

pip tries to use git when git is not installed

I have a project folder that is a git repository (has a .git folder). When I use the command pip freeze, pip tries to use git. However, I don't have git installed on my system so this causes an error:
(env) PS C:\Users\eclaird\work\myproject> pip freeze
Cannot find command 'git'
Storing complete log in C:\Users\eclaird\pip\pip.log
(env) PS C:\Users\eclaird\work\myproject>
pip.log:
------------------------------------------------------------
C:\Users\eclaird\work\env\Scripts\pip-script.py run on 01/09/14 11:54:42
Cannot find command 'git'
Exception information:
Traceback (most recent call last):
File "C:\Users\eclaird\work\env\lib\site-packages\pip\basecommand.py", line 134, in main
status = self.run(options, args)
File "C:\Users\eclaird\work\env\lib\site-packages\pip\commands\freeze.py", line 73, in run
req = pip.FrozenRequirement.from_dist(dist, dependency_links, find_tags=find_tags)
File "C:\Users\eclaird\work\env\lib\site-packages\pip\__init__.py", line 180, in from_dist
req = get_src_requirement(dist, location, find_tags)
File "C:\Users\eclaird\work\env\lib\site-packages\pip\vcs\__init__.py", line 249, in get_src_requirement
return version_control().get_src_requirement(dist, location, find_tags)
File "C:\Users\eclaird\work\env\lib\site-packages\pip\vcs\git.py", line 151, in get_src_requirement
repo = self.get_url(location)
File "C:\Users\eclaird\work\env\lib\site-packages\pip\vcs\git.py", line 122, in get_url
[self.cmd, 'config', 'remote.origin.url'],
File "C:\Users\eclaird\work\env\lib\site-packages\pip\vcs\__init__.py", line 110, in cmd
command = find_command(self.name)
File "C:\Users\eclaird\work\env\lib\site-packages\pip\util.py", line 108, in find_command
raise BadCommand('Cannot find command %r' % cmd)
BadCommand: Cannot find command 'git'
Is there a way to disable the git integration in pip?
(pip 1.4.1,
Python 2.7.6)

Unfortunately, no. There is no config option to enable/disable backends.
Details, found by digging in the code:
Git module is always registered: In pip/install.py, the git module is imported. At the end of it, it registers itself, and will thus be queried whenever one of the schemes declared in ´git.Git´ matches the url of the dependency.
schemes = ('git', 'git+http', 'git+https', 'git+ssh', 'git+git', 'git+file')
Exception is not handled Funnily, freezing anticipates that an error might occur when determining the dependency url to be frozen. Excerpt from pip.FrozenRequirement:
try:
req = get_src_requirement(dist, location, find_tags)
except InstallationError:
logger.warn("Error when trying to get requirement for VCS system %s, falling back to uneditable format" % ex)
InstallationError inherits from PipError. Unfortunately, an exception of type ´BadCommand´ is raised, which inherits from ´PipError´.
So, aside from hacking the source: Nothing you can do. If you need this to work, you need to install git, hack the source, or simulate a git executable. If you go for the latter, for starters you need to fake ´git config remote.origin.url´, which is called (and fails) in pip.vcs.git.Git.get_url.
Hope that helps, even though it's not a yes. ;)