We Keep Coding

Pandas read_excel

I struggled for a few hours how to read an excel file with pd.read_excel where the path is a website address. I figured out that the link doesn't go directly to the file but just triggers downloading. Is there any easy way to solve it?
Part of code:
link_energy = 'http://unstats.un.org/unsd/environment/excel_file_tables/2013/Energy%20Indicators.xls'
df_energy = pd.read_excel(link_energy)
Error message:
XLRDError: Unsupported format, or corrupt file: Expected BOF record; found b'\n\n\n<!DOC'
Probably it's not a problem of pandas but my lack of skills how do do it.

For me works everything as expected in the following code:
import pandas as pd
link_energy = 'http://unstats.un.org/unsd/environment/excel_file_tables/2013/Energy%20Indicators.xls'
df_energy = pd.read_excel(link_energy)
df_energy
without errors on the following env:
The version of the notebook server is: 5.2.2
The server is running on this version of Python:
Python 3.6.3 | packaged by conda-forge | (default, Nov 4 2017, 10:10:56)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)]
Current Kernel Information:
Python 3.6.3 | packaged by conda-forge | (default, Nov 4 2017, 10:10:56)
Type 'copyright', 'credits' or 'license' for more information
IPython 6.2.1 -- An enhanced Interactive Python. Type '?' for help.

However I am not having access to your url posted.
but pd.read_excel won't work and you need to use pd.read_csv
import pandas as pd
df = pd.read_csv('https://cib.societegenerale.com/fileadmin/indices_feeds/CTA_Historical.xls')
Now you need to see the excel file what it contains what is the separator used, if there are any other values in any columns then it needs to be skipped in order to load and read useful data.

OpenCV-python and ArUco library estimatePoseSingleMarker() module has no attribute

working with OpenCV-python and the aruco library for a tracking project.
The following error is thrown when using
rvecs,tvecs = aruco.estimatePoseSingleMarker(corners, 0.1765, cameraMatrix, distcoefs)
error:
AttributeError: 'module' object has no attribute 'estimatePoseSingleMarker
others work fine
aruco.Dictionary_get(aruco.Dict_6x6_250)
aruco.DetectParameters_create()
any idea why its throwing that error?
>>> cv2.__version__
'3.2.0-dev'
odroid#odroid:~$ python --version
Python 2.7.12
odroid#odroid:~$ uname -a
Linux odroid 3.10.104-126 #1 SMP PREEMPT Tue Nov 29 22:24:16 UTC 2016
armv7l armv7l armv7l GNU/Linux
edit: its in the help documentation
help(cv2.aruco)
Help on module cv2.aruco in cv2:
NAME
cv2.aruco
FILE
(built-in)
FUNCTIONS
estimatePoseSingleMarkers(...)
estimatePoseSingleMarkers(corners, markerLength, cameraMatrix,
distCoeffs[, rvecs[, tvecs[, _objPoints]]]) -> rvecs, tvecs,
_objPoints

the module is aruco.estimatePoseSingleMarkers with an s at the end

pylint 1.6.3 reports E1101(no-member) for xml

We are using the ElementTree library in our project. This comes from the standard library, but for some reason pylint chokes on it. It always reports an E1101 on any reference on an ElementTree object. Here is a sample bit of code to demonstrate this:
import xml.etree.ElementTree as ET
test = ET.parse('test.xml')
config = test.find('.//configuration')
print(config)
for role in config.findall('.//role'):
print(role)
The output from pylint always throws an E1101:
$ pylint --rcfile=pylint.cfg -E test.py
************* Module orion-scripts.test
E: 7,12: Instance of 'int' has no 'findall' member (no-member)
But the program runs just fine:
$ python test.py
<Element 'role' at 0x1052dfe50>
<Element 'configuration' at 0x1052dfe10>
I found a number of solutions to similar problems, but so far none have worked for this one. I've tried:
Setting an ignored class with ignored-classes=xml.etree.ElementTree
Setting it as a generated member with generated-members=ET
Allowing any extension to run with unsafe-load-any-extension=yes
Nothing has worked and I'm currently just ignoring E1101 messages. However, this is a really bad option and I would much prefer to just ignore this particular class. If it helps, this is the version info for pylint:
$ pylint --version
No config file found, using default configuration
pylint 1.6.3,
astroid 1.4.7
Python 2.7.11 (default, Jan 22 2016, 08:28:37)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)]

Scipy test issue - pointer being freed was not allocated

I installed Numpy and Scipy on Mac os X (mountain lion) by compiling from the source.
The numpy test result is great. But there is an issue with scipy test.
In [1]: import scipy
In [2]: scipy.test()
Running unit tests for scipy
NumPy version 1.6.2
NumPy is installed in /usr/local/lib/python2.7/site-packages/numpy
SciPy version 0.11.0
SciPy is installed in /usr/local/lib/python2.7/site-packages/scipy
Python version 2.7.3 (default, Dec 4 2012, 00:30:38) [GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))]
nose version 1.2.1
.....................................................................................................................................................................................
F.FFPython(66098,0x7fff74ae5180) malloc: *** error for object 0x7fe9cb2e07b0: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6
Any one knows how to solve this problem?
It seems to be a bug thank to pv.
test_definition (test_basic.TestDoubleIFFT) ... FAIL
test_definition_real (test_basic.TestDoubleIFFT) ... ok
test_djbfft (test_basic.TestDoubleIFFT) ... FAIL
test_random_complex (test_basic.TestDoubleIFFT) ... FAIL
Python(66990,0x7fff74ae5180) malloc: *** error for object 0x7f9f53d8dec0: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
I am curious about the rest of the tests. It would be fine if this failed test is the only issue.
How can I skip this failed test and continue with the rest?

Run with scipy.test("full", verbose=2), storing the output to a file, and file a bug report:
http://projects.scipy.org/scipy/ (but try searching first if the issue is known)

How do I use the TarFile dereference flag in Python 2.4?

I wrote a script that collects all our log files to one tar file. The problem is that it doesn't contain soft link's data. I tried to use the dereference flag but it is not recognized for python 2.4.3.
I can't use the command line since there's a length limit and I want to support many log files.
user ~/Desktop]$python
Python 2.4.3 (#1, Jun 11 2009, 14:09:37)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-44)] on linux2
>>> import tarfile
>>> tarfile.TarFile('test.tar.gz', mode='w', dereference=True)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: __init__() got an unexpected keyword argument 'dereference'
>>>

According to the Python 2.4 docs dereference = True is supported. It appears to have been supported since tarfile was added in Python 2.3.
Unless you're on a non-Posix system (Windows) you must be doing it wrong. Post your code and the error you get with dereference = True so we can tell you what that is.
Also, by soft links I assume you mean symbolic links? Because that's what dereference = True allows to work.
Edit: I just looked at the code for tarfile on Python 2.4. It does not support the dereference parameter to the constructor, but it does seem to have the needed code to actually dereference (Checked against the source for Python 2.6). So,
import tarfile
tf = tarfile.TarFile('test.tar.gz', mode='w')
tf.dereference = True
should work. Please, update with your results.

Couldn't you just use the -T or --files-from option on tar?