I want to use Entrez API in Python to grab some informations from Pubmed,this is just a example of Official Tutorial.
Entrez.email = "example#xx.com" # Always tell NCBI who you are
handle = Entrez.einfo()
record = handle.read()
print record
This is just a simple example.But sometimes I encounter this error,and don't know how to deal with it.
Traceback (most recent call last):
File "/Users/Rabbit/PycharmProjects/test_2.7/Biopython.py", line 25, in <module>
handle = Entrez.einfo()
File "/Library/Python/2.7/site-packages/biopython-1.66-py2.7-macosx-10.10-intel.egg/Bio/Entrez/__init__.py", line 258, in einfo
return _open(cgi, variables)
File "/Library/Python/2.7/site-packages/biopython-1.66-py2.7-macosx-10.10-intel.egg/Bio/Entrez/__init__.py", line 516, in _open
handle = _urlopen(cgi)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 431, in open
response = self._open(req, data)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 449, in _open
'_open', req)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1227, in http_open
return self.do_open(httplib.HTTPConnection, req)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1197, in do_open
raise URLError(err)
urllib2.URLError: <urlopen error [Errno 65] No route to host>
Can anyone tell me what's going on.Thanks in advance!
urllib2.URLError: <urlopen error [Errno 65] No route to host>
I never used Entrez but the above error indicates that urllib2 is unable to connect to the host. So check whether you have an active internet connection or your are offline.
This could be due to a limit set by NCBI API. They have a guideline for using the API. Please check that you aren't exceeding the limit. If you hit that limit, this error will occur regardless of your internet connectivity.
If you have >300 requests get a custom API key from NCBI. You can do that by signing up to NCBI website.
Also, if you are using this script to download files, it's better to use efetch() rather than einfo().
To avoid the network error related stop, I've been recommended to use the "try and except" functions in python.
Such as
try:
net_handle = Entrez.efetch(db="nucleotide", id=id_list[ax], rettype="fasta", retmode="text")
except (IOError):
do something to manage the error/move on to next step
It hasn't been a foolproof solution but it works most of the time.
Related
I am trying to download a dataset from a web API for my work project which requires using python. I used python 3.4 and the library urllib to open the request. This does not work:
from urllib import request
r = request.urlopen(SOME_URL)
This gives error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Anaconda3\lib\urllib\request.py", line 161, in urlopen
return opener.open(url, data, timeout)
File "C:\Anaconda3\lib\urllib\request.py", line 463, in open
response = self._open(req, data)
File "C:\Anaconda3\lib\urllib\request.py", line 481, in _open
'_open', req)
File "C:\Anaconda3\lib\urllib\request.py", line 441, in _call_chain
result = func(*args)
File "C:\Anaconda3\lib\urllib\request.py", line 1210, in http_open
return self.do_open(http.client.HTTPConnection, req)
File "C:\Anaconda3\lib\urllib\request.py", line 1184, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time,
or established connection failed because connected host has failed to respond>
But when I used RStudio with the same URL, it works:
dt = read.csv(SOME_URL)
This gives me the exact dataset I want.
For the project we want to keep a unified tech stack (only use python throughout the process), does anyone have idea why the URL can be open in R but not python? Is there any special set-up I need to configure for python?
Thanks
The following should do the job:
urllib2.urlopen(SOME_URL).read()
In my project we are trying to use CMIS to get the folder repository and I use python script to test it; below is the piece of code I used
from cmislib.model import CmisClient
client = CmisClient('http://localhost/CMIS/Service/servicedoc', 's', 's')
repo = client.defaultRepository
info = repo.info
for k,v in info.items():
print "%s:%s" % (k,v)
somefld = repo.getObject('idf_96_Z2CMIS')
props = somefld.properties
for k,v in props.items():
print "%s:%s" % (k,v)
This code works perfectly fine. However now the service is SSL enabled so (https//localhost/CMIS/Service/servicedoc) when I change the URL in CmisClient it is throwing the below error
c:\Python27>python.exe cmis.py
CMIS client connection to https://localhost/Cmis/Service/servicedoc
Traceback (most recent call last):
File "cmis.py", line 4, in <module>
repo = client.defaultRepository
File "c:\Python27\lib\site-packages\cmislib-0.5.1-py2.7.egg\cmislib\model.py",
line 179, in getDefaultRepository
File "c:\Python27\lib\site-packages\cmislib-0.5.1-py2.7.egg\cmislib\model.py",
line 206, in get
File "c:\Python27\lib\site-packages\cmislib-0.5.1-py2.7.egg\cmislib\net.py", l
ine 145, in get
File "c:\Python27\lib\urllib2.py", line 404, in open
response = self._open(req, data)
File "c:\Python27\lib\urllib2.py", line 422, in _open
'_open', req)
File "c:\Python27\lib\urllib2.py", line 382, in _call_chain
result = func(*args)
File "c:\Python27\lib\urllib2.py", line 1222, in https_open
return self.do_open(httplib.HTTPSConnection, req)
File "c:\Python27\lib\urllib2.py", line 1184, in do_open
raise URLError(err)
urllib2.URLError: <urlopen error [Errno 10054] An existing connection was forcibly closed by the remote host>
How do I use CMISClient library to connect to SSL enabled website. Thanks in advance.
I changed my URL to have the <> instead of localhost https://<>/Cmis/Service/servicedoc and it worked.
I'm working my way through The Programming Historian 2, a self-tutorial in coding for historians focusing on HTML and Python. I am attempting to complete the lesson Working with Files and Web Pages but am stuck on the Opening URLs with Python unit. I am running the following program:
# open-webpage.py
import urllib2
url = 'http://www.oldbaileyonline.org/print.jsp?div=t17800628-33'
response = urllib2.urlopen(url)
webContent = response.read()
print webContent[0:300]
Every time I run the program Komodo Edit 7 returns the following error message:
Traceback (most recent call last):
File "open-webpage.py", line 7, in <module>
response = urllib2.urlopen(url)
File "C:\Python27\lib\urllib2.py", line 126, in urlopen
return _opener.open(url, data, timeout)
File "C:\Python27\lib\urllib2.py", line 400, in open
response = self._open(req, data)
File "C:\Python27\lib\urllib2.py", line 418, in _open
'_open', req)
File "C:\Python27\lib\urllib2.py", line 378, in _call_chain
result = func(*args)
File "C:\Python27\lib\urllib2.py", line 1207, in http_open
return self.do_open(httplib.HTTPConnection, req)
File "C:\Python27\lib\urllib2.py", line 1180, in do_open
r = h.getresponse(buffering=True)
File "C:\Python27\lib\httplib.py", line 1030, in getresponse
response.begin()
File "C:\Python27\lib\httplib.py", line 407, in begin
version, status, reason = self._read_status()
File "C:\Python27\lib\httplib.py", line 365, in _read_status
line = self.fp.readline()
File "C:\Python27\lib\socket.py", line 447, in readline
data = self._sock.recv(self._rbufsize)
socket.error: [Errno 10054] An existing connection was forcibly closed by the remote host
I have attempted the program with a number of different urls, always with the same result. The guys at Komodo think the problem is to do with my university's firewall, because I access the internet through my university's proxy. The tech people here told me to change my default browser from RockMelt (chromium) to IE, because only IE is fully supported. I did so with no change and they have no other suggestions.
Can anyone suggest an alternate explanation for the error or a way to address the firewall problem? Thanks.
from poster.encode import multipart_encode
from poster.streaminghttp import register_openers
def picscrazy(str,int):
register_openers()
datagen, headers = multipart_encode({"imagefile[]": open(str, "rb")})
request = urllib2.Request("http://www.picscrazy.com/process.php", datagen, headers)
Str is the filename and the int is just another flag.
The code is to upload a file to a image hosting website .I am using poster Poster for the post requests. The program stops after the request statement and gives an error .I cant understand the error whether its a problem in my network or in the program.
Below is the traceback of the error:
Traceback (most recent call last):
File "C:\Documents and Settings\Administrator\Desktop\for exbii\res.py", line 42, in <module>
picscrazy(fname,1)
File "C:\Documents and Settings\Administrator\Desktop\for exbii\res.py", line 14, in picscrazy
print(urllib2.urlopen(request).read())
File "C:\Python25\Lib\urllib2.py", line 121, in urlopen
return _opener.open(url, data)
File "C:\Python25\Lib\urllib2.py", line 374, in open
response = self._open(req, data)
File "C:\Python25\Lib\urllib2.py", line 392, in _open
'_open', req)
File "C:\Python25\Lib\urllib2.py", line 353, in _call_chain
result = func(*args)
File "C:\Python25\lib\poster\streaminghttp.py", line 142, in http_open
return self.do_open(StreamingHTTPConnection, req)
File "C:\Python25\Lib\urllib2.py", line 1076, in do_open
raise URLError(err)
URLError: <urlopen error (10054, 'Connection reset by peer')>
If you can't display the header coming back from the server, then, your server has simply cut you off.
It may be your request is bad -- but that's unlikely.
It may be that you've exceeded bandwidth restrictions.
It may be that your requests appear to be a DDOS attack because they're happening too frequently.
I am running my application on my production server and it is giving me this error:
File "/usr/lib/python2.6/urllib2.py", line 391, in open
response = self._open(req, data)
File "/usr/lib/python2.6/urllib2.py", line 409, in _open
'_open', req)
File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain
result = func(*args)
File "/usr/lib/python2.6/urllib2.py", line 1161, in http_open
return self.do_open(httplib.HTTPConnection, req)
File "/usr/lib/python2.6/urllib2.py", line 1136, in do_open
raise URLError(err)
URLError: <urlopen error timed out>
BTW I understand the error. But this is not interesting. The interesting part is that when I run this on my local machine or test server every thing is working great. It is just too annoying.
Every where I am using the same OS:
ubuntu 10.04
What could be the possible reason? Any help will be appreciated.
Can you retrieve the URL in question with wget at your production server? This could be a firewall problem rather than a Python bug.
It seems reasonable that the production machine may be taking significantly longer than your test or local machines perhaps due to under-provisioning or excess load on the production system so you should be checking that you know what your timeouts are and you aren't accidentally setting the global timeout too agressively.
This is probably an issue with socket.setdefaulttimeout / socket.settimeout. The call to urllib2.urlopen should accept a timeout argument in python2.6. Try urllib2.urlopen(..., timeout=None) and see if that resolves things.
I'd also confirm the value of socket.getdefaulttimeout before making your urlopen call.