I'm trying to read a raw CGI response from an old website to simply parse the data and can't seem to read it with requests or urllib.
Python throws the following exception:
Exception has occurred: requests.exceptions.ConnectionError
('Connection aborted.', BadStatusLine('60.000\n',))
The data loads from 'http://powermeter.tradesmanmfg.ca:9900/basic.cgi?sid=0.487933' and works only in IE, but I'd like to parse this raw data directly in python. I've tried this but can't get any response without the 'BadStatusLine' response. My code is below: How can I read this into python to parse?
import requests
webpage = 'http://powermeter.tradesmanmfg.ca:9900/basic.cgi?sid=0.487933'
r = requests.get(webpage, stream=True)
print(r.raw.data)
Related
I am trying to make simple script in Python to request API GET responses from Bitcoin mining machines. I need to request a certain JSON key and value {"command":"summary"}. I will extract data from the JSON payload obtained with this to monitor some machines on Zabbix.
If I send the request with bash like this:
# echo '{"cmd":"summary"}' | timeout 1.5 nc IPADDRESS PORT
I will get the data I need, and I could even process it with bash using tr, sed and jq...
However if I send the request with my Python Script I always get "invalid msg". I would much rather do it with Python because I am learning and I can imagine my error is pretty dumb.
This is the python code.
#!/usr/bin/python3
import sys
import logging
import requests
import json
server = 'http://10.136.132.140:4028'
payload = {
"command":"summary"
}
jsonpayload = json.dumps(payload)
print(payload)
print(jsonpayload)
response = requests.get(server,
jsonpayload
)
print(response.json())
This won't work no matter what I do. I have tried using directly payload as params for requests.get to send the string only. Nothing works. Everytime I get the same invalid msg. I have tried typing response = requests.get('http://IPADDRESS:PORT', '{"command":"summary"}') directly, any combination possible of single quotes or double quotes... Nothing works.
I get three exceptions when I call it. The important tracebacks are the following:
http.client.BadStatusLine: STATUS=E,When=1608811559,Code=14,Msg=invalid cmd,Description=whatsminer v1.1
urllib3.exceptions.ProtocolError: ('Connection aborted.', BadStatusLine('STATUS=E,When=1608811559,Code=14,Msg=invalid cmd,Description=whatsminer v1.1'))
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.',
BadStatusLine('STATUS=E,When=1608811559,Code=14,Msg=invalid
cmd,Description=whatsminer v1.1'))
What am I doing wrong here?
Thanks for your help in advance!
If you want to send a json payload, try this:
server = '' # server url
payload = {
"command":"summary"
}
response = requests.get(server, json=payload)
i'm trying to create a script to download a test excel (.xlsx) file. The script does output a 'test'.xlsx file, but it contains the following error message instead of the actual file contents.
Bad Request
Your browser sent a request that this server could not understand.
Additionally, a 400 Bad Request error was encountered while trying to use an ErrorDocument to handle the request.*
I did manage to download the test file using urllib, however would like to use requests instead as I require authentication for my actual use case. Here's the code i used, do let me know what's wrong, thanks!
import requests
dls = "http://www.excel-easy.com/examples/excel-files/fibonacci-sequence.xlsx"
resp = requests.get(dls)
output = open('test.xlsx', 'wb')
output.write(resp.content)
output.close()
I am using a zeep client to consume a web service that is supposed to return me a file in base64binary format. I am using the following code.
try:
client = Client('https://host.com/xmlpserver/services/v2/CatalogService?wsdl')
response = client.service.downloadObject('/~fin_impl/INVITEM20.xdo','username','password')
print(response)
except zeepExceptions.Fault as fault:
print(fault.detail)
However, I am not sure what format is this response in and how to decode it.
b'PK\x03\x04\x14\x00\x08\x08\x08\x00o\xb8\x9cP\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0b\x00\x00\x00_report.xdo\x8dT\xcbN\xdc0\x14\xdd\xf3\x15V6\xacH\x02tQ\x01\x03*/i\xa4\x99\x01\r\x94v\x87\x8cs\x87\xb1\xf0K\xf65\xc9t\xd1o\xefu^\xa2\x14\xaaI\x94\x85\x8f\x8f}\xcf\xb99\xf6\xc9Y\xa3\x15{\x05\x1f\xa45l\xc2v\xf7\xf3r\x97\x81\x11\xb6\x92\xe69\x01\x11W{_w\xcfNwN<8\xeb\x91\xd1\x02\x13&\xd9\x1a\xd1\x1d\x15E;\xca\xad\xe7BA.\xac.l\xe3\x12\xe8\xb2\x8ex\xd4\x84j$\xd7\xf4\xe4\xf5!\xd1\x9f\x8b\x83\xb2\xdc/~\xcegwb\r\x9ag\x83\x86Iv\x90\x97\x19\xab8\xf2\xb9\xad#M2\xf4\x112\x16\x03\x9cKw\xcb=\xd7\x80D\x1dp\xe7m\x15\x05\xf8\x05\xe1\x93l%\xcd\x1e}\xdc\x08\xc9\xd5\x85\xd5\xda\x1a\xa2\x0c\x8b\x1en:\xd6\x1b\xe8\x9e\x87\x97ke\xeb\x04\x8a\x18\xd0\xeaK\xaa|a\rz\xabZ\x90\x93\xbe[\xf0\xdf\x03\xf8\xa1f\x8b\xddi\xeeq\t+\x0faM\x85\xb9\n\xc3L\xa2\xbe\x9b8\xdda\x8c\x9d\x8c\xa6X\xf4\xb4y\xf1\xfbz\xbax\x9c\xceog\xc5t\xf10\xbd\xbf\x9a\xe7M\xa5\xb3\xa2\'C\x10^:\xa4\x9e\xf4\x08Yu\xe0q\xc3L\xeb"\xacm\xdd\x0b\r\xd4>\xae"\xf4\x02?\xe6[\xa3\xa4\x81\xad\x98\x0e\xccL\x9a\x97\xa9Y#\xfdC\x9a\xca\xd6\xdb,\xe3\x11\xed2\x9a\x91\xdaY\xff\x98\xdb6\xea\xd2\x8a\xa8\xc1\xe06\x9b\'\xb7\xcb6\x80I\xd9V\x86y\xd8\x18\xb1\xf6\xd6\xd8\x18\xb6\xd5\xe5\xbc\xb4^\xe2f$\x1b\xeb5W\xffu\x11=O\x7fi\\rX~F\xef\xfe\x16T\xe7\x9b\xab\x06\xbf9\xf7\x89\xa41\xe4]N)}\xe49m\x9c\xb1\x80\x1b\x95d\x0e\x94\x99\x15m\xf1#\n\xfe\x9a\x94\xff\xa2\x12\\\x1dSa\xedp\x9c\x0c\xb2\x82\xe3a{\x04\xed\x14G\x08\xac\x82\x15\x8f\n\'Y\x1f\xbf\xc7\x83\x92\xde\xfd\xb2\xfc\xd2\x05\xf6-\x9b)\xfe\x94N\xe3\xbf\xd4.\xcco\xf1\x04\xe7MJ%n\x1c\xa9\x85F\x80\xca\x98\x8d\xe8"^\xa7\x86\xe2\x08\xf6\x1a\xde\xa1\x8a\x94\'\x9f#]\xadd\xe0O\n\xe6<\xa4\xf3\xda\xcb\x19\x8e"\x17(_\xc7\xd1\xab\x84\xfa\xa6\x8d\xf9\xdf\xc1(F\xcft\x8d\x15\xdd=v\xba\xf3\x07PK\x07\x08\xe8\xc9\xba\xb2-\x02\x00\x00\xfc\x04\x00\x00PK\x03\x04\x14\x00\x08\x08\x08\x00o\xb8\x9cP\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x18\x00\x00\x00INVITEM_20201004.xls.png\x03\x00PK\x07\x08\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00PK\x03\x04\x14\x00\x08\x08\x08\x00o\xb8\x9cP\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x17\x00\x00\x00INVITEM_20201004_en.xls\xed]\x07\\SI ........'
Please suggest, Thanks
How do I access the error stream using the Python requests library? For example, using HttpURLConnection in Java, I would do something like:
InputStream errorStream = conn.getErrorStream();
Is there a function like this with requests?
I'm looking for the error message in the response that was supplied by the source, NOT the error status e.g. Internal Server Error
requests won't raise an exception if HTTP error occurs, but you can get the error message in the response content. Example:
url = 'https://stackoverflow.com/does_not_exist'
r = requests.get(url)
print(r.status_code, r.reason)
print(r.text)
I'm trying to write a small program that will simply display the header information of a website. Here is the code:
import urllib2
url = 'http://some.ip.add.ress/'
request = urllib2.Request(url)
try:
html = urllib2.urlopen(request)
except urllib2.URLError, e:
print e.code
else:
print html.info()
If 'some.ip.add.ress' is google.com then the header information is returned without a problem. However if it's an ip address that requires basic authentication before access then it returns a 401. Is there a way to get header (or any other) information without authentication?
I've worked it out.
After try has failed due to unauthorized access the following modification will print the header information:
print e.info()
instead of:
print e.code()
Thanks for looking :)
If you want just the headers, instead of using urllib2, you should go lower level and use httplib
import httplib
conn = httplib.HTTPConnection(host)
conn.request("HEAD", path)
print conn.getresponse().getheaders()
If all you want are HTTP headers then you should make HEAD not GET request. You can see how to do this by reading Python - HEAD request with urllib2.