I am building a simple Python tool that takes GPS coordinates from an external receiver via COM port and translates it to a JSON string like is returned by Google Geolocation API. The purpose is to replace the Google Geolocation provider URL in Firefox with a local one that serves this string back to the browser, thus implementing GPS-based location in my browser.
The GPS part is fine, but I am having trouble with the HTTP server to send the data to the browser. When the browser requests location from Google, it sends a POST like this:
POST https://www.googleapis.com/geolocation/v1/geolocate?key=KEY HTTP/1.1
Host: www.googleapis.com
Connection: keep-alive
Content-Length: 2
Pragma: no-cache
Cache-Control: no-cache
Content-Type: application/json
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36
Accept-Encoding: gzip, deflate, br
{}
This is my code to respond to it:
from BaseHTTPServer import BaseHTTPRequestHandler,HTTPServer
PORT_NUMBER = 8080
class myHandler(BaseHTTPRequestHandler):
def do_POST(self):
self.send_response(200)
self.send_header('Content-type','application/json; charset=UTF-8')
self.end_headers()
self.wfile.write('{"location": {"lat": 33.333333, "lng": -33.333333}, "accuracy": 5}')
return
try:
server = HTTPServer(('', PORT_NUMBER), myHandler)
print 'Started httpserver on port ', PORT_NUMBER
server.serve_forever()
except KeyboardInterrupt:
print 'Shutting down server'
server.socket.close()
So it works fine when I send a POST request from Curl that's empty, but not when the request is the one sent by the browser (i. e. '{}' in the body):
curl --data "{}" http://localhost:8080
> curl: (56) Recv failure: Connection was reset
curl --data "foo" http://localhost:8080
> curl: (56) Recv failure: Connection was reset
curl --data "" http://localhost:8080
> {"location": {"lat": 33.333333, "lng": -33.333333}, "accuracy": 5}
I'm not familiar with HTTP protocol or BaseHTTPServer at all. Why is this going wrong? How can I fix it.
The best I have been able to come up with is that I needed to do something with the posted content, so I just added these two lines to the beginning of the do_POST handler:
content_len = int(self.headers.getheader('content-length', 0))
post_body = self.rfile.read(content_len)
Related
I'm not sure how much of the code I can show, but the concept is simple. I am writing a python script that works with the TD Ameritrade API. I am getting a url for the portal from the API, and opening it in the browser. Next, I'm setting up a socket server to handle the redirect of the portal. Below is the code for the server:
serversocket = socket.socket(
socket.AF_INET, socket.SOCK_STREAM)
# get local machine name
host = "localhost"
port = 10120
serversocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
# bind to the port
serversocket.bind((host, port))
print("listening")
# queue up to 5 requests
serversocket.listen(5)
allData = ""
while True:
# establish a connection
try:
conn,addr = serversocket.accept()
except KeyboardInterrupt:
break
print("Got a connection from %s" % str(addr))
while True:
data = conn.recv(4096)
if not data:
print("done")
break
print(data.decode('utf-8', "ignore"))
conn.close()
When I go through the portal and get redirected, in the console I see the following:
Got a connection from ('127.0.0.1', 65505)
|,?2!c[N': [?`XAn] "::+/,0̨̩ / 5
jj localhost
3 + )http/1.1
ej\E<zpִ_<%q\r)+ - +
jj zz
However, if I were to copy the URL, open a new tab, paste it and go, I get the following (correct) response:
Got a connection from ('127.0.0.1', 49174)
GET /?code=<RESPONSE_TOKEN> HTTP/1.1
Host: localhost:10120
Connection: keep-alive
Cache-Control: max-age=0
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/81.0.4044.138 Safari/537.36 OPR/68.0.3618.125
Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-
exchange;v=b3;q=0.9
Sec-Fetch-Site: none
Sec-Fetch-Mode: navigate
Sec-Fetch-User: ?1
Sec-Fetch-Dest: document
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
When I go to the network overview of my browser, I see the following warning when trying to view the request headers: Provisional headers are shown
And the only difference between the http request from the redirect, and the request when I manually paste the url in is the initiator column in the network viewer is "oath" for the redirect, and "other" when manually pasted in.
I hope I've provided enough information and code. I can try to make a copy for reproducing if needed, but a TD Ameritrade Developer account would be needed to connect with the API.
Thanks in advance for any help. I've been researching for over 6 hours and wasn't able to find anything. Hopefully I didn't miss something obvious.
I think, socket is not required to handle an oauth redirect. Socket is for another kind of requeriments.
Also when you manually hit the redirect, a socket is not invoked. Just a simple http endpoint.
Try with this snippet which has the oauth code extract:
from urlparse import urlparse,parse_qsl
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
url = urlparse(self.path)
code = parse_qsl(url.query)['code']
Or this:
https://gist.github.com/willnix/daed2b57ab8d613f6bfa53c6d0b46fd3
You can get more snippets of simple http get endpoints here:
https://gist.github.com/search?q=def+do_GET+python&ref=searchresults
I'd like to call the Webiopi REST API from my angular application in a browser running on the Raspberry. As Webiopi HTTP server doesn't allow CORS request, I have created a proxy with apache that sends the Header add "Access-Control-Allow-Origin" "*" header.
This is working fine, however the call to the REST API throws many errors mainly because the browser sends an OPTIONS request to the server in case of a CORS request to check wether it is allowed or not. But the webiopi http handler doesn't handle the OPTIONS verb at all.
So I started to write it into the code myself with zero python experience. In the file python/webiopi/protocols/http.py I have added at the end:
def do_OPTIONS(self):
self.send_response(200,"ok")
self.send_header("Access-Control-Allow-Origin", "*")
self.send_header("Access-Control-Allow-Methods", "POST,GET,OPTIONS")
self.send_header("Access-Control-Allow-Headers", "Authorization")
self.send_header("Access-Control-Allow-Headers", "Content-Type")
self.end_headers()
Now it doesn't throw any error but doesn't give me the proper response to my GET request. It just stops after the OPTIONS. The request and response looks like this:
Request headers:
OPTIONS /GPIO/1/value HTTP/1.1
Host: localhost:8000
Connection: keep-alive
Access-Control-Request-Method: GET
Origin: http://192.168.1.108:51443
User-Agent: Mozilla/5.0 (X11; Linux armv7l) AppleWebKit/537.36 (KHTML, like Gecko) Raspbian Chromium/65.0.3325.181 Chrome/65.0.3325.181 Safari/537.36
Access-Control-Request-Headers: authorization
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: hu-HU,hu;q=0.9,en-US;q=0.8,en;q=0.7
Response headers:
HTTP/1.1 200 OK
Date: Fri, 23 Nov 2018 22:06:28 GMT
Server: WebIOPi/0.7.1/Python3.5
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: POST,GET,OPTIONS
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET,POST,OPTIONS
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
General (from chrome network tab):
Request URL: http://localhost:8000/GPIO/1/value
Request Method: OPTIONS
Status Code: 200 OK
Remote Address: [::1]:8000
Referrer Policy: no-referrer-when-downgrade
Where is my GET request? Why do I see only the OPTIONS which by the way I'm not initiating at all?
The request from angular:
this.http.get<number>(this.route+'GPIO/'+gpio+'/value').subscribe(result => {
resolve(result);
})
I had to enable all headers to the http server:
def do_OPTIONS(self):
self.send_response(200,"ok")
self.send_header("Access-Control-Allow-Origin", "*")
self.send_header("Access-Control-Allow-Methods", "*")
self.send_header("Access-Control-Allow-Headers", "*")
self.end_headers()
I'm trying to use Python 2 to send my own HTTP GET message to a web server, retrieve html text, and write it to an html file (no urllib, urllib2, httplib, requests, etc. allowed).
import socket
tcpSocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
tcpSocket.connect(('python.org', 80))
http_get = """GET / HTTP/1.1\r
Host: www.python.org/\r
Connection: keep-alive\r
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8\r
Upgrade-Insecure-Requests: 1\r
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36\r
Accept-Encoding: gzip, deflate, sdch\r
Accept-Language: en-US,en;q=0.8\r\n\r\n"""
tcpSocket.send(http_get)
m = tcpSocket.recv(4096)
tcpSocket.close()
print m
Output:
HTTP/1.1 301 Moved Permanently
Location: https://www.python.org//
Connection: Keep-Alive
Content-length: 0
Why does it return 301 when the location is apparently still the same? What message and to where should I send next to get the html content?
Thank you very much!
Your problem is that the url you are seeking doesn't serve over http://, but rather redirects to https://. To show that your code fundamentally works with a proper target I have changed your get request to
import socket
tcpSocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
tcpSocket.connect(('www.cnn.com', 80))
http_get = """GET / HTTP/1.1\r
Host: www.cnn.com/\r
Connection: keep-alive\r
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8\r
Upgrade-Insecure-Requests: 1\r
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36\r
Accept-Encoding: gzip, deflate, sdch\r
Accept-Language: en-US,en;q=0.8\r\n\r\n"""
http_get_minimum = """GET / HTTP/1.1\r\nHost: www.cnn.com\r\nConnection: close\r\n\r\n"""
tcpSocket.send(http_get_minimum)
m = tcpSocket.recv(4096)
tcpSocket.close()
and received
HTTP/1.1 200 OK
x-servedByHost: prd-10-60-168-42.nodes.56m.dmtio.net
Cache-Control: max-age=60
X-XSS-Protection: 1; mode=block
Content-Security-Policy: default-src 'self' http://.cnn.com: https://.cnn.com: .cnn.net: .turner.com: .ugdturner.com: .vgtf.net:; script-src 'unsafe-inline' 'unsafe-eval' 'self' *; style-src 'unsafe-inline' 'self' *; frame-src 'self' *; object-src 'self' *; img-src 'self' * data:; media-src 'self' *; font-src 'self' *; connect-src 'self' *;
Content-Type: text/html; charset=utf-8
Via: 1.1 varnish
Content-Length: 74864
Accept-Ranges: bytes
Date: Mon, 05 Oct 2015 00:39:54 GMT
Via: 1.1 varnish
Age: 170
Connection: close
X-Served-By: cache-iad2144-IAD, cache-sjc3129-SJC
X-Cache: HIT, HIT
X-Cache-Hits: 2, 95
X-Timer: S1444005594.675567,VS0,VE0
Vary: Accept-Encoding
UPDATE: Yes, there is extra functionality required from what you have presented to be able to request over HTTPS. There are some primary differences between http and https, however, beginning with the default port, which is 80 for http and 443 for https. Https works by transmitting normal http interactions through an encrypted system, so that in theory, the information cannot be accessed by any party other than the client and end server. There are two common types of encryption layers: Transport Layer Security (TLS) and Secure Sockets Layer (SSL), both of which encode the data records being exchanged.
When using an https connection, the server responds to the initial connection by offering a list of encryption methods it supports. In response, the client selects a connection method, and the client and server exchange certificates to authenticate their identities. After this is done, both parties exchange the encrypted information after ensuring that both are using the same key, and the connection is closed. In order to host https connections, a server must have a public key certificate, which embeds key information with a verification of the key owner's identity. Most certificates are verified by a third party so that clients are assured that the key is secure.
I had the same problem and changing port from 80 to 443 solved it.
I have to make a basic proxy which intercepts the requests of the browser and sends back a standard response. It doesn't work if I try to send the response in response to a https request. The code I'm using is:
#after the server socket starts listening
conn, addr = server.accept()
request = conn.recv(4096)
print request
conn.send(b"HTTP/1.1 200 OK\n\n<p>Hello</p>")
conn.close()
Now for https requests, e.g.:
Got request:
CONNECT www.google.com:443 HTTP/1.1
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:28.0) Gecko/20100101 Firefox/28.0
Proxy-Connection: keep-alive
Connection: keep-alive
Host: www.google.com
I've tried sending the same response, but the browser shows "The connection was interrupted". The response has certainly been sent though. Am I right in thinking that to overcome this, I need to get a ssl certificate and send the response through a ssl socket?
(I'm not asking this because I'm too lazy to try it out, but setting up the certificate should take some time so I'd like to verify with someone who knows before wasting hours on a wrong hypothesis)
I'm trying to connect to the twiiter API through a POST request as the docs say but I always get a 403 forbidden error.
This is my code. I'm using urlib2 in python 2.7:
def auth_API():
url = 'https://api.twitter.com/oauth2/token'
header = {}
values = {}
header['User-Agent'] = 'Mozilla/6.0 (Windows NT 6.2; WOW64; rv:16.0.1) Gecko/20121011 Firefox/16.0.1'
header['Authorization'] = 'Basic ' + B64BEARERTOKENCREDS
header['Content-Type'] = 'application/x-www-form-urlencoded;charset=UTF-8'
header['Accept-Encoding'] = 'gzip'
values['grant_type'] = 'client_credentials'
data = urllib.urlencode(values)
req = urllib2.Request(url, data, header)
try:
response = urllib2.urlopen(req)
response.read()
except urllib2.HTTPError as e:
print e
Checking the docs I found an example request wich is the same as mine:
Twitter example:
POST /oauth2/token HTTP/1.1
Host: api.twitter.com
User-Agent: My Twitter App v1.0.23
Authorization: Basic NnB1[...]9JM29jYTNFOA==
Content-Type: application/x-www-form-urlencoded;charset=UTF-8
Content-Length: 29
Accept-Encoding: gzip
grant_type=client_credentials
My request:
POST /oauth2/token HTTP/1.1
Content-Length: 29
Accept-Encoding: gzip
Connection: close
User-Agent: Mozilla/6.0 (Windows NT 6.2; WOW64; rv:16.0.1) Gecko/20121011 Firefox/16.0.1
Host: api.twitter.com
Content-Type: application/x-www-form-urlencoded;charset=UTF-8
Authorization: Basic NnB1[...]YTNFOA==
grant_type=client_credentials
Any idea with what could be wrong with this?
Regards.
PS: I know that there are some third party libs for this but I want to make it by myself.
I solved my problem, the error was with base64.encodestring() which adds an \n at the end of the string messing up the request.
Using base64.b64encode() instead worked fine.
Regards