So I was reading about these partial GET requests that would make a server timeout the connection after a while on this request. How would send a partial GET request?..
import socket, sys
host = sys.argv[1]
request = "GET / HTTP/1.1\nHost: "+host+"\n\nUser-Agent:Mozilla 5.0\n" #How could I make this into a partial request?
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host, 80))
s.sendto(request, (host, 80))
response = s.recv(1024)
How would I do this?
I think you confuse partial and incomplete requests:
partial: request for some part of a resource, that is a Range request like shown in the answer of falsetru. This will not cause a timeout but instead a response with code 206 and your requested part of the resource.
incomplete: your request is incomplete and cannot be processed by the server, thus it will wait for the rest of the request and timeout after a while if it does not get the request. In your question you already have such an incomplete request because you did not finish you request properly (it must end with \r\n\r\n and not a single \n). Other ways are just a TCP connect without sending any data or doing a POST request with a content-length and then not sending as much data as specified in the request header.
The HTTP headers ends too early. (\n\n should come after headers, before the contents)
import socket, sys
host = sys.argv[1]
request = "GET / HTTP/1.1\nHost: "+host+"\nUser-Agent:Mozilla 5.0\n\n"
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host, 80))
s.send(request)
response = s.recv(1024)
If you mean partial content retrieval, you can speicfy Range header:
"GET / HTTP/1.1\nHost: "+host+"\nUser-Agent:Mozilla 5.0\rRange: bytes=0-999\n\n"
NOTE
It should be \r\n not \n as line end, even if most (but not all) servers accept \n too.
Related
Here is what I am trying to do :
I send a DNS request (with dig or whatever), and I intercept it with socket in python. Then, I want to send this same request to the gateway, and wait for the response from the gateway.
import socket
#I redirected all DNS requests from port 53 to port 2525 so I am listening here
port = 2525
ip = '127.0.0.1'
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((ip, port))
#Gateway part
gateway = '192.168.1.1'
port_gateway = 9090
sock_gateway = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock_gateway.connect((gateway, port_gateway))
while 1:
data, addr = sock.recvfrom(512)
print(data)
#send the request from User to the gateway :
sock_gateway.sendto(data, (gateway, 53))
reply = sock_gateway.recv(512)
print(reply)
Here is the output : b'7\xa3\x01 \x00\x01\x00\x00\x00\x00\x00\x01\x06google\x03com\x00\x00\x01\x00\x01\x00\x00)\x10\x00\x00\x00\x00\x00\x00\x0c\x00\n\x00\x08\x9b+\xc1\xfa\xf4\xbc\x1c\x14'
This is the data from the DNS request I made, but then, I want to forward it to the gateway, and to listen to the response, but it doesn't get any response.
How can I do that ?
I suggest first use tools that are already built and debugged.
the DNS lookup tool i use is
DNS python
You will have to grab the request scan through the request for the data you need typically the name of server, then start up a new request using the DNS libary.
Trying to just forward a request without altering the raw data typically never works.
import dns.resolver as dns
gateway = '192.168.1.1'
port_gateway = 9090
sock_gateway = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock_gateway.connect((gateway, port_gateway))
while 1:
data, addr = sock.recvfrom(512)
## extract the request
request = extract_the_request_data_Fucntion(data)
gateway_reponse = dns.query(request, source=gateway_server_ip, source_port=port_gateway)
## construct responds code goes here
##function to parse the responds and send it back to the client
writing forwarders is not just extracting the data from the UDP packet and forwarding it, it almost always requires additional changes.
I have written a web server in python and I want to send HTTP response message codes:400 instead of the response "Website Coming Soon!" on any client-request, please tell how can I do this.
The Server Code is:
import socket
import re
HOST = "localhost"
PORT = 13555
listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
listen_socket.bind((HOST, PORT))
listen_socket.listen(1)
print ("Serving HTTP on port %s ..." % PORT)
while True:
client_connection, client_address = listen_socket.accept()
request = client_connection.recv(2048)
response = "Website Coming Soon!" #this response should be http response message code:400
http_response = "HTTP/1.1 200 OK\n"+"Content-Type: text/html\n"+"\n"+"<html><body>"+response+"</body></html>\n"
client_connection.sendall(http_response)
client_connection.close()
Try to get to know the protocol you're trying to speak :)
HTTP is fairly simple, all HTTP messages consist of 3 basic parts, of which the 3rd is optional:
The request or status line (first line)
The request headers, each on one line (or with some escaping spread over multiple), followed by an extra newline
The request body, which is optional for most requests, and for some responses.
What you want to do is change the "status line" in a response message. Since you want to send the 400 status code, the first line in your response should be
HTTP/1.1 400 Bad request
But there's two things wrong here:
You don't actually parse the request, so you can't really tell the client he's doing something wrong (all 4xx codes represent client errors)
Your sending the wrong message. Probably, what you want is something like 503 Service unavailable
Dive into the specs. They're really, really straight forward. And if you read it thoroughly, and start speaking HTTP the way it is intended, the world gets another tiny bit better ;)
In python 3.3, I want to get response headers from a youtube webpage. Using HTTP 1.0, the code below works fine:
import socket
PATH='/watch?v=GVIjOr98B7Q'
HOST='www.youtube.com'
buffer = bytes('HEAD %s HTTP/1.0\r\nHost: %s\r\n\r\n' %(PATH, HOST),'ascii')
PORT=80
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
s.sendall(buffer)
td=b''
while 1:
data = s.recv(1024)
td+=data
if not data:
break
However, if I replace HTTP 1.0 with HTTP 1.1:
buffer = bytes('HEAD %s HTTP/1.1\r\nHost: %s\r\n\r\n' %(PATH, HOST),'ascii')
and any other lines remain the same. It will stop at the while loop for a really long time (It is not looping but waiting for the end signal). Why does this happen?
The HTTP 1.1 keeps connections open unless you pass along the header Connection: close. See: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
If you want the connection to close immediately either use HTTP 1.0 or send the header Connection: close
The following code doesn't output anything(why?).
#!/usr/bin/python
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("www.python.org" , 80))
print s.recv(4096)
s.close()
What do I have to change in order to output the source code of the python website as you would see when you go to view source in a browser?
HTTP is request/response protocol. You're not sending any request, thus you're not getting any response.
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("www.python.org" , 80))
s.sendall("GET /\r\n") # you're missing this line
print s.recv(4096)
s.close()
Of course that will do the most raw HTTP/1.0 request, without handling HTTP errors, HTTP redirects, etc. I would not recommend it for actual usage beyond doing it as an exercise to familiarize yourself with socket programming and HTTP.
For HTTP Python provides few built in modules: httplib (bit lower level), urllib and urllib2 (high level ones).
You'll get a redirect (302) unless you use the full URL in your request.
Try this instead:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("www.python.org" , 80))
s.sendall("GET http://www.python.org HTTP/1.0\n\n")
print s.recv(4096)
s.close()
Of course if you just want the content of a URL this is far simpler. :)
print urllib2.urlopen('http://www.python.org').read()
I get the html with
def steal_html():
url='https://some_website.org'
with open('index.html', 'w') as FILE:
html = requests.get(url).text
FILE.write(html)
Following is the code which listens on a port for HTTP requests and sends the request packet to the server running on port 80, gets the response and sends the data back to the client. Now, everything is executing fine but the following line of code :
data = req_soc.recv(1024)
is taking too much time to execute and I have observed that, it takes long time to execute when it is going to/has received the last packet. I have also tried the same code using select.select() but the results are the same. Since I want to handle the data (raw) that is coming from the client and the actual HTTP server, I have no other choice than using sockets.
import socket
import thread
def handle_client(client):
data = client.recv(512)
request = ''
request += data
print data
print '-'*20
spl = data.split("\r\n")
print spl[0]
print spl[1]
if len(request):
req_soc = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
req_soc.connect(('localhost', 80))
req_soc.send(request)
response = ''
data = req_soc.recv(1024)
while data:
response += data
print 1
data = req_soc.recv(1024)
req_soc.close()
print response
if len(response):
client.send(response)
client.close()
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(('localhost', 4422))
server.listen(5)
print("Server is running...\n")
MSGLEN = 1024
while 1:
client, address = server.accept()
thread.start_new_thread(handle_client, (client, ))
Clients can do multiple commands (eg: GET) within one connection. You cannot wait for the client to send all the commands because based on what you return it could request more (eg: images of a web page). You have to parse the parts (commands) of request, find the boundary, forward that request to the server and write back the answer to the client. All this in a way that doesn't block on reading the client.
I'm not sure what's the best way to do this in python, but if you spend 5 minutes of googling you'll find a perfect HTTP proxy library.