Im trying to make a web proxy in python which is able to get the texts from the main server but not the images. The url http://gaia.cs.umass.edu/wireshark-labs/HTTP-wireshark-file1.html contains a line of text i am able to view in the browser and the url http://images.mid-day.com/images/2017/feb/15-salman-khan.jpg contains a image which i am not able to display in the browser. Im using Google Chrome. Below is my code. (I have hard-coded the hostname of the the image url for this post). Can anyone help me fix the problem.
from socket import *
client= socket(AF_INET, SOCK_STREAM)
proxy_port = 8880
client.bind(("", proxy_port ))
client.listen(10)
while 1:
client_connection, client_address = CLIENT.accept()
request = client_connection.recv(102400).decode()
if request.startswith("GET"):
try:
print(request)
web = socket(AF_INET, SOCK_STREAM)
web.connect(("images.mid-day.com", 80))
web.send(request.encode())
reply = web.recv(102400).decode()
print(reply)
client_connection.send(reply.encode())
web.close()
except:
print("illegal req")
client.close()
This is my get request from the browser:
You have only read 102400 bytes from the upstream server, but the image response is (at least) 567702 bytes. You should read until upstream shutdown connection, besides use sendall() to make sure all data has been sent:
reply = b''
while True:
data = web.recv(4096)
if not data:
break
reply += data
client_connection.sendall(reply)
Related
I am running a simple form input on my localhost:port using socket programming.
Currently, I have a form running on my chrome, just a text box on localhost:2333, I am able to see the text box input on my wireshark like this
The input message I typed is testesest.
After which, I put the <form action="http://localhost:2333"> such that the entered form data can flow back to my localhost:port. However, my 2nd r= recv(1024)is not receiving anything.
import socket
import sys
import os
Addr = ''
PORT = 2333
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((Addr, PORT))
s.listen()
The above is the standard part.
while(1):
try:
print("waiting for connection")
conn, address = s.accept()
print("New client connected from IP address {} and port number {}".format(*address))
received = conn.recv(1024)
#print("Request received")
#This is what i am hosting
#A webpage with a form
conn.send(b'\r\n')
#This is the webpage content
#The code will stuck here at recv
print("Waiting for form input from client")
r = conn.recv(1024)
print(r.decode())
print("Form input received")
print("HTTP response sent")
except KeyboardInterrupt:
conn.close()
s.close()
conn.close()
s.close()
break
Can I get some help please?
Input data sent via GET is attached to the URI (/?work=<data>), which is sent as a new request:
import socket
import sys
import os
Addr = ''
PORT = 2333
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((Addr, PORT))
s.listen()
while (1):
try:
print("waiting for connection")
conn, address = s.accept()
print(
"New client connected from IP address {} and port number {}".format(
*address
)
)
request = conn.recv(1024)
print("Request received")
method, uri, _ = request.decode().split(' ', 2)
print(method, uri)
#This is what i am hosting
#A webpage with a form
response = ""
conn.send(b'HTTP/1.1 200 OK\r\n')
conn.send(b'Content-Type: text/html\r\n')
conn.send(b'Host: localhost:2333\n')
conn.send(b'\r\n')
if uri == '/':
response = """<html>
<body><form action="http://localhost:2333/" method="GET">
<input type="text" name="work"></form></body>
</html>"""
elif uri.startswith('/?work'):
response = f"<html><body><h2>recevied: {uri[uri.find('/?work=')+7:]}</h2></body></html>"
conn.send(response.encode())
conn.send(b"\r\n")
print("Form input received")
#print("HTTP response sent")
except KeyboardInterrupt:
conn.close()
s.close()
#conn.close()
#s.close()
#break
Out:
waiting for connection
New client connected from IP address 127.0.0.1 and port number 55941
Request received
GET /?work=TestInput
<html><body><h2>recevied: TestInput</h2></body></html>
Form input received
waiting for connection
...
Note:
You might want to have a look at the protocol specs and/or use any existing library to get rid of this low level stuff.
Whenever we submit any form, browser makes new http request instead of using the existing connection, so you need to handle it in new http request/connection.
Another thing is, r = conn.recv(1024) is not letting the current connection close, that's why pressing enter in textfield is also not working.
I've set up a python client and server with socket in Python, that allows the server to send text to the client and I've been trying to extend it so that images can be sent to the client.
Server code:
import socket
#setup and bind server socket
s_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)#setup socket
s_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)#reuses same port (allows reconnection)
s_socket.bind(('192.168.178.52', 9001))
s_socket.listen(1)
#connects and prints clients data and send message
clientsocket, address = s_socket.accept()
print('Connection from {}'.format(address))
clientsocket.send(bytes('Welcome to the server', 'utf-8'))
#Loops for server to sent text data to client
while True:
m = input('Enter')
try:
file = open(m, 'rb')
b = file.read(2048)
clientsocket.send(b)
except:
clientsocket.send(bytes(m, 'utf-8'))
Client code:
import socket
import webbrowser
import os
import pyautogui
#setup and bind client socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.connect(('----------', 9001))#ip swapped for post
while True:
message = s.recv(2048)#recieves all messages sent with buffer size
if message:
txt = str(message)
with open('plswork.png', 'wb') as file:
file.write(message)
file.close()
The problem I'm having is that it will send the file over and create it perfectly fine, but only part of the image will load in when i open it (see image) I am pretty sure this is something to do with the buffer size however when I increase it, it wont recognise the file at all and I'll get an error trying to open the photo (preferably you would be able to send most photos). New to python sockets so any help would be appreciated!
(at the moment trying to send a pic of tux...)
https://i.stack.imgur.com/lBblq.png
I don't know the size of the file, but shouldn't you read the file until it is read completely and send data in chunks?
while True:
m = input('Enter')
try:
file = open(m, 'rb')
while True:
b = file.read(2048)
if not b:
break
clientsocket.send(b)
except:
clientsocket.send(bytes(m, 'utf-8'))
Client side had to be adapted as well.
Most network protocols add more information to simplify reception.
It could for example be a good idea, if you first send the number of bytes, that the welcome message contains, then the welcome message, then some indicator, that you will send a file, then some information, how many bytes you will send for the image and only then the bytes of the image
You will also find out, that it is complicated for the client to know what is the text message and what is part of the png file.
In fact if you would remove the input() command from the server and hard code a file name you might probably notice. that the welcome message and the png file could arrive combined at the client side. and it would have difficulties separating the two.
So to make your code robust, there's a lot of work to do.
I am writing a very simple Python socket program to read an HTML body from the server. If I create a HelloWorld.html file and open it with the designated host and port, I can open the file in my browser with the following server and read the message in the HTML file. However, I am having trouble reading in the same information from my client.
Server
from socket import *
serverSocket = socket(AF_INET,SOCK_STREAM)
host = '127.0.0.1'
port = 6789
serverSocket.bind((host,port))
serverSocket.listen(5)
print("server started...")
(connectionSocket, addr) = serverSocket.accept()
try:
message = connectionSocket.recv(1024).decode()
filename = message.split()[1]
f = open(filename[1:]) # Throws IOError if file not found
print(filename, "found")
connectionSocket.send("HTTP/1.0 200 OK\r\n".encode())
connectionSocket.send("Content-Type: text/html\r\n".encode())
connectionSocket.send(message.encode())
outputdata = f.read()
for i in range(0, len(outputdata)):
connectionSocket.send(outputdata[i].encode())
connectionSocket.send("\r\n".encode())
connectionSocket.close()
print(filename, "delivered")
except IOError:
print(filename, "NOT found")
connectionSocket.send('HTTP/1.0 404 NOT FOUND\r\n')
connectionSocket.close()
print("file not found message delivered")
serverSocket.close()
print("server closed...")
My server seems to be working. However, when my client tries to send the HTML object path to the socket and have the server read it, it does not seem to be picking up the message. I have just started socket programming in Python and I am trying to understand how the server receives the message from the socket. My initial thought was if I send the path of the HTML object (located in same directory as client and server) to the socket, the server should be able to read that information, open it, and return the information to the client.
Client
from socket import *
import sys
client = socket(AF_INET, SOCK_STREAM)
host = sys.argv[1]
port = sys.argv[2]
obj = sys.argv[3]
port = int(port)
client.connect((host, port))
print(client.getsockname())
request = obj
client.send("hello".encode())
client.send(request.encode())
s = client.recv(1024).decode()
print(s)
For my client, I accept the host,port, and the path to the HTML from a commandline argument and establish a connection.
When I run the browser for my HTML file with the url http://127.0.0.1:6789/HelloWorld.html, the server responds well. However, when I run the server and run the client with the command py capClient.py 127.0.0.1 6789 HelloWorld.html on the shell, it returns the filename = message.split()[1] IndexError: list index out of range error. I am assuming that this problem is coming from the server not being able to split the message coming in from the connectionSocket as an acceptable HTML object path.
What are some tips on modifying the client code to receive HTML file from servers?
This trouble is because you await message string is 'hello HelloWorld.html', but it is 'helloHelloWorld.html' and split() get you list ['helloHelloWorld.html'] where index 1 not exists.
f = open(filename[1:])
# must be replaced with [1:] give you a list, not string
f = open(filename)
# there needs encode()
connectionSocket.send('HTTP/1.0 404 NOT FOUND\r\n'.encode())
I can't seem to figure out why my code can't handle the exception of reporting an error if my web server does not contain a file. In the directory of my server I have the code for it and HelloWorld.html. For other files it should report an error. I'm looking through my code and it would seem that it is reading any file and just saying that its contents are blank without actually throwing an error that the file is not on the server. What is going on here?
#Tasks: Create a socket, bind to a specific address and port, send and receive an HTTP packet.
#Description: Web server should handle one HTTP request at a time. So the serve closes its TCP connection after response.
#Accept and parse the HTTP request, get the requested file from the server (i.e. HelloWorld.html), create a response
#message with the requested file and header lines, then send the response to the client.
#Error handling: If file not found then send HTTP "404 Not Found" Message back to client.
#import socket module: here we are using a low-level networking class from Python
from socket import *
#create the socket that belongs to the server.
#AF_INTET represents the address families and protocols.
#SOCK_STREAM represents the socket type
serverSocket = socket(AF_INET, SOCK_STREAM)
#Prepare a server socket
#Define variable for serverPort; we'll use the one in the helper page of the book
serverPort = 51350
#Define host address
serverHost = ''
#Bind the socket to the local host machine address and port
serverSocket.bind((serverHost, serverPort))
#Listen for TCP connections from the client
serverSocket.listen(1)
#Verify setup for receiving
print 'Server is ready to receive'
while True:
#Establish the connection
print 'Ready to serve...'
#When the server receive a request from the client it must establish a new connectionSocket and begin taking in the data.
connectionSocket, addr = serverSocket.accept()
try:
#Take data from connectionSocket and place in message.
#.recvfrom doesn't work because it expects data and return address variables.
message = connectionSocket.recv(1024)
#uncomment for header information
#print message
#parse the message
filename = message.split()[1]
f = open(filename[1:])
outputdata = f.read();
#Send one HTTP header line into socket
connectionSocket.send('HTTP/1.1 200 OK\r\n\r\n')
#Send the content of the requested file to the client
for i in range(0, len(outputdata)):
connectionSocket.send(outputdata[i])
connectionSocket.close()
except IOError:
#Send response message for file not found
connectionSocket.send('404 Not Found')
connectionSocket.close()
#Close client socket
serverSocket.close()
Perhaps you need "HTTP/1.1 404 Not Found\r\n\r\n" instead of "404 Not Found".
Also, you seem to close serverSocket within the loop, thus next accept() fails.
Following is the code which listens on a port for HTTP requests and sends the request packet to the server running on port 80, gets the response and sends the data back to the client. Now, everything is executing fine but the following line of code :
data = req_soc.recv(1024)
is taking too much time to execute and I have observed that, it takes long time to execute when it is going to/has received the last packet. I have also tried the same code using select.select() but the results are the same. Since I want to handle the data (raw) that is coming from the client and the actual HTTP server, I have no other choice than using sockets.
import socket
import thread
def handle_client(client):
data = client.recv(512)
request = ''
request += data
print data
print '-'*20
spl = data.split("\r\n")
print spl[0]
print spl[1]
if len(request):
req_soc = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
req_soc.connect(('localhost', 80))
req_soc.send(request)
response = ''
data = req_soc.recv(1024)
while data:
response += data
print 1
data = req_soc.recv(1024)
req_soc.close()
print response
if len(response):
client.send(response)
client.close()
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(('localhost', 4422))
server.listen(5)
print("Server is running...\n")
MSGLEN = 1024
while 1:
client, address = server.accept()
thread.start_new_thread(handle_client, (client, ))
Clients can do multiple commands (eg: GET) within one connection. You cannot wait for the client to send all the commands because based on what you return it could request more (eg: images of a web page). You have to parse the parts (commands) of request, find the boundary, forward that request to the server and write back the answer to the client. All this in a way that doesn't block on reading the client.
I'm not sure what's the best way to do this in python, but if you spend 5 minutes of googling you'll find a perfect HTTP proxy library.