We Keep Coding

Python proxy server fails to connect to host

I'm making a python proxy server for a school assignment and I've got the code below. When I run it in my command prompt and attempt to connect to google, the code doesn't make it past connecting the server socket, but the page still connects. I honestly have no idea why it doesn't even go through the connection step. Thoughts?
EDIT: And yeah there's been other homework posts about this but none of them seem to have addressed the fact the sys.exit() on line 8 ends the script (to my knowledge anyway) and whenever we comment it out, the script still does not get past connecting the server socket and hits the "illegal request" exception.
from socket import *
from urllib2 import HTTPError #Used for 404 Not Found error
import sys
import requests
if len(sys.argv) <= 1:
print 'Usage : "python ProxyServer.py server_ip"\n[server_ip : It is the IP Address Of Proxy Server]'
#sys.exit(2)
#POST request extension
print 'Fetching webpage using POST'
r = requests.post('http://httpbin.org/post', data = {'key':'value'})
print 'Printing webpage body'
print r.text
print 'Creating and binding socket for proxy server'
# Create a server socket, bind it to a port and start listening
tcpServerSock = socket(AF_INET, SOCK_STREAM)
# Fill in start.
tcpServerSock.bind(('',8888))
tcpServerSock.listen(10) #the number is the maximum number of connections we want to have
# Fill in end.
while 1:
# Start receiving data from the client
print 'Ready to serve...'
tcpClientSock, addr = tcpServerSock.accept()
print 'Received a connection from:', addr
# Fill in start.
message = tcpClientSock.recv(4096) #receive data with buffer size 4096
# Fill in end.
print 'Printing message'
print message
# Extract the filename from the given message
print message.split()[1]
filename = message.split()[1].partition("/")[2]
print '\n'
print 'Printing file name'
print filename
fileExist = "false"
filetouse = "/" + filename
print '\n'
print 'Printing file to use'
print filetouse
print '\n'
try:
# Check whether the file exist in the cache
f = open(filetouse[1:], "r")
outputdata = f.readlines()
fileExist = "true"
# ProxyServer finds a cache hit and generates a response message
tcpClientSock.send("HTTP/1.0 200 OK\r\n")
tcpClientSock.send("Content-Type:text/html\r\n")
# Fill in start.
for x in range(0,len(outputdata)):
tcpClientSock.send(outputdata[x])
# Fill in end.
print 'Read from cache\n'
# Error handling for file not found in cache
except IOError:
if fileExist == "false":
# Create a socket on the proxyserver
# Fill in start.
print 'Creating server socket\n'
c = socket(AF_INET, SOCK_STREAM)
# Fill in end.
hostn = filename
#hostn = filename.replace("www.","",1)
print 'Printing host to connect'
print hostn
print '\n'
print 'Attempting to connect to hostn\n'
try:
# Connect to the socket to port 80
# Fill in start.
c.connect((hostn,80)) #port 80 is used for http web pages
# Fill in end.
# Create a temporary file on this socket and ask port 80
# for the file requested by the client
fileobj = c.makefile('r', 0)
fileobj.write("GET "+"http://" + filename + "HTTP/1.0\n\n")
# Show what request was made
print "GET "+"http://" + filename + " HTTP/1.0"
# Read the response into buffer
# Fill in start.
buff = fileobj.readlines() #reads until EOF and returns a list with the lines read
# Fill in end.
# Create a new file in the cache for the requested file.
# Also send the response in the buffer to client socket
# and the corresponding file in the cache
tmpFile = open("./" + filename,"wb") #creates the temp file for the requested file
# Fill in start.
for x in range(0, len(buff)):
tmpFile.write(buff[x]) #writes the buffer response into the temp file (cache?)
tcpClientSock.send(buff[x]) #sends the response saved in the buffer to the client
# Fill in end.
tmpFile.close()
except:
print "Illegal request\n"
else:
# HTTP response message for file not found
# Fill in start.
print 'File not found'
# Fill in end.
#404 not found error handling
except HTTPError as e:
print 'The server couldn\'t fulfill the request.'
print 'Error code: ', e.code
# Close the client and the server sockets
tcpClientSock.close()
# Fill in start.
tcpServerSock.close()
# Fill in end

I'm aware this question is old, and Jose M's assignment is probably long past due.
if len(sys.argv) <= 1: checks for an additional argument that needs to be passed, which is the IP of the server. Commenting out the exit essentially removes the error checking.
A fix for the code above is to change line 20 from this tcpSerSock.bind(('', 8888)) to this tcpSerSock.bind((sys.argv[1], tcpSerPort))
You must then call the script correctly python ProxyServer.py 127.0.0.1.

TCP multi threading web server in python

I have created a TCP multithreading webserver but it gives me the following
If the file is found
1. It says "int obj has no attribute encode"
2. Later, it will show list index out of range
If the file is not found
1. It does not display the html error message on the web browser
2. Later, it will show list index out of range
My server code is
import socket
import threading
import os
import sys
#to convert bytes into string
def bytestoString(stringToRead):
stringToRead = bytes.decode(stringToRead)
type(stringToRead)
return(stringToRead)
#to conver string into bytes
def stringToBytes(bytesToSend1):
bytesToSend1= str.encode(bytesToSend1)
type (bytes)
return(bytesToSend1)
#to retreive a file
def retrFile(name,sock):
message=sock.recv(1024)
message_string = bytestoString(message)
print(message_string)
fileName=message_string.split()[1]
fileName=fileName.split("/")[1]
#stringLength=len(fileName)
print(fileName)
if os.path.isfile(fileName):
print ('Ready to send the file................')
with open(fileName,'rb') as fileRead:
data= fileRead.read()
print (data)
exists_Bytes=stringToBytes('HTTP/1.1 200 OK\nContent-type=text/html')
sock.send(exists_Bytes)
for i in range(0 ,len(data)):
sock.send((data[i]).encode())
print('file sent succesfully')
sock.close()
else :
httpResponse=stringToBytes('HTTP/1.1 404 not Found')
sock.send(httpResponse)
#errorDisplayPage=stringToBytes('<html><title>404 Error</title><body>404 Error- Page cannot be found </body></html>')
sock.send(b'<html><title>404 Error</title><body>404 Error- Page cannot be found </body></html>')
print ('error message displayed')
sock.close()
def Main(serverPort):
#creating a server socket type TCP
serverSocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
localHost=socket.gethostname()
#binding the server to the client
serverSocket.bind((localHost,serverPort))
serverSocket.listen(5)
print ('***Server is ready to recieve***')
while True:
connectionSocket, addr = serverSocket.accept()
print ('got connection from:<', addr,'>')
t=threading.Thread(target=retrFile,args=('retrThread',connectionSocket))
t.start()
connectionSocket.send('thank you for connecting')
connectionSocket.close()
if __name__ == '__main__':
#getting server hostname and port number from the user
serverPort=int(sys.argv[1])
Main(serverPort)

You have opened your file as binary in your code:
with open(fileName,'rb') as fileRead:
data= fileRead.read()
However, later you try to do some funny things. Now, this differs in Python 2 and 3.You are using Python 3, so this following block is problematic:
for i in range(0 ,len(data)):
sock.send((data[i]).encode())
In python 2 you would iterate over individual chars and it would work fine. However, in python 3 you are iterating over a bytes object, so data[i] will be an int object.
Instead you can just:
sock.sendall(data)
Also according to wikipedia, the HTTP-protocol assumes that the status line is followed by an empty line. That is why you do not see the error message, and will not see the file.
So, postfix your status codes with "\n\n", so like: 'HTTP/1.1 404 not Found\n\n'. The same goes for the OK-message.

.recv function Socket programming TCP Server in Python

Im having trouble getting my very basic and simple TCP Server to properly work with http requests. This is what I have so far
from socket import *
import sys
serverSocket = socket(AF_INET, SOCK_STREAM)
serverSocket.bind(('', 4567))
serverSocket.listen(1)
while True:
print('Ready to serve...')
connectionSocket, addr = serverSocket.accept()
print("connected from: ", addr)
try:
message = connectionSocket.recv(1024)
filename = message.split()[1]
f = open(filename[1:])
outputdata = f.read()
connectionSocket.send("HTTP/1.1 200 OK\r\n")
for i in range(0, len(outputdata)):
connectionSocket.send(outputdata[i].encode())
connectionSocket.send("\r\n".encode())
connectionSocket.close()
except IOError:
connectionSocket.send("file not found")
serverSocket.close()
sys.exit()
The error comes from the open statement. I dont fully understand how this line of code's return value is organized.
message = connectionSocket.recv(1024)
I know that the return value is in bytes but when I try to use a fuction to turn it into a string like decode() i get errors as well
I have the .py file and the html file sitting in the same directory on my local machine and the way I test this is I just run this and open up a browser and type in
http://127.0.0.1:4567/helloworld.html
My code then promptly crashes after receiving the HTTP request.
Any and all help will be greatly appreciated!

There are numerous problems with your code and since you don't state what specific issues you are concerned about, here is what I see:
connectionSocket.send(outputdata[i].encode())
connectionSocket.send("\r\n".encode())
That appears to send a newline after every character you send back to the client.
Also, it doesn't deal with the client disconnecting because you're sending back invalid data.
Even if what you were trying to do didn't have these errors in it, you don't appear to be attempting to send back a valid http response.
https://www.w3.org/Protocols/rfc2616/rfc2616-sec6.html

Python WebServer breaks with .send message?

Inserting a .send to send an OK message apparently makes the rest of the code not work?
If I remove the client.send messages from the following code, it works. But with it, nothing happens in the browser, checking in Firefox, it says that the request went through, but there isn't any page displayed... it's just blank. Why would .send messages cause nothing to happen?
from socket import *
server = socket(AF_INET, SOCK_STREAM)
port = 12030
server.bind((gethostname(), port))
server.listen(1)
while True:
print 'Ready to serve'
conection, addr = server.accept()
try:
print 'Working'
message = conection.recv(1024)
conection.send("HTTP/1.0 200 OK\r\n")
conection.send("Content-Type:text/html\r\n")
filename = message.split()[1]
print "FILENAME", filename
f = open(filename[1:]) #cuts off the '/' in the request page
outputdata = f.read()
print "OUTDATA: ", outputdata
for i in range(0, len(outputdata)):
conection.send(outputdata[i])
conection.close()
except IOError:
print 'IO ERROR'
conection.send("404 NOT FOUND")
print message
conection.close()
except KeyboardInterrupt:
server.close()
conection.close()
break;
As seen here, it doesn't affect the data stream at all..
user ##$$ python webServer.py
Ready to serve
Working
FILENAME /HelloWorld.html
OUTDATA: <html>Hello World</html>
Ready to serve

Python web server - IndexError when splitting filename

I have successfully been able to access my web-server from a browser, downloaded a file on the server and properly viewed it with chrome. However, when the server standbys for approx. 20 seconds, it'll crash with an IndexError.
from socket import *
serverport = 972
serverSocket = socket(AF_INET, SOCK_STREAM)
serverSocket.bind(('', serverport))
serverSocket.listen(1)
print 'Standing by...'
while True:
#Establish the connection
connectionSocket, addr = serverSocket.accept()
try:
message = connectionSocket.recv(1024)
filename = message.split()[1]
f = open(filename[1:])
outputdata = f.read()
for i in range(0, len(outputdata)):
connectionSocket.send(outputdata[i])
print 'Success! File sent!'
connectionSocket.close()
except IOError:
errormessage = 'Error 404 - File not found'
connectionSocket.send(errormessage)
The output i get is the following:
Standing by..
Success! File sent! #sent everytime i request the webpage on the client localhost:80/helloworld.html
Traceback (most recent call last):
File "C:/Users/Nikolai/Dropbox/NTNU/KTN/WebServer/TCPServer.py", line 14, in <module>
filename = message.split()[1]
IndexError: list index out of range

This is probably the client closing the connection. When the connection is finished, an empty string '' is received.
''.split()[1] will fail with index out of range. My advice is to try with this patch:
message = connectionSocket.recv(1024)
if not message:
# do something like return o continue
As an aside, you should recv from your socket until you get the empty string. In your code, what happens if the request is larger than 1024? Something like this can be done:
try:
message = ''
rec = connectionSocket.recv(1024)
while rec:
rec = connectionSocket.recv(1024)
message += rec
if not message:
connectionSocket.close()
continue
filename = message.split()[1]
f = open(filename[1:])
outputdata = f.read()
for i in range(0, len(outputdata)):
connectionSocket.send(outputdata[i])
print 'Success! File sent!'
connectionSocket.close()
You should read Socket Programming HOWTO, specially the part of creating multithreaded servers, which is probably how you want to do yours :)
Hope this helps!