I wrote a simple proxyserver code that connects to a client (currently the local machine) and retrieves data from a server (again running on the local machine) through HTTP GET requests that I incorporate into the curl command
Here's the curl command
curl -x http://localhost:12345 http://127.0.0.1:20000/1.txt
But despite making the connection, the curl command doesn't seem to extract the GET request. It doesn't seem to be a problem with the code I've written as it seems to work on other PCs, it just won't work on mine
Here's the data I received from the socket that is supposed to contain the GET request
listening to host: ('127.0.0.1', 58274)
GET http:/// HTTP/1.1
Host:
User-Agent: curl/7.55.1
Accept: */*
Proxy-Connection: Keep-Alive
And here's a snippet of the code I used to just make the the connection through the socket
import socket, sys, os
from thread import *
import operator
try:
listening_port = 12345
except KeyboardInterrupt:
print "\nUser Requested an interrupt"
print "Application Exiting ..."
sys.exit()
max_conn = 5
buffer_size = 40960
def main():
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(('',listening_port))
s.listen(max_conn)
except Exception, e:
print "\nUnable to initialize socket"
sys.exit(2)
while 1:
try:
conn, addr = s.accept()
print "Connection with client established"
print "listening to host: ", addr
print conn.recv(1024)
except KeyboardInterrupt:
print "\nProxy Server shutting down ..."
sys.exit(1)
s.close()
Does anybody have any idea as to why this is happening ? I'm new to sockets and networking in general, I'd really appreciate it if I could get some help.
Thanks
Related
I'm trying to make basic synchronous hello world with sockets
(server is supposed to send some message as answer for any message from client).
I bind localhost:5000 to socket
I'm trying to receive console input with sock.recv(4096)
I try to connect to socket from the console using curl localhost:5000, but I can't write to the console. Also, server sends message when i connect to it, but nothing more
here is the code:
import socket
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server_socket.bind(("localhost", 5000))
server_socket.listen()
def accept_connection(server_socket):
while True:
client_socket, addr = server_socket.accept()
print("connection from", addr)
send_message(client_socket)
def send_message(client_socket):
while True:
request = client_socket.recv(4096)
if request:
response = "request recieved\n".encode()
client_socket.send(response)
else:
break
print("done with sending stuff")
client_socket.close()
if __name__ == "__main__":
print("stuff_started")
accept_connection(server_socket)
server output:
C:\Users\USER\Desktop\stuff\pth>python testing.py
stuff_started
connection from ('127.0.0.1', 53053)
client output:
C:\Users\USER\Desktop\stuff\pth>curl localhost:5000
request recieved
As written in the comments, curl is used to HTTP servers and not for generic sockets.
You can use netcat for that:
nc 127.0.0.1 5000
or just plain old Python in your favorite shell like so:
> py -c "import socket; s = socket.create_connection(('localhost', 5000)); s.sendall(b'data'); print(s.recv(1024))"
b'request recieved\n'
I have a python network server code.
import socket
HOST, PORT = '', 5000
listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
listen_socket.bind((HOST, PORT))
listen_socket.listen(1)
print('Serving HTTP on port %s ...' % PORT)
while True:
client_connection, client_address = listen_socket.accept()
request = client_connection.recv(1024)
print(request)
http_response = """\
HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
<H1>Hello, World!</H1>
"""
client_connection.sendall(http_response.encode())
client_connection.close()
I have a client code that accesses the server.
import socket
HOST = '127.0.0.1'
PORT = 5000 # The port used by the server
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print "Socket successfully created"
s.connect((HOST, PORT))
s.sendall('GET /')
data = s.recv(1000)
print('Received', repr(data))
s.close
except socket.error as err:
print "socket creation failed with error %s" %(err)
It works fine with the expected output when I executed the server and client.
Socket successfully created
('Received', "'HTTP/1.1 200 OK\\nContent-Type: text/html; charset=utf-8\\n\\n<H1>Hello, World!</H1>\\n'")
Then, I tried to execute the python server using ngrok.
Session Status online
Account ...
Version 2.3.34
Region United States (us)
Web Interface http://127.0.0.1:4040
Forwarding http://d2fccf7f.ngrok.io -> http://localhost:5000
Using curl, I could access the webserver with ngrok.
> curl http://d2fccf7f.ngrok.io
<H1>Hello, World!</H1>
However, when I tried to use the same client code with minor modifications, the server doesn't seem to respond.
import socket
ip = socket.gethostbyname('d2fccf7f.ngrok.io')
print(ip)
HOST = ip
PORT = 5000
# the rest of the code is the same
I changed the PORT to 80 or 8080, but I had same results.
What might be wrong?
Might I suggest trying something like pyngrok to programmatically manage your ngrok tunnel for you? Full disclosure, I am the developer of it. Socket and other TCP examples are here.
From oguz ismail's hint, I made the following REQUEST header to make it work. I see that the Host information and blank line should be required.
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print "Socket successfully created"
s.connect((HOST, PORT))
header = '''GET / HTTP/1.1\r\nHost: d2fccf7f.ngrok.io\r\n\r\n'''
...
When I try to execute Python server socket program and client socket program (both running in same linux machine) in Linux terminal I don't get any error, server accepts connection and sends data to client successfully.
But when I execute the same programs in python IDLE I get "[Errno 111] Connection refused" error.
What is the difference in both execution?
I'm using serversock.bind(('',port#)) in server
and in client i'm using clientsock.connect(('localhost',port#))
Server.py
import socket
serversock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
host = ''
print host
port = 5800
serversock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
serversock.bind((host,port))
serversock.listen(2)
try:
while True:
csock,addr = serversock.accept()
print 'Recieved address from %s '% str(addr)
csock.send('hello')
csock.close()
except Exception,e:
print e
client.py
import socket
try:
c = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
host = 'localhost'
port = 5800
c.connect((host,port))
data = c.recv(1024)
print data
except Exception,e:
print e
finally:
c.close()
Server side you must use:
serversock.bind(('',port#)) # binds to port
serversock.listen(5) # actually listen to port and allow 5 incoming calls
conv = serversock.accept() # accept first incoming call
The connection will only be accepted after the listen call, before it, you have only bound a socket, but have not declared that you were ready to accept incoming calls, so they are refused.
With added code, another possible error cause is that you close connection (server side) immediately after sending data. That is bad: the close condition can destroy the socket before the data has actually been sent.
You should use a graceful shutdown:
server side:
csock.send('hello')
csock.shutdown(socket.SHUT_WR) # shutdown the socket
csock.read() # wait the close from peer
csock.close()
client side: you can leave the code as is in your use case you do not send anything to server, so when the client has finished reading it can safely close the socket
Anyway you should close serversock when everything is finished
try:
...
except ...:
...
finally:
serversock.close()
So i got to the web-server and i can display the info with the next code
#!/usr/bin/env python
import socket
import sys
HOST = 'www.inf.utfsm.cl'
GET = '/~mvaras/tarea1.php'
UA = 'tarea1'
PORT = 80
try:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except socket.error, msg:
sys.stderr.write("[ERROR] %s\n" % msg[1])
sys.exit(1)
try:
sock.connect((HOST, PORT))
except socket.error, msg:
sys.stderr.write("[ERROR] %s\n" % msg[1])
sys.exit(2)
sock.send("GET %s HTTP/1.1\r\nHost: %s\r\n\r\nUser-Agent: %s\r\n\r\n" % (GET, HOST, UA))
sock.send("POST Alexis Ahumada 17536441-2HTTP/1.1\r\n\r\nUser-Agent: tarea1\r\n\r\n")
data = sock.recv(1024)
string = ""
while len(data):
string = string + data
data = sock.recv(1024)
sock.close()
print string
sys.exit(0)
but the thing is the info i send (Alexis Ahumada 17536441-2) never writes on the server log (www.inf.utfsm.cl/~mvaras/tarea1.log) i'd want to know what i'm doing wrong. Any help is greatly appreciated i've really looked everywhere by now :(
change
TCP_IP = ('http://www.inf.utfsm.cl/~mvaras/tarea1.php')
to
TCP_IP = 'www.inf.utfsm.cl'
and then you will need send a HTTP request for "~mvaras/tarea1.php"
The trouble is that you are trying to communicate in the HTTP protocol over a TCP connection - HTTP is a much higher level protocol.
instead of using socket you need to use the requests library for this.
I am trying to connect to URL https://www.ssehl.co.uk/HALO/publicLogon.do in Python.
The simple solution using requests fails:
import requests
r = requests.get('https://www.ssehl.co.uk/HALO/publicLogon.do')
print r.text
with error
File "c:\Python27\lib\site-packages\requests\adapters.py", line 327, in send
raise ConnectionError(e)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='www.ssehl.co.uk', port=443): Max retries exceeded with url: /HALO/publicLogon.do (Caused by <class 'httplib.BadStatusLine'>: '')
so I tried to get the raw response from the server using library socket:
import socket #for sockets
import sys #for exit
#create an INET, STREAMing socket
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except socket.error:
print 'Failed to create socket'
sys.exit()
print 'Socket Created'
host = 'www.ssehl.co.uk';
port = 443;
try:
remote_ip = socket.gethostbyname(host)
except socket.gaierror:
#could not resolve
print 'Hostname could not be resolved. Exiting'
sys.exit()
#Connect to remote server
s.connect((remote_ip , port))
print 'Socket Connected to ' + host + ' on ip ' + remote_ip
#Send some data to remote server
message = "GET /HALO/publicLogon.do HTTP/1.1\r\n\r\n"
try :
#Set the whole string
s.sendall(message)
except socket.error:
#Send failed
print 'Send failed'
sys.exit()
print 'Message send successfully'
#Now receive data
reply = s.recv(4096)
print reply
will output:
Socket Created
Socket Connected to www.ssehl.co.uk on ip 161.12.7.194
Message send successfully
Reply:
after reply there is some garbage which I can't paste, however this is a sublime console screenshot:
Screenshot
Is there any way to get a 200 response from the server, just like a browser?
For some reason when you use either Python's built in stuff (urllib2, requests, httplib) or even command line stuff (curl, wget) over https the server spazzes out and gives an erroneous response.
However when you request the page over regular http, it works fine, for example:
import urllib2
print urllib2.urlopen('http://www.ssehl.co.uk/HALO/publicLogon.do').getcode()
prints out
>> 200
My guess is that their servers are configured wrong and your browser somehow deals with it silently.
It worked for me when I used port 80. Sooo:
port = 80;
There must be some error when using HTTPS servers thought Python...
Also, you are sending wrong request. You are not sending the hostname. Fixed request:
message = "GET /HALO/publicLogon.do HTTP/1.1\r\nHostname: %s\r\n\r\n"%host
So here is working code.
I think the problem exists, because port 443 is encrypted. And Python doesn't support encryption (probably).
You should use ssl.wrap_socket if you want support https.
See http://docs.python.org/2/library/ssl.html for details.