I have a device that sends commands to a webserver. I've redirected those commands to my own server, with the goal of use the device to run another part of my system. In short, the device sends commands, I intercept them and use them.
The commands are sent to my server,but are not valid HTTP requests. I'm trying to use flask to read them with python, because I'd like to have these commands go straight into another web app.
Note that I can't change how the commands are sent.
Using sockets, I can read the data. For instance, here is a version of the data sent via socket (data is meaningless, just for illustration):
b'#123456#A'
In constrast, as HTTP message looks like this:
b'POST / HTTP/1.1\r\nHost: 123.123.123.123:12345\r\nRequest info here'
I know how to filter these (they always start with a #). Can I hook flask to let me handle these request differently, before they are parsed as HTTP requests?
Update: The code I used to read the requests, in case it provides context:
import socket
host = ''
port = 5000
backlog = 5
size = 1024
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((host,port))
s.listen(backlog)
while 1:
client, address = s.accept()
data = client.recv(size)
print("Request:")
print(data)
print("\n\n")
if data:
client.send(data)
client.close()
Related
I would like to know if anyone was able to change this code so that instead of it running on a wifi network it ran on http. I did all i could but i do not think there are any tutorials on how to preform this changing.
Server:
import socket
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host = ''
port = 1234
server.bind((host, port))
server.listen(5)
run = True
client, addr = server.accept()
print('Got connection from',addr)
while run:
try:
data = input('>>>')
client.send(data.encode('UTF-8'))
msg = client.recv(1024)
print(msg.decode('UTF-8'))
except ConnectionResetError:
print('Client lost server connection')
print('Trying to connect . . .')
client, addr = server.accept()
print('Got connection from',addr)
Client:
import socket
import os
server = socket.socket()
host = '127.0.0.1'
port = 1234
run = True
server.connect((host,port))
while run:
msg = server.recv(1024)
os.popen(msg.decode('UTF-8'))
server.send('Client online . . .'.encode('UTF-8'))
I think you are confused on what TCP and HTTP is. TCP is part of layer 4 of the OSI model. HTTP is layer 7 of the OSI model.
Http is a protocol/standard used to exchange data with another Http enabled endpoint. Http is built on top of TCP. Http uses TCP to send data. The server has a TCP socket listening for data and once it receives data it then passes it along to another "process/program/line of code" to then parse that data using Http standards.
If you are wanting to use http to send/receive data you will need a web framework like Flask, Django or Starlette. They are Http servers with added features to allow you to code what you want to do with the data rather than handling the lower level stuff of parsing the http data being sent over TCP.
Http is a HUGE protocol/standard. If you try to do the parsing from scratch it is a big project to undertake.
Here is what I am trying to do :
I send a DNS request (with dig or whatever), and I intercept it with socket in python. Then, I want to send this same request to the gateway, and wait for the response from the gateway.
import socket
#I redirected all DNS requests from port 53 to port 2525 so I am listening here
port = 2525
ip = '127.0.0.1'
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((ip, port))
#Gateway part
gateway = '192.168.1.1'
port_gateway = 9090
sock_gateway = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock_gateway.connect((gateway, port_gateway))
while 1:
data, addr = sock.recvfrom(512)
print(data)
#send the request from User to the gateway :
sock_gateway.sendto(data, (gateway, 53))
reply = sock_gateway.recv(512)
print(reply)
Here is the output : b'7\xa3\x01 \x00\x01\x00\x00\x00\x00\x00\x01\x06google\x03com\x00\x00\x01\x00\x01\x00\x00)\x10\x00\x00\x00\x00\x00\x00\x0c\x00\n\x00\x08\x9b+\xc1\xfa\xf4\xbc\x1c\x14'
This is the data from the DNS request I made, but then, I want to forward it to the gateway, and to listen to the response, but it doesn't get any response.
How can I do that ?
I suggest first use tools that are already built and debugged.
the DNS lookup tool i use is
DNS python
You will have to grab the request scan through the request for the data you need typically the name of server, then start up a new request using the DNS libary.
Trying to just forward a request without altering the raw data typically never works.
import dns.resolver as dns
gateway = '192.168.1.1'
port_gateway = 9090
sock_gateway = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock_gateway.connect((gateway, port_gateway))
while 1:
data, addr = sock.recvfrom(512)
## extract the request
request = extract_the_request_data_Fucntion(data)
gateway_reponse = dns.query(request, source=gateway_server_ip, source_port=port_gateway)
## construct responds code goes here
##function to parse the responds and send it back to the client
writing forwarders is not just extracting the data from the UDP packet and forwarding it, it almost always requires additional changes.
Sorry for the long post. I am writing a TCP socket server which communicates with two different clients (client1 and client2). The server receives request from client1 and on receiving the request, the server sends data to client2 and in turn client2 responds with required data.
In detail, client1 is an android device requesting some location information from the server. On receiving the request, the server communicates with the computer (client2) that hosts the location information and the information is given from client2 to server. The received information is further communicated from server to client1 (android device).
For implementing this, I have the following steps written:
1. Since there may be number of android devices (client1) connecting to the server, there is a thread handling the connections with client1 over a port (port1).
2. Once the server gets a request from android device, it communicates with location client (client2) and get the required data.
3. Communication with client2 happens over same port and that works fine.
4. After receiving the data from client2, the server has to transfer the obtained data back to client1.
I am facing problem in step4. ie, I created a socket inside the function that transfers the data to client2 in a different port (port2) and transferred the data. But I am getting the following error "Address already in use".
To avert this, I tried transferring the information over the socket connection (created on port1) from within the function. The data transfer is failing, that is client2 is not receiving the data sent over the socket.
class ClientThread(Thread):
def __init__(self,ip,port,obj_socket_connect):
Thread.__init__(self)
self.obj_socket_connect = obj_socket_connect
def run(self):
read_data = conn.recv(2048)
if read_data == "loc_info_req":
self.obj_socket_connect.request_for_location()
Another class that communicates the data from client2 and send the data to client1.
class socket_connect:
def __init__(self):
self.write_android_request_port = Port2
def request_for_location(self):
# Information requested to client 2
self.read_loc_data(c_write)
def read_loc_data(self,c_read):
read_data = c_read.recv(15)
# Data processing for sending the info to client1
self.write_to_android_dev_data(read_data)
# Writing the required information to android on another port
# The problem of "Address already in use occurs here"
def write_to_android_dev_data(self,data_to_write):
s2 = socket.socket() # Create a socket object
s2.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
host = socket.gethostname()
s2.bind(('', self.write_android_request_port))
s2.listen(5)
c_write,address = self.s2.accept()
c_write.send(data_to_write)
s2.close()
The main function is given below:
if __name__ == "__main__":
obj_socket_connect = socket_connect()
print "Read write ports assigned"
# Multithreaded Python server : TCP Server for android devices connection
TCP_PORT = Port1
BUFFER_SIZE = 20
tcpServer = socket.socket()
tcpServer.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
tcpServer.bind(('', TCP_PORT))
threads = []
# Thread is created everytime when new Android device is connected (client1)
while True:
tcpServer.listen(4)
print "Multithreaded Python server : Waiting for connections from TCP clients..."
(conn, (ip,port)) = tcpServer.accept()
# Calling the thread function and creating the object
newthread = ClientThread(ip,port,obj_socket_connect)
newthread.start()
threads.append(newthread)
for t in threads:
t.join()
All the above pieces of code, belongs to same file and this acts as the TCPserver.
Client1 : (Android device)
The android device sends the request through writing on the Port1, sends the string "loc_info_req". This works fine as the information is available in server after request. On receiving the request the server communicates with client 2 (another computer)
Client 2 is another computer sending data on request.
Server - Client2 data transfer works fine, but the problem is when data has to be transferred back to client1 from server.
For receiving the data on Android device, I have the following lines of code:
# When tried to receive the data on Port1, the information is not getting
# transferred here. So, receiving data on another port.
android_loc_client = new Socket(TCP_IP, Port2);
android_loc_client.setSoTimeout(100000);
BufferedReader input_info = new BufferedReader(new
InputStreamReader(android_loc_client.getInputStream()));
message = input_info.readLine();
Note: I am not having enough credits to upload images for better explanation. Or I have a pictorial representation of what is written here. The data transfer works fine for few iterations and after sometime, the error of "Address already in use" error comes. Tried closing and shutdown the socket inside server. Still the error persists.
I'm trying to understand how send and receive are working.
I was trying to send continuously data to a server and i noticed that the server would receive mixed bytes because i was sending to much data at a time. See my code:
Server:
import socket, struct
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(("",1996))
server.listen(0)
c,d = server.accept()
while True:
data = c.recv(1024)
print( struct.unpack("i", data)[0] )
Client:
import socket, struct
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.connect(("192.168.1.4",1996))
while True:
data = 1
server.send( struct.pack("i", data) )
Then i change the while loops to this:
Server:
data = c.recv(1024)
print( struct.unpack("i", data)[0] )
c.send( str.encode("Server received your message. You now can continue
sending more data") )
Client:
data = 1
server.send( struct.pack("i", data) )
#Wait to secure the send.
server.recv(1024)
This is working. I'm making sure that the client won't send data before the
server already receive the previous send.
But what if i want to do the same for the server too? How can i make sure that the server will send bytes to the client in a safe way?
I already tried this and i notice that i created an infinity loop because(I used multi-threading in order to send and receive at the same time on the server):
client was sending some data and then waiting to get a signal from the server
that he can send again.
the server was getting some data then sending the signal and after that waiting for a signal from the user that he can send again.
But because the client was actually sending data again, the whole thing was going on again and this caused me an infinity talk-reply loop.
So what can i do to make a continuously conversation between two sockets without mixing the bytes together?
Your problem is caused by Nagle algorithm which works by combining a number of small outgoing messages, and sending them all at once as TCP is a stream protocol. You can enable TCP_NODELAY socket option by calling sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1) to sent data as soon as possible, even if there is only a small amount of data. And on the receiver side, it isn't going to get one packet at a time either, you must implement message boundaries itself if you want "continuous conversation between two sockets without mixing the bytes together".
Following is the code which listens on a port for HTTP requests and sends the request packet to the server running on port 80, gets the response and sends the data back to the client. Now, everything is executing fine but the following line of code :
data = req_soc.recv(1024)
is taking too much time to execute and I have observed that, it takes long time to execute when it is going to/has received the last packet. I have also tried the same code using select.select() but the results are the same. Since I want to handle the data (raw) that is coming from the client and the actual HTTP server, I have no other choice than using sockets.
import socket
import thread
def handle_client(client):
data = client.recv(512)
request = ''
request += data
print data
print '-'*20
spl = data.split("\r\n")
print spl[0]
print spl[1]
if len(request):
req_soc = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
req_soc.connect(('localhost', 80))
req_soc.send(request)
response = ''
data = req_soc.recv(1024)
while data:
response += data
print 1
data = req_soc.recv(1024)
req_soc.close()
print response
if len(response):
client.send(response)
client.close()
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(('localhost', 4422))
server.listen(5)
print("Server is running...\n")
MSGLEN = 1024
while 1:
client, address = server.accept()
thread.start_new_thread(handle_client, (client, ))
Clients can do multiple commands (eg: GET) within one connection. You cannot wait for the client to send all the commands because based on what you return it could request more (eg: images of a web page). You have to parse the parts (commands) of request, find the boundary, forward that request to the server and write back the answer to the client. All this in a way that doesn't block on reading the client.
I'm not sure what's the best way to do this in python, but if you spend 5 minutes of googling you'll find a perfect HTTP proxy library.