Man in the middle - proxy - too slow - python

I've developed a simple transparent proxy script in Python that changes a string in the server-client traffic.
Source Here
It was used with SMTP and now I have changed it to other application.
My problem is that after it changes the 1st defined string, I need it to stop processing the messages, only pass them to the correct host.
Or...If there's any other software that does this, it would be great.
Not that I don't trust my own code, but something well tested, and with better performance would be great.
Also, keep in mind, the original code would handle little traffic, just a few e-mails, this new one, will handle a lot, several SOAP messages passing around between the hosts, and with multiple clients.
It's working, but, every now and then I get a "server disconnected" message in the client app.
(Sorry my lousy english, will update the post if needed, to better understanding)

Related

How can I connect front-end with back-end?

I have a web page (front-end) created with ASP.NET (visual studio) on one laptop and a server (back-end) for my web page on another laptop. In the first place, I have a login page and I want to take the username and the password from textBox-es (front-end) and somehow transfer the data to the other laptop to verify if it's correct and then receive if the data it's correct or not. The front-end is on Windows and the server is on Linux. My teammate is using python on back-end.
I'm working with HTML, CSS and a little bit of C# on the front-end and I don't know how to transfer that username and password to my teammate and how to receive an answer from him.
He has the database on his laptop and he's the one who will verify if the pair is correct or not and send me an answer, also he wants to code in python.
We think that we can do some magic connection between ASP.NET and Node.js.
How do I do this data transfer ?
It's my first time doing this so please be gentle because I really don't know what to do.
Your question is quite broad and while I am not that fluent in web applications I will share what advice I can. Because the question is broad, you cannot get an exact, short and concise, answer.
Some clarifications and feedback regarding your question.
I have a web page (front-end) created with ASP.NET ...
You actually have a web application here (that, in your case, includes one, or more, web pages). Give more attention to such little details, programming in itself is a form of grammar natzy-ism but it’s not grammar and you are not being corrected by a person, but by machines and life. A nice definition of web application: https://blog.stackpath.com/web-application/
We think that we can do some magic connection between ASP.NET and Node.js.
I am not specialized in web application but it appears to me that this line comes completely out of the blue, you have not indicated that you are using or intend to use Node.js in your back-end; you indicated that you were using Python, so I will ignore this statement. Oh and there is no magic, there are only race conditions.
Structurally speaking, I would split this question into several smaller ones. Some suggestions follow.
Because you are attempting a somewhat more complicated task you cannot escape de idea of a design pattern (especially because of using ASP.NET which is designed to be patterning friendly).
So your first question should be similar to
how to communicate with a web server asp.net
Two-way communication between ASP.NET Web App and C# Application
back-end is not python but read the answer, it gives you a valid indicator, that request-response pattern could help you
search for questions regarding how to implement that pattern in C#/ASP.NET
Another possible way of how to-ing would be
how to transmit data to server C#
a bit old but reading might give you an understanding of how the ecosystem works
C# server and client communication - send/receive data
maybe helps?
https://codingvision.net/networking/c-sending-data-using-get-or-post
https://www.csharp-examples.net/socket-send-receive/
From you explication I presume that you already have implemented the UI forms that will retrieve the input data so I will skip UI related questions.
Since you wish to send user SENSITIVE data over the internet then you MUST think about security. Further questions
how to store password in database securely
this question will actually lead you to examples of transmitting the password and maybe even receiving the content
https://www.meziantou.net/how-to-store-a-password-in-a-web-application.htm
How to store passwords in database securely?
There are 2 more issues I wish to discuss.
While searching the internet for tutorials you will probably see a
lot working with client and server applications on the same machine.
Those are still valid tutorials for you! Read them! I will explain
further down why it's (almost) the same for you even if your case is
a remote server.
You need to somehow be able to test your application.
Addressing #1
Why working with localhost (127.0.0.1) is for you the same as working with a real server (in your case).
Let's say you finish the application, the back-end is also made and you wish to communicate with one another. You read on the internet that you need to known the IP of the server so you ask your partner to for him to give his IP. You search the wild wild web and reach the conclusion that to find your IP you would try:
executing ipconfig.exe (or what tool you use depending on your platform) and getting the IPv4 Address. . . . . . . . . . . : 192.168.1.12 value.
you would presume that that is the IP you need to put in your application to connect to. This is FALSE.
reaching the conclusion that that does not work, you find that you can get your IP with online sites such as https://whatismyipaddress.com/
and use that IP for your server address. This (in 99% cases) is FALSE.
A bit of poetry
Consider IP as you address in the world and your local (private) IP as the exact place in your house. The IP address you see there is a local IP and it would indicate in which chair you are sitting in, let’s say the 12 from the already provided example. Anyone in the world (who has 12 chairs anyway) can sit in the 12th chair in their house. You cannot use this information to identify the address of someone.
How would you know that that IP is a local IP?
One way would be to look at the the full output of ipconfig.exe (or whatever tool you are using) where you see a Wireless LAN adapter Wi-Fi: above from where you got your IP, the LAN there (hxxps://en.wikipedia.org/wiki/Local_area_network) is an indicator of that.
The other (more straightforward) is by recognizing the IP range (or visual pattern, if IP range is not a clear concept for you just yet). If it starts with 198.168. then it's a local/private IP address. Please read here more: https://en.wikipedia.org/wiki/Private_network
IPs are a valuable commodity (I joke you not). As the world has evolved you cannot now simply plug your computer with an internet cable, and voila you have your own IP. This was (relatively) true when the internet first started decades ago but now the IPs are managed by big (huge) companies that they allocate them as they consider.
The IP address space is managed globally by the Internet Assigned Numbers Authority (IANA), and by five regional Internet registries (RIRs) responsible in their designated territories for assignment to local Internet registries, such as Internet service providers, and other end users.
Wiki quote: https://en.wikipedia.org/wiki/IP_address
I also highly (please do) recommend you read the following answer
hxxps://superuser.com/questions/323801/how-can-i-own-an-ip-address
If you ask your (real world) neighbors to look up https://whatismyipaddress.com/ (and they have the same ISP and monthly plan as you) there is a strong possibility that they will see the same IP as you see. To put it simply, the IP you see there, when accessing sites such as whatismyipaddress, is the IP that that your ISP has assigned to the network node you are in. It would be a waste of IPs to assign each user an IP (not to mention there actually aren’t mathematically that many IPs to do so).
Regarding your current predicament. I can see 3 options.
Option 1 - buy hosting
If your response is "I really really wanna have a separate IP" (or domain) then you will need to buy from a hosting service an IP (the good kind) that will point to a device the hosting company has. You usually buy domains that point to IPs, but let's not got there for now. You will connect to that machine/devices and install your code on the server there. Then, and only then, you could get the IP/domain of that machine and use it in your code and it will work (this is the 1% cases where sites like whatismyipaddress will actually help you).
Recommended searches: server hosting cheap :D (or something free for students if you are students)
Option 2 - work locally
If your project is used for academia and there is no actual need for a separate server then you and your partner will run the client/server applications while connected to the same wireless modem. Ask him to get his IP in this way (here you actually need the LAN IP), you hardcoded it in your code (preferably a separate config file, this way no extra compilation is required) and voila, it works.
Option 3 - LAN over the internet
Technically this would be hard to describe but the idea is you use programs that communicate over the internet, you each install it on your computer, somehow connect with each other (usually a name/password pair) and that software make is seem for the programs on your computer that you are actually in a LAN setup (Option 2). In my days I used hamachi so that we could play CS online :D.
hxxps://www.howtogeek.com/172762/how-to-play-old-lan-games-over-the-internet/
Regarding #2
Until your partner provides a server you will need to test your code. So you will need to somehow fake a server response so that you can actually continue development.
Quite simply, the operation of faking something, a return/a class, in your code, so that you can correctly develop and test your code is known as mocking
how to mock server response asp.net
hxxps://gingter.org/2018/07/26/how-to-mock-httpclient-in-your-net-c-unit-tests/
For beginning programmers I think this is one of the most complicated parts. The concept of mocks and tests, while they do appear simple, are usually not engraved into the mind of young, innocent, previously untainted, future programmers. They are left stuck with printfs until graduations (even a dose of hate towards tests is sprinkled in the mix, what a pity).
As a courtesy, I wish to further extend that madness. Please, for all that is good and sacred in this world, do not use this code in production, burn it after you used it to understand some of the basics, BURN IT!
I modified the code from here Python 3.x BaseHTTPServer or http.server so that you could run a local HTTP server on your machine and test your application. You simply run it (python3) and test the sending of user name and password to the server using the following pagan syntax:
127.0.0.1:4242/?user=aba&password=Aba1 which would return I'al cohol you later
#!/usr/bin/python
from http.server import BaseHTTPRequestHandler,HTTPServer
from urllib.parse import urlparse, parse_qs
HOST = '127.0.0.1' # Standard loopback interface address (localhost)
PORT_NUMBER = 4242 # Port to listen on (non-privileged ports are > 1023)
# !!!!!!!!!!!!!!! THIS IS ONLY FOR DIDACTICAL PURPOSE< NEVER EVER DO SOMETHING LIKE THIS IN PRODUCTION!!!!!!!
# NEVER EVER STORE PASSWORDS LIKE THIS !!!!!!!
# LITERRLAY NEVER!!!!!!
data_base = [
{
'user': 'ionzapada',
'password': 'neaomat',
'data': b"I like pie!"
},
{
'user': 'aba',
'password': 'Aba1',
'data': b"I'al cohol you later"
}
]
class myHandler(BaseHTTPRequestHandler):
#Handler for the GET requests
def do_GET(self):
self.send_response(200)
self.send_header('Content-type','text/html')
self.end_headers()
# Send the html message
query_components = parse_qs(urlparse(self.path).query)
# DO NOT USE THIS LOGIC ON ANY PRODUCTION PLATFORM!!!!
# NEVER SEND PASSWORDS THIS WAY!!!
# NEVER SEND PASSWORDS!!!!
# NEVER SEND HASHES WITHOUT SALT!!!!
# NEVER IMPLEMENT YOUR OWN TRICKY ALGORITHM!!!!
user_name = query_components.get("user",[None])[0]
password = query_components.get("password",[None])[0]
if user_name and password:
for data_base_entry in data_base:
if data_base_entry['user'] == user_name:
print('Found searched user: {}'.format(user_name))
if data_base_entry['password'] == password:
print('Valid password for user {} given; returning db data'.format(user_name))
self.wfile.write(data_base_entry['data'])
else:
self.wfile.write(b"Not supported!")
else:
self.wfile.write(b"Not supported!")
return
try:
server = HTTPServer((HOST, PORT_NUMBER), myHandler)
print('Started httpserver on port', PORT_NUMBER)
server.serve_forever()
except KeyboardInterrupt:
print('CTRL+C received, shutting down the web server')
server.socket.close()
I burned my finger writing that, do not ever send user names and passwords in this way!!! DO NOT EVEN SEND PASSWORDS FOR THE LOVE OF GOD!!!!!!!! search for standard and secure ways of doing this!!!!
Please also visit the links indicated by #amc they are an also good start.
P.S. I am not allowed to have more then 8 link in one post, on some links that I have posted change the hxxp to http for them to work.

Efficient way to send results every 1-30 seconds from one machine to another

Key points:
I need to send roughly ~100 float numbers every 1-30 seconds from one machine to another.
The first machine is catching those values through sensors connected to it.
The second machine is listening for them, passing them to an http server (nginx), a telegram bot and another program sending emails with alerts.
How would you do this and why?
Please be accurate. It's the first time I work with sockets and with python, but I'm confident I can do this. Just give me crucial details, lighten me up!
Some small portion (a few rows) of the core would be appreciated if you think it's a delicate part, but the main goal of my question is to see the big picture.
Main thing here is to decide on a connection design and to choose protocol. I.e. will you have a persistent connection to your server or connect each time when new data is ready to it.
Then will you use HTTP POST or Web Sockets or ordinary sockets. Will you rely exclusively on nginx or your data catcher will be another serving service.
This would be a most secure way, if other people will be connecting to nginx to view sites etc.
Write or use another server to run on another port. For example, another nginx process just for that. Then use SSL (i.e. HTTPS) with basic authentication to prevent anyone else from abusing the connection.
Then on client side, make a packet every x seconds of all data (pickle.dumps() or json or something), then connect to your port with your credentials and pass the packet.
Python script may wait for it there.
Or you write a socket server from scratch in Python (not extra hard) to wait for your packets.
The caveat here is that you have to implement your protocol and security. But you gain some other benefits. Much more easier to maintain persistent connection if you desire or need to. I don't think it is necessary though and it can become bulky to code break recovery.
No, just wait on some port for a connection. Client must clearly identify itself (else you instantly drop the connection), it must prove that it talks your protocol and then send the data.
Use SSL sockets to do it so that you don't have to implement encryption yourself to preserve authentication data. You may even rely only upon in advance built keys for security and then pass only data.
Do not worry about the speed. Sockets are handled by OS and if you are on Unix-like system you may connect as many times you want in as little time interval you need. Nothing short of DoS attack won't inpact it much.
If on Windows, better use some finished server because Windows sometimes do not release a socket on time so you will be forced to wait or do some hackery to avoid this unfortunate behaviour (non blocking sockets and reuse addr and then some flo control will be needed).
As far as your data is small you don't have to worry much about the server protocol. I would use HTTPS myself, but I would write myown light-weight server in Python or modify and run one of examples from internet. That's me though.
The simplest thing that could possibly work would be to take your N floats, convert them to a binary message using struct.pack(), and then send them via a UDP socket to the target machine (if it's on a single LAN you could even use UDP multicast, then multiple receivers could get the data if needed). You can safely send a maximum of 60 to 170 double-precision floats in a single UDP datagram (depending on your network).
This requires no application protocol, is easily debugged at the network level using Wireshark, is efficient, and makes it trivial to implement other publishers or subscribers in any language.

Sending a chat message to a LAN game with Python

This might be a confusing question...
But take for example the game Battlefield2.
Is it in any way possible to send chat messages to the game through Python? The reason I am asking this is because I've seen messages appear from no where on various games, and want to know if it's possible. Now, the way I understand a packet is like this...
It's a small collection of data with a sender IP address and a recipient IP address. Contained within the packet is the data. So, in theory... If the 'packet name' for the chat message is like:
SRV_CHAT|<Sender_Info>|<Message Text>
(For example): SRV_CHAT|10.1.1.5,Player1|hello, how's the game?
Does that mean that I can create a Python script to send my own message to a LAN game and have it appear? If it is possible, how can I go about dissecting the actual packet information in order to discover any parameters? What I mean by this, is finding that data contained in there...
It's just a thought and a question I've had for a long time.
Thanks!
Yes, it is technically possible. But not very feasible.
What you said about packets is essentially correct, and you can read more about that here.
However, beyond packets, there are entire networks of protocols that determine where they go, and who receives them, you can read about that here.
This means that if you were to manage to emulate the connection to the games server, you could possibly send data to that server, but most likely the server does not support random connections simply sending messages, and the server expects to receive connections which are made by other game clients, and unless you can emulate the clients responses and requests correctly, it will probably not work.
This means that your idea to use Python to directly connect to a server and send messages in the format similar to what you suggested would not work.
However. The server most definitely supports server messages, messages that are sent FROM the server itself into the game, usually if someone hosts a game for instance, they can send messages as the game server (host) of the game. Or, if it is a dedicated server, they might not be in the game itself and send messages to the players in the game through a management console.
It is most likely that many 'plugins' or methods to hook into this control are available, meaning that you could send a message to the application running the game server, telling it that it should send a message into the game.
This would be less emulation and more implementation of the games management system.
It is important to note, that many games are different, and their operation is different, meaning this may or may not work, depending on what options are available. It is my experience that games like Battlefield2 (which I have not played) usually have these tools builtin.
Specifically with BattleField2, there are some links I found that you might like.
http://www.gamefront.com/files/listing/pub2/Battlefield_2/Official_Server_Files/Dedicated_Server/
http://www.bf2cc.com/

Sending image to server: http POST vs custom tcp protocol

I am working out how to build a python app to do image processing. A client (not a web browser) sends an image and some text data to the server and the server's response is based on the received image.
One method is to use a web server + WSGI module and have clients make a HTTP POST request (using multipart/form-data). The http server then 'works out' the uploaded image and other data that the program can use.
Another method is to create a protocol that only sends the needed data and is handled within the application. The application would be doing everything (listening on the port, etc).
Is one of these a stand-out 'best' way (if yes, which one?), or is it more up to preference (or is there another way which is better)?
I believe it's more up to your needs, the size of the images, and your general knowledge of network programming.
In terms of simplicity, posting an image to the webserver using WSGI would be fairly simple, and you wouldn't have to worry about handling connections, sockets, error handling due to busy network ports, etc.
Another argument in favor of this approach is that you can easily reuse this "feature" if you already have it working on a webserver, say, by including a browser client. It might not be one of your needs now, but the door is left open.
This would be my choice.
Also, in Python you have a huge plethora of web frameworks to choose from, from the likes of Django, which is probably a huge overkill for your needs, to something alot simpler, like http://flask.pocoo.org/ which might just suit your needs and is really simple to set up.
In my opinion HTTP is an ideal protocol for sending files or large data, and its very common use, easy to suit any situation. If you use a self-created protocol, you may find it hard to transform when you get other client needs, like a web API.
Maybe the discussions about HTTP's lack of instantaneity and agility make you hesitate about choosing HTTP, but that mostly something about instant messaging and server push, there are better protocols. But when it comes to stability and flexiblity, HTTP is always a good choice.

How would I keep a constant piece of data updated through a socket in Python?

I have a client and a server, both written in Python 2.7.
Lets say I wanted to make a multiplayer game server (which I don't at the moment but I'm working towards it). I would need to keep the server up to date (and other clients) on my characters whereabouts, correct?
How would I do this with sockets? Send or request information only when it is needed (e.g the character moves, or another players character moves and the server sends the information to other clients) or would I keep a constant socket open to send data real-time of EVERYBODY's movement regardless of if they have actually done something since the last piece of data was sent or not.
I won't struggle coding it, I just need help with the concept of how I would actually do it.
With TCP sockets it is more typical to leave the connections open, given the teardown & rebuild cost.
Eventually when scaling you will do look into NewIO\RawIO.
If you do not, imagine that the game client might take a step & not get confirmation if sending it to the server & other players.
Definitely keep the socket open, but you should consider using something like ZeroMQ which gives you more kinds of sockets to work with.

Categories