I want to create a python program that can communicate with another python program running on another machine. They should communicate via network. For me, it's super simple using BasicHTTPServer. I just have to direct my message to http:// server2 : port /my/message and server2 can do whatever action needed based on that message "/my/message". It is also very time-efficient as I do not have to check a file every X seconds or something similar. (My other idea was to put text files via ssh to the remote server and then read that file..)
The downside is, that this is not password protected and not encrypted. I would like to have both, but still keep it that simple to transfer messages.
The machines that are communicating know each other and I can put key files on all those machines.
I also stumbled upon twisted, but it looks rather complicated. Also gevent looks way too complicated with gevent.ssl.SSLsocket, because I have to check for byte length of messages and stuff..
Is there a simple example on how to set something like this up?
You should consider using HTTPS, as it does the job you want.
The good part is that you won't need to change the code as the connection between the two parties is encrypted. The downside is that you have to set up a server with an HTTP certificate (there are lot of resources on the Internet) and you will need sometimes (depending of your implementation) to accept this certificate in order to make a successful connection.
You can combine it, of course, with using password protected files.
if you have no problem rolling out a key file to all nodes ...
simply throw your messages into AES, and move the output like you moved the unencrypted messages ...
on the other side ... decrypt, and handle the plaintext like the messages you handled before ...
Related
Need sugestions on how to implement/what to use/what or about what to read in order to know how, a mail server in Python, which would let me to generate temporary mailboxes.
Let’s say I have a domain test.com, I would like to be able to setup for example a mailbox temporary1#test.com, which would receive and store mails (even in txt files).
What do I need to be able to do that?
(I’m interested in links or explanations rather than implementations)
Python 3 ships with the smtpd module that can be used to implement this.
You'll probably want to trivially subclass smtpd.SMTPServer (maybe even derive from or at least be inspired by smtpd.DebuggingServer, but have it print out to files instead of standard output).
Once that's up and running (you can use the smtplib clients to test things out), you'll need to have that server process running on a machine on port 25 (the standard SMTP port), and you'll need to point the MX record of test.com to that machine's address.
Key points:
I need to send roughly ~100 float numbers every 1-30 seconds from one machine to another.
The first machine is catching those values through sensors connected to it.
The second machine is listening for them, passing them to an http server (nginx), a telegram bot and another program sending emails with alerts.
How would you do this and why?
Please be accurate. It's the first time I work with sockets and with python, but I'm confident I can do this. Just give me crucial details, lighten me up!
Some small portion (a few rows) of the core would be appreciated if you think it's a delicate part, but the main goal of my question is to see the big picture.
Main thing here is to decide on a connection design and to choose protocol. I.e. will you have a persistent connection to your server or connect each time when new data is ready to it.
Then will you use HTTP POST or Web Sockets or ordinary sockets. Will you rely exclusively on nginx or your data catcher will be another serving service.
This would be a most secure way, if other people will be connecting to nginx to view sites etc.
Write or use another server to run on another port. For example, another nginx process just for that. Then use SSL (i.e. HTTPS) with basic authentication to prevent anyone else from abusing the connection.
Then on client side, make a packet every x seconds of all data (pickle.dumps() or json or something), then connect to your port with your credentials and pass the packet.
Python script may wait for it there.
Or you write a socket server from scratch in Python (not extra hard) to wait for your packets.
The caveat here is that you have to implement your protocol and security. But you gain some other benefits. Much more easier to maintain persistent connection if you desire or need to. I don't think it is necessary though and it can become bulky to code break recovery.
No, just wait on some port for a connection. Client must clearly identify itself (else you instantly drop the connection), it must prove that it talks your protocol and then send the data.
Use SSL sockets to do it so that you don't have to implement encryption yourself to preserve authentication data. You may even rely only upon in advance built keys for security and then pass only data.
Do not worry about the speed. Sockets are handled by OS and if you are on Unix-like system you may connect as many times you want in as little time interval you need. Nothing short of DoS attack won't inpact it much.
If on Windows, better use some finished server because Windows sometimes do not release a socket on time so you will be forced to wait or do some hackery to avoid this unfortunate behaviour (non blocking sockets and reuse addr and then some flo control will be needed).
As far as your data is small you don't have to worry much about the server protocol. I would use HTTPS myself, but I would write myown light-weight server in Python or modify and run one of examples from internet. That's me though.
The simplest thing that could possibly work would be to take your N floats, convert them to a binary message using struct.pack(), and then send them via a UDP socket to the target machine (if it's on a single LAN you could even use UDP multicast, then multiple receivers could get the data if needed). You can safely send a maximum of 60 to 170 double-precision floats in a single UDP datagram (depending on your network).
This requires no application protocol, is easily debugged at the network level using Wireshark, is efficient, and makes it trivial to implement other publishers or subscribers in any language.
This might be a confusing question...
But take for example the game Battlefield2.
Is it in any way possible to send chat messages to the game through Python? The reason I am asking this is because I've seen messages appear from no where on various games, and want to know if it's possible. Now, the way I understand a packet is like this...
It's a small collection of data with a sender IP address and a recipient IP address. Contained within the packet is the data. So, in theory... If the 'packet name' for the chat message is like:
SRV_CHAT|<Sender_Info>|<Message Text>
(For example): SRV_CHAT|10.1.1.5,Player1|hello, how's the game?
Does that mean that I can create a Python script to send my own message to a LAN game and have it appear? If it is possible, how can I go about dissecting the actual packet information in order to discover any parameters? What I mean by this, is finding that data contained in there...
It's just a thought and a question I've had for a long time.
Thanks!
Yes, it is technically possible. But not very feasible.
What you said about packets is essentially correct, and you can read more about that here.
However, beyond packets, there are entire networks of protocols that determine where they go, and who receives them, you can read about that here.
This means that if you were to manage to emulate the connection to the games server, you could possibly send data to that server, but most likely the server does not support random connections simply sending messages, and the server expects to receive connections which are made by other game clients, and unless you can emulate the clients responses and requests correctly, it will probably not work.
This means that your idea to use Python to directly connect to a server and send messages in the format similar to what you suggested would not work.
However. The server most definitely supports server messages, messages that are sent FROM the server itself into the game, usually if someone hosts a game for instance, they can send messages as the game server (host) of the game. Or, if it is a dedicated server, they might not be in the game itself and send messages to the players in the game through a management console.
It is most likely that many 'plugins' or methods to hook into this control are available, meaning that you could send a message to the application running the game server, telling it that it should send a message into the game.
This would be less emulation and more implementation of the games management system.
It is important to note, that many games are different, and their operation is different, meaning this may or may not work, depending on what options are available. It is my experience that games like Battlefield2 (which I have not played) usually have these tools builtin.
Specifically with BattleField2, there are some links I found that you might like.
http://www.gamefront.com/files/listing/pub2/Battlefield_2/Official_Server_Files/Dedicated_Server/
http://www.bf2cc.com/
I want to write a Python Twisted server that serves text to its clients, and I want the clients to be able to write text back to manipulate the server. I will use Telnet, and the clients will use Putty or some similar terminal...I would also be open to using SSH if it is easier to do this.
My question is, how do I configure the server so that the client can send raw, unbuffered bytes (I don't want the user to have to press enter after a command)? Also, is there a way to change the configuration mid-session so that I can change back and forth to and from buffered/unbuffered bytes?
I think it is Telnet option 34 "Linemode" --- http://www.freesoft.org/CIE/RFC/1700/10.htm
I just don't know how to set up Twisted to use that...
Any help setting this up for Telnet or SSH is appreciated!!!
Thanks!
twisted.conch.telnet.TelnetBootstrapProtocol is a good example of how to do some option negotiation. It also happens to perform some LINEMODE negotiation. Take a look at the implementation for details, but here's a snippet that shows the server asking the client to enable linemode, naws, and sga:
for opt in (LINEMODE, NAWS, SGA):
self.transport.do(opt).addErrback(log.err)
A real server might want to do more error handling than log.err if the negotiation fails, since the client will be left in a state that is presumably not ideal for use with the server.
Also take a look at some of the funky terminal demos that come with Twisted. These do lots of character-at-a-time processing.
I have been creating an application using UDP for transmitting and receiving information. The problem I am running into is security. Right now I am using the IP/socketid in determining what data belongs to whom.
However, I have been reading about how people could simply spoof their IP, then just send data as a specific IP. So this seems to be the wrong way to do it (insecure). So how else am I suppose to identify what data belongs to what users? For instance you have 10 users connected, all have specific data. The server would need to match the user data to this data we received.
The only way I can see to do this is to use some sort of client/server key system and encrypt the data. I am curious as to how other applications (or games, since that's what this application is) make sure their data is genuine. Also there is the fact that encryption takes much longer to process than unencrypted. Although I am not sure by how much it will affect performance.
Any information would be appreciated. Thanks.
One solution is to use TCP because it is immune to spoofing the source address over the open internet because of the three-way-handshake (More information on why TCP source address spoofing is impossible.). If you still want to use UDP, you could have a simulated three way handshake to begin the connection. A session id could then be added to each UDP packet. This will increase the connection overhead by 2 packets and a few bits per packet, however you will still gain from UDP's speed for the rest of the session when compared to tcp.
However, using TCP or UDP as a transport layer still leaves you open to other attacks such as Sniffing and Man in The Middle attacks using arp spoofing or dns cache poising. Another problem is if both the attacker and the gamers are on the same local lan, such as a wireless network or another broadcast network then you are able to receive traffic regardless of the source/dest address and ONLY THEN does spoofing a three way handshake become possible (and an hmac can't help!). The best soltuion is to use SSL/TLS as your transport layer which solves all of these problems.
You should not reinvent the wheal, but if you need to encrypt UDP for some reason you should use a Stream Cipher like RC4-drop1024 or even better a Block Cipher like AES 256 in OFB Mode. This will save bandwidth over other modes of encryption because they round up to the largest block size.
EDIT:
Based on Marts comment for (Datagram Transport Layer Security)DTLS I did some digging and I found there is an official RFC and its supported by OpenSSL and should be exposed using the pyOpenSSL library. I recommend using the RC4-SHA cipher suite to reduce overhead, this suite is supported by SSL 3.0 (newest). However DTLS will probably have more overhead (LAG!) then TCP.
You can look at HMAC
Wikipedia:
In cryptography, HMAC (Hash-based
Message Authentication Code), is a
specific construction for calculating
a message authentication code (MAC)
involving a cryptographic hash
function in combination with a secret
key. As with any MAC, it may be used
to simultaneously verify both the data
integrity and the authenticity of a
message.
Each client would need to get a unique token which only they know. Every message they send, they'll make a hash based on the token and the message and send it along with the message itself. Then you can verify that the message came from a specific client.
If you absolutely need to verify that a particular user is a particular user then you need to use some form of encryption where the user signs their messages. This can be done pretty quickly because the user only needs to generate a hash of their message and then sign (encrypt) the hash.
For your game application you probably don't need to worry about this. Most ISPs wont allow their users to spoof IP addresses thus you need to only worry about users behind NAT in which you may have multiple users running from the same IP address. In this case, and the general one, you can fairly safely identify unique users based on a tuple containing ip address and UDP port.
DTLS is likely the best solution, however, it appears to be very poorly documented. I've been looking for a similar solution for a while now and all of the references I've seen to OpenSSL's DTLS implementation suggests that you'll be needing to dig through the OpenSSL examples & source code to figure out how to use it... which, to me, means I'm going to make 10 serious security mistakes when I try to set it up. Also, I don't believe the pyOpenSSL liberary exports this functionality.
An alternative I've been considering is the Secure Remote Password Protocol. The advantage of this solution is that it gives you strong mutual authentication (on par with the security of Kerberos according to the docs) and, just as importantly in your case, it provides both ends with a shared session key that can be used for encryption.
Given the shared key, each packet could contain AES256_CBC( <random starter block for CBC><user-id><sequence_number><application data> ) If the decryption succeeds in providing the anticipated user-id, the packet is authenticated as coming from your user and the sequence number can be used for avoiding replay attacks.
One downside to SRP is that, in Python, the number crunching is pretty slow. I modified the demo Python code into something a bit more usable and found that it took about 300ms to perform a single client-server SRP exchange (2Ghz CPU). However, a straight-forward implementation in C++ of the SRP algorithim using the BigNumber support in the OpenSSL took only 2ms. So, if you intend to go this route, I'd highly recommend using a C/C++ implementation of the algorihim for production code. Otherwise, you'll likely only be able to handle a few logins per second.
I'm breaking this down into four levels of security.
Extremely Insecure - Anyone on the network can spoof a valid request/response with generally available prior knowledge. (ie syslog)
Very Insecure - Anyone on the network can spoof a valid request/response only if they have at least read access to the wire. (Passive MITM) (ie http accessable forum with browser cookies)
Somewhat Insecure - Anyone in the network can spoof a valid request/response if they can read AND make changes to the wire (Active MITM) (ie https site with self-signed cert)
Secure - Requests/Responses cannot be spoofed even with full access to the
wire. (ie https accessable ecommerce site)
For Internet games the very insecure solution might actually be acceptable (It would be my choice) It requires no crypto. Just a field in your apps UDP packet format with some kind of random practically unguessable session identifier ferried around for the duration of the game.
Somewhat insecure requires a little bit of crypto but none of the trust/PKI/PSK needed to prevent Active-MITM of the secure solution. With somewhat insecure if the data payloads were not sensitive you could use an integrity only cipher with (TCP) TLS/ (UDP) DTLS to reduce processing overhead and latency at the client and server.
For games UDP is a huge benefit because if there is packet loss you don't want the IP stack to waste time retransmitting stale state - you want to send new state. With UDP there are a number of clever schemes such as non-acknowledged frames (world details which don't matter so much if their lost) and statistical methods of duplicating important state data to counter predictable levels of observed packet loss.
At the end of the day I would recommend go very insecure or somewhat insecure /w DTLS integrity only.
I would look into the Garage Games networking library. It is written in C++ and uses UDP. It is designed for low latency and is considered one of the best for games.
If I remember correctly they would actually calculate the likely position of the player both on the client side and the server side. It would do this for many aspects to ensure integrity of the data. It also would do a crc check on the client software and compare against the server software to make sure they matched.
I am not sure you can license it separately anymore so you may have to license the game engine (100 bucks). It would at least give you some insight on a proven approach to UDP for games. Another possibility is looking into the PyGame networking code. It may have already addressed the issues you are facing.