We Keep Coding

Simple way for message passing in distributed system

I am implementing a small distributed system (in Python) with nodes behind firewalls. What is the easiest way to pass messages between the nodes under the following restrictions:
I don't want to open any ports or punch holes in the firewall
Also, I don't want to export/forward any internal ports outside my network
Time delay less than, say 5 minutes, is acceptable, but closer to real time would be nice, if possible.
1+2 → I need to use a third party, accessible by all my nodes. From this follows, that I probably also want to use encryption
Solutions considered:
Email - by setting up separate or a shared free email accounts (e.g. Gmail) which each client connects to using IMAP/SMTP
Google docs - using a shared online spreadsheet (e.g. Google docs) and some python library for accessing/changing cells using a polling mechanism
XMPP using connections to a third party server
IRC
Renting a cheap 5$ VPS and setting up a Zero-MQ publish-subscribe node (or any other protocol) forwarded over SSH and having all nodes connect to it
Are there any other public (free) accessible message queues available (or platforms that can be misused as a message queue)?
I am aware of the solution of setting up my own message broker (RabbitMQ, Mosquito) etc and make it accessible to my nodes somehow (ssh-forwardning to a third host etc). But my questions is primarily about any solution that doesn't require me to do that, i.e. any solutions that utilizes already available/accessible third party infrastructure. (i.e. are there any public message brokers I can use?)

How about Mosquitto: message broker that implements the MQ Telemetry Transport protocol versions 3.1 and 3.1.1. MQTT provides a lightweight method of carrying out messaging using a publish/subscribe model. This makes it suitable for "machine to machine" messaging. It supports encryption. Time to setup: approximatively 15 mins you should be up and running. Since it is a message broker, you can write your own code to ensure you can communicate with 3rd party solutions. Also, it achieves soft real-time, but depending on your setup you can achieve hard real-time. After you look into Mosquitto have a look at Paho, which is a port of Mosquito to Eclipse Foundation.
Paho also provides a Python Client, which offers support for both MQTT v3.1 and v3.1.1 on Python 2.7 or 3.x. It also provides some helper functions to make publishing one off messages to an MQTT server very straightforward. Plenty of documentation and examples to get you up and running.

I would recommend RabbitMQ or Redis (RabbitMQ preferred because it is a very mature technology and insanely reliable). ZMQ is an option if you want a single hop messaging system instead of a brokered messaging system such as RabbitMQ but ZMQ is harder to use than RabbitMQ. Depending on how you want to utilize the message passing (is it a task dispatch in which case you can use Celery or if you need a slightly more low-level access in which case use Kombu with librabbitmq transport )

Found https://www.cloudamqp.com/ which offers a free plan with a cloud based installation of RabbitMQ. I will try that and see if it fulfill my needs.

How can I communicate between a Siemens S7-1200 and python?

I am running a process on a S7-1200 plc and I need it to send a start signal to my python script, after the script is done running it needs to send something back to the plc to initiate the next phase. Oh, and it has to be done in ladder.
Is there a quick and dirty way to send things over profibus or am I better off using just a RS232 thing?

I would go with SNAP7 and python-snap7 libraries. Example on how to install it on a Raspberry pi found in this blog:
http://simplyautomationized.blogspot.com/2014/12/raspberry-pi-getting-data-from-s7-1200.html
Update:
Video walkthrough on how to set it up: https://youtu.be/yJNEsI5KJxs

After failing with libnodave and OPC, I created a TCON,TSEND and TRECV communication thing. It transmits a byte over TCP and it works.

I would use libnodave on PC side to change value of some M tag or a DB tag in a PLC. You can easily make a ladder rung which will monitor this tag and execute further actions. Although libnodave doesn't support Python natively, you can find a python wrapper here.

Well, a quick google of 'python rs232' turned up some site talking about doing robotics controls through rs232 using pySerial. So if you know how to hook your widget up to rs232, you could go that route.

There is a commercial library called "S7connector" by Rothenbacher GmbH (obviously it's not the "s7connector" on sourceforge).
It is for the .NET framework, so could be used with IronPython.
It does work with S7-1200 PLCs. You just have to make sure a DB you want to read from / write to is not an optimized S7-1200 style DB, but a S7-300/400 compatible one, an option which you can set when creating a DB in TIA portal.
This lib also allows to read and write all I/O ports - the "shadow registers" (not sure what they're called officially) and directly as well, overriding the former.

There are several Modbus TCP Clients available for Python, I have used pyModbusTCP successfully. TIA Portal the IDE for the S7-1200 has Modbus Server function that you can load to your PLC that will allow you to read and write directly to the PLC.
https://support.industry.siemens.com/cs/ww/en/view/83130159

Ther best way to communicate with S7-1200 PLC cpu's is with OPC UA or Classic OPC (ommonly known as OPC DA. ) Libnodave is made for S7-300 and S7-400 not for S71200 (2.x firmware).
If you use a third party solution to communicate with S7-1200 (or S7-1500) you have to decrease the security level at the PLC by allowing the put and get mechanism. Put and get are pure evil to use. You open the memory of the CPU for every process. Don’t use them anymore. Siemens should actually block this.
This applies for all firmware release for S7-1200.
Siemens pushes people you use OPC UA as default communication from PLC. What makes sense, because OPC UA is the protocol for industry 4.0 and IIoT.
Edit: rewrite everything. Info was heavily outdated.
If you use a firmware 2 or 3 1200, consider replacement or upgrade. These versions are no longer supported and contains the worm issue.

Interprocess messaging between two Python programs

We have two Python programs running on two linux servers. Now we want to send messages between these Python programs. The best idea so far is to create a TCP/IP server and client architecture, but this seems like a very complicate way to do it. Is this really best practice for doing such a thing?

I like zeromq for simple messaging, it's really lightweight and fast...very flexible as well. Using AMQP messaging isn't a bad idea either depending on the specifics of your situation, I've found kombu to be a very nice pythonic library for that. You could also use xmlrpclib or setup a simple REST API with bottle or flask. Every option has it's place, so I'd investigate all your options.

This really depends on the kind of messaging you want and the roles of the two processes. If it's proper "client/server", I would probably create a SimpleHTTPServer and then use HTTP to communicate between the two. You can also use XMLRPCLib and the client to talk between them. Manually creating a TCP server with your own custom protocol sounds like a bad idea to me. You might also consider using a message queue system to communicate between them.

You could have a mulitprocessing.managers. As doc says :"A manager object controls a server process which manages shared objects. Other processes can access the shared objects by using proxies."
In your case, you could create a master process that control your other processes, each of those processes will call the master to grab the data.

Python inter-computer communication

This whole topic is way out of my depth, so forgive my imprecise question, but I have two computers both connected to one LAN.
What I want is to be able to communicate one string between the two, by running a python script on the first (the host) where the string will originate, and a second on the client computer to retrieve the string.
What is the most efficient way for an inexperienced programmer like me to achieve this?

First, lets get the nomenclature straight. Usually the part that initiate the communication is the client, the parts that is waiting for a connection is a server, which then will receive the data from the client and generate a response. From your question, the "host" is the client and the "client" seems to be the server.
Then you have to decide how to transfer the data. You can use straight sockets, in which case you can use SocketServer, or you can rely on an existing protocol, like HTTP or XML-RPC, in which case you will find ready to use library packages with plenty of examples (e.g. xmlrpclib and SimpleXMLRPCServer)

There are about a million ways.
If I were doing it, I'd use the SocketServer library, because it's not too insane, fairly well documented, and most importantly, I've used it before.
There are a couple of examples here: http://docs.python.org/library/socketserver.html#examples

File share and polling filesystem every minute. No joke. Of course, it depends on what are requirements for your applications and what lag is acceptable but in practice using file shares is quite common.

Creating connection between two computers in python

The question: How do I create a python application that can connect and send packets over the internet to another computer running the same application? Is there any existing code/library I could use?
The background: I am pretty new to programming (HS senior). I've created a lot of simple things in python but I've recently decided to start on a bigger project. I'm considering creating a Magic: the Gathering booster draft simulator, but I'm not sure if it is feasible given my skill set so I'm asking around before I get started. The application would need to send data between computers about which cards are being picked/passed.
Thanks!

Twisted is a python event-driven networking engine licensed under MIT. Means that a single machine can communicate with one or more other machines, while doing other things between data being received and sent, all asynchronously, and running a in a single thread/process.
It supports many protocols out of the box, so you can just as well using an existing one. That's better because you get support for the protocol from 3rd party software (i.e. using HTTP for communication means middleware software that uses HTTP will be compatible: proxies etc.)
It also makes easy to create your own communication protocol, if that's what you want.
The documentation is filled with examples.

The standard library includes SocketServer (documented here), which might do what you want.
However I wonder if a better solution might be to use a message queue. Lots of good implementations already exist, including Python interfaces. I've used RabbitMQ before. The idea is that the computers both subscribe to the queue, and can either post or listen for events.

A great place to start looking is the Python Standard Library. In particular there are a few sections that will be relevant:
18. Interprocess Communication and Networking
19. Internet Data Handling
21. Internet Protocols and Support
Since you mentioned that you have no experience with this, I'd suggest starting with a HTTP based implementation. The HTTP protocol is fairly simple and easy to work with. In addition, there are nice frameworks to support this operation, such as webpy for the server and HTTPLib from the standard library for the client.
If you really want to dive into networking, then a socket based implementation might be educational. This will teach you the underlying concepts that are used in lots of networking code while resulting in an interface that is similar to a file stream.

Also, check out Pyro (Python remoting objects). Se this answer for more details.
Pyro basically allows you to publish Python object instances as services that can be called remotely. Pyro is probably the easiest way to implement Python-to-python process communication.

It's also worth looking at Kamaelia for this sort of thing - it's original usecase was network systems, and makes building such things relatively intuitive.
Some links: Overview of basic TCP system, Simple Chat server, Building a layered protocol, walk-through of how to evolve new components. Other extreme: P2P radio system: source, peer.
If it makes any difference, we've tested whether the system is accessible to novices through involvement in google summer of code 3 years in a row, actively taking on both experienced and inexperienced developers. All of them managed to build useful systems.
Essentially, if you've ever played with unix pipelines the ideas should be familiar.
Caveat: I wrote major chunks of Kamaelia :-)
If you want to learn how to do these things though, playing with a few different approaches makes sense, and you should definitely check out Twisted (the standard answer to this question), Pyro & the standard library tools. Each has a different approach, and learning them will definitely benefit you!
However, like nosklo, I would recommend against using the socket library directly and use a library instead - simply because it is much much harder to get sockets programming correct than people tend to realise.

Communication will take place with sockets, one way or another. Just a question of whether you use existing higher-level libraries, or roll your own.
If you're doing this as a learning experience, probably want to start as low-level as you can, to see the real nuts and bolts. Which means you probably want to start with a SocketServer, using a TCP connection (TCP is basically guaranteed delivery of data; UDP is not).
Google for some simple example code. Setting one up is very easy. But you will have to define all the details of your communications protocol: which end sends when and what, which end listens and when, what exactly the listener will expect, does it reply to confirm receipt, etc.