I'm building a network music player with my Raspberry Pi and I'm trying to come up with a scheme that will allow me to send a "command" to my Pi that will allow it to do various things over the network (such as transport control).
This is what I'm thinking on the receiver (in sort-of pseudo-code):
while True:
while nothingIsRecvD:
do_stuff()
do_something_with(theDataRecvDfromSocket)
Is there some basic code for beginners I can look at?
You'll need to use the socket module and the select module.
To set up the socket, you'll need to
Use socket.socket to create a socket. You'll probably want to use the AF_INET address family. For TCP, use SOCK_STREAM; for UDP, use SOCK_DGRAM.
bind the socket to the interface and port you want to listen on.
For TCP, call listen on the socket. 5 is the typical backlog value used.
If you're using TCP, you've just created a listening socket. In order to actually receive data, you'll need to accept a connection using accept. With a connected socket you can recv or send data.
UDP is similar, except accepting is not necessary and you'll use recvfrom and sendto rather than recv and send.
These methods block, however, and if I understand you correctly, you don't want that. select.select lets you wait for an event to occur on any of a given set of sockets. You can also provide a zero timeout if you want to just check if there is some activity. Once it has detected activity, you can usually perform the appropriate action once without blocking.
Once you're done with sockets, be polite and close them after shutting down any connected sockets.
You could consider using sockets to communicate between the music player and server. The recv() call (typically used with TCP sockets) or recvfrom() call (typically used with UDP sockets) are blocking -- so they should provide a nice blocking context to your nothingIsRecvd case and would allow you to get rid of the "while True" loop. You can find examples on Python Library reference: http://docs.python.org/release/2.5.2/lib/socket-example.html
Related
I have an application, foo which takes in data, does stuff to it, and then publishes the new treated data over AMQ for another downstream application to grab. Until this point, foo has always gotten its data by connecting to another AMQ server which another script is publishing packetized data to (a lot of handwaving here, but the specifics don't really matter).
Recently a change has been made, and foo needs to be able to grab its data from a UDP socket. Is AMQ able to connect to this socket and receive/listen to the data being transmitted over it? From my understanding, AMQ uses TCP to establish connection to the client, and some initial research points me to this UDP Transport documentation from Apache, but not much else.
Alternatively, I could develop a rough UDP socket listener in Python, and then publish those messages to AMQ for foo to grab, but it would be optimal to have it all included in foo itself.
Not necessarily looking for an exhaustive solution here; quick and dirty would be enough to get me started.
Thanks!
ActiveMQ itself is a broker and therefore doesn't connect to sockets and listen for messages. It is the job of a client to connect to the broker and send and/or receive messages.
The UDP transport documentation is just theoretical as far as I know. It is technically possible to use UDP as the base of a traditional messaging protcol, but I've never actually seen it done since UDP is unreliable. The documentation even says, "Note that by default UDP is not reliable; datagrams can be lost so you should add a reliability layer to ensure the JMS contract can be implemented on a non-reliable transport." Adding a "reliability layer" is impractical when TCP can simply be used instead. All of the protocols which ActiveMQ supports (i.e. AMQP, STOMP, MQTT, OpenWire) fundamentally require a reliable network transport.
I definitely think you'll need some kind of intermediary process to read the data from the UDP socket and push it to the broker.
I'm trying to make a python function that scans a range of addresses. I started a socket and pass the socket as an argument to the function that connects to it:
def scan(socket, address, port):
c = socket.connect_ex((address, port))
print(c)
then I call scan for each address, each in its own thread. I'm getting Error 114: Operation already in progress..
Do I need to start a new socket for each connection? I'm trying to read about socket reusage, and I found that there exists flags like SO_ADDREUSE or something like that. I tried to insert but it didn't work.
I'm trying to think how a socket works. I think the moment I create one, it choses a tcp source port, and then when I create a connection, it sends to a destination port. I think I can't reuse the same socket because the source port would be the same for all destination ports, so the clients would answer to the same port and would cause confusion.
So do I need to create a new socket for each connection?
You can not connect stream socket multiple times.
One of the connect possible errors is EISCONN.
The socket is already connected.
This goes for stream sockets.
man bind also has this:
[EINVAL] The socket is already bound to an address, and the
protocol does not support binding to a new address; or
the socket has been shut down.
Again, this goes for stream sockets.
From the man connect:
Generally, stream sockets may successfully connect() only once; datagram sockets may use connect() multiple times to change their association.
I made emphasis on the important line.
stream sockets can not be connected multiple times. datagram sockets can be connected multiple times. Generally speaking, BSD sockets have multiple protocols, types, domains avaible. You shall read documentation for your particular case.
P.S Get yourself familiar with the readings that were suggested in the comment to your question. That will explain enough to manipulate socket family of functions.
Do I need to start a new socket for each connection?
Yes.
I'm trying to read about socket reusage
There is no such thing as 'socket reusage'. There is port reuse. Not the same thing. You cannot reconnect an existing socket once you've tried to connect it, even if the connect attempt failed.
I found that there exists flags like SO_ADDREUSE or something like that
SO_REUSEADDR means to reuse the port. Not the socket.
I'm trying to think how a socket works. I think the moment I create one, it choses a tcp source port,
Between creating a socket using the socket() system call and using it to create an outgoing connection with the connect() system call, there is an opportunity to optionally use the bind() system call to set source IP address and/or port if you want to. If you don't use bind(), the operating system will automatically bind the socket to the first available port in the appropriate range when you use the connect() system call. In this case, the source IP address is normally selected to match the network interface that provides the shortest route to the specified destination according to the routing table.
At least, that's how it works at the system call level. Some programming languages or libraries may choose to combine some of these operations into one.
To your actual question, man 7 ip says:
A TCP local socket address that has been bound is unavailable for some
time after closing, unless the SO_REUSEADDR flag has been set. Care
should be taken when using this flag as it makes TCP less reliable.
The idea is to delay the re-use of a port until any possible re-sent copies of packages that belonged to the closed connection have for sure expired on the network.
According to the bind() man page, trying to re-bind a socket that is already bound to an address will result in an EINVAL error. So "recycling" a socket using bind(socket, INADDR_ANY, 0) (after ending a connection that used SO_REUSEADDR) does not seem to be possible.
And even if that would be possible, when you're using multiple threads on a modern multi-core system, you end up (very probably) doing multiple things in parallel. A socket can be used for just one outgoing connection at a time. Each of your scan threads will need its own socket.
How can I create a UDP server in Python that is possible to know when a client has disconnected? The server needs to be fast because I will use in an MMORPG. Never did a UDP server then I have a little trouble.
There is no such thing as a connection in UDP. Because of this, it becomes your responsibility to detect if the client has disconnected. Generally speaking, your protocol should implement a way to notify the server that it is ending its session. Additionally, you will need to implement some type of timeout functionality such that after a certain period of interactivity, the session is ended.
Note that UDP is more difficult to work with than TCP because packets are not always guaranteed to be delivered. Depending on what you are doing, you might need to implement some type of check to ensure that packets that are not delivered are sent again. TCP does this for you, but it also has the side effect of making the protocol slower.
This answer provides some more considerations: https://stackoverflow.com/a/57489/4250606
UDP is not connection-based. Since no connection exists when using UDP, there is nothing to disconnect. Since there is nothing to disconnect, you can't ever know when something disconnects. It never will because it was never connected in the first place.
Using a Lantronix UDS-1100 serial to IP converter. The goal is to write a small proof of concept piece in Python to capture serial data output by this device over IP.
I've done a couple test projects using sockets in python, but they were all done between python processes (python > python): listen() on one end, and connect(), sendall() etc on the other.
I think I can use sockets for this project, but before I invest a bunch of time into it, wanted to make sure it is a viable solution.
Can python sockets be used to capture IP traffic when the traffic is originating from a non-python source? I have full control over the IP and port that the device sends the serial data to, but there will be no python connect() initiated by the client. I can pre-pend then serial data with some connect() string if needed.
If sockets won't work, please recommend another solution...guessing it will be REST or similar.
Of course. TCP/IP is supposed to be cross-platform and cross-language, so in theory you should be able to communicate with every kind of device as long as you manage to process and send the expected protocol.
I wrote a server based on Twisted, and I encountered a problem, some of the clients are disconnected not gracefully. For example, the user pulls out the network cable.
For a while, the client on Windows is disconnected (the connectionLost is called, and it is also written in Twisted). And on the Linux server side, my connectionLost of twisted is never triggered. Even it try to writes data to client when the connection is lost. Why Twisted can't detect those non-graceful disconnection (even write data to client) on Linux? How to makes Twisted detect non-graceful disconnections? Because the feature Twisted can't detect non-graceful, I have lots of zombie user on my server.
---- Update ----
I thought it might be the feature of socket of unix-like os, so, what is the behavior of socket on unix-like for handling situation like this?
Thanks.
Victor Lin.
You're describing the behavior of TCP connections on an unreliable network. Twisted is merely exposing this behavior: after all, when you set up a TCP connection with Twisted, it is nothing more than a TCP connection.
You're mistaken when you say that the connectionLost callback isn't invoked even if you try to send data over it. After two minutes, the underlying TCP connection will disappear and Twisted will inform you of this by calling connectionLost.
If you need to detect this condition more quickly than that, then you can implement your own timeouts using reactor.callLater.
Seconding what Jean-Paul said, if you need more fine grained TCP connection management, just use reactor.CallLater. We have exactly that implementation on a Twisted/wxPython trading platform, and it works a treat. You might also want to tweak the behaviour of the ReconnectingClientFactory in order to achieve the results I understand your looking for.