Using a Lantronix UDS-1100 serial to IP converter. The goal is to write a small proof of concept piece in Python to capture serial data output by this device over IP.
I've done a couple test projects using sockets in python, but they were all done between python processes (python > python): listen() on one end, and connect(), sendall() etc on the other.
I think I can use sockets for this project, but before I invest a bunch of time into it, wanted to make sure it is a viable solution.
Can python sockets be used to capture IP traffic when the traffic is originating from a non-python source? I have full control over the IP and port that the device sends the serial data to, but there will be no python connect() initiated by the client. I can pre-pend then serial data with some connect() string if needed.
If sockets won't work, please recommend another solution...guessing it will be REST or similar.
Of course. TCP/IP is supposed to be cross-platform and cross-language, so in theory you should be able to communicate with every kind of device as long as you manage to process and send the expected protocol.
Related
I am trying to create a chess game between two different computers that are not in the same LAN. I am having trouble connecting the two via a TCP connection (UDP would probably be sufficient as well if the packets are arriving, but ideally TCP).
I am new to a lot of networking and am unaware of many different tools that may be useful and I am also in university and therefore don't have control over the router to update firewall rules. What can I do to work around the router firewall to connect the two devices.
I am primarily using the Python socket library at the moment to implement the connection.
Any information about how I can send messages between the two computers outside of a LAN would be very useful. Thank you for your help!
I have ensured that the client side is using the public IP of the server and the server is using "" for its socket host. I also checked that the connection was working when utilizing a LAN without issue. I included a batch file that enables the specific port used for the game at the beginning of runtime and disables it at the end of the program. If I am not mistaken, that only impacts the computer's firewall rules not the router's. I have looked into receive the packets through port 80 and redirecting it to my specific program, but was unsuccesful in finding a solution of that type.
If the server is behind a router/firewall you'll have to use some sort of hole punching method to create the connection. STUN is one of the most common, though I've never actually used it in a Python program so I don't know what Python implementations are out there.
Key points:
I need to send roughly ~100 float numbers every 1-30 seconds from one machine to another.
The first machine is catching those values through sensors connected to it.
The second machine is listening for them, passing them to an http server (nginx), a telegram bot and another program sending emails with alerts.
How would you do this and why?
Please be accurate. It's the first time I work with sockets and with python, but I'm confident I can do this. Just give me crucial details, lighten me up!
Some small portion (a few rows) of the core would be appreciated if you think it's a delicate part, but the main goal of my question is to see the big picture.
Main thing here is to decide on a connection design and to choose protocol. I.e. will you have a persistent connection to your server or connect each time when new data is ready to it.
Then will you use HTTP POST or Web Sockets or ordinary sockets. Will you rely exclusively on nginx or your data catcher will be another serving service.
This would be a most secure way, if other people will be connecting to nginx to view sites etc.
Write or use another server to run on another port. For example, another nginx process just for that. Then use SSL (i.e. HTTPS) with basic authentication to prevent anyone else from abusing the connection.
Then on client side, make a packet every x seconds of all data (pickle.dumps() or json or something), then connect to your port with your credentials and pass the packet.
Python script may wait for it there.
Or you write a socket server from scratch in Python (not extra hard) to wait for your packets.
The caveat here is that you have to implement your protocol and security. But you gain some other benefits. Much more easier to maintain persistent connection if you desire or need to. I don't think it is necessary though and it can become bulky to code break recovery.
No, just wait on some port for a connection. Client must clearly identify itself (else you instantly drop the connection), it must prove that it talks your protocol and then send the data.
Use SSL sockets to do it so that you don't have to implement encryption yourself to preserve authentication data. You may even rely only upon in advance built keys for security and then pass only data.
Do not worry about the speed. Sockets are handled by OS and if you are on Unix-like system you may connect as many times you want in as little time interval you need. Nothing short of DoS attack won't inpact it much.
If on Windows, better use some finished server because Windows sometimes do not release a socket on time so you will be forced to wait or do some hackery to avoid this unfortunate behaviour (non blocking sockets and reuse addr and then some flo control will be needed).
As far as your data is small you don't have to worry much about the server protocol. I would use HTTPS myself, but I would write myown light-weight server in Python or modify and run one of examples from internet. That's me though.
The simplest thing that could possibly work would be to take your N floats, convert them to a binary message using struct.pack(), and then send them via a UDP socket to the target machine (if it's on a single LAN you could even use UDP multicast, then multiple receivers could get the data if needed). You can safely send a maximum of 60 to 170 double-precision floats in a single UDP datagram (depending on your network).
This requires no application protocol, is easily debugged at the network level using Wireshark, is efficient, and makes it trivial to implement other publishers or subscribers in any language.
I have 2 devices communicating with each other using Modbus and I would like to sniff this communication line using Python for logging purposes. I have connected my computer to the communication bus via a Modbus dongle, but I don't know how I can actually read the data.
I have used Pymodbus before, but I don't think that would work in this case since it only allows for master or slave devices (as far as I know)
Are there any other libraries that I can use for my purpose? Would it be possible for me to implement something like a read only socket on the COM port that doesn't interfere with existing communication on the bus?
Thank you!
You can try a MODBUS simulator like this one: It has an option to see the activity on the bus.
If you want to use another python MODBUS I recommend using modbus_tk. It has an example on how to create an MODBUS simulator. But to be able sniff the packets sent to other devices you will need to do some modifications to disable the automatic response.
If the only thing you want to do is to see the activity on the bus I recommend the first option.
I hope this helps you.
[Edit]:To be more specific you will need to download the following software : Modbus Poll - MODBUS slave simulator(which works on Windows) and plug your MODBUS dongle in the port you intend to use. After you do all the settings for the serial communications go to the Display tab and click on Communication. You will be able to see the traffic on the line.
This is the problem I'm trying to solve,
I want to write an application that will read outbound http request packets on the same machine's network card. This would then be able to extract the GET url from it.On basis of this information, I want to be able to stop the packet, or redirect it , or let it pass.
However I want my application to be running in promiscuous mode (like wireshark does), and yet be able to eat up (stop) the outbound packet.
I have searched around a bit on this..
libpcap / pcap.h allows to me read packets at the network card, however I haven't yet been able to figure out a way to stop these packets or inject new ones into the network.
Certain stuff like twisted or scapy in python, allows me set up a server that is listening on some local port, I can then configure my browser to connect to it, using proxy configurations. This app can then do the stuff.. but my main purpose of being promiscuous is defeated here..
Any help on how I could achieve this would be greatly appreciated ..
I'd suggest that you approach this at the application layer and use a transparent proxy (e.g. squid) and iptables based interception of outbound port-80 traffic.
The reason I suggest this is that that it will avoid issues with the request being split between packets.
However, if you still want to go ahead with packet interception, you can do it in userspace using netfilters in netlink. I believe there are python wrappers for libnl around.
Essentially you create an iptables rule pointing to "QUEUE" for the traffic you want to intercept and write a program using a netlink library to process the queue, accepting, rejecting and/or modifying packets.
Using pcap you cannot stop the packets, if you are under windows you must go down to the driver level... but you can stop only packets that your machine send.
A solution is act as a pipe to the destination machine: You need two network interfaces (without address possibly), when you get a packet that you does not found interesting on the source network card you simply send it on the destination network card. If the packet is interesting you does not send it, so you act as a filter. I have done it for multimedia performance test (adding jitter, noise, etc.. to video streaming)
You are confusing several things here:
"Promiscuous" usually refers to a mode of a hardware ethernet network card where it delivers all packets in its collision domain up to the kernel network stack and have it sort out delivery (vs. just unicast to given MAC, subscribed multicast, and broadcast in normal operating mode of the card). This is on the receive path.
All the data outbound from your machine will go through (one of) the network cards on the machine, so "promiscuous" does not at all apply here.
You are working on filtering TCP-based protocol (HTTP), but talk in terms of packets. This is wrong. TCP connection is a stream that could be (as far as socket readers and writers are concerned) arbitrarily split into IP datagrams. That URL from HTTP request header could be split across multiple link-layer frames. You would have to stitch them back together and parse the stream anyway. Then you have no chance even at that if SSL is in use.
If you are interested in HTTP filtering then read HTTP RFCs, and read existing open-source code, e.g. squid, nginx, etc.
If you are digging through network stack for better understaning then read W. Richard Stevens books, look into existing code in open-source operating systems, check out BPF and netlink.
Hope this clears it a little.
I have implemented this module in Windows by using two separate NICs and using a socket/pipe(whatever you like) between them in this thread
Is there anyway to do this without getting a "COM PORT IN USE" error? I have a a service that listens to gps GPRMC sentences on a com port. But I don't have this device on my testing computer. So I wanted to write a python script to simulate GPRMC sentences on the port while my other python script listens to the same port and parses.
Writing to a serial port does not leave a message on the serial port to be read by the same device. This just isn't how a serial port works and is not how most OSes are written to allow as a buffer behavior. What you really need is a virtual serial port.
Check out this section of a wikipedia article on COM port redirectors and see if any of it will fulfill your needs. Otherwise I recommend searching for COM port emulator, serial port virtualization, etc. until you find software that will work for your use case and operating system. This might be hard, especially if timing is important to your simulations.
Edit: To make this slightly more clear, let's talk about what the pySerial library is actually doing to communicate with python. pySerial is just communicating to the OS's API for the serial port. The OS will, generally, model this as a location to write information to and a location to read information from (buffered in just about all modern computing systems). What's important to understand is that from the point of view of the OS (how the serial port is modeled), the write location can ONLY be written to and the read location can only be read from. This may or may not be how the actual serial hardware interfaces with the machine, in most serial port hardware and interface designs that I've worked with, this is the case for the sake of simplicity and reduced cost. Because of this, you are down to two basic choices.
Give the OS a virtual serial port that you can read to AND write from somehow
Possibly simpler, put a null modem adapter on one of your computer's serial ports and, using a serial cable, connect the two ports. You can now have your service on one port and your simulated device script on the other.