Currently, I've got a tcp socket running, I was thinking on adding the capability for the server side to write to a log file, what has been sent/received/errors over the socket. My server script is running as a service on boot.
I'm wondering if its possible to do it? I've looked up the logging facilities on the python docs, but it doesn't seem to have much details on it(or I just can't understand). I hope someone could point me in the right direction.
Using the logging python class to write to a file?
If you do not want to use the logging class, you could simply open a file and write error messages to it.
Another option if on Linux is doing something like this:
How can I log the stdout of a process started by start-stop-daemon?
Tried netcat?
e.g.
netcat -l 0.0.0.0 1999 |tee /tmp/log.txt
This opens a listening TCP socket on port 1999 on your host open to the world. It then will write a copy of anything incoming to /tmp/log.txt.
Related
I have an application to stream a video on web using flask. Something like an example. But sometimes when the user closes its connection flask does not recognize the disconnection and keeps the socket open. Each socket in Linux is a file-descriptor and the maximum number of open file-descriptors in Linux is 1024 by default. After a while (e.g. 24 hours) new users cannot see the video stream because flask cannot create new socket (which is a file-descriptor).
The same happens when I use flask sockets: from flask_sockets import Sockets. I dont know what happens to these sockets. Most of the time when user refreshes the browser or closes it normally, the socket get closed on server.
I made a test and removed the network cable from my laptop (as a client) and realized that in this case sockets continue to be open and flask does not recognize this kind of disconnection.
Any help will be appreciated.
Update 1:
I put this code on top of main script and it can recognize some of disconnections. But the main problem still is there.
import socket
socket.setdefaulttimeout(1)
Update 2:
Seems duplicated with this post but no solution. I checked the sockets status(by sudo lsof -p your_process_id | egrep 'CLOSE_WAIT') and all of them are "CLOSE_WAIT".
I'm trying to listen on a certain ip address so I use a socket but when I specify the udp_IP, I get "address already in use".
Is there a way I can reuse the ip address for both my application and my listener?
I don't have a very good networking knowledge, any help would be great.
Usually, you cannot use the same port or socket with different programs. If what you want is stopping the listener and start the application in that port, open a terminal and type:
sudo netstat -nlp | grep:yourSocketNumberGoesHere
This is to get the process ID of the program running. Then just use kill followed by the process ID you get to kill the program. Then, try starting your application.
If what you want is using 2 different programs in the same socket or port, you can't, without maybe doing something more complex (which I do not know).
You could use 2 ports that communicate with each other if you want the 2 programs to interact.
I am creating a python application using twisted which reads lines from a serial port.
In order to (unit)test that app without having to connect an actual device to the serial port (also on pc's without an actual serial port) I would like to create a python script/app that sets up a virtual serial port and writes to it, so the twisted app can connect to the other end of the virtual serial port and read from it. This way I can write some unittests.
I found this is possible using pseudo terminals in linux. I also found a working example script on https://askubuntu.com/questions/9396/virtual-serial-port-for-testing-purpose.
I would like to change that script to a class on which I can call a write method to write data to the serial port, and then test the twisted app.
This example script does a lot of stuff with poll and select and a linux stty command which I don't really understand. I hope someone can fill the gap in my knowledge or provide some hints.
Cheers,
Dolf.
In addition to what Jean-Paul Calderone said (which was the correct answer mostly), I also made the following script in python, using socat.
This can be imported and instantiated into an interpreter, and then you can use it's writeLine method to write data to a (vritual) serial port, which is connected through socat to another (virtual) serial port, on which another twisted app can be listening. But as Jean-Paul Calderone said: if it's just unittesting you want, you don't really need to do this stuff. Just read the docs he mentioned.
import os, subprocess, serial, time
from ConfigParser import SafeConfigParser
class SerialEmulator(object):
def __init__(self,configfile):
config=SafeConfigParser()
config.readfp(open(configfile,'r'))
self.inport=os.path.expanduser(config.get('virtualSerialPorts','inport'))
self.outport=os.path.expanduser(config.get('virtualSerialPorts','outport'))
cmd=['/usr/bin/socat','-d','-d','PTY,link=%s,raw,echo=1'%self.inport,'PTY,link=%s,raw,echo=1'%self.outport]
self.proc=subprocess.Popen(cmd,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
time.sleep(3)
self.serial=serial.Serial(self.inport)
self.err=''
self.out=''
def writeLine(self,line):
line=line.strip('\r\n')
self.serial.write('%s\r\n'%line)
def __del__(self):
self.stop()
def stop(self):
self.proc.kill()
self.out,self.err=self.proc.communicate()
You don't need a pty to test your protocol. You don't even need any kind of file descriptor. Follow the guidelines at http://twistedmatrix.com/documents/current/core/howto/trial.html, particularly the Testing a protocol section.
A better approach is probably to use a software null modem emulator.
You can get it from github for linux and from sourceforge for windows.
On linux it is called tty0tty and you simply type
make
to build everything. Then you would need to type
sudo insmod module/tty0tty.ko
to install the virtual driver and
./pts/tty0tty
to launch the application, which opens you 2 virtual ports: /dev/pts/4 and /dev/pts/6.
You can then open the /dev/pts/4 serial port in your python unit tests and open the /dev/pts/6 in your application.
In your python unit test, you would just type something like:
import serial
ser = serial.Serial('/dev/pts/4', 19200)
I want to write a Python Twisted server that serves text to its clients, and I want the clients to be able to write text back to manipulate the server. I will use Telnet, and the clients will use Putty or some similar terminal...I would also be open to using SSH if it is easier to do this.
My question is, how do I configure the server so that the client can send raw, unbuffered bytes (I don't want the user to have to press enter after a command)? Also, is there a way to change the configuration mid-session so that I can change back and forth to and from buffered/unbuffered bytes?
I think it is Telnet option 34 "Linemode" --- http://www.freesoft.org/CIE/RFC/1700/10.htm
I just don't know how to set up Twisted to use that...
Any help setting this up for Telnet or SSH is appreciated!!!
Thanks!
twisted.conch.telnet.TelnetBootstrapProtocol is a good example of how to do some option negotiation. It also happens to perform some LINEMODE negotiation. Take a look at the implementation for details, but here's a snippet that shows the server asking the client to enable linemode, naws, and sga:
for opt in (LINEMODE, NAWS, SGA):
self.transport.do(opt).addErrback(log.err)
A real server might want to do more error handling than log.err if the negotiation fails, since the client will be left in a state that is presumably not ideal for use with the server.
Also take a look at some of the funky terminal demos that come with Twisted. These do lots of character-at-a-time processing.
I currently have a small Python script that I'm using to spawn multiple executables, (voice chat servers), and in the next version of the software, the servers have the ability to receive heartbeat signals on the UDP port. (There will be possibly thousands of servers on one machine, ranging from ports 7878 and up)
My problem is that these servers might (read: will) be running on the same machine as my Python script and I had planned on opening a UDP port, and just sending the heartbeat, waiting for the reply, and voila...I could restart servers when/if they weren't responding by killing the task and re-loading the server.
Problem is that I cannot open a UDP port that the server is already using. Is there a way around this? The project lead is implementing the heartbeat still, so I'm sure any suggestions in how the heartbeat system could be implemented would be welcome also. -- This is a pretty generic script though that might apply to other programs so my main focus is still communicating on that UDP port.
This isn't possible. What you'll have to do is have one UDP master program that handles all UDP communication over the one port, and communicates with your servers in another way (UDP on different ports, named pipes, ...)
I'm pretty sure this is possible on Linux; I don't know about other UNIXes.
There are two ways to propagate a file descriptor from one process to another:
When a process fork()s, the child inherits all the file descriptors of the parent.
A process can send a file descriptor to another process over a "UNIX Domain Socket". See sendmsg() and recvmsg(). In Python, the _multiprocessing extension module will do this for you; see _multiprocessing.sendfd() and _multiprocessing.recvfd().
I haven't experimented with multiple processes listening on UDP sockets. But for TCP, on Linux, if multiple processes all listen on a single TCP socket, one of them will be randomly chosen when a connection comes in. So I suspect Linux does something sensible when multiple processes are all listening on the same UDP socket.
Try it and let us know!