I would like to directly access to syslog messages from Python by reading /dev/log.
My (very limited) understanding is that the correct way is to read from there is to bind a datagram socket.
import socket
sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
sock.bind('/dev/log')
sock.listen(1)
while True:
data, addr = sock.recvfrom(1024)
print(data)
Apparently /dev/log is in use:
Traceback (most recent call last):
File "readlog.py", line 4, in <module>
sock.bind('/dev/log')
OSError: [Errno 98] Address already in use
How should I read /dev/log from Python?
EDIT: per #Barmar's comment - only one process can access /dev/log so that part is clear, the device must be clean before reading from it. sudo lsof /dev/log does not show anything.
A answer in a Java thread around this subject mentioned that syslog should be shut down before. I also tried that, lsof | grep "/dev/log" was empty but I got the error nevertheless.
Isn't it possible to have several processes reading from /dev/log?
There is a socket flag to set, in order to prevent this:
socket.SO_REUSEADDR
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
This flag tells the kernel to reuse a local socket in TIME_WAIT state, without waiting for its natural timeout to expire.
Ref: https://docs.python.org/3/library/socket.html
Related
So before anyone says its a duplicate, I have seen multiple questions with that error, but could not notice any of that being the same as my problem.
I am trying to make a small project including a socket over SSL, and when trying to catch if a user is trying to connect with a raw socket and not ssl wrapped socket (which is raising a ConnectionResetError) I get a different error.
My code:
import socket
from classes import ClientThread
import ssl
from time import sleep
server = 'localhost'
port = 12345
threads = []
context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
context.load_cert_chain(certfile="cert.pem", keyfile="cert.pem")
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind((server, port))
print(f"[*] server started, listening on port {port}")
while True:
s.listen()
with context.wrap_socket(s, server_side=True) as ssock:
try:
conn, addr = ssock.accept()
client = ClientThread(conn=conn, ip=addr[0], port=addr[1])
client.start()
threads.append(client)
print(f'Threads running: {len(threads)}')
except ConnectionResetError:
print(f'Could not establish ssl handshake with a client.')
The error i get is:
Traceback (most recent call last):
File "C:/Users/x/x/server.py", line 17, in <module>
s.listen()
OSError: [WinError 10038] An operation was attempted on something that is not a socket
I tried setting some sleep time after the exception maybe it needed to reset the socket but didnt hlep, tried to play a bit with the placement of the While True, and while resetting the entire socket help, I dont want to reset all my clients thread just because of a client who didnt try to log in with a SSL socket.
I think it has something to do with the wrap_socket because it modified the socket instance passed to it , but couldnt find a way to unwrap.
Thank you in advance!
listen enables a socket to take incoming connection requests (also called a "passive socket") and establishes a backlog of how many of those requests can be pending in the network stack at any given time. accept accepts one of those connections. You call listen once and accept many times.
Pull the listen outside of the while so that is only called once to establish this as a listening socket.
Ok guys, I can't understand this code:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
What does:
socket.AF_INET
socket.SOCK_STREAM
do?
I really read everything abount them, but I can't understand what dows they do, could you please explain me, them in simple words?Thanks for read, have a great day!
TL;DR
socket.AF_INET = The type of socket address
socket.SOCK_STREAM = The type of socket
Explanation
Whenever you provide socket.AF_INET to your server you instruct it to listen to an IPv4 address, and to your client to connect to an IPv4 address. This will work. Same for IPv6. However mixing them up doesn't.
That would be the same me waiting for you to talk to me on StackOverflow while you send me messages by email. We are not looking at the same spot, so we won't be able to communicate.
Same for socket.SOCK_STREAM (and the other values). The difference lies in the fact that this tells python's socket how we are going to communicate. socket.SOCK_STREAM will be TCP, while socket.SOCK_DGRAM will be UDP.
Let's come back to our "real world" example and let's imagine we agreed on communicating by email. I could expect either one email from you (explaining me everything you have to tell) or several (with a part of what you have to say in each email). That's the same as TCP vs UDP.
References
Well, I guess you have read both:
python's socket module
python's socket How to
Potentially:
SO: What is Address Family?
Wikipedia: IPv4
Also, I guess:
Difference Between Socket Types
(and the references therein)
Wikipedia: Network socket - Types
Super long explanation but mostly testing
So. If after all that you don't understand. Let's try:
# server.py
import socket
s = socket.socket()
s.bind(('localhost', 5050))
s.listen(5)
while True:
(clientsocket, address) = s.accept()
print 'client connected'
And:
# client.py
import socket
s = socket.socket(socket.AF_INET)
s.connect(('127.0.0.1', 5050))
print "Yeah! I'm connected :)"
So far, everything as in the how to.
We launch our server:
$ python server.py
And then we launch our client:
$ python client.py
Yeah! I'm connected :)
Everything works fine. That's good.
Now, lets change our client:
# client.py
import socket
s = socket.socket(socket.AF_INET6)
s.connect(('127.0.0.1', 5050))
print "Yeah! I'm connected :)"
And relaunch our new client:
$ python client.py
Traceback (most recent call last):
File "client.py", line 4, in <module>
s.connect(('127.0.0.1', 5050))
File "/.../lib/python2.7/socket.py", line 228, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 65] No route to host
Aie! Everything breaks!
But what happens? 127.0.0.1:5050 is an IPv4 address, hence the socket module tells us it's not happy about what we are doing! Indeed, we said our connection will be using an IPv6 address but we are providing it with an IPv4 address.
Okay... But if I use the same address but in IPv6, will it work? Let's try by changing our client (you could check out this SO answer for the equivalent of 127.0.0.1 for IPv6):
# client.py
import socket
s = socket.socket(socket.AF_INET6)
s.connect(('::1', 5050))
print "Yeah! I'm connected :)"
and our server:
# server.py
import socket
s = socket.socket(socket.AF_INET6)
s.bind(('::1', 5050))
s.listen(5)
while True:
(clientsocket, address) = s.accept()
print 'client connected'
We relaunch our server and our client:
$ python client.py
Yeah! I'm connected :)
Success!
The same procedure could be used to understand/test the socket.SOCK_STREAM parameter (but I think my answer is already long enough).
Hope this helped :).
socket.STREAM is the kind of socket you want. In this case, you are looking to stream bytes to and from the host you want to connect to, rather than just issue a one-time call. This means that the socket will listen for bytes until it receives an empty byte b'', at which point it will terminate the connection (because the stream is complete).
I would imagine you aren't too worried about the type of socket, so low-level understanding here isn't paramount, nor could I give you a particularly satisfactory answer to that, either.
socket.AF_INET is the AddressFamily, ipv4 or ipv6. This tells sockets what kind of address to expect. You will most likely use ipv4, so (host, port) tuple will work just fine.
AF_INET is well described in there. It is basically the method you are using for sending data over to the other socket. SOCK_STREAM basically describes that you are sending using TCP and essentially describes rules for the endpoint to which you are sending and recieving data (IP adress and port number).
But since you seem confused over these terms I'd suggest just just think of them as specifications on how you are going to transmit data between your two socket endpoints.
I am experimenting with python sockets to try to understand the whole concept better, but I have run into a problem. I have a simple server and a client, where the client sends a list to the server, and then waits for the server to send a string signaling the process is complete.
This is the client file:
import socket
import json
host = '192.168.1.102'
port = 14314
def request():
print 'Connecting'
clientsocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
clientsocket.connect((host, port))
print 'Sending request'
clientsocket.sendall(json.dumps([1, 2, 3, 4, 5, 6, 7, 8, 9]))
print 'Receiving data'
data = clientsocket.recv(512)
print 'Received: {}'.format(data)
request()
and here is the server file:
import socket
import json
host = '192.168.1.102'
port = 14314
def run():
print 'Binding socket'
serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serversocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
serversocket.bind((host, port))
print 'Waiting for client'
serversocket.listen(1)
clientsocket, addr = serversocket.accept()
print 'Receiving data'
raw_data = ''
while True:
tmp = clientsocket.recv(1024)
if not tmp:
break
raw_data += tmp
data = json.loads(raw_data)
print 'Received: {}'.format(data)
print 'Sending data'
clientsocket.sendall('done')
run()
The problem is that while the client is done sending the list, the server is stuck in the recv loop, waiting for nothing. The whole data has been received in the first iteration and in the second iteration there is nothing to be received because the client has moved on to the receiving part.
The weird part is that if I comment out the receive part from the client and the send part from the server, the process completes successfully. So, what am I doing wrong? Why is this not working?
Thanks.
The Docs for socket.recv talk about additional flags being able to be passed in to the recv function described in the unix documentation. So turning to that documentation, I found the following message:
If no messages are available at the socket, the receive calls wait for
a message to arrive, unless the socket is nonblocking (see fcntl(2)),
in which case the value -1 is returned
So once again, we're directed to another page. The documentation for fcntl says
Performs one of the operations described below on the open file
descriptor
So, normally the socket.recv function is blocking (it will wait indefinitely for new data), unless we use a file descriptor. How do we do that? Well there is a socket.makefile function that gives us a file descriptor attached to the socket. Cool. This SO question gives us an example of how we can read and write to a socket, using a file descriptor.
Well what if we don't want to use a file descriptor. Reading further into the unix documentation for the recv function, I see that I can use the MSG_DONTWAIT flag. This doesn't work in Windows, but I did find out that we can use socket.setbocking(False) to permamently change the socket to non-blocking mode. You would then need to ignore any "A non-blocking socket operation could not be completed immediately" errors. Those are normal and non-fatal(error #10035 of this page mentions it is non-fatal).
Another possible implementation would be to multi-thread your program, you can implement a receiving and a sending thread for your socket. This might give you the best performance, but it would be a lot of work to setup.
Python is awesome. I just found some libraries Python has that does asynchronous sockets too. There's asyncore, asynchat which have both been deprecated in favor of asyncio if that is available in the version of Python you are using.
Sorry for throwing so much out there. I don't know a whole lot about sockets. I used them once with the Paramiko library, and that was it. But it looks like there are a lot of ways of implementing them.
I found this script of TCP server which "echoes" back the data to the client.
#!/usr/bin/env python
import socket
host = ''
port = 50000
backlog = 5
size = 1024
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((host,port))
s.listen(backlog)
while 1:
client, address = s.accept()
data = client.recv(size)
if data:
client.send(data)
client.close()
I'm trying to test & understand it before I will be able to do something on my own and modify, but I'm having some problems. When I'm trying to run the .py script I get the following error in my Terminal (using Ubuntu 14.04 LTS)
> Traceback (most recent call last):
File "echo.py", line 14, in <module>
s.bind((host,port))
File "/usr/lib/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 98] Address already in use
My Python version is 2.7.6
is there something wrong with the code or I'm doing something wrong?
UPDATE:
it gets worse, any script I run with bind(host, port) gives me the same error.
any help would be appreciated
Perhaps you accidentally ran the EchoServer twice in different windows? You can only bind one receiver to a port/address combination.
Seems like there is some other application running on those ports.
Can you try checking if there are any other app listening on same port using:
netstat -ntlpu | grep 50000
To bind a server it's a little confusing but you have to use a tuple. The correct way is server.bind((host, port))
I am having some problems using raw IPv6 sockets in python. I connect via:
if self._socket != None:
# Close out old socket first
self._socket.close()
self._socket = socket.socket(socket.AF_INET6, socket.SOCK_RAW)
self._socket.bind((self._interface,0))
self._socket.sendall(data)
where self._interface is my local address; specifically "fe80::fa1e:dfff:fed6:221d". When trying this, I get the following error:
File "raw.py", line 164, in connect
self._socket.bind((self._interface,0))
File "<string>", line 1, in bind
socket.error: [Errno 49] Can't assign requested address
If I use my ipv6 localhost address for self._interface ("::1") I can actually bind the address, but can not send anything:
self._socket.sendall(data)
File "<string>", line 1, in sendall
socket.error: [Errno 39] Destination address required
Why would a raw socket need a destination address? Has anyone worked with raw IPv6 sockets in python, and can help me understand why this is happening?
Although this is an old question, i thought of adding an answer that works and helps any one who stumbles upon it latter.
The key problems are:
Raw sockets are not bound and connected to other sockets. Also sendto is the correct api to use.
Moreover, 4 tuple structure for destination address is required for ipv6 packets as opposed to two tuple ones for ipv4.
Lastly, the stack (at least on Linux mint 15) is more strict on ipv6 packets. If you try sending an empty icmpv4 echo request, python allows it and sends a meaning less packet on wire. Where as in case of ipv6, it simply gives error of 'invalid argument' when you try sending an empty packet. Hence a valid request is also required in case of ipv6. Following example does that all for ipv6 and sends a valid ping echo request to loop back address.
import socket
def main(dest_name):
addrs = socket.getaddrinfo(dest_name, 0, socket.AF_INET6, 0, socket.SOL_IP)
print addrs
dest = addrs[0]
# A minimal ICMP6-echo message (thanks to abarnert)
data = '\x80\0\0\0\0\0\0\0'
icmp = socket.getprotobyname('ipv6-icmp')
#print icmp
send_socket = socket.socket(socket.AF_INET6, socket.SOCK_RAW, icmp)
print "sent to " + str(dest[4])
send_socket.sendto(data, dest[4])
send_socket.close()
if __name__ == '__main__':
main('::1')
I don't understand your combination of bind and sendall. In my understanding, bind is for server sockets and sendall requires a connection. Did you mean connect instead of bind?
Anyway, the IPv6 equivalent of INADDR_ANY is, according to the man page, IN6ADDR_ANY_INIT. Python does not define a constant for it, but this is the same as '::' (all zero).
This worked for me (as root):
>>> import socket
>>> s = socket.socket(socket.AF_INET6, socket.SOCK_RAW, socket.IPPROTO_RAW)
>>> s.bind(('::', 0))
EDIT:
Oops, I first did not saw that you actually managed to bind the socket to an address. However your second problem is obvious: You must first connect to some address before you can send data. Or use sendto with an address. This is not different from IPv4.
This code provides a raw socket with L2 access. Unfortunately OSX does not support the socket.PF_PACKET...
soc = socket.socket(socket.PF_PACKET, socket.SOCK_RAW) #create the raw-socket
soc.bind(("lo0", 0))
soc.send(packet)