We Keep Coding

Pyserial - COM disconnection and reconnection possible?

new to Python but it really makes fun to work with :-)
Using the pyserial lib and works fine so far. BUT...
...is there a way to ignore the following problem: During a serial communication I disconnect the COM-cable for a short time. I got the following errormessage then:
**Traceback (most recent call last):
File "C:\Users\greulich\PycharmProjects\arduino_serial\main.py", line 48, in <module>
functions.receiveWithStartMarkers()
File "C:\Users\greulich\PycharmProjects\arduino_serial\functions.py", line 30, in receiveWithStartMarkers
receivedChar = serialPort.read(1) # read 1 byte
File "C:\Users\greulich\PycharmProjects\arduino_serial\venv\lib\site-packages\serial\serialwin32.py", line 275, in read
raise SerialException("ClearCommError failed ({!r})".format(ctypes.WinError()))
serial.serialutil.SerialException: ClearCommError failed (PermissionError(13, 'Das Gerät erkennt den Befehl nicht.', None, 22))**
My code looks like that:
while serialPort.is_open is True and newData is False:
#try:
receivedChar = serialPort.read(1) # read 1 byte
print(str(date.time()) + ' >>> ' + 'I got the following byte: ' + str(receivedChar))
I opened up port initially in that module:
try:
serialPort = serial.Serial('COM12', 115200)
except:
print('COM-Port not available!')
print('Will exit not the Python program!')
exit() #quits the complete Python program
serialPort.timeout = 3
Is there a way to define kind of a timeout until this error will hit me where the user has the chance to reconnect the cable?
In a nutshell: I want to be able to disconnect the com cable for a short time and connect it again without an error showing :-)
Thanks,
Markus

Fastest pinging method via python?

I'm looking for the fastest pinging method via python. I need to ping over 100,000 servers and my current procedure below takes approximately 85 minutes to complete. I've read small snippets about scapy, along with general ICMP and python ping. I need to know a definitive method, or at least a solid way to test, which is the fastest. I cannot test python - ping from work as it is not an approved package. I also tried a code snippet for scapy, but got an error:
OSError: Windows native L3 Raw sockets are only usable as administrator !
Install 'Winpcap/Npcap to workaround !
So I'm admittedly looking for code snippets I can test at home or ways around that error from more experienced persons
To prove I've tried, here are some related posts, as well as my current code
Current code:
import pandas as pd
import subprocess
import threading
raw_list = []
raw_list2 = []
def ping(host):
raw_list.append(host+ ' '+ str((subprocess.run('ping -n 3 -w 800 '+host).returncode)))
with open(r"FILEPATH", "r") as server_list_file:
hosts = server_list_file.read()
hosts_list = hosts.split('\n')
num_threads = 100
num_threads2 = 10
num_threads3 = 1
number = 0
while number<len(hosts_list):
print(number)
if len(hosts_list)>number+num_threads:
for i in range(num_threads):
t = threading.Thread(target=ping, args=(hosts_list[number+i],))
t.start()
t.join()
number = number + num_threads
elif len(hosts_list)>(number+num_threads2):
for i in range(num_threads2):
t = threading.Thread(target=ping, args=(hosts_list[number+i],))
t.start()
t.join()
number = number + num_threads2
elif len(hosts_list)>(number+num_threads3-1):
for i in range(num_threads3):
t = threading.Thread(target=ping, args=(hosts_list[number+i],))
t.start()
t.join()
number = number + num_threads3
else:
number = number+1
for x in range(len(raw_list)):
if(raw_list[x][-1] == '0'):
raw_list2.append(raw_list[x][0:-2])
to_csv_list = pd.DataFrame(raw_list2)
to_csv_list.to_csv('ServersCsv.csv', index = False, header = False)
to_csv_list.to_csv(r'ANOTHERFILEPATH', index = False, header = False)
subprocess.call(r'C:\ProgramData\Anaconda3\python.exe "A_PROGRAM_THAT_INSERTS_INTO_SQL"')
This does exactly what I need, however, it does not do it quickly enough.
I've tried the very small snippet:
from scapy.all import *
packets = IP(dst=["www.google.com", "www.google.fr"])/ICMP()
results = sr(packets)
resulting in gaierror: [Errno 11001] getaddrinfo failed
I've also tried:
TIMEOUT = 2
conf.verb = 0
packet = IP("ASERVERNAME", ttl=20)/ICMP()
reply = sr1(packet, timeout=TIMEOUT)
if not (reply is None):
print(reply.dst + "is online")
else:
print("Timeout waiting for %s") % packet[IP].dst
resulting in:
OSError: Windows native L3 Raw sockets are only usable as administrator !
Install Winpcap/Npcap to workaround !
A few links I looked at but could not garner a solid answer from:
Ping a site in Python?
Fastest way to ping a host in python?

This only solves the Python part. The comments are very right.
OSError: Windows native L3 Raw sockets are only usable as administrator ! Install Winpcap/Npcap to workaround !
I find this pretty damn explicit. If you follow's Scapy documentation for windows it says you need to install Npcap.https://nmap.org/npcap/
Other than that,
packets = IP(dst=["www.google.com", "www.google.fr"])/ICMP()
results = sr(packets)
Is likely the cleanest way to go. Works on my machine.. make sure you're using the latest development version from GitHub (unzip it and install it via python setup.py install).
If you are using the latest version, you might even want to turn on threaded=True in sr() to send and receive packets on two threads, as pointed out by the comments. You might also want to use prn and store=False to not store the answers (100k is a lot)

PySerial client unable to write data

I'm trying to write a python program which can communicate over a serial interface using PySerial module as follows:
import serial
if __name__ == '__main__':
port = "/dev/tnt0"
ser = serial.Serial(port, 38400)
print ser.name
print ser.isOpen()
x = ser.write('hello')
ser.close()
print "Done!"
But if I execute the above I get the following error:
/dev/tnt0
True
Traceback (most recent call last):
File "/home/root/nested/test.py", line 15, in <module>
x = ser.write('hello')
File "/usr/local/lib/python2.7/dist-packages/serial/serialposix.py", line 518, in write
raise SerialException('write failed: %s' % (v,))
serial.serialutil.SerialException: write failed: [Errno 22] Invalid argument
I referred to the pyserial documentation and according to that this should work without an issue. Please let me know what i'm doing wrong in this.
TIA!

For some reason, in order to use the module tty0tty, you need to open both /dev/tnt0 and /dev/tnt1, or any of the other pairs (e.g /dev/tnt2 and /dev/tnt3).
The code below works:
import time
import serial
def main():
vserial0 = serial.Serial(port='/dev/tnt0', baudrate=9600, bytesize=8, parity=serial.PARITY_EVEN, stopbits=1)
vserial1 = serial.Serial(port='/dev/tnt1', baudrate=9600, bytesize=8, parity=serial.PARITY_EVEN, stopbits=1)
n_bytes = 0
while n_bytes == 0:
vserial0.write('test')
n_bytes = vserial1.inWaiting()
time.sleep(0.05)
print vserial1.read(n_bytes)
if __name__ == '__main__':
main()

/dev/tntX are emulated port pairs, and to perform a successful read or write you need to open both ports from a pair.
Think of it as a pipe - if one end is closed, you will be not able to push the data through.

Finding Bluetooth low energy with python

Is it possible for this code to be modified to include Bluetooth Low Energy devices as well? https://code.google.com/p/pybluez/source/browse/trunk/examples/advanced/inquiry-with-rssi.py?r=1
I can find devices like my phone and other bluetooth 4.0 devices, but not any BLE. If this cannot be modified, is it possible to run the hcitool lescan and pull the data from hci dump within python? I can use the tools to see the devices I am looking for and it gives an RSSI in hcidump, which is what my end goal is. To get a MAC address and RSSI from the BLE device.
Thanks!

As I said in the comment, that library won't work with BLE.
Here's some example code to do a simple BLE scan:
import sys
import os
import struct
from ctypes import (CDLL, get_errno)
from ctypes.util import find_library
from socket import (
socket,
AF_BLUETOOTH,
SOCK_RAW,
BTPROTO_HCI,
SOL_HCI,
HCI_FILTER,
)
if not os.geteuid() == 0:
sys.exit("script only works as root")
btlib = find_library("bluetooth")
if not btlib:
raise Exception(
"Can't find required bluetooth libraries"
" (need to install bluez)"
)
bluez = CDLL(btlib, use_errno=True)
dev_id = bluez.hci_get_route(None)
sock = socket(AF_BLUETOOTH, SOCK_RAW, BTPROTO_HCI)
sock.bind((dev_id,))
err = bluez.hci_le_set_scan_parameters(sock.fileno(), 0, 0x10, 0x10, 0, 0, 1000);
if err < 0:
raise Exception("Set scan parameters failed")
# occurs when scanning is still enabled from previous call
# allows LE advertising events
hci_filter = struct.pack(
"<IQH",
0x00000010,
0x4000000000000000,
0
)
sock.setsockopt(SOL_HCI, HCI_FILTER, hci_filter)
err = bluez.hci_le_set_scan_enable(
sock.fileno(),
1, # 1 - turn on; 0 - turn off
0, # 0-filtering disabled, 1-filter out duplicates
1000 # timeout
)
if err < 0:
errnum = get_errno()
raise Exception("{} {}".format(
errno.errorcode[errnum],
os.strerror(errnum)
))
while True:
data = sock.recv(1024)
# print bluetooth address from LE Advert. packet
print(':'.join("{0:02x}".format(x) for x in data[12:6:-1]))
I had to piece all of that together by looking at the hcitool and gatttool source code that comes with Bluez. The code is completely dependent on libbluetooth-dev so you'll have to make sure you have that installed first.
A better way would be to use dbus to make calls to bluetoothd, but I haven't had a chance to research that yet. Also, the dbus interface is limited in what you can do with a BLE connection after you make one.
EDIT:
Martin Tramšak pointed out that in Python 2 you need to change the last line to print(':'.join("{0:02x}".format(ord(x)) for x in data[12:6:-1]))

You could also try pygattlib. It can be used to discover devices, and (currently) there is a basic support for reading/writing characteristics. No RSSI for now.
You could discover using the following snippet:
from gattlib import DiscoveryService
service = DiscoveryService("hci0")
devices = service.discover(2)
DiscoveryService accepts the name of the device, and the method discover accepts a timeout (in seconds) for waiting responses. devices is a dictionary, with BL address as keys, and names as values.
pygattlib is packaged for Debian (or Ubuntu), and also available as a pip package.

Listing available com ports with Python

I am searching for a simple method to list all available com port on a PC.
I have found this method but it is Windows-specific: Listing serial (COM) ports on Windows?
I am using Python 3 with pySerial on a Windows 7 PC.
I have found in the pySerial API (http://pyserial.sourceforge.net/pyserial_api.html) a function serial.tools.list_ports.comports() that lists com ports (exactly what I want).
import serial.tools.list_ports
print(list(serial.tools.list_ports.comports()))
But it seems that it doesn't work. When my USB to COM gateway is connected to the PC (I see the COM5 in the Device Manager), this COM port isn't included in the list returned by list_ports.comports(). Instead I only get COM4 which seems to be connected to a modem (I don't see it in the COM&LPT section of Device Manager)!
Do you know why it doesn't work? Have you got another solution which is not system specific?

This is the code I use.
Successfully tested on Windows 8.1 x64, Windows 10 x64, Mac OS X 10.9.x / 10.10.x / 10.11.x and Ubuntu 14.04 / 14.10 / 15.04 / 15.10 with both Python 2 and Python 3.
import sys
import glob
import serial
def serial_ports():
""" Lists serial port names
:raises EnvironmentError:
On unsupported or unknown platforms
:returns:
A list of the serial ports available on the system
"""
if sys.platform.startswith('win'):
ports = ['COM%s' % (i + 1) for i in range(256)]
elif sys.platform.startswith('linux') or sys.platform.startswith('cygwin'):
# this excludes your current terminal "/dev/tty"
ports = glob.glob('/dev/tty[A-Za-z]*')
elif sys.platform.startswith('darwin'):
ports = glob.glob('/dev/tty.*')
else:
raise EnvironmentError('Unsupported platform')
result = []
for port in ports:
try:
s = serial.Serial(port)
s.close()
result.append(port)
except (OSError, serial.SerialException):
pass
return result
if __name__ == '__main__':
print(serial_ports())

Basically mentioned this in pyserial documentation
https://pyserial.readthedocs.io/en/latest/tools.html#module-serial.tools.list_ports
import serial.tools.list_ports
ports = serial.tools.list_ports.comports()
for port, desc, hwid in sorted(ports):
print("{}: {} [{}]".format(port, desc, hwid))
Result :
COM1: Communications Port (COM1) [ACPI\PNP0501\1]
COM7: MediaTek USB Port (COM7) [USB VID:PID=0E8D:0003 SER=6 LOCATION=1-2.1]

You can use:
python -c "import serial.tools.list_ports;print serial.tools.list_ports.comports()"
Filter by know port:
python -c "import serial.tools.list_ports;print [port for port in serial.tools.list_ports.comports() if port[2] != 'n/a']"
See more info here:
https://pyserial.readthedocs.org/en/latest/tools.html#module-serial.tools.list_ports

one line solution with pySerial package.
python -m serial.tools.list_ports

A possible refinement to Thomas's excellent answer is to have Linux and possibly OSX also try to open ports and return only those which could be opened. This is because Linux, at least, lists a boatload of ports as files in /dev/ which aren't connected to anything. If you're running in a terminal, /dev/tty is the terminal in which you're working and opening and closing it can goof up your command line, so the glob is designed to not do that. Code:
# ... Windows code unchanged ...
elif sys.platform.startswith ('linux'):
temp_list = glob.glob ('/dev/tty[A-Za-z]*')
result = []
for a_port in temp_list:
try:
s = serial.Serial(a_port)
s.close()
result.append(a_port)
except serial.SerialException:
pass
return result
This modification to Thomas's code has been tested on Ubuntu 14.04 only.

refinement on moylop260's answer:
import serial.tools.list_ports
comlist = serial.tools.list_ports.comports()
connected = []
for element in comlist:
connected.append(element.device)
print("Connected COM ports: " + str(connected))
This lists the ports that exist in hardware, including ones that are in use. A whole lot more information exists in the list, per the pyserial tools documentation

Probably late, but might help someone in need.
import serial.tools.list_ports
class COMPorts:
def __init__(self, data: list):
self.data = data
#classmethod
def get_com_ports(cls):
data = []
ports = list(serial.tools.list_ports.comports())
for port_ in ports:
obj = Object(data=dict({"device": port_.device, "description": port_.description.split("(")[0].strip()}))
data.append(obj)
return cls(data=data)
#staticmethod
def get_description_by_device(device: str):
for port_ in COMPorts.get_com_ports().data:
if port_.device == device:
return port_.description
#staticmethod
def get_device_by_description(description: str):
for port_ in COMPorts.get_com_ports().data:
if port_.description == description:
return port_.device
class Object:
def __init__(self, data: dict):
self.data = data
self.device = data.get("device")
self.description = data.get("description")
if __name__ == "__main__":
for port in COMPorts.get_com_ports().data:
print(port.device)
print(port.description)
print(COMPorts.get_device_by_description(description="Arduino Leonardo"))
print(COMPorts.get_description_by_device(device="COM3"))

Please, try this code:
import serial
ports = serial.tools.list_ports.comports(include_links=False)
for port in ports :
print(port.device)
first of all, you need to import package for serial port communication,
so:
import serial
then you create the list of all the serial ports currently available:
ports = serial.tools.list_ports.comports(include_links=False)
and then, walking along whole list, you can for example print port names:
for port in ports :
print(port.device)
This is just an example how to get the list of ports and print their names, but there some other options you can do with this data. Just try print different variants after
port.

something simple but I use it a lot.
import serial.tools.list_ports as ports
com_ports = list(ports.comports()) # create a list of com ['COM1','COM2']
for i in com_ports:
print(i.device) # returns 'COMx'

try this code
import serial.tools.list_ports
for i in serial.tools.list_ports.comports():
print(i)
it returns
COM1 - Port de communication (COM1)
COM5 - USB-SERIAL CH340 (COM5)
if you just wont the name of the port for exemple COM1
import serial.tools.list_ports
for i in serial.tools.list_ports.comports():
print(str(i).split(" ")[0])
it returns
COM1
COM5
as in my case
py 3.7 64bits

Works only on Windows:
import winreg
import itertools
def serial_ports() -> list:
path = 'HARDWARE\\DEVICEMAP\\SERIALCOMM'
key = winreg.OpenKey(winreg.HKEY_LOCAL_MACHINE, path)
ports = []
for i in itertools.count():
try:
ports.append(winreg.EnumValue(key, i)[1])
except EnvironmentError:
break
return ports
if __name__ == "__main__":
ports = serial_ports()

Several options are available:
Call QueryDosDevice with a NULL lpDeviceName to list all DOS devices. Then use CreateFile and GetCommConfig with each device name in turn to figure out whether it's a serial port.
Call SetupDiGetClassDevs with a ClassGuid of GUID_DEVINTERFACE_COMPORT.
WMI is also available to C/C++ programs.
There's some conversation on the win32 newsgroup and a CodeProject, er, project.

One thing to note, codes like this:
for i in serial.tools.list_ports.comports():
print(i)
Return the following:
COM7 - Standard Serial over Bluetooth link (COM7) COM1 - Communications Port (COM1) COM8 - Standard Serial over Bluetooth link (COM8) COM4 - USB-SERIAL CH340 (COM4)
If you want the ports listed in order, and only the ones available to you, try:(credit to tfeldmann)
def serial_ports():
""" Lists serial port names
:raises EnvironmentError:
On unsupported or unknown platforms
:returns:
A list of the serial ports available on the system
"""
if sys.platform.startswith('win'):
ports = ['COM%s' % (i + 1) for i in range(256)]
elif sys.platform.startswith('linux') or sys.platform.startswith('cygwin'):
# this excludes your current terminal "/dev/tty"
ports = glob.glob('/dev/tty[A-Za-z]*')
elif sys.platform.startswith('darwin'):
ports = glob.glob('/dev/tty.*')
else:
raise EnvironmentError('Unsupported platform')
result = []
for port in ports:
try:
s = serial.Serial(port)
s.close()
result.append(port)
except (OSError, serial.SerialException):
pass
return result
This returns the following:
['COM1', 'COM4', 'COM8']
So unlike the first example, where the result was ['COM7', 'COM1', 'COM8', 'COM4'], this time I get all of the com ports in order, and only the ones available. Very handy if you need them in order, and tested to see if they're available.