We Keep Coding

socketcan J1939 filter use in python

In Python, I am trying to use the J1939 filtering as mentionned in the linux kernel docs: https://www.kernel.org/doc/html/latest/networking/j1939.html
The following code fails at the setsockopt() line (setting up filters):
import socket
import struct
def pack_J1939_filters(can_filters):
can_filter_fmt = "=" + "2Q2B2I" * len(can_filters)
filter_data = []
for can_filter in can_filters:
name = can_filter['name']
name_mask = can_filter['name_mask']
addr = can_filter['addr']
addr_mask = can_filter['addr_mask']
pgn = can_filter['pgn']
pgn_mask = can_filter['pgn_mask']
filter_data.append(name)
filter_data.append(name_mask)
filter_data.append(addr)
filter_data.append(addr_mask)
filter_data.append(pgn)
filter_data.append(pgn_mask)
return struct.pack(can_filter_fmt, *filter_data)
s = socket.socket(socket.PF_CAN, socket.SOCK_DGRAM, socket.CAN_J1939)
interface = "vcan0"
src_name = socket.J1939_NO_NAME
src_pgn = socket.J1939_NO_PGN
src_addr = 0x81
src_sck_addr = (interface, src_name, src_pgn, src_addr)
s.bind(src_sck_addr)
filters = [{"name": 0, "name_mask":0, "addr":0, "addr_mask":0, "pgn": 0, "pgn_mask": 0}]
packed_filters = pack_J1939_filters(filters)
# socket.SOL_CAN_J1939 does not seem to exist
SOL_CAN_BASE = 100
CAN_J1939 = 7
SOL_CAN_J1939 = SOL_CAN_BASE + CAN_J1939
s.setsockopt(SOL_CAN_J1939, socket.SO_J1939_FILTER , packed_filters)
s.recvfrom(128)
s.close()
First, the kernel documentation mentions to use SOL_CAN_J1939 as the first argument. However socket.SOL_CAN_J1939 does not exist in the socket package. So looking at the code at this location I was able to understand that this int value should be 107: http://socket-can.996257.n3.nabble.com/RFC-v3-0-6-CAN-add-SAE-J1939-protocol-td7571.html
As for the setsockopt() third argument, I packed the filters to match the j1939_filter structure (26 bytes as described in the code from the previous link). This is similar to what is done in can.interfaces.socketcan.utils for raw CAN.
What am I doing wrong to cause setsockopt() to fail?

The first issue was with the struct.pack format (can_filter_fmt) being wrong. I first assumed that the kernel j1939_filter structure size was the sum of the members. This is wrong since the compiler adds padding. This can be added to the struct.pack format as x such as 2Q2I2B6x. Please see Why isn't sizeof for a struct equal to the sum of sizeof of each member?
The second issue was that can_filter_fmt is not packed as 2Q2B2I but as 2Q2I2B6x (the addr member is in the middle).
As for SOL_CAN_J1939 I was correct and needs to be created in file because it is not yet in the package.
The final code is the following:
#!/usr/bin/env python3
import socket
import struct
def pack_J1939_filters(can_filters=None):
if can_filters is None:
# Pass all messages
can_filters = [{}]
can_filter_fmt = "=" + "2Q2I2B6x" * len(can_filters)
filter_data = []
for can_filter in can_filters:
if 'name' in can_filter:
name = can_filter['name']
else:
name = 0
if 'name_mask' in can_filter:
name_mask = can_filter['name_mask']
else:
name_mask = 0
if 'pgn' in can_filter:
pgn = can_filter['pgn']
else:
pgn = 0
if 'pgn_mask' in can_filter:
pgn_mask = can_filter['pgn_mask']
else:
pgn_mask = 0
if 'addr' in can_filter:
addr = can_filter['addr']
else:
addr = 0
if 'addr_mask' in can_filter:
addr_mask = can_filter['addr_mask']
else:
addr_mask = 0
filter_data.append(name)
filter_data.append(name_mask)
filter_data.append(pgn)
filter_data.append(pgn_mask)
filter_data.append(addr)
filter_data.append(addr_mask)
return struct.pack(can_filter_fmt, *filter_data)
def print_msg(data, sck_addr):
print(f"SA:{hex(sck_addr[3])} PGN:{hex(sck_addr[2])}")
for j in range(len(data)):
if j % 8 == 0 and j != 0:
print()
if j % 8 == 0:
print(f"bytes {j} to {j+7}: ", end="")
print(f"{hex(data[j])} ", end="")
print()
print()
def main():
s = socket.socket(socket.PF_CAN, socket.SOCK_DGRAM, socket.CAN_J1939)
# allows to receive broadcast messages
s.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)
interface = "vcan0"
src_name = socket.J1939_NO_NAME
src_pgn = socket.J1939_NO_PGN # always no PGN for source, unless filtering is needed
src_addr = 0x81 # recvfrom() will not return destination specific messages for other addresses
src_sck_addr = (interface, src_name, src_pgn, src_addr)
s.bind(src_sck_addr)
packed_filters = pack_J1939_filters()
SOL_CAN_BASE = 100
CAN_J1939 = 7
SOL_CAN_J1939 = SOL_CAN_BASE + CAN_J1939
s.setsockopt(SOL_CAN_J1939, socket.SO_J1939_FILTER , packed_filters)
(recv_data, recv_sck_addr) = s.recvfrom(128)
print_msg(recv_data, recv_sck_addr)
s.close()
if __name__ == "__main__":
main()
Thank you.

For J1939 to work with SocketCAN you need two things:
kernel 5.4+
can-j1939 kernel module enabled
Testing for can-1939:
If you install can-utils and after sudo modprobe can-j1939 all you get is fatal error, or if you start testj1939 from can-utils and you get error that protocol is not supported, then it means that can-j1939 was not enabled in your kernel and you need to compile it manually.
Here are my instructions for enabling can-j1939 in Debian 10 kernel:
https://github.com/linux-can/can-utils/blob/master/can-j1939-install-kernel-module.md

How to Grab IP address from ping in Python

I am currently using python 2.7 and will need to ping windows and linux.
I want to create a function that will return the IP address from a ping within a python script. I currently have this function
def ping(host):
"""
Returns True if host responds to a ping request
"""
import subprocess, platform
# Ping parameters as function of OS
ping_str = "-n 1" if platform.system().lower()=="windows" else "-c 1"
args = "ping " + " " + ping_str + " " + host
need_sh = False if platform.system().lower()=="windows" else True
# Ping
return subprocess.call(args, shell=need_sh) == 0
Right now it just returns true or false but is there a way I can run ping(google.com) and have it return 216.58.217.206. I have a list of servers and IPs and I need to make sure that the IP addresses match the FQDN.

You can use the socket to get the IP of the host.
import socket
print(socket.gethostbyname('www.example.com'))

Not sure how no-one has tried this method yet (for windows anyways)!
Use a WMI query of W32_PingStatus
With this method we return an object full of goodies
import wmi
# new WMI object
c = wmi.WMI()
# here is where the ping actually is triggered
x = c.Win32_PingStatus(Address='google.com')
# how big is this thing? - 1 element
print 'length x: ' ,len(x)
#lets look at the object 'WMI Object:\n'
print x
#print out the whole returned object
# only x[0] element has values in it
print '\nPrint Whole Object - can directly reference the field names:\n'
for i in x:
print i
#just a single field in the object - Method 1
print 'Method 1 ( i is actually x[0] ) :'
for i in x:
print 'Response:\t', i.ResponseTime, 'ms'
print 'TTL:\t', i.TimeToLive
#or better yet directly access the field you want
print '\npinged ', x[0].ProtocolAddress, ' and got reply in ', x[0].ResponseTime, 'ms'
Screenshot of output:

Variable repeating depending on loop

I'm searching a string into 2 websites.
Each website have the same string, now the main problem is:
s = "name:'(.*)',"
WebType = re.findall(s, html1)
for name in WebType:
if 'RoundCube' or 'SquirrelMail' in name:
print host + "--> " + name
So basically, I think that the results repeat depending on loop results.
The results are:
https://website1.com--> Roundcube
https://website1.com--> SquirrelMail
https://website2.com--> Roundcube
https://website2.com--> SquirrelMail
How can I make the results to be:
https://website1.com--> RoundCube, SquirrelMail
https://website2.com--> Roundcube, SquirrelMail

dursk's solution above should have worked. However, a longer solution is:
s = "name:'(.*)',"
WebType = re.findall(s, html1)
results = []
for name in WebType:
if 'RoundCube' or 'SquirrelMail' in name:
results.append(name)
print host + "--> " + ', '.join(results)

Try this code:
import random
hosts = ['https://website1.com', 'https://website2.com']
WebList = ['RoundCube', 'SquirrelMail'] + ['webtype_{}'.format(x) for x in range(2)]
WebType = WebList[random.randint(0, len(WebList)):]
name_list = ['RoundCube', 'SquirrelMail']
for host in hosts:
name = filter(set(WebType).__contains__, name_list)
if len(name):
print host + "--> " + str(name).lstrip('[').rstrip(']').replace('\'', '')
I created WebList for testing purposes.
In your code, you will not need to import random unless you want to test what's here.
It outputs:
https://website1.com--> RoundCube, SquirrelMail
https://website2.com--> RoundCube, SquirrelMail
It also outputs:
https://website1.com--> SquirrelMail
https://website2.com--> SquirrelMail
You might need to adjust it to your needs.

why does python loop continue work differently from other programming languages

in other programming languages when a continue is met in a loop it doesn't run the code below it and just does the next loop based on the condition set.
Yet in python it would actually not trigger the continue as much as 3 times on the same exact values until the continue actually triggers finally can someone tell me why it's like this?
function
def get_section(self, address):
for section in self.sections:
section_base = section.image_base + section.VirtualAddress
section_end = section_base + section.Misc_VirtualSize
print 'section_base= 0x%x' % section_base, ' section_end = 0x%x' % section_end
print 'VirtualAdderss = 0x%x' % section.VirtualAddress, 'Misc_virtualSize = 0x%x' % section.Misc_VirtualSize
if address < section_base or address >= section_end:
print 'continuued'
continue
print 'not continuued'
print 'Section name = ', section.section_name
return section
raise NotImplementedError()
Here is the log
address = 0x4013f8
section_base= 0x401000 section_end = 0x5574e5
VirtualAdderss = 0x1000 Misc_virtualSize = 0x1564e5
not continuued
Section name = text
address = 0x4013f8
section_base= 0x401000 section_end = 0x5574e5
VirtualAdderss = 0x1000 Misc_virtualSize = 0x1564e5
not continuued
Section name = text
address = 0x55869c
section_base= 0x401000 section_end = 0x5574e5
VirtualAdderss = 0x1000 Misc_virtualSize = 0x1564e5
continuued
section_base= 0x558000 section_end = 0x5818ac
VirtualAdderss = 0x158000 Misc_virtualSize = 0x298ac
not continuued
Section name = rdata
as you can see it didn't continue 2 times only on the 3rd time it continued and I can't figure out why shouldn't it work from the first time?

The first two times, the if condition was not met; therefore the continue statement was not executed.

How do I get Boto to return EC2 instances - S3 works fine

I'm having some issues with the EC2 bit of Boto (Boto v2.8.0, Python v2.6.7).
The first command returns a list of S3 Buckets - all good! The second command to get a list of EC2 instances blows up with a 403 with "Query-string authentication requires the Signature, Expires and AWSAccessKeyId parameters"
s3_conn = S3Connection(AWSAccessKeyId, AWSSecretKey)
print s3_conn.get_all_buckets()
ec2_conn = EC2Connection(AWSAccessKeyId, AWSSecretKey)
print ec2_conn.get_all_instances()
Also, my credentials are all good (Full admin) - I tested them using the Ruby aws-sdk, both EC2 and S3 work fine.
I also noticed that the host attribute in the ec2_conn object is s3-eu-west-1.amazonaws.com, "s3"...? Surely thats wrong? I've tried retro fixing it to the correct endpoint but no luck.
Any help would be great appreciate
Thanks

Here's some working code I use to list all my instances across potentially multiple regions.
Its doing a lot more than you need, but maybe you can pare it down to what you want.
#!/usr/bin/python
import boto
import boto.ec2
import sys
class ansi_color:
red = '\033[31m'
green = '\033[32m'
reset = '\033[0m'
grey = '\033[1;30m'
def name(i):
if 'Name' in i.tags:
n = i.tags['Name']
else:
n = '???'
n = n.ljust(16)[:16]
if i.state == 'running':
n = ansi_color.green + n + ansi_color.reset
else:
n = ansi_color.red + n + ansi_color.reset
return n
def pub_dns( i ):
return i.public_dns_name.rjust(43)
def pri_dns( i ):
return i.private_dns_name.rjust(43)
def print_instance( i ):
print ' ' + name(i) + '| ' + pub_dns(i) + ' ' + pri_dns(i)
regions = sys.argv[1:]
if len(regions)==0:
regions=['us-east-1']
if len(regions)==1 and regions[0]=="all":
rr = boto.ec2.regions()
else:
rr = [ boto.ec2.get_region(x) for x in regions ]
for reg in rr:
print "========"
print reg.name
print "========"
conn = reg.connect()
reservations = conn.get_all_instances()
for r in reservations:
# print ansi_color.grey + str(r) + ansi_color.reset
for i in r.instances:
print_instance(i)

There is the connect_to_region command:
import boto.ec2
connection = boto.ec2.connect_to_region('eu-west-1', aws_access_key_id=AWSAccessKeyId,
aws_secret_access_key=AWSSecretKey)
The Boto tutorial gives another way. That method would basically work like this:
import boto.ec2
for region in boto.ec2.regions():
if region.name == 'my-favorite-region':
connection = region.connect()
break
This has not been working on older versions of Boto.

Do you have your IAM credentials in order? The given access key should have rights for EC2. If you're not sure, you can add the policy AmazonEC2FullAccess to test, and later tune this down.