We Keep Coding

minimalmodbus read multiple register on same device

Is there a way where I can read a set of register on a device with the python library minimalmodbus? Or is it one by one I need to read_register like this?
instrument.read_register(0x0033,1)

Look at the minimalmodbus documentation for read_registers(). You shouldn't need to change the functioncode argument.
Assuming you want the first one hundred and twenty-five registers, starting at register zero:
registers = instrument.read_registers(0, 125)
If you wanted to print those registers:
for idx in range(len(registers)):
print(f"Register {idx:3d} : {registers[idx]:5d} ({hex(registers[idx])})")
This will output something like:
Register 0 : 0 (0x0)
Register 1 : 1 (0x1)
Register 2 : 2 (0x2)
Register 3 : 3 (0x3)
Register 4 : 4 (0x4)
…
EDIT: Looking at page 9 of the specification document, there are sixteen and thirty-two bit registers co-mingled. It will be easier to read them explicitly. Otherwise you'll need to shift and combine the two sixteen-bit registers, which is annoying and minimalmodbus has functions to make it easier and less error prone for you.
E.g.
# 0000h 2 V L1-N INT32 Value weight: Volt*10
# read_long(registeraddress: int, functioncode: int = 3, signed: bool = False, byteorder: int = 0) → int
L1_N_voltage = instrument.read_long(0, signed=True) / 10
# 0028h 2 W sys INT32 Value weight: Watt*10
sys_wattage = instrument.read_long(0x28, signed=True) / 10
Note that read_long() doesn't have number_of_decimals support, so you need to manually divide the reading by ten. For the current and power factor you'll need to divide by 1,000.

How to generate a time-ordered uid in Python?

Is this possible? I've heard Cassandra has something similar : https://datastax.github.io/python-driver/api/cassandra/util.html
I have been using a ISO timestamp concatenated with a uuid4, but that ended up way too large (58 characters) and probably overkill.
Keeping a sequential number doesn't work in my context (DynamoDB NoSQL)
Worth noticing that for my application it doesn't matter if items created in batch/same second are in a random order, as long as the uid don't collapse.
I have no specific restriction on maximum length, ideally I would like to see the different collision chance for different lengths, but it needs to be smaller than 58 (my original attempt)
This is to use with DynamoDB(NoSQL Database) as Sort-key

Why uuid.uuid1 is not sequential
uuid.uuid1(node=None, clock_seq=None) is effectively:
60 bits of timestamp (representing number of 100-ns intervals after 1582-10-15 00:00:00)
14 bits of "clock sequence"
48 bits of "Node info" (generated from network card's mac-address or from hostname or from RNG).
If you don't provide any arguments, then System function is called to generate uuid. In that case:
It's unclear if "clock sequence" is sequential or random.
It's unclear if it's safe to be used in multiple processes (can clock_seq be repeated in different processes or not?). In Python 3.7 this info is now available.
If you provide clock_seq or node, then "pure python implementation is used". IN this case even with "fixed value" for clock_seq:
timestamp part is guaranteed to be sequential for all the calls in current process even in threaded execution.
clock_seq part is randomly generated. But that is not critical annymore because timestamp is sequential and unique.
It's NOT safe for multiple processes (processes that call uuid1 with the same clock_seq, node might return conflicting values if called during the "same 100-ns time interval")
Solution that reuses uuid.uuid1
It's easy to see, that you can make uuid1 sequential by providing clock_seq or node arguments (to use python implementation).
import time
from uuid import uuid1, getnode
_my_clock_seq = getrandbits(14)
_my_node = getnode()
def sequential_uuid(node=None):
return uuid1(node=node, clock_seq=_my_clock_seq)
# .hex attribute of this value is 32-characters long string
def alt_sequential_uuid(clock_seq=None):
return uuid1(node=_my_node, clock_seq=clock_seq)
if __name__ == '__main__':
from itertools import count
old_n = uuid1() # "Native"
old_s = sequential_uuid() # Sequential
native_conflict_index = None
t_0 = time.time()
for x in count():
new_n = uuid1()
new_s = sequential_uuid()
if old_n > new_n and not native_conflict_index:
native_conflict_index = x
if old_s >= new_s:
print("OOops: non-sequential results for `sequential_uuid()`")
break
if (x >= 10*0x3fff and time.time() - t_0 > 30) or (native_conflict_index and x > 2*native_conflict_index):
print('No issues for `sequential_uuid()`')
break
old_n = new_n
old_s = new_s
print(f'Conflicts for `uuid.uuid1()`: {bool(native_conflict_index)}')
Multiple processes issues
BUT if you are running some parallel processes on the same machine, then:
node which defaults to uuid.get_node() will be the same for all the processes;
clock_seq has small chance to be the same for some processes (chance of 1/16384)
That might lead to conflicts! That is general concern for using
uuid.uuid1 in parallel processes on the same machine unless you have access to SafeUUID from Python3.7.
If you make sure to also set node to unique value for each parallel process that runs this code, then conflicts should not happen.
Even if you are using SafeUUID, and set unique node, it's still possible to have non-sequential (but unique) ids if they are generated in different processes.
If some lock-related overhead is acceptable, then you can store clock_seq in some external atomic storage (for example in "locked" file) and increment it with each call: this allows to have same value for node on all parallel processes and also will make id-s sequential. For cases when all parallel processes are subprocesses created using multiprocessing: clock_seq can be "shared" using multiprocessing.Value
As a result you always have to remember:
If you are running multiple processes on the same machine, then you must:
Ensure uniqueness of node. The problem for this solution: you can't be sure to have sequential ids from different processes generated during the same 100-ns interval. But this is very "light" operation executed once on process startup and achieved by: "adding" something to default node, e.g. int(time.time()*1e9) - 0x118494406d1cc000, or by adding some counter from machine-level atomic db.
Ensure "machine-level atomic clock_seq" and the same node for all processes on one machine. That way you'll have some overhead for "locking" clock_seq, but id-s are guaranteed to be sequential even if generated in different processes during the same 100-ns interval (unless you are calling uuid from several threads in the same process).
For processes on different machines:
either you have to use some "global counter service";
or it's not possible to have sequential ids generated on different machines during the same 100-ns interval.
Reducing size of the id
General approach to generate UUIDs is quite simple, so it's easy to implement something similar from scratch, and for example use less bits for node_info part:
import time
from random import getrandbits
_my_clock_seq = getrandbits(14)
_last_timestamp_part = 0
_used_clock_seq = 0
timestamp_multiplier = 1e7 # I'd recommend to use this value
# Next values are enough up to year 2116:
if timestamp_multiplier == 1e9:
time_bits = 62 # Up to year 2116, also reduces chances for non-sequential id-s generated in different processes
elif timestamp_multiplier == 1e8:
time_bits = 60 # up to year 2335
elif timestamp_multiplier == 1e7:
time_bits = 56 # Up to year 2198.
else:
raise ValueError('Please calculate and set time_bits')
time_mask = 2**time_bits - 1
seq_bits = 16
seq_mask = 2**seq_bits - 1
node_bits = 12
node_mask = 2**node_bits - 1
max_hex_len = len(hex(2**(node_bits+seq_bits+time_bits) - 1)) - 2 # 21
_default_node_number = getrandbits(node_bits) # or `uuid.getnode() & node_mask`
def sequential_uuid(node_number=None):
"""Return 21-characters long hex string that is sequential and unique for each call in current process.
Results from different processes may "overlap" but are guaranteed to
be unique if `node_number` is different in each process.
"""
global _my_clock_seq
global _last_timestamp_part
global _used_clock_seq
if node_number is None:
node_number = _default_node_number
if not 0 <= node_number <= node_mask:
raise ValueError("Node number out of range")
timestamp_part = int(time.time() * timestamp_multiplier) & time_mask
_my_clock_seq = (_my_clock_seq + 1) & seq_mask
if _last_timestamp_part >= timestamp_part:
timestamp_part = _last_timestamp_part
if _used_clock_seq == _my_clock_seq:
timestamp_part = (timestamp_part + 1) & time_mask
else:
_used_clock_seq = _my_clock_seq
_last_timestamp_part = timestamp_part
return hex(
(timestamp_part << (node_bits+seq_bits))
|
(_my_clock_seq << (node_bits))
|
node_number
)[2:]
Notes:
Maybe it's better to simply store integer value (not hex-string) in the database
If you are storing it as text/char, then its better to convert integer to base64-string instead of converting it to hex-string. That way it will be shorter (21 chars hex-string → 16 chars b64-encoded string):
from base64 import b64encode
total_bits = time_bits+seq_bits+node_bits
total_bytes = total_bits // 8 + 1 * bool(total_bits % 8)
def int_to_b64(int_value):
return b64encode(int_value.to_bytes(total_bytes, 'big'))
Collision chances
Single process: collisions not possible
Multiple processes with manually set unique clock_seq or unique node in each process: collisions not possible
Multiple processes with randomly set node (48-bits, "fixed" in time):
Chance to have the node collision in several processes:
in 2 processes out of 10000: ~0.000018%
in 2 processes out of 100000: 0.0018%
Chance to have single collision of the id per second in 2 processes with the "colliding" node:
for "timestamp" interval of 100-ns (default for uuid.uuid1 , and in my code when timestamp_multiplier == 1e7): proportional to 3.72e-19 * avg_call_frequency²
for "timestamp" interval of 10-ns (timestamp_multiplier == 1e8): proportional to 3.72e-21 * avg_call_frequency²

In the article you've linked too, the cassandra.util.uuid_from_time(time_arg, node=None, clock_seq=None)[source] seems to be exactly what you're looking for.
def uuid_from_time(time_arg, node=None, clock_seq=None):
"""
Converts a datetime or timestamp to a type 1 :class:`uuid.UUID`.
:param time_arg:
The time to use for the timestamp portion of the UUID.
This can either be a :class:`datetime` object or a timestamp
in seconds (as returned from :meth:`time.time()`).
:type datetime: :class:`datetime` or timestamp
:param node:
None integer for the UUID (up to 48 bits). If not specified, this
field is randomized.
:type node: long
:param clock_seq:
Clock sequence field for the UUID (up to 14 bits). If not specified,
a random sequence is generated.
:type clock_seq: int
:rtype: :class:`uuid.UUID`
"""
if hasattr(time_arg, 'utctimetuple'):
seconds = int(calendar.timegm(time_arg.utctimetuple()))
microseconds = (seconds * 1e6) + time_arg.time().microsecond
else:
microseconds = int(time_arg * 1e6)
# 0x01b21dd213814000 is the number of 100-ns intervals between the
# UUID epoch 1582-10-15 00:00:00 and the Unix epoch 1970-01-01 00:00:00.
intervals = int(microseconds * 10) + 0x01b21dd213814000
time_low = intervals & 0xffffffff
time_mid = (intervals >> 32) & 0xffff
time_hi_version = (intervals >> 48) & 0x0fff
if clock_seq is None:
clock_seq = random.getrandbits(14)
else:
if clock_seq > 0x3fff:
raise ValueError('clock_seq is out of range (need a 14-bit value)')
clock_seq_low = clock_seq & 0xff
clock_seq_hi_variant = 0x80 | ((clock_seq >> 8) & 0x3f)
if node is None:
node = random.getrandbits(48)
return uuid.UUID(fields=(time_low, time_mid, time_hi_version,
clock_seq_hi_variant, clock_seq_low, node), version=1)
There's nothing Cassandra specific to a Type 1 UUID...

You should be able to encode a timestamp precise to the second for a time range of 135 years in 32 bits. That will only take 8 characters to represent in hex. Added to the hex representation of the uuid (32 hex characters) that will amount to only 40 hex characters.
Encoding the time stamp that way requires that you pick a base year (e.g. 2000) and compute the number of days up to the current date (time stamp). Multiply this number of days by 86400, then add the seconds since midnight. This will give you values that are less than 2^32 until you reach year 2135.
Note that you have to keep leading zeroes in the hex encoded form of the timestamp prefix in order for alphanumeric sorting to preserve the chronology.
With a few bits more in the time stamp, you could increase the time range and/or the precision. With 8 more bits (two hex characters), you could go up to 270 years with a precision to the hundredth of a second.
Note that you don't have to model the fraction of seconds in a base 10 range. You will get optimal bit usage by breaking it down in 128ths instead of 100ths for the same number of characters. With the doubling of the year range, this still fits within 8 bits (2 hex characters)
The collision probability, within the time precision (i.e. per second or per 100th or 128th of a second) is driven by the range of the uuid so it will be 1 in 2^128 for the chosen precision. Increasing the precision of the time stamp has the most impact on reducing the collision chances. It is also the factor that has the lowest impact on total size of the key.
More efficient character encoding: 27 to 29 character keys
You could significantly reduce the size of the key by encoding it in base 64 instead of 16 which would give you 27 to 29 characters (depending on you choice of precision)
Note that, for the timestamp part, you need to use an encoding function that takes an integer as input and that preserves the collating sequence of digit characters.
For example:
def encode64(number, size):
chars = "+-0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
result = list()
for _ in range(size):
result.append(chars[number%64])
number //= 64
return "".join(reversed(result))
a = encode64(1234567890,6) # '-7ZU9G'
b = encode64(9876543210,6) # '7Ag-Pe'
print(a < b) # True
u = encode64(int(uuid.uuid4()),22) # '1QA2LtMg30ztnugxaokVMk'
key = a+u # '-7ZU9G1QA2LtMg30ztnugxaokVMk' (28 characters)
You can save some more characters by combining the time stamp and uuid into a single number before encoding instead of concatenating the two encoded values.
The encode64() function needs one character every 6 bits.
So, for 135 years with precision to the second: (32+128)/6 = 26.7 --> 27 characters
instead of (32/6 = 5.3 --> 6) + (128/6 = 21.3 --> 22) ==> 28 characters
uid = uuid.uuid4()
timeStamp = daysSince2000 * 86400 + int(secondsSinceMidnight)
key = encode64( timeStamp<<128 | int(uid) ,27)
with a 270 year span and 128th of a second precision: (40+128)/6 = 28 characters
uid = uuid.uuid4()
timeStamp = daysSince2000 * 86400 + int(secondsSinceMidnight)
precision = 128
timeStamp = timeStamp * precision + int(factionOfSecond * precision)
key = encode64( timeStamp<<128 | int(uid) ,28)
With 29 characters you can raise precision to 1024th of a second and year range to 2160 years.
UUID masking: 17 to 19 characters keys
To be even more efficient, you could strip out the first 64 bits of the uuid (which is already a time stamp) and combine it with your own time stamp. This would give you keys with a length of 17 to 19 characters with practically no loss of collision avoidance (depending on your choice of precision).
mask = (1<<64)-1
key = encode64( timeStamp<<64 | (int(uid) & mask) ,19)
Integer/Numeric keys ?
As a final note, if your database supports very large integers or numeric fields (140 bits or more) as keys, you don't have to convert the combined number to a string. Just use it directly as the key. The numerical sequence of timeStamp<<128 | int(uid) will respect the chronology.

The uuid6 module (pip install uuid6) solves the problem. It aims at implementing the corresponding draft for a new uuid variant standard, see here.
Example code:
import uuid6
for i in range(0, 30):
u = uuid6.uuid7()
print(u)
time.sleep(0.1)
The package suggests to use uuid6.uuid7():
Implementations SHOULD utilize UUID version 7 over UUID version 1 and
6 if possible.
UUID version 7 features a time-ordered value field derived from the
widely implemented and well known Unix Epoch timestamp source, the
number of milliseconds seconds since midnight 1 Jan 1970 UTC, leap
seconds excluded. As well as improved entropy characteristics over
versions 1 or 6.

SlidingWindows Python Apache Beam duplicate the data

The problem
Each time the system receive a message from pubsub with a Sliding Windows it been duplicated
The code
| 'Parse dictionary' >> beam.Map(lambda elem: (elem['Serial'], int(elem['Value'])))
| 'window' >> beam.WindowInto(window.SlidingWindows(30, 15),accumulation_mode=AccumulationMode.DISCARDING)
| 'Count' >> beam.CombinePerKey(beam.combiners.MeanCombineFn())
The output
If I only send one message from pub/sub and try to print what I have after the sliding window finish with the code:
class print_row2(beam.DoFn):
def process(self, row=beam.DoFn.ElementParam, window=beam.DoFn.WindowParam,timestamp=beam.DoFn.TimestampParam):
print row, timestamp2str(float(window.start)), timestamp2str(float(window.end)),timestamp2str(float(timestamp))
The result
('77777', 120.0) 2018-11-16 08:21:15.000 2018-11-16 08:21:45.000 2018-11-16 08:21:45.000
('77777', 120.0) 2018-11-16 08:21:30.000 2018-11-16 08:22:00.000 2018-11-16 08:22:00.000
If I print the message before 'window' >> beam.WindowInto(window.SlidingWindows(30, 15)) I only get once
The process in "graphic mode:
time: ----t+00---t+15---t+30----t+45----t+60------>
: : : : :
w1: |=X===========| : :
w2: |==============| :
...
The message X was sent only once at the begining of the slidingwindow, it should only be received once, but is been receiving twice
I have tried with both AccumulationMode values, also with a trigger=AftyerWatermark but i can not fix the problem.
What could be wrong?
Extra
With FixedWindows this is the correct code for my porpouse:
| 'Window' >> beam.WindowInto(window.FixedWindows(1 * 30))
| 'Speed Average' >> beam.GroupByKey()
| "Calculating average" >> beam.CombineValues(beam.combiners.MeanCombineFn())
or
| 'Window' >> beam.WindowInto(window.FixedWindows(1 * 30))
| "Calculating average" >> beam.CombinePerKey(beam.combiners.MeanCombineFn())

All elements that belong to the window are emitted. If an element belongs to multiple windows it will be emitted in each window.
Accumulation mode only matters if you plan to handle late data/multiple trigger firings. In this case discarding mode gives you only new elements in the window when the trigger fires again, i.e. emits only the elements that arrived in the same window since previous trigger firing, the elements that were already emitted are not emitted again and are discarded. In accumulating mode the whole window will be emitted for every trigger firing, it will include old elements that were already emitted last time and new elements that have arrived since then.
If I understand your example you have sliding windows, they have a length of 30 seconds, and they start every 15 seconds. So they overlap for 15 seconds:
time: ----t+00---t+15---t+30----t+45----t+60------>
: : : : :
w1: |=============| : :
w2: |==============| :
w3: |===============|
...
So any element in your case will belong to at least two windows (except for first and last windows).
E.g. in your example, if your message was sent between 17:07:15 and 17:07:30 it will appear in both windows.
Fixed windows don't overlap so element can belong only to one window:
time: ----t+00---t+15---t+30----t+45----t+60------>
: : :
w1: |=============| :
w2: |===============|
w3: |====...
...
More about windows here: https://beam.apache.org/documentation/programming-guide/#windowing

I have exactly the same issue, however in java. I have a window with 10 seconds duration and a step of 3 seconds. When an event is emitted from the mqtt topic, that I subscribe to, it looks like the ParDo function that I have runs and emits the first and only event to all of the three "constructed" windows.
X is the event that i send at a random timestamp: 2020-09-15T21:17:57.292Z
time: ----t+00---t+15---t+30----t+45----t+60------>
: : : : :
w1: |X============| : :
w2: |X=============| :
w3: |X==============|
...
Even the same timestamp is assigned to them!! I must really doing something completely wrong.
I use Scala 2.12 and BEAM 2.23 with a Direct Runner.
[Hint]: I use states in the processElement function! Where the state is being hold per key + window. Maybe there is a bug there? I will try to test it without states.
UPDATE: Removed the state fields and the single event is assigned to one window.

Sampling in RasPi3 and ADS1115 using Sample count or time.time()

I'm using Raspberry Pi 3 and ADS1115, my project requires me to get evenly spaced samples so as to plot and analyse. The other posts were about achieving 10k and 50k sps but I only require 500SPS and that isn't working either. Is there a way to run my code for 120 seconds with 500 sps and get 60,000 samples in the end from both A0 and A1 channel at the same time? I have attached the code for reference. Thanks in advance
from Adafruit_ADS1x15 import ADS1x15
import time
import numpy as np
pga = 2/3 # Set full-scale range of programmable gain
# amplifier (page 13 of data sheet), change
#depending on the input voltage range
ADS1115 = 0x01 # Specify that the device being used is the
# ADS1115, for the ADS1015 used 0x00
adc = Adafruit_ADS1x15.ADS1015() # Create instance of the class ADS1x15
# Function to print sampled values to the terminal
def logdata():
print "sps value should be one of: 8, 16, 32, 64, 128, 250, 475, 860,
otherwise the value will default to 250"
frequency = input("Input sampling frequency (Hz): ") # Get
#sampling frequency from user
sps = input("Input sps (Hz) : ") # Get
# ads1115 sps value from the user
time1 = input("Input sample time (seconds): ") # Get how
#long to sample for from the user
period = 1.0 / frequency # Calculate sampling period
datapoints = int(time1*frequency) # Datapoints is the total number
#of samples to take, which must be an integer
startTime=time.time() # Time of first sample
t1=startTime # T1 is last sample time
t2=t1 # T2 is current time
for x in range (0,datapoints) : # Loop in which data is sampled
while (t2-t1 < period) : # Check if t2-t1 is less then
#sample period, if it is then update t2
t2=time.time() # and check again
t1+=period # Update last sample time by the
# sampling period
print adc.read_adc(0, pga, data_rate=sps), "mV ", ("%.2f" %
(t2-startTime)) , "s" # Print sampled value and time to the terminal
# Call to logdata function
logdata()

1) Are you using the ADS1115 ???
adc = Adafruit_ADS1x15.ADS1015() # should be adc = Adafruit_ADS1x15.ADS1115()
2) you can't read two or more single-ended channels at the same time. In differential mode one can compare two channels to yield one value.
To read a value from channel 1, besides channel 0 you would have to add another call in your loop:
print adc.read_adc(0, pga, data_rate=sps) ..... # original call for channel 0
print adc.read_adc(1, pga, data_rate=sps) ..... # new call for channel 1
3) before a value can be read, the ADS must be configured with several parameters, like channel, rate, gain etc. After some time, which is needed for doing an analog/digital conversion, values can be read, once, or over and over again, when in continuous mode.
In the original ADAFruit library this time is calculated from the data rate (insecure) and in the recent port to CircuitPython this time will most often be around 0.01 s, because the conversion will most probably not have finished directly after configuration (check the _read method).
4) reading at 500SPS is about the fastest the ADS1115 can read. The reference states that conversion at 860SPS takes 1.2 ms. Adding time for configuration and reading you will not be able to read two or more values continuosly every 0.002s, even if you were receiving notifications for conversion, like I described on my homepage, instead of waiting for a fixed period of time.
5) I think the closest you can get with Python is to run 2 daisy-chained ADS1115 in continuous mode with GPIO notifications, but I have no experience with that.

How is an ICMP packet constructed in python

For the sake of learning I am currently trying to create a simple python porgram to send a ICMP ping packet to some device. To get started I looked through the source code of the python module Pyping: https://github.com/Akhavi/pyping/blob/master/pyping/core.py
I am trying to understand all that is going on when sending and constructing the packet however i have managed to get stuck on one part of the code and can't seem to figure out exactly what its fucntion and use is. I have been looking into ICMP packets and i understand that they contain Type code checksum and data now the piece of code that puzzles me is:
self.own_id = os.getpid() & 0xFFFF
header = struct.pack(
"!BBHHH", ICMP_ECHO, 0, checksum, self.own_id, self.seq_number
)
padBytes = []
startVal = 0x42
for i in range(startVal, startVal + (self.packet_size)):
padBytes += [(i & 0xff)] # Keep chars in the 0-255 range
data = bytes(padBytes)
My questions would be:
What is the use of adding the self.own_id and self.seq_number to the header?
What is being calculated in the for-loop, and why does it have a specific start value of 0x42?
I am new to networking and any help would be really appreciated.

Description of ICMP Echo Request packets
The ICMP Echo Request PDU looks like this:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type(8) | Code(0) | Checksum |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Identifier | Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Payload |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
And here's a description of the various fields from the wiki link above:
The Identifier and Sequence Number can be used by the client to match the reply with the request that caused the reply.
In practice, most Linux systems use a unique identifier for every ping process, and sequence number is an increasing number within that process. Windows uses a fixed identifier, which varies between Windows versions, and a sequence number that is only reset at boot time.
Description of pyping Code
Header Generation
Look at the full function body of send_one_ping, which is where your code is from. I will annotate it with some information:
def send_one_ping(self, current_socket):
"""
Send one ICMP ECHO_REQUEST
"""
# Header is type (8), code (8), checksum (16), id (16), sequence (16)
# Annotation: the Type is 8 bits, the code is 8 bits, the
# header checksum is 16 bits
# Additional Header Information is 32-bits (identifier and sequence number)
# After that is Payload, which is of arbitrary length.
So this line
header = struct.pack(
"!BBHHH", ICMP_ECHO, 0, checksum, self.own_id, self.seq_number
)
This line creates the packet header using struct with layout !BBHHH, which means:
B - Unsigned Char (8 bits)
B - Unsigned Char (8 bits)
H - Unsigned Short (16 bits)
H - Unsigned Short (16 bits)
H - Unsigned Short (16 bits)
And so the header will look like this:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ICMP_ECHO | 0 | checksum |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| self.own_id | self.seq_number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Note this:
self.own_id sets the identifier of the application sending this data. For this code, it just uses the program's Program Identifier number.
self.seq_number sets the sequence number. This helps you identify which ICMP request packet this is if you were to send multiple in a row. It would help you do things like calculate ICMP packet loss.
Both the Identifier and Sequence Number fields combined can be used by a client to match up echo replies with echo requests.
Payload Generation
Now let's move on to the Payload portion. Payloads are of arbitrary length, but the Ping class this code is taken from defaults to a total packet payload size of 55 bytes.
So the portion below just creates a bunch of arbitrary bytes to stuff into the payload section.
padBytes = []
startVal = 0x42
# Annotation: 0x42 = 66 decimal
# This loop would go from [66, 66 + packet_size],
# which in default pyping means [66, 121)
for i in range(startVal, startVal + (self.packet_size)):
padBytes += [(i & 0xff)] # Keep chars in the 0-255 range
data = bytes(padBytes)
At the end of it, byte(padBytes) actually looks like this:
>> bytes(padBytes)
b'BCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwx'
Why 0x42 was chosen?
As far as I know, 0x42 has no actual significance as a Payload identifier, so this seems rather arbitrary. The payload here is actually pretty meaningless. As you can see from the Payload Generation section, it just generates a contiguous sequence that doesn't really mean anything. They could have just decided to fill the entire packet payload with 0x42 bytes if they wanted.

Use scapy http://www.secdev.org/projects/scapy/ !
Scapy is packet manipulation framework written in python.
You can forge a lot of kind of packets (http, tcp, ip, udp, icmp, etc...)

ICMP echo request and echo reply are better known as ping.
The ID and the sequence number allow to match the replies with the requests. They are necessary beacuse the ICMP protocol does not have source and destinations ports like TCP and UDP do, only a source and a destination IP address. Multiple processes can ping the same host and the replies must be delivered to the correct process. For this reason, each reply contains data literally copied from the corresponding request. The ID identifies the process sending the requests. The sequence number helps to match replies with requests within the process. That is necessary to calculate the RTT (round-trip time) and to detect unanswered pings.
The data computed in the loop is payload which is also literally copied from a request to a reply. The payload is optional and a ping implementation can use it for whatever it wants.
Why 0x42? I guess the author was probably a Douglas Adams fan.