Arbitrary byte value in struct.pack() - python

I am trying to understand how struct work?
I have two confusions :
how struct is different from * and ** or *args and **kwargs ? where should i use struct and where should i use * and ** ?
My code is :
list(b'stackoverflow')
[115, 116, 97, 99, 107, 111, 118, 101, 114, 102, 108, 111, 119]
second doubt is suppose if someone send me this code :
[115, 116, 97, 99, 107, 111, 118, 101, 114, 102, 108, 111, 119]
i have to read it after unpacking it so general way is :
>>> struct.pack(b'bbbbbbbbbbbbb',115, 116, 97, 99, 107, 111, 118, 101, 114, 102, 108, 111, 119)
but i suppose i sentence is too long and i don't want to add many "bbbbbb" as argument so how can i use something which automatically detect the length of sentence ? something like
struct.pack(b'*magic',115, 116, 97, 99, 107, 111, 118, 101, 114, 102, 108, 111, 119)
can we do it?

The struct module has nothing to do with argument passing, so I simply can't guess what you're asking about in the first part.
To your second question, you shouldn't be using struct at all for that purpose. What you want is a bytes object corresponding to the list of little integers, and that's very easy to get:
>>> somelist = [115, 116, 97, 99, 107, 111, 118, 101, 114, 102, 108, 111, 119]
>>> bytes(somelist)
b'stackoverflow'
If for some reason you're determined to use struct.pack instead for this purpose, then this is the easiest way:
>>> struct.pack("b" * len(somelist), *somelist)
b'stackoverflow'

Related

how to take values out of a list to an array in python?

My Data This picture shows my data. See this: [1, 0,list([32, 64, 117, 115, 101, 114, 32, 119, 104, 101, 110,....], I want this data to be like : [1,0,32,64,117,115......]. I want data out of the list and become a member of the array. How can I do this easily and quickly? I have tried a long method but there are many errors. The method I have tried: Take data out of the list one by one and append in the particular row then move to the next data row and do the same thing there for the list.
you can try something like this :
`for i in range(len(data)) :
data[i][2].insert(0,data[i][1])
data[i][2].insert(0,data[i][0])
data[i].pop(1)
data[i].pop(0)`
You could do something like below.
We iterate through the given list, checking each type. If an element is a type list we will iterate through that and append each of those elements to a list called new. If an element we have iterated through in the given list is a type != list then we can directly append the element to new.
x = [1, 0,list([32, 64, 117, 115, 101, 114, 32, 119, 104, 101, 110])]
new = []
for num in x:
if type(num) == list:
for num2 in num:new.append(num2)
else:new.append(num)
You can flatten this jagged nested array with numpy.hstack() as follows:
import numpy as np
original = [1, 0,list([32, 64, 117, 115, 101, 114, 32, 119, 104, 101, 110])]
f = np.hstack(np.array(original, dtype=object)).tolist()
print(f)
#[1, 0, 32, 64, 117, 115, 101, 114, 32, 119, 104, 101, 110]
Let first consider this example
# lets take this example value for data
data = [1, 0,list([32, 64, 117, 115, 101, 114, 32, 119, 104, 101, 110])]
print("data before modification:", data)
print()
extract_items = data.pop(2) # Since in the picture, the inner list is always the third element we can use the index 2
for item in extract_items: # loop through the inner list and append items to outer
data.append(item)
print("extract_items", extract_items)
print("data", data)
**output for example case: **
data before modification: [1, 0, [32, 64, 117, 115, 101, 114, 32, 119, 104, 101, 110]]
extract_items: [32, 64, 117, 115, 101, 114, 32, 119, 104, 101, 110]
data: [1, 0, 32, 64, 117, 115, 101, 114, 32, 119, 104, 101, 110]
Which is what we would like.
Thus, the Final Solution:
for element in data:
extract_items = element.pop(2)
for item in extract_items:
element.append(item)

I have several lists, each containing 206 elements. When I attempt to reference an index within those 206, I'm given an error

I have several corresponding lists each containing exactly 206 elements. The number of elements and values of those elements can be different depending on what's inside the file the program reads.
One of those lists is comprised entirely of integers, and is used as the basis to determine if elements should be removed from all lists. My code follows:
toRemove = []
for element in myList:
if element < int(cutoff): # cutoff is specified by the user
toRemove.append(myList.index(element))
toRemove = [int(element) for element in toRemove]
for element in toRemove:
del myList2[element]
del myList3[element]
...
When I run >>>len(myList), Python returns 206 for all lists, minus toRemove as toRemove is determined according to user input. However, when I use toRemove to remove those indexes from all lists Python greets me with the following:
Traceback (most recent call last):
File "Test.py", line 104, in <module>
del myList2[element]
IndexError: list assignment index out of range
The highest value in toRemove is 204 -- there are 205 indices in this list. toRemove does not contain any values lower than 0. Typecasting [element] into [int(element)] throws the same error.
The list of indexes to remove:
[0, 1, 2, 3, 4, 5, 6, 7, 9, 12, 15, 16, 17, 18, 20, 21, 22, 25, 26, 28, 29, 30, 31, 33, 34, 37, 38, 39, 40, 41, 45, 46, 47, 48, 51, 52, 54, 56, 57, 58, 59, 60, 61, 62, 63, 65, 66, 67, 68, 69, 72, 73, 74, 75, 77, 80, 84, 86, 87, 88, 89, 90, 91, 92, 94, 95, 96, 97, 98, 100, 101, 102, 103, 104, 106, 107, 108, 109, 110, 111, 113, 114, 115, 116, 117, 118, 119, 120, 121, 124, 126, 127, 128, 129, 130, 133, 134, 135, 136, 138, 139, 140, 141, 142, 144, 145, 147, 149, 151, 110, 153, 154, 155, 157, 158, 161, 162, 163, 164, 165, 166, 167, 168, 169, 171, 173, 174, 175, 177, 178, 179, 180, 181, 182, 183, 184, 186, 187, 188, 189, 190, 192, 194, 195, 196, 197, 198, 200, 202, 203, 204]
As you can see, none less than 0 or greater than 205.
The list of numbers to check:
[1032, 3882, 4182, 6880, 1170, 1968, 2085, 3548, 9111, 1122, 11987, 12718, 7606, 8391, 8890, 2574, 1447, 6257, 1457, 11430, 7202, 6891, 1495, 12372, 12907, 2243, 3984, 11236, 3462, 4790, 1103, 1044, 8805, 1404, 6259, 8655, 13695, 6505, 3758, 2378, 3303, 3360, 9102, 14768, 12154, 3079, 3307, 2416, 7043, 9910, 10050, 6497, 2407, 13477, 4118, 13762, 1351, 3645, 4092, 3785, 1565, 3721, 4982, 4166, 11234, 7306, 3608, 2437, 4022, 2688, 11411, 9126, 3526, 1398, 5657, 2336, 8775, 4073, 14731, 8885, 7264, 8399, 10268, 12131, 2129, 11420, 1334, 3675, 1055, 3544, 5301, 6608, 1482, 14221, 2254, 4288, 1178, 5032, 4973, 19573, 3855, 4360, 6408, 3040, 3220, 11735, 3591, 3807, 2590, 6169, 2300, 1332, 8996, 6680, 3537, 1048, 3505, 5960, 3480, 1486, 4782, 4607, 18269, 8258, 4514, 8069, 5698, 1753, 6314, 1634, 3688, 9249, 18783, 5514, 1409, 7197, 3789, 8172, 2718, 5535, 3508, 1769, 2503, 8178, 2414, 1175, 13069, 1916, 12297, 1732, 69609, 3047, 2300, 5752, 1106, 1522, 11687, 3020, 3929, 9407, 13449, 4644, 2399, 4317, 4917, 5476, 1194, 4016, 1496, 7788, 9365, 1223, 12289, 1624, 1410, 3321, 12930, 1806, 7154, 4961, 2798, 5571, 1931, 7912, 4944, 10963, 2427, 7514, 2425, 2649, 1303, 13568, 2923, 11225, 5822, 4268, 5962, 2422, 6978, 12393, 1331, 12749, 7460, 1683, 6403, 11972]
You can substitute cutoff with 8000, as that is the number I am using to test the program.
What am I doing wrong, and how do I correct this problem?
The issue you're encountering is that the list shrinks as you remove elements from it. Since you're removing them in order from front to back, the later indexes may end up being off the end of the list.
Consider this simpler scenario than your list with 200+ values:
lst = [0, 1, 2, 3]
indexes_to_remove = [2, 3]
for i in indexes_to_remove:
del lst[i]
You'll get the same exception as in your example. The reason is that after you remove index 2, the last value (3) is also at index 2, since the list got shorter.
There are a few ways to fix this. One option would be to remove the elements starting at the end of the list, and working forwards. That way the indexes you remove later on will always still be valid, as nothing before them will have been removed. Since you're guaranteed to be adding the indexes in order (more or less, see my note below), you can just iterate in reverse over them with reversed:
for element in reversed(toRemove):
del myList2[element]
del myList3[element]
Another option would be to compute a modified index as you go about removing some of the values from the lists. This is not too difficult:
for i, element in enumerate(toRemove):
del myList2[element-i]
del myList3[element-i]
Finally, it might be more efficient to rebuild your whole lists, rather than using del to remove some values from them. del some_list[some_index] takes O(N) time to complete, so even if you have to loop over the indexes to skip several times, it's probably going to be faster than doing a bunch of del operations:
toRemove = set(toRemove)
myList2 = [v for i, v in enumerate(myList2) if i not in toRemove]
myList3 = [v for i, v in enumerate(myList3) if i not in toRemove]
A final note: Your current code may have another error finding all the indexes to remove if myList can have multiple copies of the same values. That's because myList.index(element) will always find the first occurrence of the value element, even if you were just iterating over it when it appears later in the list. Instead you should probably use enumerate to get the indexes as you iterate:
for i, element in enumerate(myList):
if element < int(cutoff): # cutoff is specified by the user
toRemove.append(i)
It might also be a good idea to change some of your variable names. You're currently using element for all your loops, and it sometimes refers to an index (in another list) and other times refers to a value. A more descriptive name would help distinguish between those cases!
As comments said, when you del something, the size of the list decreases and further elements shift (there's no gap).
What you can do is:
1. Use what you already used, but going from the highest index first.
2. Use your loop to mark elements for deletion (e.g. by assigning a None or other value not used in the list) rather than physically delete it - and after the loop filter them out (e.g. using myList2 = [elem for elem in myList2 if elem]).
3. Use list comprehension and avoid using loop altogether.
myList2 = [value for (index, value) in enumerate(myList2) if index not in toRemove]
I would zip all lists, having the first (int-valued) list as the first list, then filter the list of tuples based on the first element of each tuple and unzip the filtered list of tuples back into several lists.
In [2]: list1 = [1, 2, 3, 4, 5]
In [3]: list2 = list('abcde')
In [4]: list3 = list('12345')
In [5]: list(map(list, zip(*[x for x in zip(list1, list2, list3) if x[0] >= 3])))
Out[5]: [[3, 4, 5], ['c', 'd', 'e'], ['3', '4', '5']]
Is it what you'd like to have?
Reversing the list using myList.sort(reverse=True) solved my problem. Thanks!

How to get a list of combinations in z3py?

I have a list in python of integers like this:
myList = [97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57]
I want to get Z3 to output various sets or lists of numbers which are all members of myList... Essentially, I want to use Z3 to get additional lists of numbers which all exist in myList but are in various different orders. Put differently, I want to get various outputs from Z3 which contain numbers in the set myList above.
I am having trouble doing this with Z3py because I do not know how to have z3 return a list or a set as the model when I call s.model(), assuming s = Solver().
from z3 import *
myList = [97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57]
s = Solver ()
pick = []
for (i, v) in enumerate(myList):
b = Bool ('pick_%d' % i)
s.add(b == b)
pick += [(b, v)]
while (s.check() == sat):
m = s.model()
chosen = []
block = []
for (p, v) in pick:
if m.eval(p, model_completion=True):
chosen += [v]
block.append(Not(p))
else:
block.append(p)
print chosen
s.add(Or(block))
Note that this will print 2^n solutions where n is the number of elements in your list; so it'll take a while to finish if n is large!

Best way to append new number to the `range` list and sort the new list

I've been writing Python for a while. What are the opinions on what is the best way to make these three lines concise?
my_range = range(97, 123) # I want a-z
my_range.append(32) # I want space as well
my_range = sorted(my_range) # I want it sorted (whilst not needed)
~~~~~~~~~~~~~~~~~~~~~
import random
count = 0
string = ""
my_range = range(97, 123)
my_range.append(32)
my_range = sorted(my_range)
while len(string) != 27:
string = string + chr(random.choice(my_range))
print string
~~~~~~~~~~~~~~~~
In Python 2, you can do:
>>> my_range = sorted([32]+range(97,123))
>>> my_range
[32, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122]
Single Line Solution
In Python 2.7, in order to achieve this in single line, you may simply do:
>>> sorted(range(97, 123) + [32])
[32, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122]
whereas in Python 3.x, you need to type-cast range to list as it returns object of range type (which doesn't have + operator defined for it):
>>> sorted(list(range(97, 123)) + [32])
[32, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122]
Performance Efficient Solution (using bisect.bisect)
However, the performance efficient way to achieve this will be using bisect.bisect which gives the same result without performing sorting on the list. This solution is based on the fact that range list is already sorted. Hence, insert the new number at the correct position in the sorted list maintaining the sortedness (without explicitly sorting the list). Below are the example to illustrate the behavior:
from bisect import bisect
# Example 1: when new number falls inside the range
my_range = list(range(31, 35))
my_range.insert(bisect(my_range, 32), 32)
# where `my_range` will hold `[31, 32, 32, 33, 34]`
# Example 2: when new number falls before the range
my_range = list(range(97, 102))
my_range.insert(bisect(my_range, 32), 32)
# where `my_range` will hold `[32, 97, 98, 99, 100, 101]`
# Example 3: when new number falls after the range
my_range = list(range(25, 30))
my_range.insert(bisect(my_range, 32), 32)
# where `my_range` will hold `[25, 26, 27, 28, 29, 32]`
Performance Comparison
Below is the timeit comparison of both the above solution in Python 3. As the size of range will increase, the time difference between both the solution will increase proportionally.
using sorted(...) (1.91 usec per loop)
mquadri$ python3 -m timeit "sorted(list(range(97, 123)) + [32])"
100000 loops, best of 3: 1.91 usec per loop
using bisect.bisect(...) (1.18 usec per loop)
mquadri$ python3 -m timeit -s "from bisect import bisect" "my_range = list(range(97, 123)); my_range.insert(bisect(my_range, 32), 32)"
1000000 loops, best of 3: 1.18 usec per loop

Repeated Random Numbers

I have created a bingo game where random numbers are generated and called for a list.
bingo_num = random.randint(1,100)
How would I stop a random number being generated more than once
I would suggest, random.shuffle
from random import shuffle
my_list = range(100)
shuffle(my_list)
print my_list
But if you need only specific amount of unique numbers, you can use random.sample, like this
from random import sample
my_list = range(100)
print sample(my_list, 10)
You can take a sample of a range from the random library
>>> import random
>>> nums = random.sample(range(0,200),100)
>>> nums
[143, 149, 52, 183, 161, 179, 180, 155, 163, 157, 139, 15, 154, 181, 56, 29, 31,
14, 77, 82, 165, 32, 35, 92, 109, 172, 69, 99, 54, 3, 88, 76, 11, 126, 78, 162,
198, 145, 124, 75, 114, 174, 136, 100, 190, 193, 148, 153, 167, 113, 38, 17, 16
8, 0, 196, 73, 47, 164, 184, 6, 140, 30, 58, 74, 4, 79, 147, 178, 191, 21, 112,
13, 27, 57, 199, 116, 28, 104, 111, 71, 23, 85, 170, 25, 141, 156, 91, 7, 182, 1
34, 94, 169, 175, 166, 137, 160, 129, 36, 67, 135]
Just do it the same way they do it in a real Bingo game. They do not roll dices, but put all the numbers in a big bag, shake it, and pull out numbers one at a time, until all the numbers are used up.
numbers = list(range(1, 101)) # all the numbers in the bag, from 1 to 100
random.shuffle(numbers) # shake the bag
bingo_num = numbers.pop() # pull out next number (inside your loop)
You could first create a list containing all possible numbers. Then pick a random number from that list, add that to a result list and finally remove it from the list of possible numbers.
For example if you want 5 different numbers from 0-9:
possible_numbers = range(10)
numbers = []
for i in range(5):
index = random.randint(0, len(possible_numbers) - 1)
numbers.append(possible_numbers[index])
del possible_numbers[index]
Something like this should work:
numbers = []
while len(numbers) < 100:
bingo_num = random.randint(1,100)
if not bingo_num in numbers:
numbers.append(bing_num)
You could just create a new number if already in use
advantage: easy code
disadvantage: will run long if nearly all numbers are use (there are better solutions)
Sample code:
bingo_num_list = []
# init num
bingo_num = random.randint(1,100)
# create new numbers till "find" a not used one
while bingo_num in bingo_num_list:
bingo_num = random.randint(1,100)
bingo_num_list.append(bingo_num)
Build list of all the numbers then select a random 1, remove it from the list then select the next one:
#!/usr/bin/python
import random
myNumz=[]
xIdx=1
while xIdx<101:
myNumz.insert(xIdx,xIdx)
xIdx+=1
xIdx=100
while xIdx>0:
xIdx-=1
baseNum=random.randint(0,xIdx)
print myNumz[baseNum]
myNumz.remove(myNumz[baseNum])

Categories