Related
I have a tuple like this (note that first element can be of any size (big but not extremely big, ie. 2**12 - 1 is OK), and second will always be in range [0, 255]).
t = [(0, 137), (0, 80), (0, 78), (0, 71), (0, 13), ...]
I want to store these numbers as bytes on the file system (for compression). That means I also want to later use these bits to recover the tuple. Also note that it is a requirement that the Big endian is used.
for idx, v in compressed:
if v:
f.write(struct.pack(">I", idx))
f.write(struct.pack(">I", v))
However, when I try to get the numbers, like this:
with open(filepath, 'rb') as file:
data = file.read(4)
nums = []
while data:
num = struct.unpack(">I", data)[0]
print(num)
data = file.read(4)
nums.append(num)
I am not getting the numbers above (I am for some numbers, but later it gets messed up, probably because of bit padding).
How to stay consistent with bit padding? How can I add something with struct.pack('>I, ...) that I can later reliably get?
Update:
For the following tuple
[(0, 137), (0, 80), (0, 78), (0, 71), (0, 13), (0, 10), (0, 26), (6, 0), (0, 0), (9, 13), (0, 73), (0, 72), (0, 68), (0, 82), (9, 0), (0, 1), (0, 44), (15, 1), (17, 8), (0, 2), (15, 0), (0, 246) ...]
I get the following numbers using my approach:
[0, 137, 0, 80, 0, 78, 0, 71, 0, 13, 0, 10, 0, 26, 9, 13, 0, 73, 0, 72, 0, 68, 0, 82, 0, 1, 0, 44, 15, 1, 17, 8, 0, 2, 0, 246 ...]
See, at the (6,0) it starts to diverge. Until then it's fine. But it corrects itself?? at 9,13 and continues to do well.
Your code seems to work fine. However, you do have the following line in there:
if v:
v will be False for some of those tuples where the second element is 0 which won't be written to the file and therefore you won't see them when reading from that file, again.
Also, since you're writing your elements in pairs anyways, you could use >II as your format:
from struct import pack, unpack, calcsize
original = [(0, 137), (0, 80), (0, 78), (0, 71), (0, 13), (0, 10), (0, 26), (6, 0), (0, 0), (9, 13), (0, 73), (0, 72), (0, 68), (0, 82), (9, 0), (0, 1), (0, 44), (15, 1), (17, 8), (0, 2), (15, 0), (0, 246)]
filename = "test.txt"
fileformat = ">II"
with open(filename, "wb") as fp:
for element in original:
fp.write(pack(fileformat, *element))
with open(filename, "rb") as fp:
elements = iter(lambda: fp.read(calcsize(fileformat)), b"")
readback = [unpack(fileformat, element) for element in elements]
print(readback == original)
Given the following input:
compressed = [(0, 137), (0, 80), (0, 78), (0, 71), (0, 13), (0, 10), (0, 26), (6, 0), (0, 0), (9, 13), (0, 73), (0, 72), (0, 68), (0, 82), (9, 0), (0, 1), (0, 44), (15, 1), (17, 8), (0, 2), (15, 0), (0, 246)]
try this code for wrting data:
import struct
with open('file.dat', 'wb') as f:
for idx, v in compressed:
f.write(struct.pack(">I", idx))
f.write(struct.pack(">I", v))
and this code for reading:
with open('file.dat', 'rb') as f:
data = f.read(4)
nums = []
while data:
idx = struct.unpack(">I", data)[0]
data = f.read(4)
v = struct.unpack(">I", data)[0]
data = f.read(4)
nums.append((idx,v))
and nums contains:
[(0, 137), (0, 80), (0, 78), (0, 71), (0, 13), (0, 10), (0, 26), (6, 0), (0, 0), (9, 13), (0, 73), (0, 72), (0, 68), (0, 82), (9, 0), (0, 1), (0, 44), (15, 1), (17, 8), (0, 2), (15, 0), (0, 246)]
Which is the same as the input, in fact nums == compressed gives True.
We have a given number from 2 to N. Each number divisible should appear as below:
[(2,4), (2,6), (2,8),(3,9),(3,12),(3,15),(4,8),(4,12),(4,16) ...] upto N
numbers.
I tried it myself (see below), but I am not getting the expected output as above (which is the one I want).
>>> [(a, b) for a in range(5) for b in range(5) if a%2 == 0 and b %2==0]
>>> [(0, 0), (0, 2), (0, 4), (2, 0), (2, 2), (2, 4), (4, 0), (4, 2), (4, 4)]
NOTE: Any number divisible by n (where n is a whole number - 1, 2, 3, 4 ...) is a multiple of n.
You can try:
n = int(input("Enter a number: "))
multiples = [(a, b) for a in range(2, n + 1) for b in range(2, n + 1) if b%a == 0 and a != b]
print (multiples)
where n is the number "below" which the number of multiples is printed for a specific number, but using n + 1 prints the number of multiples "up to" n (only if it is possible).
For example, when n = 10, it will give this output, [(2, 4), (2, 6), (2, 8), (2, 10), (3, 6), (3, 9), (4, 8), (5, 10)].
This has two conditions: b%a == 0, which makes sure that a is not zero, as b / 0 == math error and b%a checks whether the second number is a factor of the first number, or not.
Without b%a == 0, you would have:
[(2, 3), (2, 4), (2, 5), (2, 6), (2, 7), (2, 8), (2, 9), (3, 2), (3, 4), (3, 5), (3, 6), (3, 7), (3, 8), (3, 9), (4, 2), (4, 3), (4, 5), (4, 6), (4, 7), (4, 8), (4, 9), (5, 2), (5, 3), (5, 4), (5, 6), (5, 7), (5, 8), (5, 9), (6, 2), (6, 3), (6, 4), (6, 5), (6, 7), (6, 8), (6, 9), (7, 2), (7, 3), (7, 4), (7, 5), (7, 6), (7, 8), (7, 9), (8, 2), (8, 3), (8, 4), (8, 5), (8, 6), (8, 7), (8, 9), (9, 2), (9, 3), (9, 4), (9, 5), (9, 6), (9, 7), (9, 8)]
Since you do not want a equal to b (3 == 3), you can use a != b, so don't have to worry about getting (3, 3) or (4, 4) etc.
To get the maximum number of multiples for a good range of numbers ( 2 to n) , I would use a larger sample such as n = 50, which will give this output:
[(2, 4), (2, 6), (2, 8), (2, 10), (2, 12), (2, 14), (2, 16), (2, 18), (2, 20), (2, 22), (2, 24), (2, 26), (2, 28), (2, 30), (2, 32), (2, 34), (2, 36), (2, 38), (2, 40), (2, 42), (2, 44), (2, 46), (2, 48), (2, 50), (3, 6), (3, 9), (3, 12), (3, 15), (3, 18), (3, 21), (3, 24), (3, 27), (3, 30), (3, 33), (3, 36), (3, 39), (3, 42), (3, 45), (3, 48), (4, 8), (4, 12), (4, 16), (4, 20), (4, 24), (4, 28), (4, 32), (4, 36), (4, 40), (4, 44), (4, 48), (5, 10), (5, 15), (5, 20), (5, 25), (5, 30), (5, 35), (5, 40), (5, 45), (5, 50), (6, 12), (6, 18), (6, 24), (6, 30), (6, 36), (6, 42), (6, 48), (7, 14), (7, 21), (7, 28), (7, 35), (7, 42), (7, 49), (8, 16), (8, 24), (8, 32), (8, 40), (8, 48), (9, 18), (9, 27), (9, 36), (9, 45), (10, 20), (10, 30), (10, 40), (10, 50), (11, 22), (11, 33), (11, 44), (12, 24), (12, 36), (12, 48), (13, 26), (13, 39), (14, 28), (14, 42), (15, 30), (15, 45), (16, 32), (16, 48), (17, 34), (18, 36), (19, 38), (20, 40), (21, 42), (22, 44), (23, 46), (24, 48), (25, 50)]
Hope this helps!
[(a, b) for a in range(5) for b in range(5)]
gives tuples containing all the combinations of pairs of numbers starting from, and including 0, up to (but not including) 5.
a%2 == 0 is checking for a being even. Likewise for b%2 == 0, so you are finding tuples of even numbers from the unfiltered list above, hence your output.
If you don't want zero, start at 1, e.g. range(1,5) or range(2, 5).
In fact, you need to avoid zeros.
To check the second number is a factor of the first, check b%a, having made sure a won't be zero.
So more like this:
>>> [(a, b) for a in range(2, 5) for b in range(2, 5) if b%a == 0]
[(2, 2), (2, 4), (3, 3), (4, 4)]
To avoid (3,3) or other cases add a != b to the condition in the list comprehension.
[(a, b) for a in range(2, 5) for b in range(2, 5) if b%a == 0 and a!=b]
[(2, 4)]
If you then want each second number to just appear once, you can subsequently filter that list.
I am working on a capacitated vehicle routing problem and have found an optimal solution with the following set of arcs on my graph active:
[(0, 1),
(0, 4),
(0, 5),
(0, 6),
(0, 7),
(0, 10),
(1, 0),
(2, 13),
(3, 9),
(4, 12),
(5, 0),
(6, 14),
(7, 8),
(8, 0),
(9, 0),
(10, 11),
(11, 0),
(12, 3),
(13, 0),
(14, 2)]
the list is called arcsDelivery.
I would like to restructure this list to come to my found routes stored in the list routesdelivery:
[[0,1,0],[0,4,12,3,9,0],[0,5,0],[0,6,14,2,13,0],[0,7,8,0],[0,10,11,0]]
However, I have been struggeling to do so, anyone with some helpful tips?
Here is a way to do it (considering that the arcsdelivery list is sorted in ascending order based on the first element of each tuple):
def findTuple(elem):
for t in arcsDelivery:
if t[0]==elem:
return t
return None
arcsDelivery = [(0, 1),
(0, 4),
(0, 5),
(0, 6),
(0, 7),
(0, 10),
(1, 0),
(2, 13),
(3, 9),
(4, 12),
(5, 0),
(6, 14),
(7, 8),
(8, 0),
(9, 0),
(10, 11),
(11, 0),
(12, 3),
(13, 0),
(14, 2)]
routesDelivery = []
startRoutes = len(list(filter(lambda elem: elem[0]==0, arcsDelivery)))
for i in range(startRoutes):
tempList = []
currentTuple = arcsDelivery[i]
tempList.append(currentTuple[0])
tempList.append(currentTuple[1])
while True:
if currentTuple[1]==0:
break
else:
nextTuple = findTuple(currentTuple[1])
currentTuple = nextTuple
tempList.append(currentTuple[1])
routesDelivery.append(tempList)
print(routesDelivery)
Output:
[[0, 1, 0], [0, 4, 12, 3, 9, 0], [0, 5, 0], [0, 6, 14, 2, 13, 0], [0, 7, 8, 0], [0, 10, 11, 0]]
I have a list of tuples similar to A:
A = [[(90, 1, 5), (126, 1, 3), (139, 1, 3), (1000, 1, 5), (111, 1, 2), (176, 1, 5)],
[(160, 2, 5), (1000, 2, 5), (111, 1, 2)],
[(134, 3, 5), (126, 1, 3), (128, 3, 4), (139, 1, 3)],
[(128, 3, 4)],
[(90, 1, 5), (160, 2, 5), (134, 3, 5), (1000, 2, 5), (1000, 1, 5), (176, 1, 5)]]
In each row of this list, there might be tuples which their second and third elements are the same. For example in A[0]:
A[0] = [(90, 1, 5), (126, 1, 3), (139, 1, 3), (1000, 1, 5), (111, 1, 2), (176, 1, 5)]
(90, 1, 5), (1000, 1, 5) and (176, 1, 5) have the same second and third elements. Among these, I need to keep the one which has the max value for the first element and remove the other two. So, I should be able to keep (1000, 1, 5) and remove (90, 1, 5) and (176, 1, 5) from A[0].
It would be better to keep the ordering of the list.
Is there any way to do that iteratively for all the rows in A? Any help would be appreciated!
If I understand correctly, here's an itertools.groupby solution. I'm assuming order in the final result does not matter.
from itertools import groupby
def keep_max(lst, groupkey, maxkey):
'groups lst w.r.t. to groupkey, keeps maximum of each group w.r.t. maxkey'
sor = sorted(lst, key=groupkey)
groups = (tuple(g) for _, g in groupby(sor, key=groupkey))
return [max(g, key=maxkey) for g in groups]
In action:
>>> from operator import itemgetter
>>> groupkey = itemgetter(1, 2)
>>> maxkey = itemgetter(0)
>>> A = [[(90, 1, 5), (126, 1, 3), (139, 1, 3), (1000, 1, 5), (111, 1, 2), (176, 1, 5)], [(160, 2, 5), (1000, 2, 5), (111, 1, 2)], [(134, 3, 5), (126, 1, 3), (128, 3, 4), (139, 1, 3)], [(128, 3, 4)], [(90, 1, 5), (160, 2, 5), (134, 3, 5), (1000, 2, 5), (1000, 1, 5), (176, 1, 5)]]
>>>
>>> [keep_max(sub, groupkey, maxkey) for sub in A]
[[(111, 1, 2), (139, 1, 3), (1000, 1, 5)],
[(111, 1, 2), (1000, 2, 5)],
[(139, 1, 3), (128, 3, 4), (134, 3, 5)],
[(128, 3, 4)],
[(1000, 1, 5), (1000, 2, 5), (134, 3, 5)]]
This solution keeps the original ordering of the tuples assuming each tuple (as a whole) is unique; in the case there are duplicates tuples this will return the last appearance of each tuple:
from operator import itemgetter
A = [[(90, 1, 5), (126, 1, 3), (139, 1, 3), (1000, 1, 5), (111, 1, 2), (176, 1, 5)],
[(160, 2, 5), (1000, 2, 5), (111, 1, 2)],
[(134, 3, 5), (126, 1, 3), (128, 3, 4), (139, 1, 3)],
[(128, 3, 4)],
[(90, 1, 5), (160, 2, 5), (134, 3, 5), (1000, 2, 5), (1000, 1, 5), (176, 1, 5)]]
def uniques(lst):
groups = {}
for t in lst:
groups.setdefault(t[1:], []).append(t)
lookup = {t: i for i, t in enumerate(lst)}
index = lookup.get
first = itemgetter(0)
return sorted(map(lambda x: max(x, key=first), groups.values()), key=index)
result = [uniques(a) for a in A]
print(result)
Output
[[(139, 1, 3), (1000, 1, 5), (111, 1, 2)], [(1000, 2, 5), (111, 1, 2)], [(134, 3, 5), (128, 3, 4), (139, 1, 3)], [(128, 3, 4)], [(134, 3, 5), (1000, 2, 5), (1000, 1, 5)]]
If you can afford to ignore ordering, you can use itertools.groupby to group elements by the 2nd and 3rd elements on a list sorted by ascending order of 2nd and 3rd elements and descending order of the first element. Then the first element of each group is your desired choice:
from itertools import groupby
A = [[(90, 1, 5), (126, 1, 3), (139, 1, 3), (1000, 1, 5), (111, 1, 2), (176, 1, 5)],
[(160, 2, 5), (1000, 2, 5), (111, 1, 2)],
[(134, 3, 5), (126, 1, 3), (128, 3, 4), (139, 1, 3)],
[(128, 3, 4)],
[(90, 1, 5), (160, 2, 5), (134, 3, 5), (1000, 2, 5), (1000, 1, 5), (176, 1, 5)]]
def max_duplicate(lst):
res = []
for k, g in groupby(sorted(lst, key=lambda x: (x[1], x[2], -x[0])), key=lambda x: (x[1], x[2])):
res.append(next(g))
return res
result = [max_duplicate(l) for l in A]
for r in result:
print(r)
Output
[(111, 1, 2), (139, 1, 3), (1000, 1, 5)]
[(111, 1, 2), (1000, 2, 5)]
[(139, 1, 3), (128, 3, 4), (134, 3, 5)]
[(128, 3, 4)]
[(1000, 1, 5), (1000, 2, 5), (134, 3, 5)]
You can do this by using a hashmap as follows:
d = {}
for a in A:
for aa in a:
v, k1, k2 = aa
if (k1, k2) in d:
d[(k1, k2)] = max(v, d[(k1, k2)])
else:
d[(k1, k2)] = v
l = [[v, k1, k2] for (k1, k2), v in d.iteritems()]
Using dictionaries:
fin = []
for row in A:
dict = {}
for tup in row:
dict[tup[1:2]] = tup[0]
fin.append(dict)
A = [[value, t1, t1] for (t1, t2), value in dict.iteritems()]
Using this, your dict will transform A[0] from
A[0] = [(90, 1, 5), (126, 1, 3), (139, 1, 3), (1000, 1, 5), (111, 1, 2), (176, 1, 5)]
to
{ (1,5): 1000, (1,3): 139, (1,2): 111 } (as a dict)
and can then be converted back to an array using iteritems
This way, the order will also be preserved.
I'm trying to parse items in a text file and store them into a list. The data looks something like this:
[(0, 0, 0), (1, 0, 0), (2, 0, 0), (3, 0, 0), (4, 0, 0), (5, 0, 0), (6, 0, 0)]
[(10, 3, 1), (11, 3, 1), (12, 3, 1), (13, 3, 1), (13, 4, 1)]
[(10, 3, 5), (11, 3, 5), (12, 3, 5), (13, 3, 5), (13, 4, 5), (13, 5, 5), (13, 6, 5)]
[(6, 13, 5), (7, 13, 5), (8, 13, 5), (8, 14, 5), (7, 14, 5), (6, 14, 5), (6, 14, 6)]
I was able to strip the "[" and "]" but couldn't store the rest of information into list as such format:
(x, y, z). Any help?
def dataParser(fileName):
zoneList=[]; zone=[]
input=open(fileName,"r")
for line in input:
vals = line.strip("[")
newVals = vals.strip("]\n")
print newVals
v=newVals[0:9]
zone.append(v)
input.close()
return zone
In this particular case, you can use ast.literal_eval:
>>> with open("list.txt") as fp:
... data = [ast.literal_eval(line) for line in fp if line.strip()]
...
>>> data
[[(0, 0, 0), (1, 0, 0), (2, 0, 0), (3, 0, 0), (4, 0, 0), (5, 0, 0), (6, 0, 0)], [(10, 3, 1), (11, 3, 1), (12, 3, 1), (13, 3, 1), (13, 4, 1)], [(10, 3, 5), (11, 3, 5), (12, 3, 5), (13, 3, 5), (13, 4, 5), (13, 5, 5), (13, 6, 5)], [(6, 13, 5), (7, 13, 5), (8, 13, 5), (8, 14, 5), (7, 14, 5), (6, 14, 5), (6, 14, 6)]]
It's the "safe" version of eval. It's not as general, though, for precisely that reason. If you're generating this input, you might want to look into a different way to save your data ("serialization"), whether using pickle or something like JSON -- there are lots of examples of using both you can find on SO and elsewhere.
You can do it without eval, using the string split method and the tuple constructor:
>>> st = "[(0,0,0), (1,0,0)]"
>>> splits = st.strip('[').strip(']\n').split(', ')
>>> splits
['(0,0,0)', '(1,0,0)']
>>> for split in splits:
... trimmed = split.strip('(').strip(')')
... tup = tuple(trimmed.split(','))
... print tup, type(tup)
...
('0', '0', '0') <type 'tuple'>
('1', '0', '0') <type 'tuple'>
>>>
From there, it's just appending to a list.
some might not like using eval() here, but you can do this in one line using it:
In [20]: lis=eval("[(0, 0, 0), (1, 0, 0), (2, 0, 0), (3, 0, 0), (4, 0, 0), (5, 0, 0), (6, 0, 0)]")
In [23]: lis
Out[23]: [(0, 0, 0), (1, 0, 0), (2, 0, 0), (3, 0, 0), (4, 0, 0), (5, 0, 0), (6, 0, 0)]
using a text file:
In [44]: with open('data.txt') as f:
....: lis=[eval(x.strip()) for x in f]
....: print lis
....:
....:
[[(0, 0, 0), (1, 0, 0), (2, 0, 0), (3, 0, 0), (4, 0, 0), (5, 0, 0), (6, 0, 0)], [(10, 3, 1), (11, 3, 1), (12, 3, 1), (13, 3, 1), (13, 4, 1)], [(10, 3, 5), (11, 3, 5), (12, 3, 5), (13, 3, 5), (13, 4, 5), (13, 5, 5), (13, 6, 5)], [(6, 13, 5), (7, 13, 5), (8, 13, 5), (8, 14, 5), (7, 14, 5), (6, 14, 5), (6, 14, 6)]]
The following is a bad idea if you're getting this data from any source that you don't trust completely, but if the data will always be in this format (and only contain numbers as the elements) something like this is quite straightforward:
collect = []
for line in input:
collect.append(eval(line))
The other answers work just fine and are a simple solution to this specific problem. But I am assuming that if you were having problems with string manipulation, then a simple eval() function won't help you out much the next time you have this problem.
As a general rule, the first thing you want to do when you are approached with a problem like this, is define your delimiters.
[(0, 0, 0), (1, 0, 0), (2, 0, 0), (3, 0, 0), (4, 0, 0), (5, 0, 0), (6, 0, 0)]
Here you can see that "), (" is a potential delimiter between groups and a simple comma (",") is your delimiter between values. Next you want to see what you need to remove, and as you pointed out, brackets ("[" and "]") provide little information. We can also see that because we are dealing with numeric values, all spacing gives us little information and needs to be removed.
Building on this information, I set up your dataParser function in a way that returns the values you were looking for:
fileName= "../ZoneFinding/outputData/zoneFinding_tofu_rs1000.txt"
def dataParser(fileName):
with open(fileName,"r") as input
zoneLst = []
for line in input:
#First remove white space and the bracket+parenthese combos on the end
line = line.replace(" ","").replace("[(","").replace(")]","")
#Now lets split line by "),(" to create a list of strings with the values
lineLst = line.split("),(")
# At this point lineLst = ["0,0,0" , "1,0,0", "2,0,0", ...]
#Lastly, we will split each number by a "," and add each to a list
zone = [group.split(",") for group in lineLst]
zoneLst.append(zone)
return zoneLst
In the example above, all of the values are stored as strings. You could also replace the
definition of zone with the code below to store the values as floats.
zone = [ [float(val) for val in group.split(",")] for group in lineLst]