Related
This question already has answers here:
Why are slice and range upper-bound exclusive?
(6 answers)
Closed last month.
>>> range(1,11)
gives you
[1,2,3,4,5,6,7,8,9,10]
Why not 1-11?
Did they just decide to do it like that at random or does it have some value I am not seeing?
Because it's more common to call range(0, 10) which returns [0,1,2,3,4,5,6,7,8,9] which contains 10 elements which equals len(range(0, 10)). Remember that programmers prefer 0-based indexing.
Also, consider the following common code snippet:
for i in range(len(li)):
pass
Could you see that if range() went up to exactly len(li) that this would be problematic? The programmer would need to explicitly subtract 1. This also follows the common trend of programmers preferring for(int i = 0; i < 10; i++) over for(int i = 0; i <= 9; i++).
If you are calling range with a start of 1 frequently, you might want to define your own function:
>>> def range1(start, end):
... return range(start, end+1)
...
>>> range1(1, 10)
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
Although there are some useful algorithmic explanations here, I think it may help to add some simple 'real life' reasoning as to why it works this way, which I have found useful when introducing the subject to young newcomers:
With something like 'range(1,10)' confusion can arise from thinking that pair of parameters represents the "start and end".
It is actually start and "stop".
Now, if it were the "end" value then, yes, you might expect that number would be included as the final entry in the sequence. But it is not the "end".
Others mistakenly call that parameter "count" because if you only ever use 'range(n)' then it does, of course, iterate 'n' times. This logic breaks down when you add the start parameter.
So the key point is to remember its name: "stop".
That means it is the point at which, when reached, iteration will stop immediately. Not after that point.
So, while "start" does indeed represent the first value to be included, on reaching the "stop" value it 'breaks' rather than continuing to process 'that one as well' before stopping.
One analogy that I have used in explaining this to kids is that, ironically, it is better behaved than kids! It doesn't stop after it supposed to - it stops immediately without finishing what it was doing. (They get this ;) )
Another analogy - when you drive a car you don't pass a stop/yield/'give way' sign and end up with it sitting somewhere next to, or behind, your car. Technically you still haven't reached it when you do stop. It is not included in the 'things you passed on your journey'.
I hope some of that helps in explaining to Pythonitos/Pythonitas!
Exclusive ranges do have some benefits:
For one thing each item in range(0,n) is a valid index for lists of length n.
Also range(0,n) has a length of n, not n+1 which an inclusive range would.
It works well in combination with zero-based indexing and len(). For example, if you have 10 items in a list x, they are numbered 0-9. range(len(x)) gives you 0-9.
Of course, people will tell you it's more Pythonic to do for item in x or for index, item in enumerate(x) rather than for i in range(len(x)).
Slicing works that way too: foo[1:4] is items 1-3 of foo (keeping in mind that item 1 is actually the second item due to the zero-based indexing). For consistency, they should both work the same way.
I think of it as: "the first number you want, followed by the first number you don't want." If you want 1-10, the first number you don't want is 11, so it's range(1, 11).
If it becomes cumbersome in a particular application, it's easy enough to write a little helper function that adds 1 to the ending index and calls range().
It's also useful for splitting ranges; range(a,b) can be split into range(a, x) and range(x, b), whereas with inclusive range you would write either x-1 or x+1. While you rarely need to split ranges, you do tend to split lists quite often, which is one of the reasons slicing a list l[a:b] includes the a-th element but not the b-th. Then range having the same property makes it nicely consistent.
The length of the range is the top value minus the bottom value.
It's very similar to something like:
for (var i = 1; i < 11; i++) {
//i goes from 1 to 10 in here
}
in a C-style language.
Also like Ruby's range:
1...11 #this is a range from 1 to 10
However, Ruby recognises that many times you'll want to include the terminal value and offers the alternative syntax:
1..10 #this is also a range from 1 to 10
Consider the code
for i in range(10):
print "You'll see this 10 times", i
The idea is that you get a list of length y-x, which you can (as you see above) iterate over.
Read up on the python docs for range - they consider for-loop iteration the primary usecase.
Basically in python range(n) iterates n times, which is of exclusive nature that is why it does not give last value when it is being printed, we can create a function which gives
inclusive value it means it will also print last value mentioned in range.
def main():
for i in inclusive_range(25):
print(i, sep=" ")
def inclusive_range(*args):
numargs = len(args)
if numargs == 0:
raise TypeError("you need to write at least a value")
elif numargs == 1:
stop = args[0]
start = 0
step = 1
elif numargs == 2:
(start, stop) = args
step = 1
elif numargs == 3:
(start, stop, step) = args
else:
raise TypeError("Inclusive range was expected at most 3 arguments,got {}".format(numargs))
i = start
while i <= stop:
yield i
i += step
if __name__ == "__main__":
main()
The range(n) in python returns from 0 to n-1. Respectively, the range(1,n) from 1 to n-1.
So, if you want to omit the first value and get also the last value (n) you can do it very simply using the following code.
for i in range(1, n + 1):
print(i) #prints from 1 to n
It's just more convenient to reason about in many cases.
Basically, we could think of a range as an interval between start and end. If start <= end, the length of the interval between them is end - start. If len was actually defined as the length, you'd have:
len(range(start, end)) == start - end
However, we count the integers included in the range instead of measuring the length of the interval. To keep the above property true, we should include one of the endpoints and exclude the other.
Adding the step parameter is like introducing a unit of length. In that case, you'd expect
len(range(start, end, step)) == (start - end) / step
for length. To get the count, you just use integer division.
Two major uses of ranges in python. All things tend to fall in one or the other
integer. Use built-in: range(start, stop, step). To have stop included would mean that the end step would be assymetric for the general case. Consider range(0,5,3). If default behaviour would output 5 at the end, it would be broken.
floating pont. This is for numerical uses (where sometimes it happens to be integers too). Then use numpy.linspace.
Here is the problem I'm trying to solve:
Given an int, ops, n, create a function(int, ops, n) and slot operators between digits of int to create equations that evaluates to n. Return a list of all possible answers. Importing functions is not allowed.
For example,
function(111111, '+-%*', 11) => [1*1+11/1-1 = 11, 1*1/1-1+11 =11, ...]
The question recommended using interleave(str1, str2) where interleave('abcdef', 'ab') = 'aabbcdef' and product(str1, n) where product('ab', 3) = ['aaa','aab','abb','bbb','aba','baa','bba'].
I have written interleave(str1, str2) which is
def interleave(str1,str2):
lsta,lstb,result= list(str1),list(str2),''
while lsta and lstb:
result += lsta.pop(0)
result += lstb.pop(0)
if lsta:
for i in lsta:
result+= i
else:
for i in lstb:
result+=i
return result
However, I have no idea how to code the product function. I assume it has to do something with recursion, so I'm trying to add 'a' and 'b' for every product.
def product(str1,n):
if n ==1:
return []
else:
return [product(str1,n-1)]+[str1[0]]
Please help me to understand how to solve this question. (Not only the product it self)
General solution
Assuming your implementation of interleave is correct, you can use it together with product (see my suggested implementation below) to solve the problem with something like:
def f(i, ops, n):
int_str = str(i)
retval = []
for seq_len in range(1, len(int_str)):
for op_seq in r_prod(ops, seq_len):
eq = interleave(int_str, op_seq)
if eval(eq) == n:
retval.append(eq)
return retval
The idea is that you interleave the digits of your string with your operators in a varying order. Basically I do that with all possible sequences of length seq_len which varies from 1 to max, which will be the number of digits - 1 (see assumptions below!). Then you use the built-in function eval to evaluate the expression returned by inteleave for a specific sequence of the operators and compare the result with the desired number, n. If the expression evaluates to n you append it to the return array retval (initially empty). After you evaluated all the expressions for all possible operator sequences (see assumptions!) you return the array.
Assumptions
It's not clear whether you can use the same operator multiple times or if you're allowed to omit using some. I assumed you can use the same operator many times and that you're allowed to omit using an operator. Hence, the r_prod was used (as suggested by your question). In case of such restrictions, you will want to use permutations (of possibly varying length) of the group of operators.
Secondly, I assumed that your implementation of the interleave function is correct. It is not clear if, for example, interleave("112", "*") should return both "1*12" and "11*2" or just "1*12" like your implementation does. In the case both should be returned, then you should also iterate over the possible ways the same ordered sequence of operators can be interleaved with the provided digits. I omitted that, because I saw that your function always returns a single string.
Product implementation
If you look at the itertools docs you can see the equivalent code for the function itertools.product. Using that you'd have:
def product(*args, repeat=1):
pools = [tuple(pool) for pool in args] * repeat
result = [[]]
for pool in pools:
result = [x+[y] for x in result for y in pool]
for prod in result:
yield tuple(prod)
a = ["".join(x) for x in product('ab', repeat=3)]
print(a)
Which prints ['aaa', 'aab', 'aba', 'abb', 'baa', 'bab', 'bba', 'bbb'] -- what I guess is what you're after.
A more specific (assuming iterable is a string), less efficient, but hopefully more understandable solution would be:
def prod(string, r):
if r < 1:
return None
retval = list(string)
for i in range(r - 1):
temp = []
for l in retval:
for c in string:
temp.append(l + c)
retval = temp
return retval
The idea is simple. The second parameter r gives you the length of the strings you want to produce. The characters in the string give you the elements from which you build the string. Hence, you first generate a string of length 1 that starts with each possible character. Then for each of those strings you generate new strings by concatenating the old string with all of the possible characters.
For example, given a pool of characters "abc", you'll first generate strings "a", "b", and "c". Then you'll replace string "a" with strings "aa", "ab", and "ac". Similarly for "b" and "c". You repeat this process n-times to get all possible strings of length r generated by drawing with replacement from the pool "abc".
I'd think it would be a good idea for you to try to implement the prod function recursively. You can see my ugly solution below, but I'd suggest you stop reading this now and try to do it without looking at my suggestion first.
SPOILER BELOW
def r_prod(string, r):
if r == 1:
return list(string)
else:
return [c + s for c in string for s in r_prod(string, r - 1)]
I am working on disassembling a firmware (Siemens C165 processor - https://www.infineon.com/dgdl/Infineon-C165-DS-v02_00-en%5B8%5D.pdf?fileId=db3a304412b407950112b43a49a66fd7) in IDA.
I have the firmware, so I can also read it via Python.
I need to find a string being permutation of
0, 1, 2, 3, 4, 5, 6, 7 (0-7)
Wrote this simple program:
from itertools import permutations
l = list(permutations(range(0,8)))
print(len(l))
with open("firm.ori", 'rb') as f:
s = f.read()
for i in l:
hexstr = '\\x'.join('{:02x}'.format(x) for x in i)
hexstrfinal = "\\0x" + hexstr
#print hexstrfinal
if(s.find(b'hexstrfinal')>0):
print "found"
However, it does not find anything
I thought the sequence will be next to each other, but maybe not.
Want to just be sure that the program is correct.
Well actually, 0-7 should be nibbles, so does it mean I need to search for, for example as a one combination:
0x01, 0x23, 0x45, 0x67
Above is bytes.
Can somebody confirm this and advise how to search for this?
Update 1:
Tried 2nd variant
from itertools import permutations
l = list(permutations(range(0,8)))
print(len(l))
with open("firm.ori", 'rb') as f:
s = f.read()
for item in l:
str1 = ''.join(str(e) for e in item)
n = 2
out = [str1[i:i+n] for i in range(0, len(str1), n)]
hexstr = '\\x'.join(e for e in out)
hexstrfinal = "\\x" + hexstr
#print hexstrfinal
if(s.find(b'hexstrfinal')>0):
print "found"
But also no hits ...
Any ideas what I do wrong?
There are a few misunderstandings in your code, and a major inefficiency. Let's start with the misunderstandings.
Since firm.ori is opened in binary mode (rb), the result of s = f.read() is a bytes object. Despite having similar methods to a string, this is not a string! It contains bytes, not characters. When you display it, the \x... outputs do not indicate that the bytes object containing ASCII backslashes and xes. Instead, each \x... is an escape sequence used to represent the hex value of a given byte that does not correspond to an ASCII printable character.
Inside your loop, you deal exclusively with strings: hexstr = '\\x'.join('{:02x}'.format(x) for x in i) takes your permutation and formats it to look like the string representation of a bytes object. Hopefully you understand from the previous paragraph why this won't work.
s.find(b'hexstrfinal') searches for the literal ASCII array b'hexstrfinal', not for the variable named hexstrfinal. The latter wouldn't work of course, because hexstrfinal has type str, not bytes. If you were to convert it to bytes using a simple hexstrfinal.encode('ascii'), you'd get b'\\x...', which is not at all what you want. The proper way would be
s.find(hexstrfinal.encode('ascii').decode('unicode-escape').encode('latin1'))
Hopefully you can see why it is unnecessarily inefficient to convert a string three times to get the bytes you want. Any time you start using strings as a crutch to manipulate numbers is a good time to evaluate your approach. That begins our discussion of the inefficiencies in your code.
You are currently attempting to iterate through all the possible permutations of 0-7 rather than looking for the permutations that are actually there. Given that the file is only 200KB in size, it would be unreasonable to expect all or even most permutations to appear in it. Moreover, you are searching the entire file for each possible permutation. For a file size N and K permutations, your code runs in O(N * K) time, while it's possible to do this in a single pass through the file, or O(N). With the appropriate data structures, even a loop written in plain Python will likely run faster than an optimized version of the current code.
The strategy is simple. Iterate through s. If the current character and the following seven make up a valid permutation, start a party. Otherwise, keep looking:
N = 8
allowed = set(range(N))
for index, b in enumerate(s):
if b in allowed and set(s[index:index + N]) == allowed:
print(f'Found sequence {s[index:index + N]} at offset {index}')
There are a host of possible optimizations possible here, and you could probably do the whole thing way more efficiently with numpy or scipy.
Things would also get more complicated if you allowed repeats in the sequence. In that case, you'd have to sort the sequences:
allowed = sorted(...)
N = len(allowed)
for index, b in enumerate(s):
if b in allowed and sorted(s[index:index + N]) == allowed:
print(f'Found sequence {s[index:index + N]} at offset {index}')
If you are going to search for nibbles, things get more complicated still. I would drop entirely the check b in allowed, and just write a custom check that could be applied at every half-step:
N = 8
def build_set(items):
output = set()
for b in items:
output.add(b & 0xF)
output.add((b >> 4) & 0xF)
return output
def matches(seq):
if len(seq) == N // 2:
return build_set(seq) == allowed
elif len(seq) == N // 2 + 1:
check = build_set(seq[1:-1])
check.add(seq[0] & 0xF)
check.add((seq[-1] >> 4) & 0xF)
return check == allowed
else:
return False
allowed = set(range())
for index, b in enumerate(s):
if matches(s[index:index + N // 2]):
print(f'Found sequence {s[index:index + N // 2]} at offset {index}.0')
if matches(s[index:index + N // 2 + 1]):
print(f'Found sequence {s[index:index + N // 2 + 1]]} at offset {index}.5')
Here, build_set just splits the nibbles into a set. matches checks either an array of 8 nibbles aligned on a byte (4 elements), or an array of 8 nibbles offset by half a byte (5 elements). Both cases are reported independently.
It is not clear what you are trying to search for but...
Each permutation of (0,1,2,3,4,5,6,7) will be a seven item tuple similar to this
t = (7, 6, 4, 1, 3, 5, 0, 2)
You can make two-item strings like this
>>> a = [''.join(map(str,thing)) for thing in zip(t,t[1:])]
>>> a
['76', '64', '41', '13', '35', '50', '02']
Then make integers of the strings and feed it to bytes
>>> b = bytes(map(int,a))
>>> b
b'L#)\r#2\x02'
Then search for it
>>> b in s
????
If it doesn't find it it's not there.
Here is a ten character bytes object (similar to your file)
>>> b = b'\xcbl\x7f|_k\x00\x9f\xa2\xcc'
It just happens to be:
>>> bytes([203, 108, 127, 124, 95, 107, 0, 159, 162, 204])
Search for 3-character (or 3-integer) sequences
>>> bytes([127,94,107]) in b
False
>>> bytes([127,95,107]) in b
False
>>> bytes([124,95,107]) in b
True
>>>
When I thing of binary files, I really think integers not characters.
A string is palindrome if it reads the same forward and backward. Given a string that contains only lower case English alphabets, you are required to create a new palindrome string from the given string following the rules gives below:
1. You can reduce (but not increase) any character in a string by one; for example you can reduce the character h to g but not from g to h
2. In order to achieve your goal, if you have to then you can reduce a character of a string repeatedly until it becomes the letter a; but once it becomes a, you cannot reduce it any further.
Each reduction operation is counted as one. So you need to count as well how many reductions you make. Write a Python program that reads a string from a user input (using raw_input statement), creates a palindrome string from the given string with the minimum possible number of operations and then prints the palindrome string created and the number of operations needed to create the new palindrome string.
I tried to convert the string to a list first, then modify the list so that should any string be given, if its not a palindrome, it automatically edits it to a palindrome and then prints the result.after modifying the list, convert it back to a string.
c=raw_input("enter a string ")
x=list(c)
y = ""
i = 0
j = len(x)-1
a = 0
while i < j:
if x[i] < x[j]:
a += ord(x[j]) - ord(x[i])
x[j] = x[i]
print x
else:
a += ord(x[i]) - ord(x[j])
x [i] = x[j]
print x
i = i + 1
j = (len(x)-1)-1
print "The number of operations is ",a print "The palindrome created is",( ''.join(x) )
Am i approaching it the right way or is there something I'm not adding up?
Since only reduction is allowed, it is clear that the number of reductions for each pair will be the difference between them. For example, consider the string 'abcd'.
Here the pairs to check are (a,d) and (b,c).
Now difference between 'a' and 'd' is 3, which is obtained by (ord('d')-ord('a')).
I am using absolute value to avoid checking which alphabet has higher ASCII value.
I hope this approach will help.
s=input()
l=len(s)
count=0
m=0
n=l-1
while m<n:
count+=abs(ord(s[m])-ord(s[n]))
m+=1
n-=1
print(count)
This is a common "homework" or competition question. The basic concept here is that you have to find a way to get to minimum values with as few reduction operations as possible. The trick here is to utilize string manipulation to keep that number low. For this particular problem, there are two very simple things to remember: 1) you have to split the string, and 2) you have to apply a bit of symmetry.
First, split the string in half. The following function should do it.
def split_string_to_halves(string):
half, rem = divmod(len(string), 2)
a, b, c = '', '', ''
a, b = string[:half], string[half:]
if rem > 0:
b, c = string[half + 1:], string[rem + 1]
return (a, b, c)
The above should recreate the string if you do a + c + b. Next is you have to convert a and b to lists and map the ord function on each half. Leave the remainder alone, if any.
def convert_to_ord_list(string):
return map(ord, list(string))
Since you just have to do a one-way operation (only reduction, no need for addition), you can assume that for each pair of elements in the two converted lists, the higher value less the lower value is the number of operations needed. Easier shown than said:
def convert_to_palindrome(string):
halfone, halftwo, rem = split_string_to_halves(string)
if halfone == halftwo[::-1]:
return halfone + halftwo + rem, 0
halftwo = halftwo[::-1]
zipped = zip(convert_to_ord_list(halfone), convert_to_ord_list(halftwo))
counter = sum([max(x) - min(x) for x in zipped])
floors = [min(x) for x in zipped]
res = "".join(map(chr, floors))
res += rem + res[::-1]
return res, counter
Finally, some tests:
target = 'ideal'
print convert_to_palindrome(target) # ('iaeai', 6)
target = 'euler'
print convert_to_palindrome(target) # ('eelee', 29)
target = 'ohmygodthisisinsane'
print convert_to_palindrome(target) # ('ehasgidihmhidigsahe', 84)
I'm not sure if this is optimized nor if I covered all bases. But I think this pretty much covers the general concept of the approach needed. Compared to your code, this is clearer and actually works (yours does not). Good luck and let us know how this works for you.
I have a list of numbers that represent the number of 1s in a row and I have an integer that represents the length of the entire sequence. So for example, if I get list [1, 2, 3] and length 8, then the only possible bit sequence is 10110111. But if the length was 9, I should get
010110111, 100110111, 101100111 and 101101110. I wonder if there is a simply pythonic way of doing this. My current method is
def genSeq(seq, length):
strSeq = [str(i) for i in seq]
strRep = strSeq + ["x" for i in xrange(length-sum(seq))]
perm = list(set(permutations(strRep)))
legalSeq = [seq for seq in perm if isLegal(seq) and [char for char in seq if char.isdigit()] == strSeq]
return [''.join(["1"*int(i) if i.isdigit() else "0" for i in seq]) for seq in legalSeq]
def isLegal(seq):
for i in xrange(len(seq)-1):
if seq[i].isdigit() and seq[i+1].isdigit(): return False
return True
print genSeq([1, 2, 3], 9)
My approach is as follows:
Figure out how many zeros will appear in the result sequences, by doing the appropriate arithmetic. We'll call this value n, and call the length of the input list k.
Now, imagine that we write out the n zeros for a given result sequence in a row. To create a valid sequence, we need to choose k places where we'll insert the appropriate number of ones, and we have n + 1 choices of where to do so - in between any two digits, or at the beginning or end. So we can use itertools.combinations to give us all the possible position-groups, asking it to choose k values from 0 up to n, inclusive.
Given a combination of ones-positions, in ascending order, we can figure out how many zeroes appear before the first group of ones (it's the first value from the combination), the second (it's the difference between the first two values from the combination), etc. As we iterate, we need to alternate between groups of zeros (the first and/or last of these might be empty) and groups of ones; so there is one more group of zeros than of ones, in general - but we can clean that up by adding a "dummy" zero-length group of ones to the end. We also need to get "overlapping" differences from the ones-positions; fortunately there's a trick for this.
This part is tricky when put all together; I'll write a helper function for it, first:
def make_sequence(one_positions, zero_count, one_group_sizes):
zero_group_sizes = [
end - begin
for begin, end in zip((0,) + one_positions, one_positions + (zero_count,))
]
return ''.join(
'0' * z + '1' * o
for z, o in zip(zero_group_sizes, one_group_sizes + [0])
)
Now we can get back to the iteration over all possible sequences:
def generate_sequences(one_group_sizes, length):
zero_count = length - sum(one_group_sizes)
for one_positions in itertools.combinations(
range(zero_count + 1), len(one_group_sizes)
):
yield make_sequence(one_positions, zero_count, one_group_sizes)
Which we could of course then fold back into a single function, but maybe you'll agree with me that it's better to keep something this clever in more manageable pieces :)
This is more fragile than I'd like - the one_positions and one_group_sizes sequences need to be "padded" to make things work, which in turn requires assuming their type. one_positions will be a tuple because that's what itertools.combinations produces, but I've hard-coded the assumption that the user-supplied one_group_sizes will be a list. There are ways around that but I'm a little exhausted at the moment :)
Anyway, the test:
>>> list(generate_sequences([1,2,3], 9))
['101101110', '101100111', '100110111', '010110111']
>>> list(generate_sequences([1,1,1], 9))
['101010000', '101001000', '101000100', '101000010', '101000001', '100101000', '
100100100', '100100010', '100100001', '100010100', '100010010', '100010001', '10
0001010', '100001001', '100000101', '010101000', '010100100', '010100010', '0101
00001', '010010100', '010010010', '010010001', '010001010', '010001001', '010000
101', '001010100', '001010010', '001010001', '001001010', '001001001', '00100010
1', '000101010', '000101001', '000100101', '000010101']