If I had list of integers say,
x = [1,2,3,4,5]
Is there an in-built function that can convert this into a single number like 12345? If not, what's the easiest way?
>>> listvar = [1,2,3,4,5]
>>> reduce(lambda x,y:x*10+y, listvar, 0)
12345
If they're digits like this,
sum(digit * 10 ** place for place, digit in enumerate(reversed(x)))
int("".join(str(X) for X in x))
You have not told us what the result for x = [1, 23, 4] should be by the way...
My answer gives 1234, others give 334
Just for fun :)
int(str(x)[1:-1].replace(', ', ''))
Surprisingly, this is even faster for large list:
$ python -m timeit -s "x=[1,2,3,4,5,6,7,8,9,0]*100" "int(str(x)[1:-1].replace(', ', ''))"
10000 loops, best of 3: 128 usec per loop
$ python -m timeit -s "x=[1,2,3,4,5,6,7,8,9,0]*100" "int(''.join(map(str, x)))"
10000 loops, best of 3: 183 usec per loop
$ python -m timeit -s "x=[1,2,3,4,5,6,7,8,9,0]*100" "reduce(lambda x,y:x*10+y, x, 0)"
1000 loops, best of 3: 649 usec per loop
$ python -m timeit -s "x=[1,2,3,4,5,6,7,8,9,0]*100" "sum(digit * 10 ** place for place, digit in enumerate(reversed(x)))"
100 loops, best of 3: 7.19 msec per loop
But for very small list (maybe more common?) , this one is slowest.
Related
I was solving a Python question on CodingBat.com. I wrote following code for a simple problem of printing a string n times-
def string_times(str, n):
return n * str
Official result is -
def string_times(str, n):
result = ""
for i in range(n):
result = result + str
return result
print string_times('hello',3)
The output is same for both the functions. I am curious how string multiplication (first function) perform against for loop (second function) on performance basis. I mean which one is faster and mostly used?
Also please suggest me a way to get the answer to this question myself (using time.clock() or something like that)
We can use the timeit module to test this:
python -m timeit "100*'string'"
1000000 loops, best of 3: 0.222 usec per loop
python -m timeit "''.join(['string' for _ in range(100)])"
100000 loops, best of 3: 6.9 usec per loop
python -m timeit "result = ''" "for i in range(100):" " result = result + 'string'"
100000 loops, best of 3: 13.1 usec per loop
You can see that multiplying is the far faster option. You can take note that while the string concatenation version isn't that bad in CPython, that may not be true in other versions of Python. You should always opt for string multiplication or str.join() for this reason - not only but speed, but for readability and conciseness.
I've timed the following three functions:
def string_times_1(s, n):
return s * n
def string_times_2(s, n):
result = ""
for i in range(n):
result = result + s
return result
def string_times_3(s, n):
"".join(s for _ in range(n))
The results are as follows:
In [4]: %timeit string_times_1('hello', 10)
1000000 loops, best of 3: 262 ns per loop
In [5]: %timeit string_times_2('hello', 10)
1000000 loops, best of 3: 1.63 us per loop
In [6]: %timeit string_times_3('hello', 10)
100000 loops, best of 3: 3.87 us per loop
As you can see, s * n is not only the clearest and the most concise, it is also the fastest.
You can use the timeit stuff from either the command line or in code to see how fast some bit of python code is:
$ python -m timeit "\"something\" * 100"
1000000 loops, best of 3: 0.608 usec per loop
Do something similar for your other function and compare.
Is it possible to return two lists from a list comprehension? Well, this obviously doesn't work, but something like:
rr, tt = [i*10, i*12 for i in xrange(4)]
So rr and tt both are lists with the results from i*10 and i*12 respectively.
Many thanks
>>> rr,tt = zip(*[(i*10, i*12) for i in xrange(4)])
>>> rr
(0, 10, 20, 30)
>>> tt
(0, 12, 24, 36)
Creating two comprehensions list is better (at least for long lists). Be aware that, the best voted answer is slower can be even slower than traditional for loops. List comprehensions are faster and clearer.
python -m timeit -n 100 -s 'rr=[];tt = [];' 'for i in range(500000): rr.append(i*10);tt.append(i*12)'
10 loops, best of 3: 123 msec per loop
> python -m timeit -n 100 'rr,tt = zip(*[(i*10, i*12) for i in range(500000)])'
10 loops, best of 3: 170 msec per loop
> python -m timeit -n 100 'rr = [i*10 for i in range(500000)]; tt = [i*10 for i in range(500000)]'
10 loops, best of 3: 68.5 msec per loop
It would be nice to see list comprehensionss supporting the creation of multiple lists at a time.
However,
if you can take an advantage of using a traditional loop (to be precise, intermediate calculations), then it is possible that you will be better of with a loop (or an iterator/generator using yield). Here is an example:
$ python3 -m timeit -n 100 -s 'rr=[];tt=[];' "for i in (range(1000) for x in range(10000)): tmp = list(i); rr.append(min(tmp));tt.append(max(tmp))"
100 loops, best of 3: 314 msec per loop
$ python3 -m timeit -n 100 "rr=[min(list(i)) for i in (range(1000) for x in range(10000))];tt=[max(list(i)) for i in (range(1000) for x in range(10000))]"
100 loops, best of 3: 413 msec per loop
Of course, the comparison in these cases are unfair; in the example, the code and calculations are not equivalent because in the traditional loop a temporary result is stored (see tmp variable). So, the list comprehension is doing much more internal operations (it calculates the tmp variable twice!, yet it is only 25% slower).
It is possible for a list comprehension to return multiple lists if the elements are lists.
So for example:
>>> x, y = [[] for x in range(2)]
>>> x
[]
>>> y
[]
>>>
The trick with zip function would do the job, but actually is much more simpler and readable if you just collect the results in lists with a loop.
What is more efficient in terms of memory and speed between
d[(first,second)]
and
d[first][second],
where d is a dictionary of either tuples or dictionaries?
Here is some very basic test data that indicates that for a very contrived example(storing 'a' a million times using numbers as keys) using 2 dictionaries is significantly faster.
$ python -m timeit 'd = {i:{j:"a" for j in range(1000)} for i in range(1000)};a = [d[i][j] for j in range(1000) for i in range(1000)];'
10 loops, best of 3: 316 msec per loop
$ python -m timeit 'd = {(i, j):"a" for j in range(1000) for i in range(1000)};a = [d[i, j] for j in range(1000) for i in range(1000)];'
10 loops, best of 3: 970 msec per loop
Of course, these tests do not necessarily mean anything depending on what you are trying to do. Determine what you'll be storing, and then test.
A little more data:
$ python -m timeit 'a = [(hash(i), hash(j)) for i in range(1000) for j in range(1000)]'
10 loops, best of 3: 304 msec per loop
$ python -m timeit 'a = [hash((i, j)) for i in range(1000) for j in range(1000)]'
10 loops, best of 3: 172 msec per loop
$ python -m timeit 'd = {i:{j:"a" for j in range(1000)} for i in range(1000)}'
10 loops, best of 3: 101 msec per loop
$ python -m timeit 'd = {(i, j):"a" for j in range(1000) for i in range(1000)}'
10 loops, best of 3: 645 msec per loop
Once again this is clearly not indicative of real world use, but it seems to me like the cost of building a dictionary with tuples like that is huge and that's where the dictionary in a dictionary wins out. This surprises me, I was expecting completely different results. I'll have to try a few more things when I have time.
A little surprisingly, the dictionary of dictionaries is faster than the tuple in both CPython 2.7 and Pypy 1.8.
I didn't check on space, but you can do that with ps.
I would like to split a string according to the title in a single call. I'm looking for a simple syntax using list comprehension, but i don't got it yet:
s = "123456"
And the result would be:
["12", "34", "56"]
What i don't want:
re.split('(?i)([0-9a-f]{2})', s)
s[0:2], s[2:4], s[4:6]
[s[i*2:i*2+2] for i in len(s) / 2]
Edit:
Ok, i wanted to parse a hex RGB[A] color (and possible other color/component format), to extract all the component.
It seem that the fastest approach would be the last from sven-marnach:
sven-marnach xrange: 0.883 usec per loop
python -m timeit -s 's="aabbcc";' '[int(s[i:i+2], 16) / 255. for i in xrange(0, len(s), 2)]'
pair/iter: 1.38 usec per loop
python -m timeit -s 's="aabbcc"' '["%c%c" % pair for pair in zip(* 2 * [iter(s)])]'
Regex: 2.55 usec per loop
python -m timeit -s 'import re; s="aabbcc"; c=re.compile("(?i)([0-9a-f]{2})");
split=re.split' '[int(x, 16) / 255. for x in split(c, s) if x != ""]'
Reading through the comments, it turns out the actual question is: What is the fastest way to parse a color definition string in hexadecimal RRGGBBAA format. Here are some options:
def rgba1(s, unpack=struct.unpack):
return unpack("BBBB", s.decode("hex"))
def rgba2(s, int=int, xrange=xrange):
return [int(s[i:i+2], 16) for i in xrange(0, 8, 2)]
def rgba3(s, int=int, xrange=xrange):
x = int(s, 16)
return [(x >> i) & 255 for i in xrange(0, 32, 8)]
As I expected, the first version turns out to be fastest:
In [6]: timeit rgba1("aabbccdd")
1000000 loops, best of 3: 1.44 us per loop
In [7]: timeit rgba2("aabbccdd")
100000 loops, best of 3: 2.43 us per loop
In [8]: timeit rgba3("aabbccdd")
100000 loops, best of 3: 2.44 us per loop
In [4]: ["".join(pair) for pair in zip(* 2 * [iter(s)])]
Out[4]: ['aa', 'bb', 'cc']
See: How does zip(*[iter(s)]*n) work in Python? for explanations as to that strange "2-iter over the same str" syntax.
You say in the comments that you want to "have the fastest execution", I can't promise you that with this implementation, but you can measure the execution using timeit. Remember what Donald Knuth said about premature optimisation, of course. For the problem at hand (now that you've revealed it) I think you'd find r, g, b = s[0:2], s[2:4], s[4:6] hard to beat.
$ python3.2 -m timeit -c '
s = "aabbcc"
["".join(pair) for pair in zip(* 2 * [iter(s)])]
'
100000 loops, best of 3: 4.49 usec per loop
Cf.
python3.2 -m timeit -c '
s = "aabbcc"
r, g, b = s[0:2], s[2:4], s[4:6]
'
1000000 loops, best of 3: 1.2 usec per loop
Numpy is worse than your preferred solution for a single lookup:
$ python -m timeit -s 'import numpy as np; s="aabbccdd"' 'a = np.fromstring(s.decode("hex"), dtype="uint32"); a.dtype = "uint8"; list(a)'
100000 loops, best of 3: 5.14 usec per loop
$ python -m timeit -s 's="aabbcc";' '[int(s[i:i+2], 16) / 255. for i in xrange(0, len(s), 2)]'
100000 loops, best of 3: 2.41 usec per loop
But if you do several conversions at once, numpy is much faster:
$ python -m timeit -s 'import numpy as np; s="aabbccdd" * 100' 'a = np.fromstring(s.decode("hex"), dtype="uint32"); a.dtype = "uint8"; a.tolist()'
10000 loops, best of 3: 59.6 usec per loop
$ python -m timeit -s 's="aabbccdd" * 100;' '[int(s[i:i+2], 16) / 255. for i in xrange(0, len(s), 2)]'
1000 loops, best of 3: 240 usec per loop
Numpy is faster for batcher larger than 2, on my computer. You can easily group the values by setting a.shape to (number_of_colors, 4), though it makes the tolist method 50% slower.
In fact, most of the time is spent converting the array to a list. Depending on what you wish to do with the results, you may be able to skip this intermeditary step, and reap some benefits:
$ python -m timeit -s 'import numpy as np; s="aabbccdd" * 100' 'a = np.fromstring(s.decode("hex"), dtype="uint32"); a.dtype = "uint8"; a.shape = (100,4)'
100000 loops, best of 3: 6.76 usec per loop
Say I wanted to create an array (NOT list) of 1,000,000 twos in python, like this:
array = [2, 2, 2, ...... , 2]
What would be a fast but simple way of doing it?
The currently-accepted answer is NOT the fastest way using array.array; at least it's not the slowest -- compare these:
[source: johncatfish (quoting chauncey), Bartek]
python -m timeit -s"import array" "arr = array.array('i', (2 for i in range(0,1000000)))"
10 loops, best of 3: 543 msec per loop
[source: g.d.d.c]
python -m timeit -s"import array" "arr = array.array('i', [2] * 1000000)"
10 loops, best of 3: 141 msec per loop
python -m timeit -s"import array" "arr = array.array('i', [2]) * 1000000"
100 loops, best of 3: 15.7 msec per loop
That's a ratio of about 9 to 1 ...
Is this what you're after?
# slower.
twosArr = array.array('i', [2] * 1000000)
# faster.
twosArr = array.array('i', [2]) * 1000000
You can get just a list with this:
twosList = [2] * 1000000
-- EDITED --
I updated this to reflect information in another answer. It would appear that you can increase the speed by a ratio of ~ 9 : 1 by adjusting the syntax slightly. Full credit belongs to #john-machin. I wasn't aware you could multiple the array object the same way you could do to a list.
A hybrid approach works fastest for me
$ python -m timeit -s"import array" "arr = array.array('i', [2]*100) * 10000"
100 loops, best of 3: 5.38 msec per loop
$ python -m timeit -s"import array" "arr = array.array('i', [2]) * 1000000"
10 loops, best of 3: 20.3 msec per loop
$ python -m timeit -s"import array" "arr = array.array('i', [2]*10) * 100000"
100 loops, best of 3: 6.69 msec per loop
$ python -m timeit -s"import array" "arr = array.array('i', [2]*100) * 10000"
100 loops, best of 3: 5.38 msec per loop
$ python -m timeit -s"import array" "arr = array.array('i', [2]*1000) * 1000"
100 loops, best of 3: 5.47 msec per loop
$ python -m timeit -s"import array" "arr = array.array('i', [2]*10000) * 100"
100 loops, best of 3: 6.13 msec per loop
$ python -m timeit -s"import array" "arr = array.array('i', [2]*100000) * 10"
10 loops, best of 3: 14.9 msec per loop
$ python -m timeit -s"import array" "arr = array.array('i', [2]*1000000)"
10 loops, best of 3: 77.7 msec per loop
Using the timeit module you can kind of figure out what the fastest of doing this is:
First off, putting that many digits in a list will kill your machine most likely as it will store it in memory.
However, you can test the execution using something like so. It ran on my computer for a long time before I just gave up, but I'm on an older PC:
timeit.Timer('[2] * 1000000').timeit()
Ther other option you can look into is using the array module which is as stated, efficient arrays of numeric values
array.array('i', (2 for i in range(0, 1000000)))
I did not test the completion time of both but I'm sure the array module, which is designed for number sets will be faster.
Edit: Even more fun, you could take a look at numpy which actually seems to have the fastest execution:
from numpy import *
array( [2 for i in range(0, 1000000)])
Even faster from the comments:
a = 2 * ones(10000000)
Awesome!
aList = [2 for x in range(1000000)]
or base on chauncey link
anArray =array.array('i', (2 for i in range(0,1000000)))
If the initial value doesn't have to be non-zero and if you have /dev/zero available on your platform, the following is about 4.7 times faster than the array('L',[0])*size solution:
myarray = array.array('L')
f = open('/dev/zero', 'rb')
myarray.fromfile(f, size)
f.close()
In question How to initialise an integer array.array object with zeros in Python I'm looking for a better way.