Both my professor and this guy claim that range creates a list of values.
"Note: The range function simply returns a list containing the numbers
from x to y-1. For example, range(5, 10) returns the list [5, 6, 7, 8,
9]."
I believe this is to be inaccurate because:
type(range(5, 10))
<class 'range'>
Furthermore, the only apparent way to access the integers created by range is to iterate through them, which leads me to believe that labeling range as a lists is incorrect.
In Python 2.x, range returns a list, but in Python 3.x range returns an immutable sequence, of type range.
Python 2.x:
>>> type(range(10))
<type 'list'>
>>> type(xrange(10))
<type 'xrange'>
Python 3.x:
>>> type(range(10))
<class 'range'>
In Python 2.x, if you want to get an iterable object, like in Python 3.x, you can use xrange function, which returns an immutable sequence of type xrange.
Advantage of xrange over range in Python 2.x:
The advantage of xrange() over range() is minimal (since xrange() still has to create the values when asked for them) except when a very large range is used on a memory-starved machine or when all of the range’s elements are never used (such as when the loop is usually terminated with break).
Note:
Furthermore, the only apparent way to access the integers created by range() is to iterate through them,
Nope. Since range objects in Python 3 are immutable sequences, they support indexing as well. Quoting from the range function documentation,
Ranges implement all of the common sequence operations except concatenation and repetition
...
Range objects implement the collections.abc.Sequence ABC, and provide features such as containment tests, element index lookup, slicing and support for negative indices.
For example,
>>> range(10, 20)[5]
15
>>> range(10, 20)[2:5]
range(12, 15)
>>> list(range(10, 20)[2:5])
[12, 13, 14]
>>> list(range(10, 20, 2))
[10, 12, 14, 16, 18]
>>> 18 in range(10, 20)
True
>>> 100 in range(10, 20)
False
All these are possible with that immutable range sequence.
Recently, I faced a problem and I think it would be appropriate to include here. Consider this Python 3.x code
from itertools import islice
numbers = range(100)
items = list(islice(numbers, 10))
while items:
items = list(islice(numbers, 10))
print(items)
One would expect this code to print every ten numbers as a list, till 99. But, it would run infinitely. Can you reason why?
Solution
Because the range returns an immutable sequence, not an iterator object. So, whenever islice is done on a range object, it always starts from the beginning. Think of it as a drop-in replacement for an immutable list. Now the question comes, how will you fix it? Its simple, you just have to get an iterator out of it. Simply change
numbers = range(100)
to
numbers = iter(range(100))
Now, numbers is an iterator object and it remembers how long it has been iterated before. So, when the islice iterates it, it just starts from the place where it previously ended.
It depends.
In python-2.x, range actually creates a list (which is also a sequence) whereas xrange creates an xrange object that can be used to iterate through the values.
On the other hand, in python-3.x, range creates an iterable (or more specifically, a sequence)
range creates a list if the python version used is 2.x .
In this scenario range is to be used only if its referenced more than once otherwise use xrange which creates a generator there by redusing the memory usage and sometimes time as it has lazy approach.
xrange is not there in python 3.x rather range stands for what xrange is for python 2.x
refer to question
What is the difference between range and xrange functions in Python 2.X?
Python3
for loop with range function
To understand what for i in range() means in Python3, we need first to understand the working of the range() function.
The range() function uses the generator to produce numbers. It doesn’t generate all numbers at once.
As you know range() returns the range object. A range object uses the same (small) amount of memory, no matter the size of the range it represents. It only stores the start, stop and step values and calculates individual items and subranges as needed.
I.e., It generates the next value only when for loop iteration asked for it. In each loop iteration, It generates the next value and assigns it to the iterator variable i.
So it means range() produces numbers one by one as the loop moves to the next iteration. It saves lots of memory, which makes range() faster and efficient.
Reference: PyNative - for loop with range()
Related
I tried the following in the IDLE of Python 3.6
print(value for value in range(1,100))
A message is produced in the IDLE which says
<generator object <genexpr> at 0x101b73a40>
I'm confused what this means. Have I done anything wrong?
(value for value in range(1,100)) produces generator object, if you want to print list, just wrap it in []
print([value for value in range(1,100)])
or you can simply
print(list(range(1,100)))
You can read what generators are HERE
A function which returns a generator iterator. It looks like a normal function except that it contains yield expressions for producing a series of values usable in a for-loop or that can be retrieved one at a time with the next() function.
Usually refers to a generator function, but may refer to a generator iterator in some contexts. In cases where the intended meaning isn’t clear, using the full terms avoids ambiguity.
print([value for value in range(1,100)])
You should include it inside a list [] to generate the values
All you wanted is to use list comprehension here but you are not using list.
Simply it should be
for i in range(1,100):
print(i)
If you want to still use list comprehension it will be like this
[print(x) for x in range(1,100)]
this will give the output like
1
2
3
....
99
100
[None, None, None, ...97 times more]
If I write
for i in range(5):
print i
Then it gives 0, 1, 2, 3, 4
Does that mean Python assigned 0, 1, 2, 3, 4 to i at the same time?
However if I wrote:
for i in range(5):
a=i+1
Then I call a, it only gives 5
But if I add ''print a'' it gives 1, 2, 3, 4, 5
So my question is what is the difference here?
Is i a string or a list or something else?
Or maybe can anyone help me to sort out:
for l in range(5):
#vs,fs,rs are all m*n matrixs,got initial values in,i.e vs[0],fs[0],rs[0] are known
#want use this foor loop to update them
vs[l+1]=vs[l]+fs[l]
fs[l+1]=((rs[l]-re[l])
rs[l+1]=rs[l]+vs[l]
#then this code gives vs,fs,rs
If I run this kind of code, then I will get the answer only when l=5
How can I make them start looping?
i.e l=0 got values for vs[1],fs[1],rs[1],
then l=1 got values for vs[2],rs[2],fs[2]......and so on.
But python gives different arrays of fs,vs,rs, correspond to different value of l
How can I make them one piece?
A "for loop" in most, if not all, programming languages is a mechanism to run a piece of code more than once.
This code:
for i in range(5):
print i
can be thought of working like this:
i = 0
print i
i = 1
print i
i = 2
print i
i = 3
print i
i = 4
print i
So you see, what happens is not that i gets the value 0, 1, 2, 3, 4 at the same time, but rather sequentially.
I assume that when you say "call a, it gives only 5", you mean like this:
for i in range(5):
a=i+1
print a
this will print the last value that a was given. Every time the loop iterates, the statement a=i+1 will overwrite the last value a had with the new value.
Code basically runs sequentially, from top to bottom, and a for loop is a way to make the code go back and something again, with a different value for one of the variables.
I hope this answered your question.
When I'm teaching someone programming (just about any language) I introduce for loops with terminology similar to this code example:
for eachItem in someList:
doSomething(eachItem)
... which, conveniently enough, is syntactically valid Python code.
The Python range() function simply returns or generates a list of integers from some lower bound (zero, by default) up to (but not including) some upper bound, possibly in increments (steps) of some other number (one, by default).
So range(5) returns (or possibly generates) a sequence: 0, 1, 2, 3, 4 (up to but not including the upper bound).
A call to range(2,10) would return: 2, 3, 4, 5, 6, 7, 8, 9
A call to range(2,12,3) would return: 2, 5, 8, 11
Notice that I said, a couple times, that Python's range() function returns or generates a sequence. This is a relatively advanced distinction which usually won't be an issue for a novice. In older versions of Python range() built a list (allocated memory for it and populated with with values) and returned a reference to that list. This could be inefficient for large ranges which might consume quite a bit of memory and for some situations where you might want to iterate over some potentially large range of numbers but were likely to "break" out of the loop early (after finding some particular item in which you were interested, for example).
Python supports more efficient ways of implementing the same semantics (of doing the same thing) through a programming construct called a generator. Instead of allocating and populating the entire list and return it as a static data structure, Python can instantiate an object with the requisite information (upper and lower bounds and step/increment value) ... and return a reference to that.
The (code) object then keeps track of which number it returned most recently and computes the new values until it hits the upper bound (and which point it signals the end of the sequence to the caller using an exception called "StopIteration"). This technique (computing values dynamically rather than all at once, up-front) is referred to as "lazy evaluation."
Other constructs in the language (such as those underlying the for loop) can then work with that object (iterate through it) as though it were a list.
For most cases you don't have to know whether your version of Python is using the old implementation of range() or the newer one based on generators. You can just use it and be happy.
If you're working with ranges of millions of items, or creating thousands of different ranges of thousands each, then you might notice a performance penalty for using range() on an old version of Python. In such cases you could re-think your design and use while loops, or create objects which implement the "lazy evaluation" semantics of a generator, or use the xrange() version of range() if your version of Python includes it, or the range() function from a version of Python that uses the generators implicitly.
Concepts such as generators, and more general forms of lazy evaluation, permeate Python programming as you go beyond the basics. They are usually things you don't have to know for simple programming tasks but which become significant as you try to work with larger data sets or within tighter constraints (time/performance or memory bounds, for example).
[Update: for Python3 (the currently maintained versions of Python) the range() function always returns the dynamic, "lazy evaluation" iterator; the older versions of Python (2.x) which returned a statically allocated list of integers are now officially obsolete (after years of having been deprecated)].
for i in range(5):
is the same as
for i in [0,1,2,3,4]:
range(x) returns a list of numbers from 0 to x - 1.
>>> range(1)
[0]
>>> range(2)
[0, 1]
>>> range(3)
[0, 1, 2]
>>> range(4)
[0, 1, 2, 3]
for i in range(x): executes the body (which is print i in your first example) once for each element in the list returned by range().
i is used inside the body to refer to the “current” item of the list.
In that case, i refers to an integer, but it could be of any type, depending on the objet on which you loop.
The range function wil give you a list of numbers, while the for loop will iterate through the list and execute the given code for each of its items.
for i in range(5):
print i
This simply executes print i five times, for i ranging from 0 to 4.
for i in range(5):
a=i+1
This will execute a=i+1 five times. Since you are overwriting the value of a on each iteration, at the end you will only get the value for the last iteration, that is 4+1.
Useful links:
http://www.network-theory.co.uk/docs/pytut/rangeFunction.html
http://www.ibiblio.org/swaroopch/byteofpython/read/for-loop.html
It is looping, probably the problem is in the part of the print...
If you can't find the logic where the system prints, just add the folling where you want the content out:
for i in range(len(vs)):
print vs[i]
print fs[i]
print rs[i]
When I read the Django tickets and pull requests, I find they have following changes:
They want to get a range about page numbers, And change from
return range(1, self.num_pages + 1) # 1
to:
return six.moves.range(1, self.num_pages + 1) # 2
then:
return list(six.moves.range(1, self.num_pages + 1)) # 3
I am confusing what's difference between 1 and 3? And why not just this, is there any difference with 3?
return list(range(1, self.num_pages + 1)) # 4
From the Python documentation:
Python 3 reorganized the standard library and moved several functions
to different modules. Six provides a consistent interface to them
through the fake six.moves module.
In the same page, it tells us that six.moves.range() calls xrange() for Python 2 and range() for Python 3. xrange() is very similar to range(), except that it returns an xrange object rather than a list. Thus the list() around the call to make it into a list.
It seems that the whole point of the change was to make the code work with both Python 2 and Python 3.
range in Python 2.X is constructs a new list. So, for example:
range(4) == [0, 1, 2, 3]
xrange in Python 2.X is an generator that lazily yields values equivalent to what would be returned by range without any of the memory overhead.
Python 3.X got rid of range with replaced it with xrange. The behavior of xrange is generally more efficient, but if you really need to construct a list using range you'll have to explicitly iterate it or use the list constructor.
range in Python 2.X returns a list. In Python 3.X:
>>> type(range(4))
<class 'range'>
To explicitly create a list with range in Python 3 just do:
list(range(4)) == [0, 1, 2, 3]
Or
my_list = [ x for x in range(4) ]
Seems that this SO question may offer some guidance: six.moves.builtins.range is not consistent in Python 2 and Python 3, it certainly did for me. It looks like they made the change for efficiency sake.
The following python code produces [(0, 0), (0, 7)...(0, 693)] instead of the expected list of tuples combining all of the multiples of 3 and multiples of 7:
multiples_of_3 = (i*3 for i in range(100))
multiples_of_7 = (i*7 for i in range(100))
list((i,j) for i in multiples_of_3 for j in multiples_of_7)
This code fixes the problem:
list((i,j) for i in (i*3 for i in range(100)) for j in (i*7 for i in range(100)))
Questions:
The generator object seems to play the role of an iterator instead of providing an iterator object each time the generated list is to be enumerated. The later strategy seems to be adopted by .Net LINQ query objects. Is there an elegant way to get around this?
How come the second piece of code works? Shall I understand that the generator's iterator is not reset after looping through all multiples of 7?
Don't you think that this behavior is counter intuitive if not inconsistent?
A generator object is an iterator, and therefore one-shot. It's not an iterable which can produce any number of independent iterators. This behavior is not something you can change with a switch somewhere, so any work around amounts to either using an iterable (e.g. a list) instead of an generator or repeatedly constructing generators.
The second snippet does the latter. It is by definition equivalent to the loops
for i in (i*3 for i in range(100)):
for j in (i*7 for i in range(100)):
...
Hopefully it isn't surprising that here, the latter generator expression is evaluated anew on each iteration of the outer loop.
As you discovered, the object created by a generator expression is an iterator (more precisely a generator-iterator), designed to be consumed only once. If you need a resettable generator, simply create a real generator and use it in the loops:
def multiples_of_3(): # generator
for i in range(100):
yield i * 3
def multiples_of_7(): # generator
for i in range(100):
yield i * 7
list((i,j) for i in multiples_of_3() for j in multiples_of_7())
Your second code works because the expression list of the inner loop ((i*7 ...)) is evaluated on each pass of the outer loop. This results in creating a new generator-iterator each time around, which gives you the behavior you want, but at the expense of code clarity.
To understand what is going on, remember that there is no "resetting" of an iterator when the for loop iterates over it. (This is a feature; such a reset would break iterating over a large iterator in pieces, and it would be impossible for generators.) For example:
multiples_of_2 = iter(xrange(0, 100, 2)) # iterator
for i in multiples_of_2:
print i
# prints nothing because the iterator is spent
for i in multiples_of_2:
print i
...as opposed to this:
multiples_of_2 = xrange(0, 100, 2) # iterable sequence, converted to iterator
for i in multiples_of_2:
print i
# prints again because a new iterator gets created
for i in multiples_of_2:
print i
A generator expression is equivalent to an invoked generator and can therefore only be iterated over once.
The real issue as I found out is about single versus multiple pass iterables and the fact that there is currently no standard mechanism to determine if an iterable single or multi pass: See Single- vs. Multi-pass iterability
If you want to convert a generator expression to a multipass iterable, then it can be done in a fairly routine fashion. For example:
class MultiPass(object):
def __init__(self, initfunc):
self.initfunc = initfunc
def __iter__(self):
return self.initfunc()
multiples_of_3 = MultiPass(lambda: (i*3 for i in range(20)))
multiples_of_7 = MultiPass(lambda: (i*7 for i in range(20)))
print list((i,j) for i in multiples_of_3 for j in multiples_of_7)
From the point of view of defining the thing it's a similar amount of work to typing:
def multiples_of_3():
return (i*3 for i in range(20))
but from the point of view of the user, they write multiples_of_3 rather than multiples_of_3(), which means the object multiples_of_3 is polymorphic with any other iterable, such as a tuple or list.
The need to type lambda: is a bit inelegant, true. I don't suppose there would be any harm in introducing "iterable comprehensions" to the language, to give you what you want while maintaining backward compatibility. But there are only so many punctuation characters, and I doubt this would be considered worth one.
This question already has answers here:
Why do these list operations (methods: clear / extend / reverse / append / sort / remove) return None, rather than the resulting list?
(6 answers)
Closed 2 years ago.
I am attempting to sort a Python list of ints and then use the .pop() function to return the highest one. I have tried a writing the method in different ways:
def LongestPath(T):
paths = [Ancestors(T,x) for x in OrdLeaves(T)]
#^ Creating a lists of lists of ints, this part works
result =[len(y) for y in paths ]
#^ Creating a list of ints where each int is a length of the a list in paths
result = result.sort()
#^meant to sort the result
return result.pop()
#^meant to return the largest int in the list (the last one)
I have also tried
def LongestPath(T):
return[len(y) for y in [Ancestors(T,x) for x in OrdLeaves(T)] ].sort().pop()
In both cases .sort() causes the list to be None (which has no .pop() function and returns an error). When I remove the .sort() it works fine but does not return the largest int since the list is not sorted.
Simply remove the assignment from
result = result.sort()
leaving just
result.sort()
The sort method works in-place (it modifies the existing list), so it returns None. When you assign its result to the name of the list, you're assigning None. So no assignment is necessary.
But in any case, what you're trying to accomplish can easily (and more efficiently) be written as a one-liner:
max(len(Ancestors(T,x)) for x in OrdLeaves(T))
max operates in linear time, O(n), while sorting is O(nlogn). You also don't need nested list comprehensions, a single generator expression will do.
This
result = result.sort()
should be this
result.sort()
It is a convention in Python that methods that mutate sequences return None.
Consider:
>>> a_list = [3, 2, 1]
>>> print a_list.sort()
None
>>> a_list
[1, 2, 3]
>>> a_dict = {}
>>> print a_dict.__setitem__('a', 1)
None
>>> a_dict
{'a': 1}
>>> a_set = set()
>>> print a_set.add(1)
None
>>> a_set
set([1])
Python's Design and History FAQ gives the reasoning behind this design decision (with respect to lists):
Why doesn’t list.sort() return the sorted list?
In situations where performance matters, making a copy of the list
just to sort it would be wasteful. Therefore, list.sort() sorts the
list in place. In order to remind you of that fact, it does not return
the sorted list. This way, you won’t be fooled into accidentally
overwriting a list when you need a sorted copy but also need to keep
the unsorted version around.
In Python 2.4 a new built-in function – sorted() – has been added.
This function creates a new list from a provided iterable, sorts it
and returns it.
.sort() returns None and sorts the list in place.
This has already been correctly answered: list.sort() returns None. The reason why is "Command-Query Separation":
http://en.wikipedia.org/wiki/Command-query_separation
Python returns None because every function must return something, and the convention is that a function that doesn't produce any useful value should return None.
I have never before seen your convention of putting a comment after the line it references, but starting the comment with a carat to point at the line. Please put comments before the lines they reference.
While you can use the .pop() method, you can also just index the list. The last value in the list can always be indexed with -1, because in Python negative indices "wrap around" and index backward from the end.
But we can simplify even further. The only reason you are sorting the list is so you can find its max value. There is a built-in function in Python for this: max()
Using list.sort() requires building a whole list. You will then pull one value from the list and discard it. max() will consume an iterator without needing to allocate a potentially-large amount of memory to store the list.
Also, in Python, the community prefers the use of a coding standard called PEP 8. In PEP 8, you should use lower-case for function names, and an underscore to separate words, rather than CamelCase.
http://www.python.org/dev/peps/pep-0008/
With the above comments in mind, here is my rewrite of your function:
def longest_path(T):
paths = [Ancestors(T,x) for x in OrdLeaves(T)]
return max(len(path) for path in paths)
Inside the call to max() we have a "generator expression" that computes a length for each value in the list paths. max() will pull values out of this, keeping the biggest, until all values are exhausted.
But now it's clear that we don't even really need the paths list. Here's the final version:
def longest_path(T):
return max(len(Ancestors(T, x)) for x in OrdLeaves(T))
I actually think the version with the explicit paths variable is a bit more readable, but this isn't horrible, and if there might be a large number of paths, you might notice a performance improvement due to not building and destroying the paths list.
list.sort() does not return a list - it destructively modifies the list you are sorting:
In [177]: range(10)
Out[177]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
In [178]: range(10).sort()
In [179]:
That said, max finds the largest element in a list, and will be more efficient than your method.
In Python sort() is an inplace operation. So result.sort() returns None, but changes result to be sorted. So to avoid your issue, don't overwrite result when you call sort().
Is there any reason not to use the sorted function? sort() is only defined on lists, but sorted() works with any iterable, and functions the way you are expecting. See this article for sorting details.
Also, because internally it uses timsort, it is very efficient if you need to sort on key 1, then sort on key 2.
You don't need a custom function for what you want to achieve, you first need to understand the methods you are using!
sort()ing a list in python does it in place, that is, the return from sort() is None. The list itself is modified, a new list is not returned.
>>>results = ['list','of','items']
>>>results
['list','of','items']
>>>results.sort()
>>>type(results)
<type 'list'>
>>>results
['items','list','of']
>>>results = results.sort()
>>>results
>>>
>>>type(results)
<type 'NoneType'>
As you can see, when you try to assign the sort() , you no longer have the list type.