hey guys am new to python app development..i have been trying to fetch only numbers from a list using a for loop..But am confused with the correct syntax..The code i have been used.is like below.
babe = [10,11,13,'vv']
int(honey) [for honey in babe]:
print honey
When i run this i got syntax error.i have tried many situations.But it didnt helped me at all.Sorry for the silly question..
do i wanna add square brackets or something on the second line ??
Am really stuck.Hope you guys can help me out..Thanks in advance
You seem to be conflating the syntax for for loops (a statement followed by a suite of statements ... otherwise known as a "block of code") and a list comprehension (an expression).
Here's a list comprehension:
#!/usr/bin/python
# Given:
b = [1,2,3,'vv']
a = [int(x) for x in b]
... that's syntactically valid. However, the semantics of that example will raise an exception because 'vv' is not a valid literal (string). It cannot be interpreted as a decimal integer.
Here's a for loop:
#!/usr/bin/python
# Given:
b = [1,2,3,'vv']
a = list()
for x in b:
try:
a.append(int(x))
except ValueError:
pass
In this case we explicitly loop over the given list (b) and ignore any ValueError exceptions raised when we try to convert each of those entries into an integer.
There is no reasonable way to handle exceptions from within a list comprehension. You could write a function which returned some sentinel value (from the expression) for any invalid input value. That would look something like this:
#/usr/bin/python
# Given:
b = [1, 2, 3, 'vv']
def mk_integer_if_possible(n):
'''Returns an integer or the sentinel value None
'''
results = None
try:
results = int(n)
except ValueError:
pass
return results
# Use that function:
a = [mk_integer_if_possible(x) for x in b if mk_integer_if_possible(x) is not None]
Note: the absurd function name is deliberate. This is an ugly way to do this and the awkwardness of having to call this putative function TWICE for each element of b is an indication that you should NOT use a list comprehension for this situation. (You have to call it once to make the conversion but again for the conditional. Saving the results from one call would, of course, be a STATEMENT, which we can't have embedded within an EXPRESSION).
Statements contain one or more expressions. Expressions are components of statements. Python strictly delineates between statements and expressions. Assignments are statements in Python. These distinctions can be nuanced and there are other programming languages where assignments are expressions rather than being strictly defined, by the language's syntax, as statements.
So, use the for loop whenever you have to handle possible exceptions while iterating over any sort of data set and usually when you need to filter on the results generated by mapping a function over a list comprehension.
Incidentally the explicit use of the expression is not None is necessary in this example. If I attempted to shorten that test to simply be if mk_integer_if_possible(x) using Python's implicit boolean handling then we'd be inadvertently filtering out any entries from b that evaluated to integer 0 as well as any that were returned as the None sentinel by my ill-advised function.
In Python it's often fine to use implicit boolean values as conditions. None and False as well as any numerically zero value, any empty string or any sort of empty list, tuple or dictionary, are all treated as "false" in a boolean context. However, when dealing with sentinel values it's best to use the is operator and explicitly test for object identity. Otherwise you'll have corner cases where your condition might be matched by values other than your sentinel.
(Handy trick: if you ever come across the need to allow None through some sort of filter or pass it along, but you need some other sentinel ... just use sentinel = object() ... you can create (instantiate) a generic Pythonobject and use is to match it for your sentinel handling. That will be unique to your code and no other Python object or type will match it. Guaranteed).
By the way ... I should note that this code it technically not "fetching only numbers from a list." It is returning integers for all entries in the list which can be converted thereto. This is a nitpick; but it's a distinction that any good engineer will notice. Do you want to return all integers from the input list? Or do you want to return all entries as integers if that can be so converted? Your code suggested that you're trying to accomplish the latter; so that's how I implemented my working examples for you. However, to implement the later semantics you'd probably want to use either the (mathematical) additive or multiplicative identity property like so:
# ... from within some function:
try:
results = x == x + 0 # Additive identity
except (TypeError, ValueError):
results = None
return results
babe = [10,11,13,'vv']
a = [honey for honey in babe if isinstance(honey, int)]
print a
See more here about list comprehension: https://docs.python.org/2/tutorial/datastructures.html#list-comprehensions
Related
I need to find a way to split a string of multiples numbers into multiples strings of those numbers and then split again to have individual digits which would allow me to test those first inputed numbers to see if they are a harshad number without using for, else, while and if.
So far i'm able to split the input string:
a = input("Multiple numbers separated by a ,: ")
a.split(",")
Then I need to split again I think I need to use the map function. Any idea how to go any further.
The python builtin functions map, filter, and reduce are going to be your friend when you are working in a more functional style.
map
The map function lets you transform each item in an iterable (list, tuple, etc.) by passing it to a function and using the return value as a new value in a new iteratable*.
The non-functional approach would use a for ... in construct:
numbers_as_strings = ["1", "12", "13"]
numbers_as_ints = []
for number in numbers_as_strings:
numbers_as_ints.append(int(number))
or more concisely a list comprehension
numbers_as_ints =[int(number) for number in numbers_as_strings]
Since you are eschewing for there is another way
numbers_as_ints = map(int, numbers_as_strings)
But you don't just want your strings mapped to integers, you want to test them for harshadiness. Since we're doing the functional thing let's create a function to do this for us.
def is_harshad(number_as_string):
return # do your harshad test here
Then you can map your numbers through this function
list(map(is_harshad, numbers_as_string)) # wrap in list() to resolve the returned map object.
>>> [True, True, False]
But maybe you want the results as a sequence of harshady number strings? Well check out filter
filter
The filter function lets you choose which items from an iterable you want to keep in a new iterable. You give it a function that operates on an single item and returns True for a keeper or False for a rejection. You also give it an iterable of items to test.
A non-functional way to do this is with a for loop
harshady_numbers = []
for number in numbers_as_strings:
if is_harshad(number):
harshady_numbers.append(number)
Or more concisely and nicely, with a list comprehension
harshady_numbers = [number for number in numbers_as_strings if is_harshady(number)]
But, since we're getting functional well use filter
harshady_numbers = filter(is_harshady, numbers_as_strings)
That's about it. Apply the same functional thinking to complete the is_harshad function and you're done.
map() can take more than one iterable argument and it returns an iterator not a list.
I am using python in operator with for loop and with if statement. My question is how is in implemented, that it behaves differently in this two cases: it iterates when using for loop and it checks if some element exists when using with if statement? Does this depend on implementation of for and if?
for i in x:
#iterates
if i in x:
#checks weather i in x or not
Membership testing with in is implemented via the method __contains__ (see Documentation). The different behaviour comes from the keyword before, for and if in your case.
Iteration with for is implemented such, that the method next is called and its return value is written to the iteration variable as long as the condition after the key word for is true. Membership testing in general is just a condition.
Code
A in B
Execution
B.__contains__(A) # returns boolean
Code
for A in B :
# Body
Execution
A = B.next()
if B.__contains__(A) :
# Body
A = B.next()
if B.__contains__(A) :
# Body
# ...
A = B.next()
if B.__contains__(A) :
# B.next() throws "StopIteration" if there is no more element
The keyword "in" in python solves different purposes based on "for" and "if". please look at this related link in stack overflow for more clarity
In many languages you'll find keywords that have multiple uses. This is simply an example of that. It's probably more helpful to think in terms of statements than thinking about the in keyword like an operator.
The statement x in y is a boolean-valued statement taking (assuming y is some appropriate collection) True if and only if the value of x is in the collection y. It is implemented with the __contains__ member function of y.
The statement for x in y: starts a loop, where each iteration x takes a different value from the collection y. This is implemented using the __iter__ member function of y and __next__ on the resulting iterator object.
There are other statements where the in keyword can appear, such as list comprehension or generator comprehension.
The reason is that for...in is something different from just in.
for x in y iterates over y.
if x in y calls y.__contains__(x).
The in keyword is an operator usually:
print(2 in [1, 2, 3]) # True
if 3 in range(7, 20):
print('3 in range!')
It corresponds to the object.__contains__ special method. The expression a in b corresponds to type(b).__contains__(a). Note that both a and b are names that are looked up.
In a for statement, in is not an operator. It is part of the for .. in .. syntax and separates the loop variable name from the iterable.
for thing in range(20):
print(thing) # thing is something else on each step
Note that for a in b only b is a name that is looked up. a is a name to bind to, similar to an assignment statement.
Python syntax has several constructs where the leading keyword defines the meaning of following keywords. For example, the as keyword has a different meaning in import and with:
# as aliases ExitStack
from contextlib import ExitStack as ContextStack
# as holds the result of ContextStack().__enter__()
with ContextStack() as stack:
...
It helps to think about such keywords not by implementation but by meaning. For example, a in b always means that "a is contained by b".
given the following code:
numbers= [1,2,3,1,2,4,5]
if 5 in numbers:
.................
we can notice we have a list(numbers) with 7 items and I want to know if the keyword in behind the scene do a loop for find a match in the list
It depends. It will call __contains__() on the container class (right side) - that can be implemented as a loop, and for some classes it can be calculated by some other faster method if possible.
You can even define it on your own class, like this ilustrative example:
class ContainsEverything:
def __contains__(self, item):
return True
c = ContainsEverything()
>>> None in c
True
>>> 4 in c
True
For container types in general, this is documented under __contains__ in the data model chapter and Membership test operations in the expressions chapter.
When you write this:
x in s
… what Python does is effectively (slightly oversimplified):
try:
return s.__contains__(x)
except AttributeError:
for value in s:
if value == x:
return True
return False
So, any type that defines a __contains__ method can do whatever it wants; any type that doesn't, Python automatically loops over it as an iterable. (Which in turn calls s.__iter__ if present, the old-style sequence API with s.__getitem__ if not.)
For the builtin sequence types list and tuple, the behavior is defined under Sequence Types — list, tuple, range and (again) Membership test operations:
True if an item of s is equal to x, else False
… which is exactly the same as the fallback behavior.
Semantically:
For container types such as list, tuple, set, frozenset, dict, or collections.deque, the expression x in y is equivalent to any(x is e or x == e for e in y).
Fpr list and tuple, this is in fact implemented by looping over all of the elements, and it's hard to imagine how it could be implemented otherwise. (In CPython, it's slightly faster because it can do the loop directly over the underlying array, instead of using an iterator, but it's still linear time.)
However, some other builtin types do something smarter. For example:
range does the closed-form arithmetic to check whether the value is in the range in constant time.
set, frozenset, and dict use the underlying hash table to look up the value in constant time.
There are some third-party types, like sorted-dict collections based on trees or skiplists or similar, that can't search in constant time but can search in logarithmic time by walking the tree. So, they'll (hopefully) implement __contains__ in logarithmic time.
Also note that if you use the ABC/mixin helpers in collections.abc to define your own Sequence or Mapping type, you get a __contains__ implementation for free. For sequences, this works by iterating over all the elements; for mappings, it works by try: self.[key].
While I find the negative number wraparound (i.e. A[-2] indexing the second-to-last element) extremely useful in many cases, when it happens inside a slice it is usually more of an annoyance than a helpful feature, and I often wish for a way to disable that particular behaviour.
Here is a canned 2D example below, but I have had the same peeve a few times with other data structures and in other numbers of dimensions.
import numpy as np
A = np.random.randint(0, 2, (5, 10))
def foo(i, j, r=2):
'''sum of neighbours within r steps of A[i,j]'''
return A[i-r:i+r+1, j-r:j+r+1].sum()
In the slice above I would rather that any negative number to the slice would be treated the same as None is, rather than wrapping to the other end of the array.
Because of the wrapping, the otherwise nice implementation above gives incorrect results at boundary conditions and requires some sort of patch like:
def ugly_foo(i, j, r=2):
def thing(n):
return None if n < 0 else n
return A[thing(i-r):i+r+1, thing(j-r):j+r+1].sum()
I have also tried zero-padding the array or list, but it is still inelegant (requires adjusting the lookup locations indices accordingly) and inefficient (requires copying the array).
Am I missing some standard trick or elegant solution for slicing like this? I noticed that python and numpy already handle the case where you specify too large a number nicely - that is, if the index is greater than the shape of the array it behaves the same as if it were None.
My guess is that you would have to create your own subclass wrapper around the desired objects and re-implement __getitem__() to convert negative keys to None, and then call the superclass __getitem__
Note, what I am suggesting is to subclass existing custom classes, but NOT builtins like list or dict. This is simply to make a utility around another class, not to confuse the normal expected operations of a list type. It would be something you would want to use within a certain context for a period of time until your operations are complete. It is best to avoid making a globally different change that will confuse users of your code.
Datamodel
object.getitem(self, key)
Called to implement evaluation of
self[key]. For sequence types, the accepted keys should be integers
and slice objects. Note that the special interpretation of negative
indexes (if the class wishes to emulate a sequence type) is up to the
getitem() method. If key is of an inappropriate type, TypeError may be raised; if of a value outside the set of indexes for the
sequence (after any special interpretation of negative values),
IndexError should be raised. For mapping types, if key is missing (not
in the container), KeyError should be raised.
You could even create a wrapper that simply takes an instance as an arg, and just defers all __getitem__() calls to that private member, while converting the key, for cases where you can't or don't want to subclass a type, and instead just want a utility wrapper for any sequence object.
Quick example of the latter suggestion:
class NoWrap(object):
def __init__(self, obj, default=None):
self._obj = obj
self._default = default
def __getitem__(self, key):
if isinstance(key, int):
if key < 0:
return self._default
return self._obj.__getitem__(key)
In [12]: x = range(-10,10)
In [13]: x_wrapped = NoWrap(x)
In [14]: print x_wrapped[5]
-5
In [15]: print x_wrapped[-1]
None
In [16]: x_wrapped = NoWrap(x, 'FOO')
In [17]: print x_wrapped[-1]
FOO
While you could subclass e.g. list as suggested by jdi, Python's slicing behaviour is not something anyone's going to expect you to muck about with.
Changing it is likely to lead to some serious head-scratching by other people working with your code when it doesn't behave as expected - and it could take a while before they go looking at the special methods of your subclass to see what's actually going on.
See: Action at a distance
I think this isn't ugly enough to justify new classes and wrapping things.
Then again it's your code.
def foo(i, j, r=2):
'''sum of neighbours within r steps of A[i,j]'''
return A[i-r:abs(i+r+1), j-r:abs(j+r+1)].sum() # ugly, but works?
(Downvoting is fun, so I've added some more options)
I found out something quite unexpected (for me): The __getslice__(i,j) does not wrap! Instead, negative indices are just ignored, so:
lst[1:3] == lst.__getslice__(1,3)
lst[-3:-1] == 2 next to last items but lst.__getslice__(-3,-1) == []
and finally:
lst[-2:1] == [], but lst.__getslice__(-2,1) == lst[0:1]
Surprising, interesting, and completely useless.
If this only needs to apply in a few specific operations, a simple & straightworward if index>=0: do_something(array[i]) / if index<0: raise IndexError would do.
If this needs to apply wider, it's still the same logic, just being wrapped in this manner or another.
I am getting new_tag from a form text field with self.response.get("new_tag") and selected_tags from checkbox fields with
self.response.get_all("selected_tags")
I combine them like this:
tag_string = new_tag
new_tag_list = f1.striplist(tag_string.split(",") + selected_tags)
(f1.striplist is a function that strips white spaces inside the strings in the list.)
But in the case that tag_list is empty (no new tags are entered) but there are some selected_tags, new_tag_list contains an empty string " ".
For example, from logging.info:
new_tag
selected_tags[u'Hello', u'Cool', u'Glam']
new_tag_list[u'', u'Hello', u'Cool', u'Glam']
How do I get rid of the empty string?
If there is an empty string in the list:
>>> s = [u'', u'Hello', u'Cool', u'Glam']
>>> i = s.index("")
>>> del s[i]
>>> s
[u'Hello', u'Cool', u'Glam']
But if there is no empty string:
>>> s = [u'Hello', u'Cool', u'Glam']
>>> if s.index(""):
i = s.index("")
del s[i]
else:
print "new_tag_list has no empty string"
But this gives:
Traceback (most recent call last):
File "<pyshell#30>", line 1, in <module>
if new_tag_list.index(""):
ValueError: list.index(x): x not in list
Why does this happen, and how do I work around it?
1) Almost-English style:
Test for presence using the in operator, then apply the remove method.
if thing in some_list: some_list.remove(thing)
The removemethod will remove only the first occurrence of thing, in order to remove all occurrences you can use while instead of if.
while thing in some_list: some_list.remove(thing)
Simple enough, probably my choice.for small lists (can't resist one-liners)
2) Duck-typed, EAFP style:
This shoot-first-ask-questions-last attitude is common in Python. Instead of testing in advance if the object is suitable, just carry out the operation and catch relevant Exceptions:
try:
some_list.remove(thing)
except ValueError:
pass # or scream: thing not in some_list!
except AttributeError:
call_security("some_list not quacking like a list!")
Off course the second except clause in the example above is not only of questionable humor but totally unnecessary (the point was to illustrate duck-typing for people not familiar with the concept).
If you expect multiple occurrences of thing:
while True:
try:
some_list.remove(thing)
except ValueError:
break
a little verbose for this specific use case, but very idiomatic in Python.
this performs better than #1
PEP 463 proposed a shorter syntax for try/except simple usage that would be handy here, but it was not approved.
However, with contextlib's suppress() contextmanager (introduced in python 3.4) the above code can be simplified to this:
with suppress(ValueError, AttributeError):
some_list.remove(thing)
Again, if you expect multiple occurrences of thing:
with suppress(ValueError):
while True:
some_list.remove(thing)
3) Functional style:
Around 1993, Python got lambda, reduce(), filter() and map(), courtesy of a Lisp hacker who missed them and submitted working patches*. You can use filter to remove elements from the list:
is_not_thing = lambda x: x is not thing
cleaned_list = filter(is_not_thing, some_list)
There is a shortcut that may be useful for your case: if you want to filter out empty items (in fact items where bool(item) == False, like None, zero, empty strings or other empty collections), you can pass None as the first argument:
cleaned_list = filter(None, some_list)
[update]: in Python 2.x, filter(function, iterable) used to be equivalent to [item for item in iterable if function(item)] (or [item for item in iterable if item] if the first argument is None); in Python 3.x, it is now equivalent to (item for item in iterable if function(item)). The subtle difference is that filter used to return a list, now it works like a generator expression - this is OK if you are only iterating over the cleaned list and discarding it, but if you really need a list, you have to enclose the filter() call with the list() constructor.
*These Lispy flavored constructs are considered a little alien in Python. Around 2005, Guido was even talking about dropping filter - along with companions map and reduce (they are not gone yet but reduce was moved into the functools module, which is worth a look if you like high order functions).
4) Mathematical style:
List comprehensions became the preferred style for list manipulation in Python since introduced in version 2.0 by PEP 202. The rationale behind it is that List comprehensions provide a more concise way to create lists in situations where map() and filter() and/or nested loops would currently be used.
cleaned_list = [ x for x in some_list if x is not thing ]
Generator expressions were introduced in version 2.4 by PEP 289. A generator expression is better for situations where you don't really need (or want) to have a full list created in memory - like when you just want to iterate over the elements one at a time. If you are only iterating over the list, you can think of a generator expression as a lazy evaluated list comprehension:
for item in (x for x in some_list if x is not thing):
do_your_thing_with(item)
See this Python history blog post by GvR.
This syntax is inspired by the set-builder notation in math.
Python 3 has also set and dict comprehensions.
Notes
you may want to use the inequality operator != instead of is not (the difference is important)
for critics of methods implying a list copy: contrary to popular belief, generator expressions are not always more efficient than list comprehensions - please profile before complaining
try:
s.remove("")
except ValueError:
print "new_tag_list has no empty string"
Note that this will only remove one instance of the empty string from your list (as your code would have, too). Can your list contain more than one?
As a one liner:
>>> s = [u'', u'Hello', u'Cool', u'Glam']
>>> s.remove('') if '' in s else None # Does nothing if '' not in s
>>> s
['Hello', 'Cool', 'Glam']
>>>
If index doesn't find the searched string, it throws the ValueError you're seeing. Either
catch the ValueError:
try:
i = s.index("")
del s[i]
except ValueError:
print "new_tag_list has no empty string"
or use find, which returns -1 in that case.
i = s.find("")
if i >= 0:
del s[i]
else:
print "new_tag_list has no empty string"
Adding this answer for completeness, though it's only usable under certain conditions.
If you have very large lists, removing from the end of the list avoids CPython internals having to memmove, for situations where you can re-order the list. It gives a performance gain to remove from the end of the list, since it won't need to memmove every item after the one your removing - back one step (1).
For one-off removals the performance difference may be acceptable, but if you have a large list and need to remove many items - you will likely notice a performance hit.
Although admittedly, in these cases, doing a full list search is likely to be a performance bottleneck too, unless items are mostly at the front of the list.
This method can be used for more efficient removal,as long as re-ordering the list is acceptable. (2)
def remove_unordered(ls, item):
i = ls.index(item)
ls[-1], ls[i] = ls[i], ls[-1]
ls.pop()
You may want to avoid raising an error when the item isn't in the list.
def remove_unordered_test(ls, item):
try:
i = ls.index(item)
except ValueError:
return False
ls[-1], ls[i] = ls[i], ls[-1]
ls.pop()
return True
While I tested this with CPython, its quite likely most/all other Python implementations use an array to store lists internally. So unless they use a sophisticated data structure designed for efficient list re-sizing, they likely have the same performance characteristic.
A simple way to test this, compare the speed difference from removing from the front of the list with removing the last element:
python -m timeit 'a = [0] * 100000' 'while a: a.remove(0)'
With:
python -m timeit 'a = [0] * 100000' 'while a: a.pop()'
(gives an order of magnitude speed difference where the second example is faster with CPython and PyPy).
In this case you might consider using a set, especially if the list isn't meant to store duplicates.In practice though you may need to store mutable data which can't be added to a set. Also check on btree's if the data can be ordered.
Eek, don't do anything that complicated : )
Just filter() your tags. bool() returns False for empty strings, so instead of
new_tag_list = f1.striplist(tag_string.split(",") + selected_tags)
you should write
new_tag_list = filter(bool, f1.striplist(tag_string.split(",") + selected_tags))
or better yet, put this logic inside striplist() so that it doesn't return empty strings in the first place.
Here's another one-liner approach to throw out there:
next((some_list.pop(i) for i, l in enumerate(some_list) if l == thing), None)
It doesn't create a list copy, doesn't make multiple passes through the list, doesn't require additional exception handling, and returns the matched object or None if there isn't a match. Only issue is that it makes for a long statement.
In general, when looking for a one-liner solution that doesn't throw exceptions, next() is the way to go, since it's one of the few Python functions that supports a default argument.
1-use the filter option
new_tag_list = [u'', u'Hello', u'Cool', u'Glam']
new_tag_list= list(filter(None, new_tag_list))
2-list comprehension works as well for elements other than None
new_tag_list = [u'', u'Hello', u'Cool', u'Glam']
[element for element in new_tag_list if element not in['']]