We Keep Coding

Lodash for Python?

Is there a library or something similar to lodash, for Python? We use the library extensively on our API and while we move on to creating a series of Python workers, it would make sense to create a similar structure to our API syntactics.

pydash is exactly like lodash, only for Python.

pydash is something which you can use as a replacement of lodash in python. Almost everything is covered under it, the names of functions are also similar just keeping python's conventions as is.
Important: I've been using pydash for around 07 months, It helped me a lot. One thing which you should not forget is, avoid using pydash functions directly on self, its behaviour is very erratic and gives random results. I'm showing one example below.
Sample Class
class MyClass:
def get(self):
self.obj = {}
self.obj['a'] = True
Usage
class TestClass:
def init(self):
self.inst = MyClass()
def test_function(self):
# **Not Recommended Method**
# _.get(self, "inst.a") # shows random behavior
# **Highly recommended**
_.get(self.inst, "a") # Will always return True

Well I'm not sure if this is exactly what you're looking for but when I think of javascript libraries like underscore and Lodash, I think of libraries that add more functional programming functions(though I do believe both underscore and lodash have a little more utility than that) to a language.
There are a bunch of Python Libraries that try and add some of the same functionality. A quick search got me pytoolz https://github.com/pytoolz/toolz which I don't have much experience with but looks interesting. If that isn't what you're looking for, try searching for other functional programming python libraries until you find one you like.
Hope that helped

Recipe for anonymous functions in python?

I'm looking for the best recipie to allow inline definition of functions, or multi-line lambda, in python.
For example, I'd like to do the following:
def callfunc(func):
func("Hello")
>>> callfunc(define('x', '''
... print x, "World!"
... '''))
Hello World!
I've found an example for the define function in this answer:
def define(arglist, body):
g = {}
exec("def anonfunc({0}):\n{1}".format(
arglist,
"\n".join(" {0}".format(line) for line in body.splitlines())), g)
return g["anonfunc"]
This is one possible solution, but it is not ideal. Desireable features would be:
be smarter about indentation,
hide the innards better (e.g. don't have anonfunc in the function's scope)
provide access to variables in the surrounding scope / captures
better error handling
and some things I haven't thought of. I had a really nice implementation once that did most of the above, but I lost in unfortunately. I'm wondering if someone else has made something similar.
Disclaimer:
I'm well aware this is controversial among Python users, and regarded as a hack or unpythonic. I'm also aware of the discussions regaring multi-line-lambdas on the python-dev mailing list, and that a similar feature was omitted on purpose. However, from the same discussions I've learned that there is also interest in such a function by many others.
I'm not asking whether this is a good idea or not, but instead: Given that one has decided to implement this, (either out of fun and curiosity, madness, genuinely thinking this is a nice idea, or being held at gunpoint) how to make anonymous define work as close as possible to def using python's (2.7 or 3.x) current facilities?
Examples:
A bit more as to why, this can be really handy for callbacks in GUIs:
# gtk example:
self.ntimes = 0
button.connect('clicked', define('*a', '''
self.ntimes += 1
label.set_text("Button has been clicked %d times" % self.ntimes)
''')
The benefit over defining a function with def is that your code is in a more logical order. This is simplified code taken from a Twisted application:
# twisted example:
def sayHello(self):
d = self.callRemote(HelloCommand)
def handle_response(response):
# do something, this happens after (x)!
pass
d.addCallback(handle_response) # (x)
Note how it seems out of order. I usually break stuff like this up, to keep the code order == execution order:
def sayHello_d(self):
d = self.callRemote(HelloCommand)
d.addCallback(self._sayHello_2)
return d
def _sayHello_2(self, response):
# handle response
pass
This is better wrt. ordering but more verbose. Now, with the anonymous functions trick:
d = self.callRemote(HelloCommand)
d.addCallback(define('response', '''
print "callback"
print "got response from", response["name"]
'''))

If you come from a javascript or ruby background, python's abilities to deal with anonymous functions may indeed seem limited, but this is for a reason. Python designers decided that clarity of code is more important than conciseness. If you don't like that, you probably don't like python at all. There's nothing wrong about that, there are many other choices - why not to try a language that tastes better to you?
Putting chunks of code into strings and interpreting them on the fly is definitely a wrong way to "extend" a language, just because none of tools you're working with - from syntax highlighters to the python interpreter itself - would be able to deal with "stringified" code in a sensible way.
To answer the question as asked: what you're doing there is essentially an attempt to construct some better-than-python programming language and compile it to python on the fly. The idea is not new in the world of scripting languages and can be productive or not (CoffeeScript is an example of a successful implementation), but your very approach is wrong. format() not the tool you're looking for when working with code. If you're writing a compiler, do it properly: use a parser (e.g. pyparsing) to read your code in an AST, walk through the AST to generate python code (or even bytecode), catch syntax errors as you go and take measures to provide better runtime feedback (e.g. error context, line numbers etc). Finally, make sure your compiler works across different python versions and implementations.
Or just use ruby.

Ruby's tap idiom in Python

There is a useful Ruby idiom that uses tap which allows you to create an object, do some operations on it and return it (I use a list here only as an example, my real code is more involved):
def foo
[].tap do |a|
b = 1 + 2
# ... and some more processing, maybe some logging, etc.
a << b
end
end
>> foo
=> [1]
With Rails there's a similar method called returning, so you can write:
def foo
returning([]) do |a|
b = 1 + 2
# ... and some more processing, maybe some logging, etc.
a << b
end
end
which speaks for itself. No matter how much processing you do on the object, it's still clear that it's the return value of the function.
In Python I have to write:
def foo():
a = []
b = 1 + 2
# ... and some more processing, maybe some logging, etc.
a.append(b)
return a
and I wonder if there is a way to port this Ruby idiom into Python. My first thought was to use with statement, but return with is not valid syntax.

Short answer: Ruby encourages method chaining, Python doesn't.
I guess the right question is: What is Ruby's tap useful for?
Now I don't know a lot about Ruby, but by googling I got the impression that tap is conceptually useful as method chaining.
In Ruby, the style: SomeObject.doThis().doThat().andAnotherThing() is quite idiomatic. It underlies the concept of fluent interfaces, for example. Ruby's tap is a special case of this where instead of having SomeObject.doThis() you define doThis on the fly.
Why I am explaining all this? Because it tells us why tap doesn't have good support in Python. With due caveats, Python doesn't do call chaining.
For example, Python list methods generally return None rather than returning the mutated list. Functions like map and filter are not list methods. On the other hand, many Ruby array methods do return the modified array.
Other than certain cases like some ORMs, Python code doesn't use fluent interfaces.
In the end it is the difference between idiomatic Ruby and idiomatic Python. If you are going from one language to the other you need to adjust.

You can implement it in Python as follows:
def tap(x, f):
f(x)
return x
Usage:
>>> tap([], lambda x: x.append(1))
[1]
However it won't be so much use in Python 2.x as it is in Ruby because lambda functions in Python are quite restrictive. For example you can't inline a call to print because it is a keyword, so you can't use it for inline debugging code. You can do this in Python 3.x although it isn't as clean as the Ruby syntax.
>>> tap(2, lambda x: print(x)) + 3
2
5

If you want this bad enough, you can create a context manager
class Tap(object):
def __enter__(self, obj):
return obj
def __exit__(*args):
pass
which you can use like:
def foo():
with Tap([]) as a:
a.append(1)
return a
There's no getting around the return statement and with really doesn't do anything here. But you do have Tap right at the start which clues you into what the function is about I suppose. It is better than using lambdas because you aren't limited to expressions and can have pretty much whatever you want in the with statement.
Overall, I would say that if you want tap that bad, then stick with ruby and if you need to program in python, use python to write python and not ruby. When I get around to learning ruby, I intend to write ruby ;)

I had an idea to achieve this using function decorators, but due to the distinction in python between expressions and statements, this ended up still requiring the return to be at the end.
The ruby syntax is rarely used in my experience, and is far less readable than the explicit python approach. If python had implicit returns or a way to wrap multiple statements up into a single expression then this would be doable - but it has neither of those things by design.
Here's my - somewhat pointless - decorator approach, for reference:
class Tapper(object):
def __init__(self, initial):
self.initial = initial
def __call__(self, func):
func(self.initial)
return self.initial
def tap(initial):
return Tapper(initial)
if __name__ == "__main__":
def tapping_example():
#tap([])
def tapping(t):
t.append(1)
t.append(2)
return tapping
print repr(tapping_example())

I partly agree with others in that it doesn't make much sense to implement this in Python. However, IMHO, Mark Byers's way is the way, but why lambdas(and all that comes with them)? can't you write a separate function to be called when needed?
Another way to do basically the same could be
map(afunction(), avariable)
but this beautiful feature is not a built-in in Python 3, I hear.

Hardly any Ruby programmers use tap in this way. In fact, all top Ruby programmers i know say tap has no use except in debugging.
Why not just do this in your code?
[].push(1)
and remember Array supports a fluent interface, so you can even do this:
[].push(1).push(2)

How to mock a free function in python?

I have a python program with a global function that is painful to test (it needs a large dataset to work properly). What is the best way to get around this while testing functions that call it?
I've found that the following works (but it make me feel dirty to use it).
module foo:
def PainLiesHere():
return 4; #guaranteed to be random
module test
import foo
def BlissLiesHere():
return 5
foo.PainLiesHere = BlissLiesHere
# test stuff

This is a perfectly fine way to do it. As long as you know that BlissLiesHere does not change the overall behavior of the unit you are testing...
EDIT:
This is what is being done, under all the nice extras they provide, by different kinds of mocking libraries, such as Mock, Mox, etc.

What are the important language features (idioms) of Python to learn early on [duplicate]

This question already has answers here:
The Zen of Python [closed]
(22 answers)
Python: Am I missing something? [closed]
(16 answers)
Closed 8 years ago.
I would be interested in knowing what the StackOverflow community thinks are the important language features (idioms) of Python. Features that would define a programmer as Pythonic.
Python (pythonic) idiom - "code expression" that is natural or characteristic to the language Python.
Plus, Which idioms should all Python programmers learn early on?
Thanks in advance
Related:
Code Like a Pythonista: Idiomatic Python
Python: Am I missing something?

Python is a language that can be described as:
"rules you can fit in the
palm of your hand with a huge bag of
hooks".
Nearly everything in python follows the same simple standards. Everything is accessible, changeable, and tweakable. There are very few language level elements.
Take for example, the len(data) builtin function. len(data) works by simply checking for a data.__len__() method, and then calls it and returns the value. That way, len() can work on any object that implements a __len__() method.
Start by learning about the types and basic syntax:
Dynamic Strongly Typed Languages
bool, int, float, string, list, tuple, dict, set
statements, indenting, "everything is an object"
basic function definitions
Then move on to learning about how python works:
imports and modules (really simple)
the python path (sys.path)
the dir() function
__builtins__
Once you have an understanding of how to fit pieces together, go back and cover some of the more advanced language features:
iterators
overrides like __len__ (there are tons of these)
list comprehensions and generators
classes and objects (again, really simple once you know a couple rules)
python inheritance rules
And once you have a comfort level with these items (with a focus on what makes them pythonic), look at more specific items:
Threading in python (note the Global Interpreter Lock)
context managers
database access
file IO
sockets
etc...
And never forget The Zen of Python (by Tim Peters)
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

This page covers all the major python idioms: http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html

An important idiom in Python is docstrings.
Every object has a __doc__ attribute that can be used to get help on that object. You can set the __doc__ attribute on modules, classes, methods, and functions like this:
# this is m.py
""" module docstring """
class c:
"""class docstring"""
def m(self):
"""method docstring"""
pass
def f(a):
"""function f docstring"""
return
Now, when you type help(m), help(m.f) etc. it will print the docstring as a help message.
Because it's just part of normal object introspection this can be used by documention generating systems like epydoc or used for testing purposes by unittest.
It can also be put to more unconventional (i.e. non-idiomatic) uses such as grammars in Dparser.
Where it gets even more interesting to me is that, even though doc is a read-only attribute on most objects, you can use them anywhere like this:
x = 5
""" pseudo docstring for x """
and documentation tools like epydoc can pick them up and format them properly (as opposed to a normal comment which stays inside the code formatting.

Decorators get my vote. Where else can you write something like:
def trace(num_args=0):
def wrapper(func):
def new_f(*a,**k):
print_args = ''
if num_args > 0:
print_args = str.join(',', [str(x) for x in a[0:num_args]])
print('entering %s(%s)' %(f.__name__,print_args))
rc = f(*a,**k)
if rc is not None:
print('exiting %s(%s)=%s' %(f.__name__,str(rc)))
else:
print('exiting %s(%s)' %(f.__name__))
return rc
return new_f
return wrapper
#trace(1)
def factorial(n):
if n < 2:
return 1
return n * factorial(n-1)
factorial(5)
and get output like:
entering factorial(5)
entering factorial(4)
entering factorial(3)
entering factorial(2)
entering factorial(1)
entering factorial(0)
exiting factorial(0)=1
exiting factorial(1)=1
exiting factorial(2)=2
exiting factorial(3)=6
exiting factorial(4)=24
exiting factorial(5)=120

Everything connected to list usage.
Comprehensions, generators, etc.

Personally, I really like Python syntax defining code blocks by using indentation, and not by the words "BEGIN" and "END" (as in Microsoft's Basic and Visual Basic - I don't like these) or by using left- and right-braces (as in C, C++, Java, Perl - I like these).
This really surprised me because, although indentation has always been very important to me, I didn't make to much "noise" about it - I lived with it, and it is considered a skill to be able to read other peoples, "spaghetti" code. Furthermore, I never heard another programmer suggest making indentation a part of a language. Until Python! I only wish I had realized this idea first.
To me, it is as if Python's syntax forces you to write good, readable code.
Okay, I'll get off my soap-box. ;-)

From a more advanced viewpoint, understanding how dictionaries are used internally by Python. Classes, functions, modules, references are all just properties on a dictionary. Once this is understood it's easy to understand how to monkey patch and use the powerful __gettattr__, __setattr__, and __call__ methods.

Here's one that can help. What's the difference between:
[ foo(x) for x in range(0, 5) ][0]
and
( foo(x) for x in range(0, 5) ).next()
answer:
in the second example, foo is called only once. This may be important if foo has a side effect, or if the iterable being used to construct the list is large.

Two things that struck me as especially Pythonic were dynamic typing and the various flavors of lists used in Python, particularly tuples.
Python's list obsession could be said to be LISP-y, but it's got its own unique flavor. A line like:
return HandEvaluator.StraightFlush, (PokerCard.longFaces[index + 4],
PokerCard.longSuits[flushSuit]), []
or even
return False, False, False
just looks like Python and nothing else. (Technically, you'd see the latter in Lua as well, but Lua is pretty Pythonic in general.)

Using string substitutions:
name = "Joe"
age = 12
print "My name is %s, I am %s" % (name, age)
When I'm not programming in python, that simple use is what I miss most.

Another thing you cannot start early enough is probably testing. Here especially doctests are a great way of testing your code by explaining it at the same time.
doctests are simple text file containing an interactive interpreter session plus text like this:
Let's instantiate our class::
>>> a=Something(text="yes")
>>> a.text
yes
Now call this method and check the results::
>>> a.canify()
>>> a.text
yes, I can
If e.g. a.text returns something different the test will fail.
doctests can be inside docstrings or standalone textfiles and are executed by using the doctests module. Of course the more known unit tests are also available.

I think that tutorials online and books only talk about doing things, not doing things in the best way. Along with the python syntax i think that speed in some cases is important.
Python provides a way to benchmark functions, actually two!!
One way is to use the profile module, like so:
import profile
def foo(x, y, z):
return x**y % z # Just an example.
profile.run('foo(5, 6, 3)')
Another way to do this is to use the timeit module, like this:
import timeit
def foo(x, y, z):
return x**y % z # Can also be 'pow(x, y, z)' which is way faster.
timeit.timeit('foo(5, 6, 3)', 'from __main__ import *', number = 100)
# timeit.timeit(testcode, setupcode, number = number_of_iterations)