We Keep Coding

Best practice for the order of function arguments in Python [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
I often write functions that take one argument with data to be manipulated, and one or more additional arguments with specifications for the manipulation. If the "specification" parameters are optional, it makes sense to place them after the obligatory data argument:
sort(data, key=..., reverse=True)
But suppose both arguments (or several of them) are obligatory? The functions of the re module place the regexp (the manipulation) before the string (the data to be manipulated). Optional arguments still come last, of course.
re.search(r"[regexp]+", text, flags=re.I)
So here's the question: Putting optional arguments aside, are there any clear conventions (official PEP or established common practice) for how to order obligatory arguments based on their function/purpose? Back when I was first learning Python, I remember reading some claim to the effect that one of Python's advantages is that it has clear conventions, inter alia on this particular thing. But I am unable to retrieve any such information now.
In case it is not clear: I am kindly asking for pointers to established conventions or standards, not for advice on which order is "best."

This is one hell of a good question and I'm already excited to see other answers. I've never come across any conventions about functions arguments in Python except in PEP 8 where it states that:
Always use self for the first argument to instance methods.
Always use cls for the first argument to class methods.
If a function argument's name clashes with a reserved keyword, it is
generally better to append a single trailing underscore rather than
use an abbreviation or spelling corruption. Thus class_ is better than
clss. (Perhaps better is to avoid such clashes by using a synonym.)
In my opinion, functions parameter should always be in order of importance. I know importance is subjective but you generally can determine which of the parameters is the most important to your function. Since StackOverflow is not about opinion take this with a grain of salt.
So putting aside opinions, I think your question is relevant but sometimes questions leads to other discussions. If you have numerous parameters, your function is probably doing too much stuff. According to a lot of ressources, you should try to use only one parameter and avoid using more than three where the latter is considered as the worst case scenario. So, most of the time ordering parameters won't impact readability that much if you keep that in mind.

I try to order parameters to help code readability (literate programming).
ex:
system.add(user, user_database) # add a user to a database
system.add(user_database, user) # add a database to a user(?)
This usually coincide with #scharette's opinion, the "most important" parameter is the object for which the function was created for. But as (he) stated it's subjective, it could be argued the database is the most important object in a database system library.
Reading code should reveal intent on first read; reading a set of function definitions should reveal what they do.

Python design mistakes [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
A while ago, when I was learning Javascript, I studied Javascript: the good parts, and I particularly enjoyed the chapters on the bad and the ugly parts. Of course, I did not agree with everything, as summing up the design defects of a programming language is to a certain extent subjective - although, for instance, I guess everyone would agree that the keyword with was a mistake in Javascript. Nevertheless, I find it useful to read such reviews: even if one does not agree, there is a lot to learn.
Is there a blog entry or some book describing design mistakes for Python? For instance I guess some people would count the lack of tail call optimization a mistake; there may be other issues (or non-issues) which are worth learning about.

You asked for a link or other source, but there really isn't one. The information is spread over many different places. What really constitutes a design mistake, and do you count just syntactic and semantic issues in the language definition, or do you include pragmatic things like platform and standard library issues and specific implementation issues? You could say that Python's dynamism is a design mistake from a performance perspective, because it makes it hard to make a straightforward efficient implementation, and it makes it hard (I didn't say completely impossible) to make an IDE with code completion, refactoring, and other nice things. At the same time, you could argue for the pros of dynamic languages.
Maybe one approach to start thinking about this is to look at the language changes from Python 2.x to 3.x. Some people would of course argue that print being a function is inconvenient, while others think it's an improvement. Overall, there are not that many changes, and most of them are quite small and subtle. For example, map() and filter() return iterators instead of lists, range() behaves like xrange() used to, and dict methods like dict.keys() return views instead of lists. Then there are some changes related to integers, and one of the big changes is binary/string data handling. It's now text and data, and text is always Unicode. There are several syntactic changes, but they are more about consistency than revamping the whole language.
From this perspective, it appears that Python has been pretty well designed on the language (syntax and sematics) level since at least 2.x. You can always argue about indentation-based block syntax, but we all know that doesn't lead anywhere... ;-)
Another approach is to look at what alternative Python implementations are trying to address. Most of them address performance in some way, some address platform issues, and some add or make changes to the language itself to more efficiently solve certain kinds of tasks. Unladen swallow wants to make Python significantly faster by optimizing the runtime byte-compilation and execution stages. Stackless adds functionality for efficient, heavily threaded applications by adding constructs like microthreads and tasklets, channels to allow bidirectional tasklet communication, scheduling to run tasklets cooperatively or preemptively, and serialisation to suspend and resume tasklet execution. Jython allows using Python on the Java platform and IronPython on the .Net platform. Cython is a Python dialect which allows calling C functions and declaring C types, allowing the compiler to generate efficient C code from Cython code. Shed Skin brings implicit static typing into Python and generates C++ for standalone programs or extension modules. PyPy implements Python in a subset of Python, and changes some implementation details like adding garbage collection instead of reference counting. The purpose is to allow Python language and implementation development to become more efficient due to the higher-level language. Py V8 bridges Python and JavaScript through the V8 JavaScript engine – you could say it's solving a platform issue. Psyco is a special kind of JIT that dynamically generates special versions of the running code for the data that is currently being handled, which can give speedups for your Python code without having to write optimised C modules.
Of these, something can be said about the current state of Python by looking at PEP-3146 which outlines how Unladen Swallow would be merged into CPython. This PEP is accepted and is thus the Python developers' judgement of what is the most feasible direction to take at the moment. Note it addresses performance, not the language per se.
So really I would say that Python's main design problems are in the performance domain – but these are basically the same challenges that any dynamic language has to face, and the Python family of languages and implementations are trying to address the issues. As for outright design mistakes like the ones listed in Javascript: the good parts, I think the meaning of "mistake" needs to be more explicitly defined, but you may want to check out the following for thoughts and opinions:
FLOSS Weekly 11: Guido van Rossum (podcast August 4th, 2006)
The History of Python blog

Is there a blog entry or some book describing design mistakes for Python?
Yes.
It's called the Py3K list of backwards-incompatible changes.
Start here: http://docs.python.org/release/3.0.1/whatsnew/3.0.html
Read all the Python 3.x release notes for additional details on the mistakes in Python 2.

My biggest peeve with Python - and one which was not really addressed in the move to 3.x - is the lack of proper naming conventions in the standard library.
Why, for example, does the datetime module contain a class itself called datetime? (To say nothing of why we have separate datetime and time modules, but also a datetime.time class!) Why is datetime.datetime in lower case, but decimal.Decimal is upper case? And please, tell me why we have that terrible mess under the xml namespace: xml.sax, but xml.etree.ElementTree - what is going on there?

Try these links:
http://c2.com/cgi/wiki?PythonLanguage
http://c2.com/cgi/wiki?PythonProblems

Things that frequently surprise inexperienced developers are candidate mistakes. Here is one, default arguments:
http://www.deadlybloodyserious.com/2008/05/default-argument-blunders/

A personal language peeve of mine is name binding for lambdas / local functions:
fns = []
for i in range(10):
fns.append(lambda: i)
for fn in fns:
print(fn()) # !!! always 9 - not what I'd naively expect
IMO, I'd much prefer looking up the names referenced in a lambda at declaration time. I understand the reasons for why it works the way it does, but still...
You currently have to work around it by binding i into a new name whos value doesn't change, using a function closure.

This is more of a minor problem with the language, rather than a fundamental mistake, but: Property overriding. If you override a property (using getters and setters), there is no easy way of getting the parent class' property.

Yeah, it's strange but I guess that's what you get for having mutable variables.
I think the reason is that the "i" refers to a box which has a mutable value and the "for" loop will change that value over time, so reading the box value later gets you the only value there is left.
I don't know how one would fix that short of making it a functional programming language without mutable variables (at least without unchecked mutable variables).
The workaround I use is creating a new variable with a default value (default values being evaluated at DEFINITION time in Python, which is annoying at other times) which causes copying of the value to the new box:
fns = []
for i in range(10):
fns.append(lambda j=i: j)
for fn in fns:
print(fn()) # works

I find it surprising that nobody mentioned the global interpreter lock.

One of the things I find most annoying in Python is using writelines() and readlines() on a file. readlines() not only returns a list of lines, but it also still has the \n characters at the end of each line, so you have to always end up doing something like this to strip them:
lines = [l.replace("\n", "").replace("\r", "") for l in f.readlines()]
And when you want to use writelines() to write lines to a file, you have to add \n at the end of every line in the list before you write them, sort of like this:
f.writelines([l + "\n" for l in lines])
writelines() and readlines() should take care of endline characters in an OS independent way, so you don't have to deal with it yourself.
You should just be able to go:
lines = f.readlines()
and it should return a list of lines, without \n or \r characters at the end of the lines.
Likewise, you should just be able to go:
f.writelines(lines)
To write a list of lines to a file, and it should use the operating systems preferred enline characters when writing the file, you shouldn't need to do this yourself to the list first.

My biggest dislike is range(), because it doesn't do what you'd expect, e.g.:
>>> for i in range(1,10): print i,
1 2 3 4 5 6 7 8 9
A naive user coming from another language would expect 10 to be printed as well.

You asked for liks; I have written a document on that topic some time ago: http://segfaulthunter.github.com/articles/biggestsurprise/

I think there's a lot of weird stuff in python in the way they handle builtins/constants. Like the following:
True = "hello"
False = "hello"
print True == False
That prints True...
def sorted(x):
print "Haha, pwned"
sorted([4, 3, 2, 1])
Lolwut? sorted is a builtin global function. The worst example in practice is list, which people tend to use as a convenient name for a local variable and end up clobbering the global builtin.

Is monkeypatching stdlib methods a good practice in Python? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
Over time I found the need to override several stdlib methods from Python in order to overcome limitation or to add some missing functionality.
In all cases I added a wrapper function and replaced the original method from the module with my wrapper (the wrapper was calling the original method).
Why I did this? Just to be sure that all the calls to the method are using my new versions, even if these are called from other third-party modules.
I know that monkeypatching can be a bad thing but my question is if this is useful if you use it with care? Meaning that:
you still call the original methods, assuring that you are not missing anything when the original module is updated
you are not changing the original "meaning" of the methods
Examples:
add coloring support to python logging module.
make open() be able to recognize Unicode BOM masks when using text mode
adding logging support to os.system() or subprocess.Popen() - letting you output to console or/and redirect to another file.
implementing methods that are missing on your platform like os.chown() or os.lchown() that are missing on Windows.
Doing things like these appear to me as decent overrides but I would like to see how others are seeing them and specially what should be considered as an acceptable monkeypatch and what not.

None of these things seem to require monkeypatching. All of them seem to have better, more robust and reliable solutions.
Adding a logging handler is easy. No monkeypatch.
Fixing open is done this way.
from io import open
That was easy. No patch.
Logging to os.system()? I'd think that a simple "wrapper" function would be far better than a complex patch. Further, I'd use subprocess.Popen, since that's the recommended replacement.
Adding missing methods to mask OS differences (like os.chown()) seems like a better use for try/except. But that's just me. I like explicit rather than implicit.
On balance, I still can't see a good reason for monkeypatching.
I'd hate to be locked in to legacy code (like os.system) because I was too dependent on my monkeypatches.
The concept of "subclass" applies to modules as well as classes. You can easily write your own modules which (a) import and (b) extend existing modules. You then use your new modules because they provided extra features. You don't need to monkeypatch.
even if these are called from other third-party modules
Dreadful idea. You can easily break another module by altering built-in features. If you have read the other module and are sure the monkeypatches won't break then what you've found is this.
The "other" module should have had room for customization. It should have had a place for a "dependency injection" or Strategy design pattern. Good thinking.
Once you've found this, the "other" module can be fixed to allow this customization. It may be as simple as a documentation change explaining how to modify an object. It may be an additional
parameter for construction to insert your customization.
You can then provide the revised module to the authors to see if they'll support your small update to their module. Many classes can use extra help supporting a "dependency injection" or Strategy design for extensions.
If you have not read the other module and are not sure your monkeypatches work... well... we still have hope that the monkeypatches don't break anything.

Monkeypatching can be "the least of evils", sometimes -- mostly, when you need to test code which uses a subsystem that is not well designed for testability (doesn't support dependency injection &c). In those cases you will be monkeypatching (very temporarily, fortunately) in your test harness, and almost invariably monkeypatching with mocks or fakes for the purpose of isolating tests (i.e., making them unit tests, rather than integration tests).
This "bad but could be worse" use case does not appear to apply to your examples -- they can all be better architected by editing the application level code to call your appropriate wrapper functions (say myos.chown rather than the bare os.chown, for example) and putting your wrapper functions in your own intermediate modules (such as myown) that stand between the application level code and the standard library (or third-party extensions that you are thus wrapping -- there's nothing special about the standard library in this respect).
One problematic situation might arise when the "application level code" isn't really under your control -- it's a third party subsystem that you'd rather not modify. Nevertheless, I have found that in such situations modifying the third party subsystem to call wrappers (rather than the standard library functions directly) is way more productive in the long run -- then of course you submit the change to the maintainers of the third party subsystem in question, they roll your change into their subsystem's next release, and life gets better for everybody (you included, since once your changes are accepted they'll get routinely maintained and tested by others!-).
(As a side note, such wrappers may also be worth submitting as diffs to the standard library, but that is a different case since the standard library evolves very very slowly and cautiously, and in particular on the Python 2 line will never evolve any longer, since 2.7 is the last of that line and it's feature-frozen).
Of course, all of this presupposes an open-source culture. If for some mysterious reasons you're using a closed-source third party subsystem, therefore one which you cannot possibly maintain, then you are in another situation where monkey patching may be the lesser evil (but that's just because the evil of losing strategic control of your development by trusting in code you can't possibly maintain is such a bigger evil in itself;-). I've never found myself in this situation with a third-party package that was both closed-source and itself written in Python (if the latter condition doesn't hold your monkeypatches would do you no good;-).
Note that here the working definition of "closed-source" is really very strict: for example, even Microsoft 12+ years ago distributed sources of libraries such as MFC with Visual C++ (as their product was then called) -- closed-source because you couldn't redistribute their sources, but still, you DID have sources at hand, so when you met some terrible limitation or bug you COULD fix it (and submit the change to them for a future release, as well as publishing your change as a diff as long as it included absolutely none of their copyrighted code -- not trivial, but feasible).
Monkeypatching well beyond the strict confines within which such an approach is "the least of evil" is a frequent mistake of users of dynamic languages -- be careful not to fall into that trap yourself!

If my python class name has an acronym, should I keep it upper case, or only the first letter? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
I have a class named SSLXMLRPCServer. Should it be that or SslXmlRpcServer?

This is a matter of personal preference, but I find the second format much easier to read. The fact that your first format has a typo in it (PRC instead or RPC) suggests that I am not the only one.

It should be SSLXMLRPCServer, to match the standard library classes like SimpleXMLRPCServer, CGIXMLRPCRequestHandler, etc.
Adopting a naming convention that differs from equivalents in the standard library is only going to confuse people.

The problem with uppercase acronyms in CamelCase names is that the word following the acronym looks like a part of it, since it begins with a capital letter. Also, when you have several in a row as in your example, it is not clear where each begins. For this reason, I would probably use your second choice.

The PEP-8 mentions nothing about acronyms. You would be safest to keep the acronyms uppercased (it's what I see most).

I normally uppercase acronyms. Twisted and a few other libraries do this as well.

As stated already, PEP-8 says to use upper-case for acronym. Now, python zen also says "readability counts" (and for me the zen has priority over the PEP :-).
My opinion in such unclear situation is to take into account the standard in the programming context, not just the language. For example, some xml-http-query class should be written XMLHttpQuery in a servlet context (w.r.t XMLHttpRequest).
I don't know your context, but it seems XMLRPCServer exists and you want to attach ssl to it. So you could choose something like:
SSL_XMLRPCServer
It would emphasized the XMLRPCServer -without changing it-.
Also, you'd stay close to PEP-8 and follow the zen :-)
My 2 cents
Note: if XMLRPCServer is not strongly related to your class and is really a standard in the domain, then you need to choose another name, as to not be confusing.

I had this problemlots of time . I uppercase Acronym but I doesn't like it because when you chain them (as in your example) it doesn't feel right. However I think the best things to do is to make a choice and stick to hit, so at least don't you know when you need to reference something how it's written without having to check (which is one of the benefit of coding standard)

How about SSL_XML_RPC_Server for acronymity and readability?
It's what I often do when I want to avoid camel-case for some reason.

Uses for Dynamic Languages [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 months ago.
Improve this question
My primary language right now is D, and I'm in the process of learning Python because it's required for a course I'm taking. While I understand why dynamic languages would be a breath of fresh air for people programming in static languages without type inference or templates (IMHO templates are to a large extent compile-time duck typing), I'm curious what the benefits are of dynamic languages even when you have those.
The bottom line is that, if I'm going to learn Python, I want to learn it in a way that really changes my thinking about programming, rather than just writing D in Python. I have not used dynamic languages since I was a fairly novice programmer and unable to appreciate the flexibility they supposedly offer, and want to learn to take full advantage of them now. What can be done easily/elegantly in a dynamically typed, interpreted language that's awkward or impossible in a static language, even with templates, polymorphism, static type inference, and maybe runtime reflection?

In theory, there's nothing that dynamic languages can do and static languages can't. Smart people put a lot of work into making very good dynamic languages, leading to a perception at the moment that dynamic languages are ahead while static ones need to catch up.
In time, this will swing the other way. Already various static languages have:
Generics, which make static types less stupid by letting it select the right type when objects are passed around, saving the programmer from having to cast it themselves
Type inference, which saves having to waste time on writing the stuff that should be obvious
Closures, which among many other things help to separate mechanism from intention, letting you pull together complicated algorithms from mostly existing ingredients.
Implicit conversions, which lets you simulate "monkey patching" without the risks it usually involves.
Code loading and easy programmatic access to the compiler, so users and third parties can script your program. Use with caution!
Syntaxes that are more conducive to the creation of Domain Specific Languages within them.
...and no doubt more to come. The dynamic movement has spawned some interesting developments in static language design, and we all benefit from the competition. I only hope more of these features make it to the mainstream.
There's one place where I don't see the dominant dynamic language being replaced, and that's Javascript in the browser. There's just too much of an existing market to replace, so the emphasis seems to be towards making Javascript itself better instead.

Here's Steve Yegge on the subject.
Guido van Rossum also linked to that talk in his take of Scala.

"I'm curious what the benefits are of
dynamic languages even when you have
those."
Compared to D programming language:
Python is a more compact language. It allows you to express as much as D but it uses many fewer different concepts to achieve it -- less is more.
Python has a powerful standard library -- batteries included.
I don't know whether D has interactive prompts but in Python an interactive shell such as ipython is an integrated part of development process.

Example in Python:
def lengths(sequence):
try:
return sum(len(item) for item in sequence)
except TypeError:
return "Wolf among the sheep!"
>>> lengths(["a", "b", "c", (1, 2, 3)])
6
>>> lengths( ("1", "2", 3) )
'Wolf among the sheep!'
How long do you think this took me to write, and how many compile-run-debug cycles?
If you think my example is trivial, I can reply by saying that dynamic languages make trivial many programming tasks.

In dynamic languages you can use values in ways that you know are correct. In a statically typed language you can only use values in ways the compiler knows are correct. You need all of the things you mentioned to regain flexibility that's taken away by the type system (I'm not bashing static type systems, the flexibility is often taken away for good reasons). This is a lot of complexity that you don't have to deal with in a dynamic language if you want to use values in ways the language designer didn't anticipate (for example, putting values of different types in a hash table).
So it's not that you can't do these things in a statically typed language (if you have runtime reflection), it's just more complicated.

I actually wrote a blog post on this: linky. But that post basically can be summed up like this:
You'd be surprised at how much of a load off your mind it is to not have to name at compile time what type your variable is. Thus, python tends to be a very productive language.
On the other hand, even with good unit tests, you'd also be surprised at what kinds of stupid mistakes you're allowing yourself to make.

One big advantage of dynamic typing when using objects is that you don't need to use class hierarchies anymore when you want several classes to have the same interface - that's more or less what is called duck typing. Bad inheritance is very difficult to fix afterwards - this makes refactoring often harder than it is in a language like python.

The point is that in a dynamic language you can implement the same functionality much quicker than in a statically typed one. Therefore the productivity is typically much higher.
Things like templates or polymorphism in principle give you lots of flexibility, but you have to write a large amount of code to make it work. In a dynamic language this flexibility almost comes for free.
So I think you look at the difference in the wrong way, productivity really is the main point here (just like garbage collection improves productivity, but otherwise does not really allow you to do new things).

With a dynamic language it's much easier to have a command line interpreter so you can test things on the command line and don't have to worry about a compile step to see if they work.

I find dynamic languages like Perl and to a lesser extent Python allow me to write quick and dirty scripts for things I need to do. The run cycle is much shorter in dynamic languages and often less code needs to be written then in a statically typed language which increases my productivity. This unfortunately comes at the cost of maintainability but that is a fault of the way I write programs in dynamic languages not in the languages them selves.

I was going to say closures but found this thread... (not that I understand how it would work in a "static" language)
Related concepts are functions-as-first-class-objects and higher-order procedures. (e.g. a function that takes a function as input and/or returns a function as output)
edit: (for the nitpickers here) I'll echo a comment I made on #David Locke's post. Dynamically-interpreted languages make it possible to use an existing software program/project in conjunction with a small function or class created at the spur-of-the-moment to explore something interactively. Probably the best example is function graphing. If I wrote a function-graphing object with a graph(f,xmin,xmax) function, I could use it to explore functions like x2 or sin(x) or whatever. I do this in MATLAB all the time; it's interpreted and has anonymous functions (#(x) x^2) that can be constructed at the interpreter prompt to pass into higher-order functions (graphing functions, derivative operators, root finders, etc).

Take a look at this e4x example in JavaScript:
var sales = <sales vendor="John">
<item type="peas" price="4" quantity="6"/>
<item type="carrot" price="3" quantity="10"/>
<item type="chips" price="5" quantity="3"/>
</sales>;
alert( sales.item.(#type == "carrot").#quantity );
alert( sales.#vendor );
for each( var price in sales..#price ) {
alert( price );
}
Especially, take a look at line:
alert( sales.item.(#type == "carrot").#quantity );
In typical static languages, you don’t get to write sales.item, since you can not know that item is property of sales until runtime.
This is not limited to e4x. You get to program in similar style when connecting when writing SOAP clients or any other underlying type you do not know until runtime.
In a static language, you would typically need to run a tool that will generate stub classes or program in a very verbose way. Then, if something changes in a web service, you need to regenerate stubs all over again. Take a look at java DOM code:
import org.dom4j.Document;
import org.dom4j.DocumentHelper;
import org.dom4j.Element;
public class Foo {
public Document createDocument() {
Document document = DocumentHelper.createDocument();
Element root = document.addElement( "root" );
Element author1 = root.addElement( "author" )
.addAttribute( "name", "James" )
.addAttribute( "location", "UK" )
.addText( "James Strachan" );
Element author2 = root.addElement( "author" )
.addAttribute( "name", "Bob" )
.addAttribute( "location", "US" )
.addText( "Bob McWhirter" );
return document;
}
}
Definitely much more verbose than your dynamic code. And, of course, it is not statically typed. There is no way to check that you misspelled “author” as "autor" until runtime. All this verbosity is essentially there to let you capture something that is dynamic in nature in static style.
I think this is one of the strong points of dynamic languages.

Compiled languages tend to be used when efficiency and type safety are the priorities. Otherwise I can't think of any reason why anyone wouldn't be using ruby :)