Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
I write tons of python scripts, and I find myself reusing lots code that I've written for other projects. My solution has been to make sure the code is separated into logical modules/packages (this one's a given). I then make them setuptools-aware and publish them on PyPI. This allows my other scripts to always have the most up-to-date code, I get a warm fuzzy feeling because I'm not repeating myself, and my development, in general, is made less complicated. I also feel good that there MAY be someone out there that finds my code handy for something they're working on, but it's mainly for selfish reasons :)
To all the pythonistas, how do you handle this? Do you use PyPI or setuptools (easy_install)? or something else?
I have been doing the same thing. Extract common functionality, pretty the code up with extra documentation and unit tests/ doctests, create an easy_install setup.py, and then release on PyPi. Recently, I created a single Google Code site where I manage the source and keep the wiki up to date.
What kind of modules are we talking about here? If you're planning on distributing your projects to other python developers, setuptools is great. But it's usually not a very good way to distribute apps to end users. Your best bet in the latter case is to tailor your packaging to the platforms you're distributing it for. Sure, it's a pain, but it makes life for end users far easier.
For example, in my Debian system, I usually don't use easy_install because it is a little bit more difficult to get eggs to work well with the package manager. In OS X and windows, you'd probably want to package everything up using py2app and py2exe respectively. This makes life for the end user better. After all, they shouldn't know or care what language your scripts are written in. They just need them to install.
I store it all offline in a logical directory structure, with commonly used modules grouped as utilities. This means it's easier to control which versions I publish, and manage. I also automate the build process to interpret the logical directory structure.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
What is the reason packages are distributed separately?
Why do we have separate 'add-on' packages like pandas, numpy?
Since these modules seem so important, why are these not part of Python itself?
Are the "single distributions" of Python to come pre-loaded?
If it's part of design to keep the 'core' separate from additional functionality, still in that case it should at least come 'pre-imported' as soon as you start Python.
Where can I find such distributions if they exist?
Many of these tools, including core Python, are separately developed and distributed by different team, so it is up to aggregators to curate them and put them into a single distribution. Here are some notable examples:
The Anaconda distribution from Continuum Analytics
The Canopy distribution from Enthought
ActivePython from ActiveState
PythonXY for scientific programming
WinPython for Windows
PyIMSL Studio from Rogue Wave Software
This is a bit like asking "Why doesn't every motor come with a car around it?"
While a car without a motor is pretty useless, the inverse doesn't hold: Most motors aren't even used for cars. Of course one could try selling a complete car to people who want to have a generator, but they wouldn't buy it.
Also the people designing cars might not be the best to build a motor and vice versa.
Similarly with python. Most python distributions are not used with numpy, scipy or pandas. Distributing python with those packages would create a massive overhead.
However, there is of course a strong demand for prebuilt distributions which combine those modules with a respective python and make sure everything interacts smoothly. Some examples are Anaconda, Canopy, python(x,y), winpython, etc. So an end user who simply wants a car that runs, best chooses one of those, instead of installing everything from scratch. Other users who do want to always have the newest version of everything might choose to tinker them together themselves.
You can make the interactive interpreted launch with "pre-imported" modules, as well as with pre-run code, using The Interactive start-up file.
Alternatively, you can use The Customization modules to pre-run code on every invocation of python.
Regarding whether pandas and numpy should be part of the standard library - it's a matter of opinion.
PyPi currently has over 100,000 libraries available. I'm sure someone thinks each of these is important.
Why do you need or want to pre-load libraries, considering how easy a pip install is especially in a virtual environment?
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
I'm working on a large-ish project. I currently have some functional tests hacked together in shell scripts. They work, but I'd like to make them a little bit more complicated. I think it will be painful to do in bash, but easy in a full-blown scripting language. The target language of the project is not appropriate for implementing the tests.
I'm the only person who will be working on my branch for the foreseeable future, so it isn't a huge deal if my tests don't work for others. But I'd like to avoid committing code that's going to be useless to people working on this project in the future.
In terms of test harness "just working" for the largest number of current and future contributors to this project as possible, am I better off porting my shell scripts to Python or Perl?
I suspect Perl, since I know that the default version of Python installed (if any) varies widely across OSes, but I'm not sure if that's just because I'm less familiar with Perl.
Every modern Linux or Unixlike system comes with both installed; which you choose is a matter of taste. Just pick one.
I would say that whatever you choose, you should try to write easily-readable code in it, even if you're the only one who will be reading that code. Now, Pythonists will tell you that writing readable code is easier in Python, which may be true, but it's certainly doable in Perl as well.
This is more personal, but I always used python and will use it till something shows me it's better to use other. Its simple, very expansible, strong and fast and supported by a lot of users.
The version of python doesn't matter much since you can update it and most OSes have python 2.5 with expands the compatibility a lot. Perl is also included in Linux and Mac OS, though.
I think that Python will be good, but if you like perl and have always worked with it, just use perl and don't complicate your life.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
I've started working on a rather big (multithreaded) Python project, with loads of (unit)tests. The most important problem there is that running the application requires a preset environment, which is implemented by a context manager. So far we made use of a patched version of the unit test runner that would run the tests inside this manager, but that doesn't allow switching context between different test modules.
Both nose and pytest do support such a thing because they support fixtures at many granularities, so we're looking into switching to nose or pytest. Both these libraries would also support 'tagging' tests and run only these tagged subsets, which is something we also would like to do.
I have been looking through the documentation of both nose and pytest a bit, and as far as I can see the bigger part of those libraries essentially support the same functionality, except that it may be named differently, or require slightly different syntax. Also, I noted some small differences in the available plugins (nose has multiprocess-support, pytest doesn't seem to for instance)
So it seems, the devil is in the detail, which means (often at least) in personal taste and we better go with the library that fits our personal taste best.
So I'd to ask for a subjective argumentation why I should be going with nose or pytest in order to choose the library/community combo that best fits our needs.
I used to use Nose because it was the default with Pylons. I didn't like it at all. It had configuration tendrils in multiple places, virtually everything seemed to be done with an underdocumented plugin which made it all even more indirect and confusing, and because it did unittest tests by default, it regularly broke with Unicode tracebacks, hiding the sources of errors.
I've been pretty happy with py.test the last couple years. Being able to just write a test with assert out of the box makes me hate writing tests way less, and hacking whatever I need atop the core has been pretty easy. Rather than a fixed plugin interface it just has piles of hooks, and pretty understandable source code should you need to dig further. I even wrote an adapter for running Testify tests under py.test, and had more trouble with Testify than with py.test.
That said, I hear nose has plugins for classless tests and assert introspection nowadays, so you'll probably do fine with either. I still feel like I can hit the ground running with py.test, though, and I can understand what's going on when it breaks.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I'm a designer who writes mostly Sass, Less (CSS pre-processors), HTML, Javascript and usually starts off with static site generators such as Jekyll, Yeoman, etc. While working with developers who code in Python, Ruby, Clojure, I help with the templates. In my free time, I design wordpress themes and write plugins in PHP. I run grunt regularly and bower helps me with components that I need for my designs.
This means my system is littered with Ruby Gems, Python libraries, Node Modules. They are either installed via gem installations, pip, brew or npm. Now you realize that my system is a mess even though it works. I really want to do stuffs in a sane manner, the right way.
So, what are the best practices for installation and management of all the libraries, core tools, etc. for a developer on Mac OS X. Point me to resources that I can read, ponder and practice.
Here is the scenario. You're a seasoned developer and I'm your friend who just got a new Mac OS X system. I'm a designer who will work with Python (mostly with Django), Ruby (with Rails), Clojure, PHP, Sass, Less, Compass, CoffeeScript, Git, NodeJS, Grunt, Bower, Jekyll, Yeoman and alike. As a friend, you know that I'm not a 'programmer' but a developer-friendly 'designer'. How can you help me setup my Mac? And I don't want to come back again when I get a new Mac in future, I should be able to just transition smoothly from my old setup.
Thanking in anticipation.
Github open sourced there developer environment setup tools. You could try that out. http://boxen.github.com/
For python I recommend using virtualenv to setup libraries instead of installing them globally. https://pypi.python.org/pypi/virtualenv
I am not sure what you meant by "How can you help me setup my Mac?". It seems that you are very much comfortable installing all the dependencies(gems and all) for your projects. If you want to automate all these environment installation setup then you may go ahead and write a generic shell script to install ruby, python and other stuff and reuse when you have a new machine :) and it has nothing to do with Mac OSX or any other OS. You just need to put correct package/version to fetch and install/compile accordingly in the script.
Would be great if you can put a specific question here in case you are facing technical problem installing any of the above packages.
If all that you are worried about is quickly setting up a new machine, use a backup software to setup the new machine. You can also try to use a custom time machine setup with just the folders that you are interested it.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
Currently I'm using a Windows batch file to build my software. It does things like running MSBuild, copying files, creating a ZIP file, running some tests, including the subversion revision number, etc.
But the problem is, batch files are evil. So I would like to change to something better. I was planning to recreate my build script in Python. Is that a smart choice? What about all those build systems, like Ant, SCons, Maven, Rake, etc. Would using any of those be a better choice?
Note: I'm not planning to replace my Visual Studio solution/project files. I only want to script everything else that's required to create a release of my software.
Edit: I have good reasons to move away from batch, that's not what my question is about. I would like to know (for example) what SCons gives me, over a normal Python script.
For a tool that is scripted with Python, I happen to think Paver is a more easily-managed and more flexible build automator than SCons. Unlike SCons, Paver is designed for the plethora of not-compiling-programs tasks that go along with managing and distributing a software project.
Batch files aren't evil - they've actually come quite a long way from the brain-dead days of command.com. The command language can be pretty expressive nowadays, it just requires a bit of effort on your part to learn it.
Unless there's an actual problem with your build script that you can't fix (and, if that's the case, that's the question you should be asking rather than some wishy-washy "What's the best replacement?" :-), my approach would be to stick with what you've got.
A vague feeling of evilness would not be reason for me to waste effort 'fixing' something that isn't broken. And it would be wasted effort unless there's a clear advantage to changing ("less evil" is not something I'd consider a clear advantage).
As you're mentioning Python and SCons, I'd say go for SCons. It is Python after all. And yes, any of the above would be a better choice than hand-rolled build scripts.
I've seen python scripts used for building releases elsewhere so it can't be bad. Actually, I've personally used perl scripts to automate release building. I guess any scripting language could easily automate that procedure. If it's gonna be easy to do (and probably better than batch scripts), why not try it?
I would suggest using NAnt for your build script instead of python.
My reasons for this are:
It has the tasks defined already, all you need to do is write the XML and point it to the right places. If you are working with people who do not know python, XML may be a little less scary than learning a new language.
NAnt is designed to work in the windows .Net environment, so it can already do MSBuild and NUnit tasks.
If you are already writing in C#, if you need to extend NAnt to do new tasks you are not adding another language to the mix of your project.
You can hook into Cruise Control .Net (for continuous builds). Which I think is the main reason why you would use NAnt.
Why should you use python? If your build script isn't broke don't fix it. If your having issues updating it to deal with new aditions to the project then you may want to look at rewriting it. I wouldn't use Python though tools like NANT or MSBuild do the job. I don't see the point in using a general purpis programming language to do something that tools have already been written to do unless you have a lot of obscure requirements existing tools can't deal with. Second what happens if you get hit by a bus or win the lotto? If you are determined to script everything I'd use powershell or some other Microsoft specific technology since your already wedded to Microsoft. If you leave will there be enough Python programmers to maintain the build scripts?
I would strongly suggest to take a look at waf. It's kind of what you want: "a Python-based framework for configuring, compiling and installing applications"
Personally I would use scripting as a last resort given that
With a bit of work you can get MSBuild to do all those things for you by extending it with additional components
There are third party equivalents to MSBuild like NANT that can do the same thing
There are entire tools like FinalBuilder that also do the same thing, and are easier to configure and extend
However, if I had to go the scripting route I would use Powershell for a couple of reasons:
Complete access to file system
You can easily access .NET objects
You can easily access COM objects
You can create custom makefiles for Microsoft nmake tool which you already have installed. Using a tool like that (SCons, Maven, etc. fall into the same category) gives you much more than regular scripts.
The main benefit is that dependencies between files are tracked and also the timestamps of changes. For example, you can make your .zip file depend on some other files, so .zip only gets repacked if some of those files have changed in the meantime. Just like with source code and its compiled form.
Python is very portable. SCons is field tested and reliable. Given what you know (from what you explained), why even ask the question?
If your maintaining something, its not just about getting it to build, its also about explaining to the user why it can NOT build, which saves you a ton of very frustrating questions while helping users to help themselves.
I can not think of a modern, production operating system that lacks Python, unless you get into the embedded / research arena.
So, I'm answering to say, you answered your own question :)
It depends on what technology your software uses. If you're building C++ programs, I'd probably say go for scons without question (unless you have weird requirements scons can't meet). On the other hand, consider the instructions for building C#: CSharpBuilder.
I would like to know (for example) what SCons gives me, over a normal Python script.
Think of scons as being more of a library than a program. It provides you with code that will prevent a lot of tedium that you will have to deal with without it. In my opinion, vanilla Python isn't the best option for any kind of shell scripting stuff (not that it can't do it).
But the problem is, batch files are evil.
Lastly, batch files are evil if they're used for a project they're not suited to handle. For the one or two file project, batch files do just fine.
It does things like running MSBuild, copying files, creating a ZIP file, running some tests, including the subversion revision number, etc.
MSBuild and PowerShell can easily do all of this with reasonably clean succinct code. You're then sticking to purely M$ products which managers tend to like. Otherwise I would suggest you could look into Rake if not only for its large community. It has a nice syntax and iron ruby support (irake).
To be honest all but the last task you have mentioned are easily done in MSBuild alone. I would suggest learning the tools you have before going elsewhere.
Check out http://msbuildtasks.tigris.org/ for some good add ons to MSBuild