We Keep Coding

clojure working with scipy and numpy

Is there any good way to call python from clojure as a means of doing data science with scipy, numpy, scikit-learn, etc.
I know about implementations of clojure which run on python instead of java, but this doeesn't work for me, as I also need to call java libraries in my project. I also know about Jython, but I don't know of a clean way to use this with Clojure.
I want to use Clojure in my projects because I prefer it as a language, but I can't deny that Python has an incredible community, and some of the most beautiful, well-designed libraries around.

Instead of trying to get Jython to play well with both Clojure and numpy/scipy, you can use Hy. It is hosted on Python and it somewhat resembles Clojure.
If I really wanted to use numpy/scipy, I would write the backend in Python (or Hy), run it as a separate service. And if I really like ring for instance, or can't live without Instaparse, I would write a frontend in Clojure.
As an aside Python has EDN libs. It would be an interesting project to integrate one of them in Hy, or write one from scratch.

Give the toolz library a try, it's a functional standard library for Python that was designed to generally adhere to the API of the Clojure standard library.
Apart from that, I'd encourage you to find the seams between your computations, and write individual tools in the Unix way in either Clojure or Python depending on which seems to fit that use case best. Serialize data between the tools, either as text/JSON through pipes or using a binary serialization format like Protobuf, which has standard APIs for both Java and Python.
If you had a gun to my head and told me to build Clojure/Python interop, I'd start with py4j and bridge the two languages through Java interfaces, using implements members in a Python class and reify on the Clojure side.

You could use Graal VM now. Although some large companies are using it in production, it's still early days. Here's an example of using Python from Clojure:
(.eval context "python" "
import time;
time.clock()
")
http://gigasquidsoftware.com/blog/2017/10/22/embedded-interop-between-clojure-r-and-python-with-graalvm/

Why does SciPy require an installation procedure?

I'm trying to wrap my head around the Python ecosystem and parts of it aren't making complete sense to me so far.
I'm coming from the Java world and when I want to make use of, say JUnit, I simply add the JUnit jar to my classpath and that's pretty much it. If I want to be nice to my users I can also easily bunch together all my dependencies into a single jar, so that all that they need to do is install a Java Runtime get a hold of my jar.
Reading through the SciPy installation guide I can't find an explanation for why all this is really necessary. And how is this ever going to work at deployment time? It's like JUnit asked me to install a new JRE just for them.

SciPy has parts written in C that require compilation for the specific platform it's being deployed too.
How can SciPy be fast if it is written in an interpreted language like Python?
Actually, the time-critical loops are usually implemented in C or
Fortran. Much of SciPy is a thin layer of code on top of the
scientific routines that are freely available at
http://www.netlib.org/. Netlib is a huge repository of incredibly
valuable and robust scientific algorithms written in C and Fortran. It
would be silly to rewrite these algorithms and would take years to
debug them. SciPy uses a variety of methods to generate “wrappers”
around these algorithms so that they can be used in Python. Some
wrappers were generated by hand coding them in C. The rest were
generated using either SWIG or f2py. Some of the newer contributions
to SciPy are either written entirely or wrapped with Cython.
Source: http://www.scipy.org/scipylib/faq.html#id12
On Linux, SciPy and NumPy libraries’ official releases are source-code
only. Installing NumPy and SciPy from source is reasonably easy;
However, both packages depend on other software, some of them which
can be challenging to install, or shipped with incompatibilities by
major Linux distributions. Hopefully, you can install NumPy and SciPy
without any software outside the necessary tools to build python
extensions, as most dependencies are optional
Source: http://www.scipy.org/scipylib/building/linux.html

Working OpenSceneGraph bindings for Python?

I'm building a rendering engine in Python for fun. I need to load 3D scenes. Any standard modern format like DAE, 3DS, or MAX would work: I can convert my files easily between standard formats.
OpenSceneGraph seems to be the most comprehensive and well-maintained solution. It would be ideal to be able to use it in Python without much hassle. Are there working Python bindings for OSG that are easy to install, work on Mac OS X (I'm on 10.8), and are compatible with the latest versions of OSG?
I searched around and came across osgswig (http://code.google.com/p/osgswig/) and PyOSG (http://sourceforge.net/projects/pyosg/), but they don't seem to be actively maintained. I don't see any recent activity related to these packages, and it seems that people had trouble running osgswig on OSX. Ideally, I'd like to find something that "just works", without major compilation hassles. I'd like to just install a package and be able to import a module that will let me load COLLADA or 3DS files.
I also came across pycollada (https://github.com/pycollada/pycollada). It seems active, but fairly early-stage. Ideally, I'd like a reasonably comprehensive package that supports specular maps, normal maps, and other reasonably advanced features. Animation would be nice as well.
In summary, I need to load 3D scenes in Python. Bindings for OSG would probably be ideal, because OSG is so comprehensive. But I need something that works on OSX. I would also prefer something that can be installed reasonably easily. Does something like this exist?
Thanks!

Take a look at Open Asset Import Library (short name: Assimp). It is a portable Open Source library to import various well-known 3D model formats in a uniform manner. http://www.assimp.org/

You should loot at panda3D (http://www.panda3d.org/), it's a game engine with extensive python bindings. It has the features you want : http://www.panda3d.org/manual/index.php/Features
I used it for a few years and it was a solid tool.

I made my own fork of a mirror of a clone of the osgswig project for a similar purpose. I have it working with OpenSceneGraph version 3.2.1 on Windows and Mac; and it's likely I will eventually polish it for linux too. I'm already delivering one product to customers based on my version of osgswig, and I'm considering making others. Find my fork here:
https://github.com/cmbruns/osgswig
If others show enough interest, I might be coaxed into creating binary installers for my version of the osgswig module, to make installation easier.
If you just want the easiest OpenSceneGraph bindings for OSG 3.2.1, you can stop reading this answer here. Read on for more of my thoughts for the future.
Though I am maintaining a fork of osgswig (as stated above), I sort of hate SWIG, and I would prefer to use bindings based on Boost.Python, rather than on SWIG. For large, complex C++ APIs, like OpenSceneGraph, Boost.Python can be much more elegant than SWIG, both for the API consumer, and for the binding maintainer (me, and me). I found one project using Boost.Python to wrap OSG, at https://code.google.com/p/osgboostpython/, but the developer is lovingly wrapping each part of the interface by hand, and has thus only completed a tiny fraction of the large OpenSceneGraph API.
Taking that Boost.Python based project as inspiration, I created yet another OpenSceneGraph Python binding project, at https://github.com/JaneliaSciComp/osgpyplusplus. Eventually, I want to use this osgpyplusplus project for all my python osg needs. And I would appreciate help in making it ready. Right now, osgpyplusplus suffers from the following weaknesses, compared to osgswig:
osgpyplusplus is not yet used in any working product
The build environment is tricky to set up, requiring both Boost.Python and Pyplusplus
I haven't paid much attention to osgpyplusplus recently, so it might rust away if I continue to ignore it.
Though osgpyplusplus probably wraps most of the OpenSceneGraph API, there are probably some important missing pieces that won't be identified until someone tries to develop a significant project with it.
It would be a lot of work for me to create a binary module installer for osgpyplusplus at this point, so please don't ask me to.

What's going on with python 3k? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 13 years ago.
Since I'm not strictly python developer please don't flame me just for the question.
I'm wondering about Python 3k, that from my point of view, might be some kind of misconception.
Or quite in-relevant step forward (I'm taking into account the 2.6 and 3k releases, which was almost one after another).
Before the flame will start, I'll explain my position at this topic and I'd assume couple facts from my work environment.
I work in a cutting edge market data solutions company,
we use mainly functional languages to high throughput data analysis.
But we also use python as a second technology for smaller tasks, scripts, process management & monitoring.
Some of my colleagues write more serious production applications based on python technologies,
but:
ALL of our customers use python 2.6,
because of above we have quite strong 2.6 toolset and internal/external support,
We still plan to develop 2.6 apps.
Every small tool development is also based on 2.6 platform.
Additional what I observe:
at this point any new linux distribution (in our infrastructure) has python 2.6 on the board,
most third party modules are developed for 2.6 version,
great part of resources in the network is also dedicated for 2.6
(I know that You can port most of this to 3.1.x, but it's too big overhead here.)
I know that Py 3k is still growing but,
there is already python 3.1.1, but no one "cares" (in my environment).
I've a strong feeling that Python 3k overheads are stopping it for moving this great technology into new dimension.

The OP appears to be surprised that a minor-release upgrade (which added some nifty features, basically broke zero existing code, and allowed trivial rebuilds of all existing third party libraries) happened overnight at their organization, while a major-release upgrade (requiring much more effort especially from the point of view of third-party library authors) didn't and won't. I think I mostly feel surprised at their surprise;-).
Even minor-release upgrades don't happen instantly in most large projects and organizations; for example, App Engine is still using Python 2.5 (apparently, upgrading its specialized "sandboxed" Python runtime and all it relies on is not a zero-effort proposition, so they prefer to keep putting their energy towards adding engine features instead) -- so I believe are implementations such as Jython and PyPy (I think IronPython's in the process of migrating to 2.6, but the current production version is still 2.5).
Totally new projects starting today should seriously consider starting with Python 3; for example, Allison Randal's pynie (Python for Parrot) project made exactly that choice (and, I think, it was the correct choice in their situation). Migrating existing projects is a harder proposition, and mostly depends on what third-party components the existing project depends on (if a new project intrinsically depends on some functionality that's only available in 3rd party libraries for 2.6, not 3.1, then the new project will probably also have to stick with the 2.6 version for the time being, of course).
Third party libraries that are under active development will probably come out with Python 3 version gradually (for example, gmpy did so relatively recently). Once enough such third party libraries are available, the chance that a missing library inhibits migrating an existing project (or, even more, starting a new project) using Python 3 starts going down pretty rapidly. This makes Python 3 ports feasible. At some point, some attractive functionality will become available in Python 3 only (for example, if and when pynie releases, that might be the case for a Parrot implementation affording smooth interoperability with Perl &c), and that will provide a strong motivation for some projects to go 3-only (pragmatically stronger than pure issues of language quality).
Even then, some sufficiently large projects and organizations will stick with Python 2 for a long time, and you can confidently expect that at some time a 2.7 will exist (possibly one or two more after that, but that's harder to predict). Hey, I sharply remember that throughout the '90s in most large projects and organizations "Fortran" still meant Fortran 77 (in fact in some places -- not many, 30+ years after than Fortran version's first release -- that's still true today!)... for all the advantages of Fortran 90 and later versions, migration costs (esp. in terms of various compilers, libraries, tools) were just perceived as being too high a price to pay for the advantages of the new language version. That's just inevitable when a language acquires a large installed base and a rich ecosystem of third party tools and libraries, as Python 2 now has. No reason for surprise!-)

Py3k is a good and necessary step in the evolution of the Python language. I like it much better than 2.x, and code in it almost exclusively. I'm fortunately not dependent on third-party libraries, few of which are there for Py3k yet. But they will be.
There is nothing wrong with using 2.6 - 2.7 will come and be even more of a bridge to 3.x, and you can start generating Py3k-like code already that will be trivial to port to 3.x once your favorite third-party library is there. I think I read somewhere that Py2k will be around for several more years.

What exactly is the question here? You observe that there's a newer version of the platform you currently use. You have good reason for staying with the version you have. This pattern is common across very many development organisations, you simply cannot chase every new version of the stuff yiou use. Nor can you stay where you are for ever, eventually you will need to migrate.
There are forces that may drive you to migrate, for example
The new version has some very major new feature you would really benefit from (eg. annotations in Java 5)
Support for a current platform (or dependent library) is being withdrawn
Meanwhile, you could attempt to future-proof your code so that migrating is easier. Are you familiar with the level of change required to move up?

Since Python 3K is not backwards compatible, you can expect that it will take more time to be accepted in corporate environments. Python has a huge code base and many important third libraries. One must wait for all dependencies to be converted to Python 3K before adopting it. This could be a real slow process.
From what I understand, the creators of Python expected this to happen, but they thought it will be worse not to make the changes and just let the language "die".

Python 3.x is not yet aimed at production use, but it will get there in time.
As a language and a runtime, Python 3.x is fine. The limitation is that a lot of important third party libraries and tools are still in the process of being ported and tested.
It is expected that the transition to Python 3.x will take up to five years.
So, if you are not library developer, you really don't need to care about Python 3.x for the time being.
Also, it is not expected of current Python users to do the switch to Python 3.x anytime soon.

There are a lot of libraries which depend on other libraries in the Python ecosystem. Nose is used to test numpy, and scipy requires numpy, and lots of scientific libraries require scipy. Also, none of the library developers can start porting until their dependencies are ported. It's going to take a long time for whole chain to port across.
A lot of the pragmatists are hoping that python 2.7 and 2.8 will gradually depreciate the bits that won't work in python 3 (introduce a print function called print_function, then depreciate the print statement, then rename print_function to print ...), so eventually python 2 will merge with python 3.

Here is a video there Guido van Rossum, at the Py4Science meeting, talks about Python 3.
http://fperez.org/py4science/2009_guido_ucb/index.html

What's the state-of-the-art in Python programming in Windows? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I'm looking to set up my development environment at home for writing Windows applications in Python.
For my first piece, I'm writing a simple, forms-based application that stores data input as XML (and can read that information back.) I do want to set up the tools I'd use professionally, though, having already done a round of didactic programming.
What tools are professional python developers using these days? In order to have a working python environment, what version of the compiler should I be using? What editor is common for professionals? What libraries are considered a must-have for every serious python developer?
Specifically, which Windowing and XML libraries are de rigeur for working in Windows?

I like Eclipse + PyDev (with extensions).
It is available on Windows, and it works very well well. However, there are many other IDEs, with strengths and weakness.
As for the interpreter (Python is interpreted, not compiled!), you have three main choices: CPython, IronPython and Jython.
When people say "Python" they usually refer to "CPython" that is the reference implementation, but the other two (based, respectively, on .Net and Java) are full pythons as well :-)
In your case, I would maybe go on IronPython, because it will allow you to leverage your knowledge of .Net to build the GUI and treating the XML, while leaving the implementation of business logic to Python.
Finally, should you decide to use CPython, finally, there are several choices for working with xml:
minidom; included in the standard library
lxml, faster and with a better API; it means an additional installation on top of Python.

Lots of questions, most hard to answer correctly. First of all, most of python development happens on unix-like platforms. You will hit many walls during development on Windows box.
Python is not a compiled lanugage, current preferred version for production is 2.5. For environement setup you should take a look at virtualenv. Editor is a personal choice, many Python developers use Vim, you can customize it pretty well to suite your needs.
About libraries, Python is very strong around this area and it's really hard to say what is a must to know. If you want to handle XML, I would preffer lxml.

If you go for CPython, make sure you get the win32 extensions by Mark Hammond, either as a separate download which you install on top of the vanilla Python installation, or as part of ActiveState's ActivePython. It includes an integrated editor and debugger.
Jython has recently reached 2.5 compliancy, but we quickly ran into recursion limit issues.
The standard distribution includes IDLE, a graphical editor and debugger.
I like shells, so I'm using IPython for interactive work, and pydb as debugger (unfortunately, I had problems getting pydb to work under Windows).

"What tools are professional python developers using these days?"
Lots
"In order to have a working python environment, what version of the compiler should I be using?"
["compiler" is meaningless. I'll assume you mean "Python"]
We use 2.5.4. We'll be upgrading to 2.6 as soon as we've done the testing.
"What editor is common for professionals?"
We use Komodo Edit.
"What libraries are considered a must-have for every serious python developer?"
We use Django, XLRD, PIL, and a few others. We don't plan this kind of thing in advance. As our requirements arrive, we start looking for libraries. We don't "pre-load" a bunch of "must-have" libraries. The very idea is silly. We load what we need to solve the problems we have.

There are no set standards in these matters, and for good reasons:
there is a fair amount of good choice
different people are productive with different tools
different tools and libraries are suited for solving different problems
That said, I think it's a valid question exactly because there is a fair amount of good choice. When there is too much choice people often do not chose at all and move on. You still need to do your own research to decide what is best for you but you may find here some good starting points.
Here is what I use professionally on windows:
python 2.5.4
latest wxPython
XRC Resource Editor from the wxPython docs & demos for the grunt of the tedious GUI design
lxml or gnosis utils for xml
WingIDE Professional

Taking the headline question literally, the answer has to be IronPython. The 2.0 releases are equivalent to CPython 2.5, and the 2.6 release (currently at beta2) is intended to match CPython 2.6 (full 2.6 release some time in the next couple of months). With either you can use the state of the art in Windows GUI frameworks, i.e. WPF; and you get the whole .net XML support libraries (excepting Linq to XML, which relies on clever bits of C# that IronPython cannot yet emulate).
I've used NetBeans Python plug-in happily as an IDE for IronPython using WPF.

The answer would depend on what you want to do with Python. If you want to do web programming, Python is blessed with many web frameworks. The most popular ones are: Django, Pylons, and Turbogears. There's also Google App Engine, where you can deploy your Python webapp (based on GAE framework) to Google's infrastructure. If you want to do Desktop programming then there is PyQT and TkInter, or you can even try using Java Swing with Jython. And if you want to do Mobile app programming then there is Python for S60 which is backed by Nokia.
Python is interpreted language, so there is no compiler (although the interpreter also compiles your python module into bytecode). I would recommend using Python 2.6 as it has some syntax and libraries that is different compared to 2.5. You can also start learning Python 3.0 too.
There is several IDE that is good for Python. You don't have to get yourself attached into one editor/IDE because most of them are good ones. For the commercial ones there is WingIDE which is really focus on making IDE for Python and I would really recommend IntelliJ IDEA with Python plugin which is really nice if you often look at the libraries in your Python environment. For the free ones (as others have said) there is Komodo Edit or you can also try Netbeans with Python plugin.
As for the must-have libraries, this is depending on what you want to do. What kind of application you want to develop with Python. But I think every Python developer should consider PIL for imaging library. I also use simplejson quite often, because I prefer using JSON rather than XML. If you are using XML though, you can use lxml as it is really fast in parsing XML.