I'm trying to wrap my head around the Python ecosystem and parts of it aren't making complete sense to me so far.
I'm coming from the Java world and when I want to make use of, say JUnit, I simply add the JUnit jar to my classpath and that's pretty much it. If I want to be nice to my users I can also easily bunch together all my dependencies into a single jar, so that all that they need to do is install a Java Runtime get a hold of my jar.
Reading through the SciPy installation guide I can't find an explanation for why all this is really necessary. And how is this ever going to work at deployment time? It's like JUnit asked me to install a new JRE just for them.
SciPy has parts written in C that require compilation for the specific platform it's being deployed too.
How can SciPy be fast if it is written in an interpreted language like Python?
Actually, the time-critical loops are usually implemented in C or
Fortran. Much of SciPy is a thin layer of code on top of the
scientific routines that are freely available at
http://www.netlib.org/. Netlib is a huge repository of incredibly
valuable and robust scientific algorithms written in C and Fortran. It
would be silly to rewrite these algorithms and would take years to
debug them. SciPy uses a variety of methods to generate “wrappers”
around these algorithms so that they can be used in Python. Some
wrappers were generated by hand coding them in C. The rest were
generated using either SWIG or f2py. Some of the newer contributions
to SciPy are either written entirely or wrapped with Cython.
Source: http://www.scipy.org/scipylib/faq.html#id12
On Linux, SciPy and NumPy libraries’ official releases are source-code
only. Installing NumPy and SciPy from source is reasonably easy;
However, both packages depend on other software, some of them which
can be challenging to install, or shipped with incompatibilities by
major Linux distributions. Hopefully, you can install NumPy and SciPy
without any software outside the necessary tools to build python
extensions, as most dependencies are optional
Source: http://www.scipy.org/scipylib/building/linux.html
Related
I have a Fortran 95 code that I want to compile to a Python library using f2py. In a matter of fact I've already done it, and it works beautifully. Does the resulting .pyd (.so) depend on numpy after compilation? Could it be used without numpy installation and are they some other options to embed the needed parts inside the final library so it has no dependencies?
I am considering this to be a library accompanying a commercial product and I want the end user to have as little as possible to install on his system, so suggesting to my future customers to install numpy does not suit me. I've searched extensively for an answer, but I cannot seem to find one.
In case it is not possible, could you please refer me to a dependence free way to wrap Fortran code using Python.
I'm afraid there is no way of doing this without numpy, as f2py internally produces numpy dependencies.
Does your code need to be FORTRAN? If it could be C/C++ there might be a chance to get around dependencies, check out chapter 7 of this book for more insights on that.
On the website fortran90.org, there are sections dedicated to the Fortran C interface. While it is of course possible to use NumPy, plain C interfacing using iso_c_binding is certainly possible. And it is part of the standard!
See Interfacing with C and the following sections. The link refers to calling C from Fortran but is directly relevant nonetheless. The next section uses Cython for calling Fortran from Python "à la C" but uses NumPy. The part using ctypes uses no NumPy.
Note: you need a Fortran 2003 compiler. Fortran 95 code is of course legal Fortran 2003 so you can just add the wrapper/interface part to your code.
I am aware that Scipy has a few ICA algorithms, like FastICA, but it can only be used if the mixed signal observations are perfectly in sync.
My application is recording audio (speech) using microphones into mono audio files. So FastICA would not work.
In my research, a few others algorithms I have came across are: Jade, AMUSE and DUET. However, I'm not sure to what extent Python has support for these algorithms. I would prefer to stay in the Python programming language if possible.
Let me just add, I highly value ease of interface, built-in functionality of the Python library, as well as computational efficiency. With that in mind, can someone with experience in using Scipy or other relevant Python libraries suggest a suitable alternative?
I have Anaconda 4.0, and am running Python 3.5 -- just let me know what I should import.
Thank you for reading
You can run the code by installing pyemma toolkit available in python.
conda config --add conda-forge
conda install pyemma
You can refer to this site for more help: http://emma-project.org/latest/generated/MSM_BPTI.html
So I am working on setting up an agent-based model that runs over a geographic map--syria in this case. I tried writing it in python, but the performance is rather slow--even after some optimization tricks. I was thinking that I should shift to just writing the model in C++, but I don't know which visualization packages can incorporate maps? I tend to use gnuplot in C++, but I have not been able to find a way to incorporate a gis basemap in that package. I am not sure if this is possible in VTK or any other packages. I would like to find a way to run my model fast in C++ but not lose the geographic information. Any suggestions?
Perhaps this project could be useful to you ?
http://code.google.com/p/vtk-grass-bridge/
If you can handle your GIS data using GRASS, it seems that project can convert it to something VTK can render, all in one C++ application.
So I actually figured out the answer to this problem and am posting the solution for everyone. The best choice if you are using python, is to just use the mayavi and tvtk packages from Enthought. Mayavi is a gui on top of the C++ VTK libraries. And tvtk is actually a wrapper for python access to VTK objects. So this allows a person to use python GIS packages--like pyshp, Shapely, and others to manipulate GIS objects and then write them to robust and fast mayavi for visualization. At the same time, if you want to stick to C++ then you can still just write your code in C++ using gdal or ogr, etc., and then run your visualization in VTK. This seems a lot easier and more intuitive then trying to run through some other packages like GRASS, QGIS, or ArcGIS.
Here is a good example of this toolset in action.
Example
What makes you believe that a C++ implementation of your model will be dramatically faster? I suggest before being concerned with how you will visualize the results you focus first on what causes your python implementation to be slow. Is it that your algorithm won't scale? If you have tried optimization tricks, what tricks were those and why do you believe they did not work?
It all eventually comes down to machine instructions being executed on hardware, whether those instructions started out as python, C++ , or some other language source code. Unless your python was running fully interpreted all the time I don't think you will find that switching languages alone will cause you to have a fundamentally different performance profile. Premature optimization is still something to be avoided.
I'm building a rendering engine in Python for fun. I need to load 3D scenes. Any standard modern format like DAE, 3DS, or MAX would work: I can convert my files easily between standard formats.
OpenSceneGraph seems to be the most comprehensive and well-maintained solution. It would be ideal to be able to use it in Python without much hassle. Are there working Python bindings for OSG that are easy to install, work on Mac OS X (I'm on 10.8), and are compatible with the latest versions of OSG?
I searched around and came across osgswig (http://code.google.com/p/osgswig/) and PyOSG (http://sourceforge.net/projects/pyosg/), but they don't seem to be actively maintained. I don't see any recent activity related to these packages, and it seems that people had trouble running osgswig on OSX. Ideally, I'd like to find something that "just works", without major compilation hassles. I'd like to just install a package and be able to import a module that will let me load COLLADA or 3DS files.
I also came across pycollada (https://github.com/pycollada/pycollada). It seems active, but fairly early-stage. Ideally, I'd like a reasonably comprehensive package that supports specular maps, normal maps, and other reasonably advanced features. Animation would be nice as well.
In summary, I need to load 3D scenes in Python. Bindings for OSG would probably be ideal, because OSG is so comprehensive. But I need something that works on OSX. I would also prefer something that can be installed reasonably easily. Does something like this exist?
Thanks!
Take a look at Open Asset Import Library (short name: Assimp). It is a portable Open Source library to import various well-known 3D model formats in a uniform manner. http://www.assimp.org/
You should loot at panda3D (http://www.panda3d.org/), it's a game engine with extensive python bindings. It has the features you want : http://www.panda3d.org/manual/index.php/Features
I used it for a few years and it was a solid tool.
I made my own fork of a mirror of a clone of the osgswig project for a similar purpose. I have it working with OpenSceneGraph version 3.2.1 on Windows and Mac; and it's likely I will eventually polish it for linux too. I'm already delivering one product to customers based on my version of osgswig, and I'm considering making others. Find my fork here:
https://github.com/cmbruns/osgswig
If others show enough interest, I might be coaxed into creating binary installers for my version of the osgswig module, to make installation easier.
If you just want the easiest OpenSceneGraph bindings for OSG 3.2.1, you can stop reading this answer here. Read on for more of my thoughts for the future.
Though I am maintaining a fork of osgswig (as stated above), I sort of hate SWIG, and I would prefer to use bindings based on Boost.Python, rather than on SWIG. For large, complex C++ APIs, like OpenSceneGraph, Boost.Python can be much more elegant than SWIG, both for the API consumer, and for the binding maintainer (me, and me). I found one project using Boost.Python to wrap OSG, at https://code.google.com/p/osgboostpython/, but the developer is lovingly wrapping each part of the interface by hand, and has thus only completed a tiny fraction of the large OpenSceneGraph API.
Taking that Boost.Python based project as inspiration, I created yet another OpenSceneGraph Python binding project, at https://github.com/JaneliaSciComp/osgpyplusplus. Eventually, I want to use this osgpyplusplus project for all my python osg needs. And I would appreciate help in making it ready. Right now, osgpyplusplus suffers from the following weaknesses, compared to osgswig:
osgpyplusplus is not yet used in any working product
The build environment is tricky to set up, requiring both Boost.Python and Pyplusplus
I haven't paid much attention to osgpyplusplus recently, so it might rust away if I continue to ignore it.
Though osgpyplusplus probably wraps most of the OpenSceneGraph API, there are probably some important missing pieces that won't be identified until someone tries to develop a significant project with it.
It would be a lot of work for me to create a binary module installer for osgpyplusplus at this point, so please don't ask me to.
Is there a good (small and light) alternative to numpy for python, to do linear algebra?
I only need matrices (multiplication, addition), inverses, transposes and such.
Why?
I am tired of trying to install numpy/scipy - it is such a pita to get
it to work - it never seems to install correctly (esp. since I have
two machines, one linux and one windows): no matter what I do: compile
it or install from pre-built binaries. How hard is it to make a
"normal" installer that just works?
I'm surprised nobody mentioned SymPy, which is written entirely in Python and does not require compilation like Numpy.
There is also tinynumpy, which is a pure python alternative to Numpy with limited features.
Given your question, I decided just factor out the matrix code from where I were using it, and put it in a publicly accessible place -
So, this is basically a pure python ad-hoc implementation of a Matrix class which can perform addition, multiplication, matrix determinant and matrix inversion - should be of some use -
Since it is in pure python, and not worried with performance at all it unsuitable for any real calculation - but it is good enough for playing around with matrices in an interactive way, or where matrix algebra is far from being the critical part of the code.
The repository is here,
https://bitbucket.org/jsbueno/toymatrix/
And you can download it straight from here:
https://bitbucket.org/jsbueno/toymatrix/downloads/toymatrix_0.1.tar.gz
I hear you, I have been there as well. Numpy/scipy are really wonderful libraries and it is a pity that installation problems get somewhat often in the way of their usage.
Also, as far as I understand there are not very many good (easier to use) options either. The only possibly easier solution for you I know about is the "Yet Another Matrix Module" (see NumericAndScientific/Libraries listing on python.org). I am not aware of the status of this library (stability, speed, etc.). The possibility is that in the long run your needs will outgrow any simple library and you will end up installing numpy anyway.
Another notable downside on using any other library is that your code will potentially be incompatible with numpy, which happens to be the de facto library for linear algebra in python. Note also that numpy has been heavily optimized - speed is something you are not guaranteed to get with other libraries.
I would really just put more effort on solving the installation/setup problems. The alternatives are potentially much worse.
Have you ever tried anaconda? https://www.anaconda.com/download
This should allow it to install those packages easily.
conda install -c conda-forge scipy
conda install -c conda-forge numpy
Apart from offering you an easy way to install them in linux/mac/linux you will get virtualenviroments management too
I sometimes have this problem..not sure if this works but I often install it using my own account then try to run it in an IDE(komodo in my case) and it doesn't work. Like your issue it says it cannot find it. The way I solve this is to use sudo -i to get into root and then install it from there.
If that does not work can you update your answer to provide a bit more info about the type of system your using(linux, mac, windows), version of python/numpy and how your accessing it so it'll be easier to help.
For people who still have the problem: Try python portable:
http://portablepython.com/wiki/Download/
Have a look: tinynumpy, tinyarray and sympy
https://github.com/wadetb/tinynumpy
https://github.com/kwant-project/tinyarray
https://docs.sympy.org/latest/index.html