So I am working on setting up an agent-based model that runs over a geographic map--syria in this case. I tried writing it in python, but the performance is rather slow--even after some optimization tricks. I was thinking that I should shift to just writing the model in C++, but I don't know which visualization packages can incorporate maps? I tend to use gnuplot in C++, but I have not been able to find a way to incorporate a gis basemap in that package. I am not sure if this is possible in VTK or any other packages. I would like to find a way to run my model fast in C++ but not lose the geographic information. Any suggestions?
Perhaps this project could be useful to you ?
http://code.google.com/p/vtk-grass-bridge/
If you can handle your GIS data using GRASS, it seems that project can convert it to something VTK can render, all in one C++ application.
So I actually figured out the answer to this problem and am posting the solution for everyone. The best choice if you are using python, is to just use the mayavi and tvtk packages from Enthought. Mayavi is a gui on top of the C++ VTK libraries. And tvtk is actually a wrapper for python access to VTK objects. So this allows a person to use python GIS packages--like pyshp, Shapely, and others to manipulate GIS objects and then write them to robust and fast mayavi for visualization. At the same time, if you want to stick to C++ then you can still just write your code in C++ using gdal or ogr, etc., and then run your visualization in VTK. This seems a lot easier and more intuitive then trying to run through some other packages like GRASS, QGIS, or ArcGIS.
Here is a good example of this toolset in action.
Example
What makes you believe that a C++ implementation of your model will be dramatically faster? I suggest before being concerned with how you will visualize the results you focus first on what causes your python implementation to be slow. Is it that your algorithm won't scale? If you have tried optimization tricks, what tricks were those and why do you believe they did not work?
It all eventually comes down to machine instructions being executed on hardware, whether those instructions started out as python, C++ , or some other language source code. Unless your python was running fully interpreted all the time I don't think you will find that switching languages alone will cause you to have a fundamentally different performance profile. Premature optimization is still something to be avoided.
Related
I am unfamiliar with visual programming. I am writing a program now. I want to show the data my program produces in another visualization program. For example, using a dynamic diagram to show my data, etc. To learn visualization program, what do I have to learn or if there any tools to be recommended? (preferably python related)
There are plenty of very useful libraries available, choosing which would depend on what you want to achieve specifically. An example of a well supported, general python library that has lots of functionality and visual programming tools is plotly. Although I suggest you do some more research.
Is there any good way to call python from clojure as a means of doing data science with scipy, numpy, scikit-learn, etc.
I know about implementations of clojure which run on python instead of java, but this doeesn't work for me, as I also need to call java libraries in my project. I also know about Jython, but I don't know of a clean way to use this with Clojure.
I want to use Clojure in my projects because I prefer it as a language, but I can't deny that Python has an incredible community, and some of the most beautiful, well-designed libraries around.
Instead of trying to get Jython to play well with both Clojure and numpy/scipy, you can use Hy. It is hosted on Python and it somewhat resembles Clojure.
If I really wanted to use numpy/scipy, I would write the backend in Python (or Hy), run it as a separate service. And if I really like ring for instance, or can't live without Instaparse, I would write a frontend in Clojure.
As an aside Python has EDN libs. It would be an interesting project to integrate one of them in Hy, or write one from scratch.
Give the toolz library a try, it's a functional standard library for Python that was designed to generally adhere to the API of the Clojure standard library.
Apart from that, I'd encourage you to find the seams between your computations, and write individual tools in the Unix way in either Clojure or Python depending on which seems to fit that use case best. Serialize data between the tools, either as text/JSON through pipes or using a binary serialization format like Protobuf, which has standard APIs for both Java and Python.
If you had a gun to my head and told me to build Clojure/Python interop, I'd start with py4j and bridge the two languages through Java interfaces, using implements members in a Python class and reify on the Clojure side.
You could use Graal VM now. Although some large companies are using it in production, it's still early days. Here's an example of using Python from Clojure:
(.eval context "python" "
import time;
time.clock()
")
http://gigasquidsoftware.com/blog/2017/10/22/embedded-interop-between-clojure-r-and-python-with-graalvm/
what is in your opinion the best way to get data (measure date for example) into modelica (dymola)? Is it possible, to import data from python to modelica (for example into a combi-time-table)?
My idea would be as follows:
pre processing of measured data in python
load the data from python into modelica (comi-time-table)
rund simulation studies (scripted in python)
I would appreciate any suggestions.
That's probably a matter of opinion. But since you have to do much of your data post- and preprocessing in Python I would definitely export my (plant) model from Dymola as a co-simulation FMU and run it in Python.
In Dymola you can export FMU's and 'execute' them on the same pc that holds the Dymola license file. If you need to run the FMU on another pc you'll have to buy a special binary export license.
There is a free Python package called PyFMI (www.pyfmi.org) which makes it easy to run an FMU in Python. See the examples at http://www.jmodelica.org/page/4924.
PyFMI can be a bit tricky to get up and running (with the right Python package dependencies and so on). So if you are not an experienced Python user I would suggest that you download the installer for JModelica.org which will do much the setting up for you.
Best regards,
Rene Just Nielsen
As Rene Just Nielsen pointed out, this is primarily opinion based.
To give another way of accomplishing your goal, try the DymolaInterface. You could either set the table-parameter via the Interface in python, our use a .txt-file which you create and alter in python, and Modelica just knows the path to the file. The interface comes with your Dymola installation under Modelica\Library\python_interface\dymola.egg where you will also find documentation for the functions.
Another python-package for using FMU's is FMPy. I image both FMPy and PyFMI have their pros and cons.
The last option which does not require any external python package would be to use mos-files to execute simulations and use the .txt-files to read in the data. If the task you described is the only thing you want to accomplish, mos-scripts are quite sufficient.
I'm trying to wrap my head around the Python ecosystem and parts of it aren't making complete sense to me so far.
I'm coming from the Java world and when I want to make use of, say JUnit, I simply add the JUnit jar to my classpath and that's pretty much it. If I want to be nice to my users I can also easily bunch together all my dependencies into a single jar, so that all that they need to do is install a Java Runtime get a hold of my jar.
Reading through the SciPy installation guide I can't find an explanation for why all this is really necessary. And how is this ever going to work at deployment time? It's like JUnit asked me to install a new JRE just for them.
SciPy has parts written in C that require compilation for the specific platform it's being deployed too.
How can SciPy be fast if it is written in an interpreted language like Python?
Actually, the time-critical loops are usually implemented in C or
Fortran. Much of SciPy is a thin layer of code on top of the
scientific routines that are freely available at
http://www.netlib.org/. Netlib is a huge repository of incredibly
valuable and robust scientific algorithms written in C and Fortran. It
would be silly to rewrite these algorithms and would take years to
debug them. SciPy uses a variety of methods to generate “wrappers”
around these algorithms so that they can be used in Python. Some
wrappers were generated by hand coding them in C. The rest were
generated using either SWIG or f2py. Some of the newer contributions
to SciPy are either written entirely or wrapped with Cython.
Source: http://www.scipy.org/scipylib/faq.html#id12
On Linux, SciPy and NumPy libraries’ official releases are source-code
only. Installing NumPy and SciPy from source is reasonably easy;
However, both packages depend on other software, some of them which
can be challenging to install, or shipped with incompatibilities by
major Linux distributions. Hopefully, you can install NumPy and SciPy
without any software outside the necessary tools to build python
extensions, as most dependencies are optional
Source: http://www.scipy.org/scipylib/building/linux.html
I'm building a rendering engine in Python for fun. I need to load 3D scenes. Any standard modern format like DAE, 3DS, or MAX would work: I can convert my files easily between standard formats.
OpenSceneGraph seems to be the most comprehensive and well-maintained solution. It would be ideal to be able to use it in Python without much hassle. Are there working Python bindings for OSG that are easy to install, work on Mac OS X (I'm on 10.8), and are compatible with the latest versions of OSG?
I searched around and came across osgswig (http://code.google.com/p/osgswig/) and PyOSG (http://sourceforge.net/projects/pyosg/), but they don't seem to be actively maintained. I don't see any recent activity related to these packages, and it seems that people had trouble running osgswig on OSX. Ideally, I'd like to find something that "just works", without major compilation hassles. I'd like to just install a package and be able to import a module that will let me load COLLADA or 3DS files.
I also came across pycollada (https://github.com/pycollada/pycollada). It seems active, but fairly early-stage. Ideally, I'd like a reasonably comprehensive package that supports specular maps, normal maps, and other reasonably advanced features. Animation would be nice as well.
In summary, I need to load 3D scenes in Python. Bindings for OSG would probably be ideal, because OSG is so comprehensive. But I need something that works on OSX. I would also prefer something that can be installed reasonably easily. Does something like this exist?
Thanks!
Take a look at Open Asset Import Library (short name: Assimp). It is a portable Open Source library to import various well-known 3D model formats in a uniform manner. http://www.assimp.org/
You should loot at panda3D (http://www.panda3d.org/), it's a game engine with extensive python bindings. It has the features you want : http://www.panda3d.org/manual/index.php/Features
I used it for a few years and it was a solid tool.
I made my own fork of a mirror of a clone of the osgswig project for a similar purpose. I have it working with OpenSceneGraph version 3.2.1 on Windows and Mac; and it's likely I will eventually polish it for linux too. I'm already delivering one product to customers based on my version of osgswig, and I'm considering making others. Find my fork here:
https://github.com/cmbruns/osgswig
If others show enough interest, I might be coaxed into creating binary installers for my version of the osgswig module, to make installation easier.
If you just want the easiest OpenSceneGraph bindings for OSG 3.2.1, you can stop reading this answer here. Read on for more of my thoughts for the future.
Though I am maintaining a fork of osgswig (as stated above), I sort of hate SWIG, and I would prefer to use bindings based on Boost.Python, rather than on SWIG. For large, complex C++ APIs, like OpenSceneGraph, Boost.Python can be much more elegant than SWIG, both for the API consumer, and for the binding maintainer (me, and me). I found one project using Boost.Python to wrap OSG, at https://code.google.com/p/osgboostpython/, but the developer is lovingly wrapping each part of the interface by hand, and has thus only completed a tiny fraction of the large OpenSceneGraph API.
Taking that Boost.Python based project as inspiration, I created yet another OpenSceneGraph Python binding project, at https://github.com/JaneliaSciComp/osgpyplusplus. Eventually, I want to use this osgpyplusplus project for all my python osg needs. And I would appreciate help in making it ready. Right now, osgpyplusplus suffers from the following weaknesses, compared to osgswig:
osgpyplusplus is not yet used in any working product
The build environment is tricky to set up, requiring both Boost.Python and Pyplusplus
I haven't paid much attention to osgpyplusplus recently, so it might rust away if I continue to ignore it.
Though osgpyplusplus probably wraps most of the OpenSceneGraph API, there are probably some important missing pieces that won't be identified until someone tries to develop a significant project with it.
It would be a lot of work for me to create a binary module installer for osgpyplusplus at this point, so please don't ask me to.