For my app, I need to determine the nearest points to some other point and I am looking for a simple but relatively fast (in terms of performance) solution. I was thinking about using PostGIS and GeoDjango but I think my app is not really that "geographic" (I still don't really know what that means though). The geographic part (around 5 percent of the whole) is that I need to keep coordinates of objects (people and places) and then there is this task to find the nearest points. To put it simply, PostGIS and GeoDjango seems to be an overkill here.
I was also thinking of django-haystack with SOLR or Elasticsearch because I am going to need a strong, strong text search capabilities and these engines have also these "geographic" features. But not sure about it either as I am afraid of core db <-> search engine db synchronisation and hardware requirements for these engines. At the moment I am more akin to use posgreSQL trigrams and some custom way to do that "find near points problem". Is there any good one?
To find points or bounding boxes that are near each other, consider using the Rtree Python package. This uses a similar spatial index technique as PostGIS, except it is not database software and can be used in software. I've tested faster speeds than from PostGIS to find near points on millions of objects.
See examples in the tutoral to get a good feel to find nearest objects.
You're probably right, PostGIS/GeoDjango is probably overkill, but making your own Django app would not be too much trouble for your simple task. Django offers a lot in terms of templating, etc. and with the built in admin makes it pretty easy to enter single records. And GeoDjango is part of contrib, so you can always use it later if your project needs it.
check out shapely. Looks like the object's project() method may be what you're looking for.
Related
I am starting web app in Django, which must provide one simple task: get all records from DB which are close enough to other record.
For example: Iam in latlang (50, 10), and I need to get all records with latlang closer than 5km from me.
I found that geodjango thing called GeoDjango, but it contains a lot of other dependencies and libraries like GEOS, POSTGIS, and other stuff which i don't really need. I need only this one range functionality.
So should I use GeoDjango, or just write my own range calculation query?
Most definitely not write your own. As you get more familiar with geographic data you will realize that this particular calculation isn't at all simple see for example this question for a detailed discussion. However most of the solutions (answers) given in that question only produce approximate results. Partly due to the fact that the earth is not a perfect sphere.
On the other hand if you use Geospatial extensions for mysql (5.7 onwards) or postgresql you can make use of the ST_DWithin function.
ST_DWithin — Returns true if the geometries are within the specified distance of one another. For geometry units are in those of spatial reference and For geography units are in meters and measurement is defaulted to use_spheroid=true (measure around spheroid), for faster check, use_spheroid=false to measure along sphere.
ST_DWithin makes use of spatial indexes which home made solutions will be unable to. WHen GeoDjango is enabled, ST_DWithin becomes available as a filter to django querysets in the form of dwithin
Last but not least, if you write your own code, you will have to write a lot of code to test it too. Whereas dwithin is thoroughly tested.
I've been googling around and looking for a standard way to handle unit conversion (weight, distance, etc..) in Python / Django.
Since it is a very common problem, I'm afraid that I've not been able to find the proper package. (Found several handmade unit conversion questions, but no package).
As far as I know, USA and Thailand are using non metric system, and this also happens in the UK for people (however they officially support the MKS international system).
Is there an existing way of integrating unit conversion with Django LocaleMiddleware, or django_countries packages?
Quick glance shows such Python packages as units.
As far Django integration, unit conversion is definitely not something django-countries can deal with. It’s also likely outside of the scope of LocaleMiddleware, which is more of a wrapper around gettext. You can look into whether you can add support conversion on gettext level, but I personally doubt it’d be worth the trouble.
In your position I’d look into defining utility functions and template tags/filters depending on your project’s specific needs, perhaps utilizing the aforementioned units.
I developed a simple package for this. Even though this question is fairly old, there still was no intuitive package around until this year. There are packages like "units" (as Anton mentioned) but they either deal with the units themselves or they are not easy to use.
My package is very simple and can only convert numeric values (metric-metric, metric-non-metric and non-metric-non-metric) but it does that very intuitively for many quantities such as pressure, mass, speed, length, voltage, etc.
https://pypi.org/project/pyunitsconverter/
I'm not really interested in generating tiles if I can help it. Instead, what I'm looking for is a means of getting "what is near me" kind of information, specifically bodies of water and green space, or even civil services.
If I had the map tiles, I suppose I could parse them for the colours I want, but I'm thinking that there must be a better/smarter way. Isn't is possible to get a list of objects near lat,lng that belong to categories A and B?
I'm a competent Python programmer, but am completely new to OSM. I understand that I can download a Very Large XML file and have all the data, but accessing it, especially for this sort of purpose is totally foreign to me.
I should however that I have at my disposal complete access to a PostgreSQL database complete with PostGIS in a GeoDjango setup.
Tiles are not necessary for this, generating tiles is just one possible way of using OSM data.
Do you need an online or offline solution? For an online solution you don't even need a local copy of the data. Instead you can directly fetch the data around a specific position. Instead of using the official API which is mainly for editing and not for bulk querying, just use the Overpass API which is way faster and features a complex query language.
Here is an example Overpass API query for querying all shops and parking places inside the given bounding box 50.6,7.0,50.65,7.05:
(
node
["shop"]
(50.6,7.0,50.65,7.05);
node
["amenity"="parking"]
(50.6,7.0,50.65,7.05);
way
["shop"]
(50.6,7.0,50.65,7.05);
way
["amenity"="parking"]
(50.6,7.0,50.65,7.05);
relation
["shop"]
(50.6,7.0,50.65,7.05);
relation
["amenity"="parking"]
(50.6,7.0,50.65,7.05);
);
(
._;
>;
);
out;
(The result can be downloaded as either XML or JSON. You can also visualize it using overpass turbo)
In order to understand the query you have to get familiar with OSM's basic elements (nodes, ways and relations) as well as the tagging system and the most common tags.
If you need an offline solution then you better set up a local database. For an instruction you can read the serving tiles howto on switch2osm and just skip the Apache/mod_tile/mapnik steps. Importing an extract instead of the whole planet will often suffice. Live-parsing a XML file instead will be very slow except you have a very small area, say a city, and you did some filtering beforehand.
There is a very beautiful package OSMnx by Geoff Boeing https://geoffboeing.com/tag/osmnx/
You can easily get all the amenities near you by OSM.
import osmnx as ox
import geopandas as gpd
place_name = "" (geocoding of polygon)
tags = {'amenity': 'charging_station'}
ox.geometries_from_place(place_name, tags)
I have been going through trying to find the best option on Google App Engine to search a Model by GPS coordinates. There seem to be a few decent options out there such as GeoModel (which is out of date) but the new Search API seems to be the most current and gives a good bit of functionality. This of course comes at the possibility of it getting expensive after 1000 searches once this leaves the experimental zone.
I am having trouble going through the docs and creating a coherent full example to be able to use the Search API to search by location and I want to know if anyone has examples they are willing to share or create to help make this process a little more straightforward. I understand how to create the actual geosearch query but I am unclear as to how to glue that together with the construction and indexing of the document.
I don't know of any examples that show geo specifically, but the process is very much the same. You can index documents containing GeoFields, each of which has a latitude and a longitude. Then, when you construct your query, you can:
limit the results by distance from a fixed point by using a query like distance(my_geo_field, geopoint(41, 65)) < 100
sort by distance from a point with a sort expression like distance(my_geo_field, geopoint(55, -20))
calculate expressions based on the distance between points by using a FieldExpression like distance(my_geo_field, geopoint(10, -30))
They work pretty much like any other field, except you can use the distance function in the query and expression languages. If you have any specific questions, feel free to ask here.
I am searching for a python package that I can use to simulate molecular dynamics in non-equilibrium situations. I need a setup that can handle a fairly large number of molecules in a primarily kinetic theory manner, and that can handle having solid surfaces present. With regards to the surfaces, I would need to be able to create arbitrary shapes and monitor pressure and other variables resulting from the molecular action. Alternatively, I could add the surface parts myself if I had molecules that could handle it.
Does anyone know of any packages that might be suitable?
Have you considered SimPy? SimPy is a rather generic Discrete Event Simulation package, but could feasibly meet your needs.
Better yet the Molecular Modelling ToolKit (MMTK) seems more specialized...
I have used neither, but this sounds like fun. Python, as a language, seems to be in privileged position for use in simulation software, whereby people can script the specific details of their model while relying on the framework for all the common logic, such as scheduling, visualization, monitoring etc. The unknown is how well such toolkits scale when fed with agent counts commensurate with biology models (BTW, how "big" is that?)
Lampps and gromacs are two well known molecular dynamics codes. These codes both have some python based wrapper stuff, but I am not sure how much functionality the wrappers expose. They may not give you enough control over the simulation.
Google for "GromacsWrapper" or google for "lammps" and "pizza.py"
Digital material and ASE are two molecular dynamics codes that expose a lot of functionality, but last time I looked, they were both fairly specialized. They may not allow you to use the force potentials that you want
Google for "digital material" and "cornell" or google for "ase" and dtu
Note to MJV: Normal MD-codes take one time step at a time, and they move all particles in each time step. Most of the time is spend calculating the total force on each atom. This involves iterating over a list of pairs of neighboring atoms. I think the best idea is to do the force calculation and a few more basics in c++ or fortran and then wrap that functionality in python. (But it could be fun to see how far one can get by using numpy matrices)
The following programs can be used to run MD symulations:
Gromacs
AMBER
charmm
OpenMM
many others...
The following Python packages are useful for preparing and analysing MD trajectories:
MDtraj and the OMNIA ecosystem
MDAnalysis
ProDy
MMTK
Another generic simulations framework is my own GarlicSim. You can try that. I could help you get a simpack up if you're serious about it.
I don't know if that programs does all the features you need but there is avogadro in the kde programs, i think it is extendable and since it is open source you could do anything with it. http://www.kde-apps.org/content/show.php/Avogadro?content=59521
It is really advanced and programmed by a friend of mine
I second MMTK, but take a look at VMD, which is the best MD software I'm aware of, and is Python-scriptable (in addition to Tk). See this for examples and tutorials.
I recommend to use molecular dynamics software to run MD simulations like Gromacs. This software is highly optimized for that particular purpose. You can also run on GPU's and you will be able to run larger systems in less time.
Afterwards, you run only the analysis with python packages using the generated trajectories.
mdtraj
pmx