Data visualization in python - after connecting to a database - python

Can you help me to connect to my postgresql database with python? I need to create graphic interface with python which will visualize shapefile data from my database (i have about 50 polygons in shapefile format in that database). Can you help me with creating such application? I am begginer in python.

For communicating with the database, use psycopg2. It's quick, easy and efficient if you are familiar with basic DB concepts.
You have several options from here. You can use shpUtils, which is supposed to be a nice package for parsing shapefiles. You can then visualize the data using numerous python graphics packages, like pil.
PIL image source code here.
Every option suits a different need, depending on what you define as "create graphic interface". If you need to create a simple graphics output, build the polygons from text using one of the graphics utilities mentioned above. If you need to create a professional-looking image, try using mapnik (mentioned in some other answers), which easily reads shapefiles. If you need to create a fully functional GUI, it's probably not a beginner's task - you should start with programming basic GUI applications before diving into shapefiles and polygons.
If, however, you just need to view the polygons - skip python and just use qgis, which will very easily visualize your polygons. It also comes with a handful of other nice features, like colors, zooms and so on.
(source: sourceforge.net)

I would approach this by breaking it up into smaller problems and solving each of them
a) How do I connect to a postgresql database with python?
https://stackoverflow.com/search?q=postgresql+database+python - Looks like psycopg2 is a good option as Adam Matan suggested.
b) Drawing shapefile data in python
postgresql and python

Mapnik is great for drawing maps. It can handle various formats and shapefiles, too. As far as I know it also supports PostgreSQL (at least PostGIS).
And least but not last: it comes with a Python interface (see Getting started)

Related

Python web app for programmatic 3D scene construction

I'm currently developing a Django app that allows students to programmatically develop SVG graphics. They can code Python in their browser with ACE editor. The code is executed on the server, stored in a database and the generated SVG (custom library) returned and displayed. An example code that displays a filled ellipse looks like so:
graph.draw(Circle(cx=0, cy=0, r=20, fill="lime").scale(2, 1)
Now I'm wondering, how I could extend this app to do some 3D. I stumbled over X3Dom, which seems promising and not too hard to generate and I could write another lightweight pythonic library for this. But, it doesn't seem to do CSG (constructive solid geometry) which is major drawback.
Any hints in what direction I should investigate for some 3D web technology that allows easy 3D scene generation with server-side python and that implements CSG?
NB: OpenJSCad is simalar to what I'd like to achieve, except that my solution allows for classroom collaboration and it must expose Python to students as the programming language. The aim is to spice up the teaching of Python programming with graphics.
I believe I used three.js to do CSG a while back. There used to be an example online. You are right that X3D does not do CSG. I was doing cross sections of the earth and found a way with X3D. You might be able to use VPython or brython in the browser if your worried about Python not running in the browser. I’ve only brython for a short time testing another person’s project and vpython not at all.
If you’re doing something like inverseCSG or CSGNet, is your class available online?
In other words, maybe try to find a Python library that does CSG instead searching for a rendering engine in JS. Don’t view the browser as limited to JS.
I only found three.js when I was looking.
Maybe search for a solution which is not a solid solution.
try checking out the library "trimesh" for python which relays mainly on watertight stl files, but allows you to do some boolean operations for CSG. You can substract one file from the other, extended and find the intersection. Plus, it has some primitive functions directly like cylinders and spheres.

Visualizing LDA topic models using Stephen Hansen's topicmodels package

I'm using Stephen Hansen's topicmodels package to create a topic model. I'm using the tutorial data available at that link to start. This tutorial uses Python 2 but only requires one changes xrange() to range() in order to have it work properly with Python 3. I was able to run through the tutorial fully but I'm struggling to visually represent the results. First, does anyone have and resources on visualizing LDA models using this package? I haven't been able to locate any.
Primarily, I'm trying to create two kinds of graphics. First, I'm trying to create a word cloud by topic. The second is something that looks like this: (full paper available here)
I have a CSV document from running the tutorial that has all the necessary information to do this that looks like the following:
The issue is that I'm unsure how to go about creating this document. I'm also unsure how I could use this to create word clouds by topic. There's not much use of topicmodels in python and I don't have very much experience creating any graphics with Python. I'm lost trying to figure out how to convert my output to something I can use with wordclouds or any other visual.
If anyone wants to replicate this data, you just need to have a C++ compiler installed on your computer, download and adjust xrange() in the tutorial, and run it through.

Creating manipulable diagrams in python

I want to create a piece of software that allows me to create manipulable diagrams (state machine alike) in python. What library should I use (does at least one exist)?
I found this js library (https://gojs.net/latest/samples/stateChart.html), but I'd like to create the whole program in python.
I did not found something that lets you work with diagrams from the UI standpoint.
I would like to be able to click on a certain diagram and edit its contents and manipulate the contests of all diagram objects.
Use pysimpleautomata library pysimpleautomata
You might have to create your own api to create interface though. For that, I'd suggest using tkinter.

Making an interactive network topology diagram using Python or something else?

I've got masses of network information held in a SQLite database. I want to draw a diagram based on this information in the style a network diagram.
I want it to be interactive, in a sense that at the highest level only network range communication can be seen, and as you move deeper into the diagram you begin to see individual nodes (switches, routers, firewalls, hosts, servers, etc) all linked together.
I'd like this process to be as smooth as possible, allowing you to zoom in using the scroll wheel in a location of the diagram and it expands as you do so. Allowing you to then click and drag around the map. However I'd like to get basics down first, so thinking that I should start with drawing the diagram in HTML and have hyper-links for nodes allowing the user to move around and deeper into the diagram using a browser.
It is also crucial that the user would be able to able to capture their view as a still image, which I'd guess would be far more easily done in HTML.
To get to the point I'm asking where I might start to do this. I've looked at PyQT, Graphviz, outputting to HTML etc. I'm just trying to decide what to use and generally how to go about doing this. I'm reasonably good with Python but I am open to suggestion of other languages.
If you guys think Python can do this, which Python? 2.7 or 3? I've been considering making a move to 3 for a while, is it time?
Thanks in advance!
The important thing here is that you want something dynamic and that can be captured by the user.
My answer will be fairly similar to this question.
Export your graph to a standard format such as GEXF and use a Javascript graph drawing library to make your graph interactive such as: SigmaJs, or VivaGraphJs.
The big advantage is that you can script your graph to respond to user event such as zoom, save as a picture or display information dynamically about nodes and edges, etc.
To resume:
First you a python graph library such as Networkx, then export your graph with its properties as JSON or GEXF.
Load the graph using a javascript graph library using the examples as baselines. List of examples using sigma.js, tutorial for VivaGraphJs.
Concerning the python version, it is really dependent on other libraries you may use. For scientific use, I wouldn't recommend to switch to Py3k but for anything else you're good to go.
I've faced in the past something similar to your problem. We have tons of routers documented in a MySQL database. We actually use Racktables and that tool stores all the information in such a way.
At one point we needed to plot networking topologies. If you want, please have a look at this:
https://notedisabbia.wordpress.com/2016/06/17/first-blog-post/
https://github.com/RackTables/racktables-contribs/tree/master/python-graph-topology
The first link is a blog I've written in order to explain what my python program (second link) does in terms of connecting to Racktables, gathering information and plotting network diagrams.
Hope this helps.
Cheers,
Lucas
I would recommend looking at library d3graph. It does not allow you to zoom using the scroll wheel but does have other features like breaking edges.
More information can be found in this blog.

Python built open node graph?

after hunting through stackoverflow, I still can't find quite what I'm looking for.
Question:
I am looking to use (or create if this doesn't exist) an open source node graph, as a node-based approach to using Python code.
Clarification
Let me explain the concept of what I mean by "Node graph".
A node graph is a place to create "nodes", represented by little boxes with connections. The user can move select them, move them around, make and break connections based on appropriate data types.
Example
For example- and I think to myself, "surely this has been done before," but I'd like to write nodes in python script that load data, process that data in some way, etc.
For example, I want to write a python node that loads time based data from a file OR a real time source, then another node that processes it, then another node that maybe visualizes it, or writes out a custom format to file.
Background I come from a background in 3D animation where we use the software packages Maya and Nuke to process data in this way using node-based workflow- however the architectures are very high-level and specialized. What I want to do is reduce this idea of workflow to a very mid-low level. All I want is a node graph with custom python nodes.
So far I have tried:
Nodebox 3 - this is a high level software built on Java and compiled code. Seems to be too high level for me (with a fully functional gui) and too java based
Nodebox Windows - a nifty OpenGL derived library, but too low level, would have to build an interactive node graph from scratch
Coral - an interesting idea but too specialized in 3D I think, for doing generic data processing as I'd like. Perhaps I could modify it, but I'm afraid the dying community might hold me back.
appreciate any other suggestions you guys may have.
** EDIT **
StackOverflow won't let me post a screenshot example.. annoying.
A visual programming language- this general idea is what I'm looking for! But what I would prefer is the ability to write functions in Python and then graph them visually like those languages.
Try PyQtGraph
It has a node-graph among a lot of other visualization Widgets. It's called flowchart and lets you create and connect nodes. Each of these contains a python function to modify its input data.
As the name implies it relies on PyQt/PySide.

Categories