I'm looking to try and visualize data using networkx as a network graph. My data looks great, but i'm wanting to add on hover and on click events to display additional information.
For example, there might be a node called "New York", when clicked will display a small table to the side of the canvas that gives information like how many cities, current population, etc.
I'm currently using pyviz with networkx. That seems to be really straightforward as far as creating the graph, but not so much on the kind of user interaction i'm looking for.
I also tried bokeh and plotly, but on the on click and hover functions while work, isn't very straightforward to implement with networkx. Here's a picture of what my graph looks like. My goal is to show relationships between systems.
pyvis graph
I maintain a python library for network visualisations called netgraph, which works nicely with networkx or igraph Graph objects. I thought this was a neat idea for a feature, so I just implemented a bare bones version on the dev branch.
Installation via pip:
pip install https://github.com/paulbrodersen/netgraph/archive/dev.zip
Code to reproduce the example above:
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
import networkx as nx
from netgraph import InteractiveGraph
g = nx.cubical_graph()
tables = dict()
for node in g.nodes:
data = np.round(np.random.rand(3,2), decimals=2)
table = pd.DataFrame(data, index=['Lorem', 'ipsum', 'dolor'], columns=['sit', 'amet'])
tables[node] = table
for edge in g.edges:
data = np.round(np.random.rand(4,1), decimals=2)
table = pd.DataFrame(data, index=['consectetur', 'adipiscing', 'elit', 'Mauris'], columns=['sed'])
tables[edge] = table
fig, ax = plt.subplots(figsize=(12,5))
fig.subplots_adjust(right=0.6) # make space for table on the right
bbox = [1.5, 0.1, 0.5, 0.8] # position of the table in axes coordinates
instance = InteractiveGraph(g, node_labels=True, tables=tables, table_kwargs=dict(edges='horizontal', fontsize=16, bbox=bbox), ax=ax)
plt.show()
Take a look at the kglab project which is an open source abstraction layer in Python that integrates both NetworkX and PyVis, along with other graph related libraries in Python. It was built for this kind of use case.
There's a class kglab.KnowledgeGraph which has transforms and inverse transforms to work these other libraries:
NetworkX examples https://derwen.ai/docs/kgl/ex6_0/
PyVis examples https://derwen.ai/docs/kgl/ex3_0/
For instance, you could:
build a graph using a KnowledgeGraph object
transform out to run NetworkX graph algorithms
use an inverse transform to populate calculated attributes on the main graph object
transform out to load and run a PyVis interactive session, which in turn can have clickable components
We've got Jupyter notebooks on the GH repo showing each of these steps. plus a developer community where other people can help for a specific use case (create a GH issue)
There is VisDCC, which requires Dash (a kind of data-science server thing).
The end result is a web server serving an HTML canvas that you can insert in a web page, for instance. (This is actually inherited from Vis.js)
VisDCC has almost no documentation but it works (I'm using it) and the usage follows that of Dash which is well-documented. You need to learn to use Dash's #callback format to write the code.
So I guess it's a good enough solution :)
Related
I have a pandas dataframe that contains 3 columns , ['source' , 'target', 'weight'].
I have created a networkx graph successfully and plotted it in the browser. The networkx graph was created using the following line of code:
Graph = nx.from_pandas_edgelist(rules,source='source',target='target').
Using the networkx library, it was easy to create the graph using only the above line of code, and setting the nodes labels, arrows that shows the sense from source to destination etc...
I want to do this using plotly dash to have an interactive plot, however, in the examples I have seen in plotly dash networkx, they always have to declare the nodes and edges manually, which will require to have a for loop over the dataframe to pass all the entries of the dataframe. In my case, I have a pretty large dataframe which will be a problem.
My question is that, is there a way that I am missing, to create the graph using dash in a simple way without the need to declare the edges and the nodes manually? a way similar to that line of code?
Graph = nx.from_pandas_edgelist(rules,source='source',target='target').
and if there is a way, can I access the nodes by their ids? and how to insert nodes labels in the graph.
Thank you in advance.
I have tries visdcc and cytoscape libraries using plotly dash. In visdcc I used a loop which took a lot of time to pass through the dataframe entries. And in cytoscape, it needs to create the nodes and edges manually, which I do not want to do, my question is, if there is a way to create the graph automatically using the dataframe as networkx and the draw option do.
Yes, it is possible to use a dataframe to create a network graph with Networkx and then render this network graph in Dash-Cytoscape (you don't have to enter nodes and edges manually). In the question I asked yesterday (here), you can find a code example to achieve this (my question was about something more specific).
I'm trying to draw a graph of any network running my script. I use scapy to collect packets and would like to have a node per computer communicating, and an edge per connection.
The issue is I can't find a way to visualize the graph well enough on my screen. So far combining networkx with matlib.pyplot managed to bring the best results, but it still seems pretty random and chaotic, the tags are hard to read, nodes are on top of each other, etc'. It is also preferable to have the ability to interact with the graph - move nodes around, hover over nodes/edges to get extra info, perhaps zoom in or even cluster together nodes so that when you click on the cluster you can see which nodes compose the cluster.
Since analyzing the network data and adding nodes&edges to the graph will be tedious for you to read, I'm adding only the relevant part here (the part that actually shows the graph I built):
pos = nx.spring_layout(Graph, scale=2)
edge_labels = nx.get_edge_attributes(Graph, "Protocol")
nx.draw(Graph,pos, with_labels=True, node_size=600, font_size=8, font_weight='bold')
nx.draw_networkx_edge_labels(Graph, pos, edge_labels=edge_labels, font_size=8)
plt.show()
(I imported networks as nx and matplotlib.pyplot as plt)
I also tried graphviz, ploty and bokeh but couldn't really make them work and after troubleshooting on Google got the impression that anyway they won't fix my problem, and I also tried adjustText - but I could not manage to fit it in my code in any way (can't find how to get the text attribute of my graph) and Holoviews - but it refuses to show an image no matter what I try (even if I copy and paste examples from their site - either python says that '%opts...' is invalid syntax, or if I try changing options any other way the code just runs until it ends and doesn't show anything on the screen.
This is what the graph looks like:
I'm finding a lot of partial solutions online so none of them work, does anybody has a comprehensive solution?
Drawing heavy graphs with plt can be a bit problematic, the problem here is not only with the data, it is also a problem for a human eye to get a lot of information in one look.
My suggestion is to use a more advanced graph visualization library, for example, ipycytoscape. you can define also styles and more features with it that will match your demands
from ipycytoscape import CytoscapeWidget
graph_draw = ipycytoscape.CytoscapeWidget()
graph_draw.graph.add_graph_from_networkx(nx_graph, directed=True)
In addition, if you will use CytoscapeWidget you can interact with the graph and match the focus of the view to the part in the graph that interests you the most.
You can tune the hyper-parameters (k and iterations) of the nx.spring_layout to arrange the nodes. Once you tune the parameters, the connected nodes will be close to each other, and not-contacted nodes will maintain a maximum possible distance.
pos = nx.spring_layout(G,k=0.1, iterations=20)
I am new to NetworkX. Right now, I manage to connect all the nodes to this particular node. What I want to do next it to make it interactive e.g. able to make each of the node move by dragging using cursor. I know I have to make use of matplotlib, but I am not sure how to use it. Can anyone help me?
My codes are:
import matplotlib.pyplot as plt
import networkx as nx
import itertools
d = [name of nodes]
f = [number per nodes]
for i in d:
G.add_edge('"' + i + '"',b)
pos=nx.fruchterman_reingold_layout(G, k=0.5, iterations=5)
nx.draw_networkx_nodes(G,pos,node_size=130, node_color="white")
nx.draw_networkx_edges(G,pos, width=0.2,alpha=1,edge_color='black')
nx.draw_networkx_labels(G,pos,font_size=7,font_family='sans-serif')
for i,j in itertools.izip(d,f):
nx.draw_networkx_edge_labels(G,pos, {('"' + i + '"',b):j}, font_size=7, label_pos= 0.80)
plt.axis('off')
plt.show()
It seems hard to do with matplotlib (it is not really been designed for that). Networkx drawing module is pretty poor it mostly uses a custom scatter plot for nodes, etc.
I suggest another solution:
Export your graph to JSON or GEXF and use a Javascript graph drawing library to make your graph interactive such as: SigmaJs, or VivaGraphJs.
You find an example of a real graph created with NetworkX embedded on a webpage on my blog. Nodes are static in this example but clicking on a node highlights its neighbors.
Official examples for the proposed interactive graph drawing libraries:
List of examples using sigma.js.
Tutorial for VivaGraphJs.
Matplotlib was designed more for static graphs and charts.
However once the NetworkX graph is exported to GEXF format there is a tool which will allow you to select areas based on position or critera in order to move it around. The tool is called Gephi. You can play with the layout to get started or go as deep as data scientists like to get.
NetworkX is powerful but I was trying to plot a graph which shows node labels by default and I was surprised how tedious this seemingly simple task could be for someone new to Networkx. There is an example which shows how to add labels to the plot.
https://networkx.github.io/documentation/latest/examples/drawing/labels_and_colors.html
The problem with this example is that it uses too many steps and methods when all I want to do is just show labels which are same as the node name while drawing the graph.
# Add nodes and edges
G.add_node("Node1")
G.add_node("Node2")
G.add_edge("Node1", "Node2")
nx.draw(G) # Doesn't draw labels. How to make it show labels Node1, Node2 along?
Is there a way to make nx.draw(G) show the default labels (Node1, Node2 in this case) inline in the graph?
tl/dr: just add with_labels=True to the nx.draw call.
The page you were looking at is somewhat complex because it shows how to set lots of different things as the labels, how to give different nodes different colors, and how to provide carefully control node positions. So there's a lot going on.
However, it appears you just want each node to use its own name, and you're happy with the default color and default position. So
import networkx as nx
import pylab as plt
G=nx.Graph()
# Add nodes and edges
G.add_edge("Node1", "Node2")
nx.draw(G, with_labels = True)
plt.savefig('labels.png')
If you wanted to do something so that the node labels were different you could send a dict as an argument. So for example,
labeldict = {}
labeldict["Node1"] = "shopkeeper"
labeldict["Node2"] = "angry man with parrot"
nx.draw(G, labels=labeldict, with_labels = True)
I feel a better answer is not to use networkx to draw. They explicitly warn you that graph visualization is hard and networkx is mainly meant for graph analysis (from https://networkx.org/documentation/stable/reference/drawing.html#module-networkx.drawing.layout):
Drawing
NetworkX provides basic functionality for visualizing graphs, but its main goal is to enable graph analysis rather than perform graph visualization. In the future, graph visualization functionality may be removed from NetworkX or only available as an add-on package.
Proper graph visualization is hard, and we highly recommend that people visualize their graphs with tools dedicated to that task. Notable examples of dedicated and fully-featured graph visualization tools are Cytoscape, Gephi, Graphviz and, for LaTeX typesetting, PGF/TikZ. To use these and other such tools, you should export your NetworkX graph into a format that can be read by those tools. For example, Cytoscape can read the GraphML format, and so, networkx.write_graphml(G, path) might be an appropriate choice.
thus my suggestion is to transform the graph to some format that has dedicated software for graph visualization and then draw (e.g. pydot, pygraphviz, graphviz etc). My suspicion is that pydot and pygraphviz are the best for some reason since networkx only supports those two. It seems from the docs in pygraphviz that it has a similar api so it might be the easiest to use if you already want like to use networkx (https://pygraphviz.github.io/documentation/stable/tutorial.html):
The API is very similar to that of NetworkX. Much of the NetworkX tutorial at https://networkx.org/documentation/latest/tutorial.html is applicable to PyGraphviz. See http://pygraphviz.github.io/documentation/latest/reference/api_notes.html for major differences.
In addition, pydot as of now does not really have docs (which personally bothers me. Idk if it's that it doesn't look nice on my browser or that it makes me feel that project is not being taken seriously by it's developers idk something just doesn't feel right even if it has a higher set of users pydot 15k vs pygraphviz 4k) reference: https://github.com/pydot/pydot/pull/241.
Also it seems that pygraphviz has more granular control than regular graphviz ref: Graphviz vs PyGraphViz. In addition, I don't know how to convert a networkx directly to a graphviz obj (since graphviz has the best docs and highest user base ~19k so I did prefer that), so I will go with pygraphviz for those reasons. Pygravix also has docs which although small make me happy (though not as good as graphviz but idk how to make graphviz graphs from networkx). It's hard to make these decisions but I can't stay on this forever and this seems mindful enough. Also, networkx is nice because I can transform dgl graphs to networkx too (and the relabeling was simple).
Considering those reasons let me give you the example code I wrote that does what you want using pygraphviz (but you could do it with pydot if you figured out how, transforming to pydot obj is trivial using networkx see my previous link):
# https://stackoverflow.com/questions/28533111/plotting-networkx-graph-with-node-labels-defaulting-to-node-name
import dgl
import numpy as np
import torch
import networkx as nx
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
from pathlib import Path
g = dgl.graph(([0, 0, 0, 0, 0], [1, 2, 3, 4, 5]), num_nodes=6)
print(f'{g=}')
print(f'{g.edges()=}')
# Since the actual graph is undirected, we convert it for visualization purpose.
g = g.to_networkx().to_undirected()
print(f'{g=}')
# relabel
int2label = {0: "app", 1: "cons", 2: "with", 3: "app3", 4: "app4", 5: "app5"}
g = nx.relabel_nodes(g, int2label)
# https://networkx.org/documentation/stable/reference/drawing.html#module-networkx.drawing.layout
g = nx.nx_agraph.to_agraph(g)
print(f'{g=}')
print(f'{g.string()=}')
# draw
g.layout()
g.draw("file.png")
# https://stackoverflow.com/questions/20597088/display-a-png-image-from-python-on-mint-15-linux
img = mpimg.imread('file.png')
plt.imshow(img)
plt.show()
# remove file https://stackoverflow.com/questions/6996603/how-to-delete-a-file-or-folder
Path('./file.png').expanduser().unlink()
# import os
# os.remove('./file.png')
output:
g=Graph(num_nodes=6, num_edges=5,
ndata_schemes={}
edata_schemes={})
g.edges()=(tensor([0, 0, 0, 0, 0]), tensor([1, 2, 3, 4, 5]))
g=<networkx.classes.multigraph.MultiGraph object at 0x7f8443e94250>
g=<AGraph <Swig Object of type 'Agraph_t *' at 0x7f846117a930>>
g.string()='graph "" {\n\tapp -- cons [key=0,\n\tid=0];\napp -- with [key=0,\nid=1];\napp -- app3 [key=0,\nid=2];\napp -- app4 [key=0,\nid=3];\napp -- app5 [key=0,\nid=4];\n}\n'
though I want to leave this link about pydot visualization since it seems very useful in general: Display graph without saving using pydot and probably provides the pydot answer for others if they need it. Though, I'd love to see arguments in favour of pydot.
Edit1: if you want to plot by attributed and not by label, see this answer: NetworkX node attribute drawing note that relabeling the way I suggested does not always have the intended semantics (e.g. it might join two nodes that were NOT meant to be joined).
Edit2: if you want to plot the attribute instead without self loops happening by accident see this answer: Draw more information on graph\nodes using PyGraphviz
I have a graph that I would like to visualize using d3.js. Specifically, I have an adjacency matrix that looks as follows (selection of data, not the full set):
Node 1 Node 2 Weight
A B 1.0
A C 1.0
A D 1.5
B D 1.0
...
I would like to be able to use a Force-Directed Graph to display my network, specifically.
I know how to create a NetworkX graph, but I would like to be able to try this in d3.js, as I want to be able to add mouseover displays of node properties later on, which I think is possible using d3 but not possible using matplotlib.
I have tried using d3py, but I can't even get the basic example to work, and I read that the NetworkX example is broken and the repo hasn't been updated for a while. Does anybody else know if there's something else I could try, or if someone could point me in a first step direction?
There is a NetworkX example here that uses d3.js force layout :http://networkx.github.io/documentation/latest/examples/javascript/force.html
It just writes a JSON format graph object from networkx that d3.js can load in a web browser to get an interactive drawing. That code is a little fancier since it will also do the web browser part for you.
All of the code (including the d3 parts) you need to run it are in the repository https://github.com/networkx/networkx/tree/master/examples/javascript
If you're fine with an alternative JavaScript framework, we've been building http://www.github.com/graphistry/pygraphistry . The idea is to use WebGL to render the big graphs (pan/zoom/etc.), and offload most of the real-time compute (layout, filter, etc.) to a GPU cloud. There's an API underneath for controlling it similarly to D3. It's easier for most uses however, as it streamlines common big graph manipulations and analyses.