We Keep Coding

Networkx - problem while working big data

I am trying to triangulate a large amount of massive data using Delaunay to scipy.spatial for triangulation and networkx to get the node adjacency relations. My code works very well on small data sets but when I try to introduce volumes of about 2 miollion points I always get the following error:
raise NetworkXError(f"The node {n} is not in the graph.") from e
NetworkXError: The node 1 is not in the graph.
It seems like my graph store the first node and nothing more. When I did my research I found that networkx is well adapted to massive data
Here is my code :
import numpy as np
import networkx as nx
import scipy.spatial
points = np.genfromtxt('las1.xyz', delimiter = ';')
xy= points[:,0:2]
z= points[:,2]
delTri = scipy.spatial.Delaunay(xy)
edges = set()
for n in range(delTri.nsimplex):
edge = sorted([delTri.vertices[n,0], delTri.vertices[n,1]])
edges.add((edge[0], edge[1]))
edge = sorted([delTri.vertices[n,0], delTri.vertices[n,2]])
edges.add((edge[0], edge[1]))
edge = sorted([delTri.vertices[n,1], delTri.vertices[n,2]])
edges.add((edge[0], edge[1]))
pts_neigh = {}
graph = nx.Graph(list(edges))
for i in range(len(xy)):
pts_neigh[i] = list(graph.neighbors(i))
I still get the edges list from my networkx graph but it seems like it fails at the level of constructing the nodes.
I will be so grateful for your help.

Although it's possible to instantiate graph with specific data, the syntax can be a bit complex. An easier option is to explicitly add edges from a list:
graph = nx.Graph()
graph.add_edges_from(list(edges))

Python Netgraph Interactive Labels

I am trying to create an interactive plot using netgraph and networkx.
I would like the plot to allow movement of the nodes and as the nodes move, the edges and edge_labels will also dynamically update.
Moving the nodes was addressed by the author of netgraph here . Now when I make a simpler plot and try to label an edge, the label stays static, and is sometimes not even on an edge.
It seems like handling edge_positions similar to node_positions on the last two lines should at least address the label not moving. Why the label isn't anchored to a particular edge is still puzzling. Does anyone know if the desired effect is achievable?
Here is a snip before moving anything:
Here is a snip after moving the bottom-right node to the top left:
Here is my current code:
import matplotlib.pyplot as plt
import networkx as nx
import netgraph # pip install netgraph
#Graph creation:
G=nx.Graph(type="")
for i in range(6):
G.add_node(i,shape="o")
#Changing shape for two nodes
G.nodes[1]['shape'] = "v"
G.nodes[5]['shape'] = "v"
#Add edges
G.add_edge(1,2)
G.add_edge(4,5)
G.add_edge(0,4)
G.add_edge(2,3)
G.add_edge(2,4)
labs={(1,2):"1 to 2"}
nx.draw_networkx_edge_labels(G, pos=nx.spring_layout(G),edge_labels=labs)
#Get node shapes
node_shapes = nx.get_node_attributes(G,"shape")
# Create an interactive plot.
# NOTE: you must retain a reference to the object instance!
# Otherwise the whole thing will be garbage collected after the initial draw
# and you won't be able to move the plot elements around.
pos = nx.layout.spring_layout(G)
######## drag nodes around #########
# To access the new node positions:
plot_instance = netgraph.InteractiveGraph(G, node_shape=node_shapes, node_positions=pos, edge_positions=pos)
node_positions = plot_instance.node_positions
edge_positions = plot_instance.edge_positions

To have it as formal answer, you need to add to the InteractiveGraph object the information that you want to draw (and move) the edge labels, i.e. the following
netgraph.InteractiveGraph(G,
node_shape=node_shapes,
node_positions=pos,
edge_positions=pos,
edge_labels=labs)
(With emphasis on the last parameter)
As you already, noticed then you don't need the call of nx.draw_networkx_edge_labels.

[networkx]Visualise some points on a graph, the points move along the link, but they are not nodes

I want to visualise some points on a graph, the points move along the link, but they are not nodes. Currently I have added some point location, but can not display them on the figure.
This is the code
# -- coding: utf-8 --
import networkx as nx
import matplotlib.pyplot as plt
import itertools
import math
#from mesa.space import NetworkGrid
#from mesa import Agent, Model
#from mesa.time import RandomActivation
#from mesa.datacollection import DataCollector
#from mesa.space import NetworkGrid
#%%Build a graph
G=nx.Graph()
G.add_node("GPs")
G.add_node("AcuteCares")
G.add_node("Waitlists")
G.add_node("newPatients")
G.add_node("Preventabledeaths")
G.add_node("ReviewPatients")
G.add_node("DeathPools")
G.add_node("DNAPool1s")
G.add_node("DNAPool2s")
G.add_node("UntreatedPopulations")
G.add_node("SAPops")
labeldict = {}
labeldict["GPs"] = "GP"
labeldict["AcuteCares"] = "Acute Care"
labeldict["Waitlists"] = "Waitlist"
labeldict["newPatients"] = "New Patients"
labeldict["Preventabledeaths"] = "Preventable Deaths"
labeldict["ReviewPatients"] = "Review Patients"
labeldict["DeathPools"] = "Natural Deaths"
labeldict["DNAPool1s"] = "First DNA"
labeldict["DNAPool2s"] = "Second DNA"
labeldict["UntreatedPopulations"] = "Untreated Population"
labeldict["SAPops"] = "General Population"
G.node["Preventabledeaths"]['pos']=(0,6)
G.node["ReviewPatients"]['pos']=(-3,5)
G.node["UntreatedPopulations"]['pos']=(3,5)
G.node["DNAPool2s"]['pos']=(-3,3)
G.node["Waitlists"]['pos']=(3,3)
G.node["AcuteCares"]['pos']=(-5,0)
G.node["DNAPool1s"]['pos']=(5,0)
G.node["GPs"]['pos']=(-3,-5)
G.node["DeathPools"]['pos']=(3,-5)
G.node["SAPops"]['pos']=(-3,-3)
G.node["newPatients"]['pos']=(3,-3)
edges=itertools.permutations(G.nodes(),2)
G.add_edges_from(edges)
pos=nx.get_node_attributes(G,'pos')
nx.draw(G,pos,labels=labeldict, with_labels = True)
plt.show()
#grid = NetworkGrid(G)
# %%
def arclen(edge):
"""
calculate the length of an edge. The format of edge is like: ('UntreatedPopulations', 'SAPops')
"""
dist_edge = math.sqrt((G.node[edge[0]]['pos'][0] - G.node[edge[1]]['pos'][0])**2 + (G.node[edge[0]]['pos'][1] - G.node[edge[1]]['pos'][1])**2)
return dist_edge
def patientcor(speed,step,edge):
"""get the coordinate of point along the edge, speed is the moving speed per step,
time is the number of steps, edge is the specific edge
"""
x=G.node[edge[0]]['pos'][0] + speed*step/arclen(edge) *(G.node[edge[1]]['pos'][0] -G.node[edge[0]]['pos'][0])
y=G.node[edge[0]]['pos'][1] + speed*step/arclen(edge) *(G.node[edge[1]]['pos'][1] -G.node[edge[0]]['pos'][1])
return (x,y)
#%% Visualise the graph, set the speed at 0.2, time is 0,1,2
edge=("SAPops","GPs")
for t in range(3):
pos[t]=patientcor(0.2, t,edge) #add the location of point on the link per step to the dict
nx.draw(G,pos, labels=labeldict,with_labels = True) #visualise pos dict along with the graph, but the additional points other than nodes do not appear on the figure
plt.show()
The graph figure only displays the nodes, but not the points that move along the edges:

The nx.draw command will only plot those nodes that are in the graph. If your dictionary pos provides locations of other points, it will silently ignore them. I believe this is the appropriate behavior and I can think of lots of times where my coding would be much more difficult if it would also plot other points that appeared in my pos dictionary.
For what you want, simply create a new list of the points you want to plot (or in your example it looks like just a single point). Then use matplotlib's scatter command.
#stuff skipped here
edge=("SAPops","GPs")
for t in range(3):
mypoint = patientcor(0.2, t,edge)
nx.draw(G,pos, labels=labeldict,with_labels = True) #visualise pos dict along with the graph, but the additional points other than nodes do not appear on the figure
plt.scatter([mypoint[0]], [mypoint[1]])
plt.show()
You'll probably want to play with the node sizes and specific locations of these points.

Is it possible to control the order that nodes are drawn using NetworkX in python?

I have a large graph object with many nodes that I am trying to graph. Due to the large number of nodes, many are being drawn one over another. This in itself is not a problem. However, a small percentage of nodes have node attributes which dictate their colour.
Ideally I would be able to draw the graph in such a way that nodes with this property are drawn last, on top of the other nodes, so that it is possible to see their distribution across the graph.
The code I have so far used to generate the graph is shown below:
import networkx as nx
import matplotlib.pyplot as plt
import numpy as np
import os
import pickle
from pathlib import Path
def openFileAtPath(filePath):
print('Opening file at: ' + filePath)
with open(filePath, 'rb') as input:
file = pickle.load(input)
return file
# Pre manipulation path
g = openFileAtPath('../initialGraphs/wordNetadj_dictionary1.11.pkl')
# Post manipulation path
# g = openFileAtPath('../manipulatedGraphs/wordNetadj_dictionary1.11.pkl')
print('Fetching SO scores')
scores = list()
for node in g.nodes:
scores.append(g.node[node]['weight'])
print('Drawing network')
nx.draw(g,
with_labels=False,
cmap=plt.get_cmap('RdBu'),
node_color=scores,
node_size=40,
font_size=8)
plt.show()
And currently the output is as shown:
This graph object itself has taken a relatively long time to generate and is computationally intensive, so ideally I wouldn't have to remake the graph from scratch.
However, I am fairly sure that the graph is drawn in the same order that the nodes were added to the graph object. I have searched for a way of changing the order that the nodes are stored within the graph object, but given directional graphs actually have an order, my searches always end up with answers showing me how to reverse the direction of a graph.
So, is there a way to dictate the order in which nodes are drawn, or alternatively, change the order that nodes are stored inside some graph object.
Potentially worthy of a second question, but the edges are also blocked out by the large number of nodes. Is there a way to draw the edges above the nodes behind them?

Piggybacking off Paul Brodersen's answer, if you want different nodes to be in the foreground and background, I think you should do the following:
For all nodes that belong in the same layer, draw the subgraph corresponding to the nodes, and set the , as follows:
pos = {...} # some dictionary of node positions, required for the function below
H = G.subgraph(nbunch)
collection = nx.draw_networkx_nodes(H, pos)
collection.set_zorder(zorder)
Do this for every group of nodes that belong in the same level. It's tedious, but it will do the trick. Here is a toy example that I created based on looking up this question as part of my own research
import matplotlib as mpl
mpl.use('agg')
import pylab
import networkx as nx
G = nx.Graph()
G.add_path([1, 2, 3, 4])
pos = {1 : (0, 0), 2 : (0.5, 0), 3 : (1, 0), 4 : (1.5, 0)}
for node in G.nodes():
H = G.subgraph([node])
collection = nx.draw_networkx_nodes(H, pos)
collection.set_zorder(node)
pylab.plot([0, 2], [0, 0], zorder=2.5)
pylab.savefig('nodes_zorder.pdf', format='pdf')
pylab.close()
This makes a graph, and then puts the each node at a successively higher level going from left to right, so the leftmost node is farthest in the background and the rightmost node is farthest in the foreground. It then draws a straight line whose zorder is 2. As a result, it comes in front of the two left nodes, and behind the two right nodes. Here is the result.

draw is a wrapper around draw_networkx_nodes and draw_networkx_edges.
Unlike draw, the two functions return their respective artists ( PathCollection and LineCollection, IIRC). These are your standard matplotlib artists, and as as such their relative draw order can be controlled via their zorder attribute.

Drawing nested networkx graphs

I would like to know if there is a way to draw nested networkx graphs in python.
I can successfully draw these graphs using the nx.draw_(...) method call as described in the networkx docs, but the case I'm using it for requires that one of the nodes itself is a graph (Imagine a network of rooms, at the top level with a network of areas/zones within those rooms at the next level down). I would like to show this using matplotlib or similar.
Any ideas would be appreciated.

Edit
You can probably do better than my original answer by defining a recursive function. Here is a rough outline of how that recursive function would look. My answer below gives a less elegant approach that can be easily tuned for a specific case, but if you're ever doing this frequently, you'll probably want this recursive version.
def recursive_draw(G,currentscalefactor=0.1,center_loc=(0,0),nodesize=300, shrink=0.1):
pos = nx.spring_layout(G)
scale(pos,currentscalefactor) #rescale distances to be smaller
shift(pos,center_loc) #you'll have to write your own code to shift all positions to be centered at center_loc
nx.draw(G,pos=pos, nodesize=nodesize)
for node in G.nodes_iter():
if type(node)==Graph: # or diGraph etc...
recursive_draw(node,currentscalefactor=shrink*currentscalefactor,center_loc=pos[node], nodesize = nodesize*shrink, shrink=shrink)
If anyone creates the recursive function, please add it as a separate answer, and give me a comment. I'll point to it from this answer.
Original answer
Here's a first pass (I'll hopefully edit to a complete answer by end of day, but I think this will get you most of the way there):
import networkx as nx
import pylab as py
G = nx.Graph()
H = nx.Graph()
H.add_edges_from([(1,2), (2,3), (1,3)])
I = nx.Graph()
I.add_edges_from([(1,3), (3,2)])
G.add_edge(H,I)
Gpos = nx.spring_layout(G)
Hpos = nx.spring_layout(H)
Ipos = nx.spring_layout(I)
scalefactor = 0.1
for node in H.nodes():
Hpos[node] = Hpos[node]*scalefactor + Gpos[H]
for node in I.nodes():
Ipos[node] = Ipos[node]*scalefactor + Gpos[I]
nx.draw_networkx_edges(G, pos = Gpos)
nx.draw_networkx_nodes(G, pos = Gpos, node_color = 'b', node_size = 15000, alpha = 0.5)
nx.draw(H, pos = Hpos, with_labels = True)
nx.draw(I, pos = Ipos, with_labels = True)
py.savefig('tmp.png')
The main additional thing I think you should do is to center each subnode. This would require identifying xmin, xmax, ymin, and ymax for each subplot and adjusting. You may also want to play with the scalefactor.