So it is clear with NetworkX that they use an algorithm in n^2 time to generate a random geometric graph. They say there is a faster algorithm possible with the use of K-D Trees. My question is how would one go about attempting to implement the K-D Tree version of this algorithm? I am not familiar with this data structure, nor would I call myself a python expert. Just trying to figure this out. All help is appreciated, thanks!
def random_geometric_graph(n, radius, dim=2, pos=None):
G=nx.Graph()
G.name="Random Geometric Graph"
G.add_nodes_from(range(n))
if pos is None:
# random positions
for n in G:
G.node[n]['pos']=[random.random() for i in range(0,dim)]
else:
nx.set_node_attributes(G,'pos',pos)
# connect nodes within "radius" of each other
# n^2 algorithm, could use a k-d tree implementation
nodes = G.nodes(data=True)
while nodes:
u,du = nodes.pop()
pu = du['pos']
for v,dv in nodes:
pv = dv['pos']
d = sum(((a-b)**2 for a,b in zip(pu,pv)))
if d <= radius**2:
G.add_edge(u,v)
return G
Here is a way that uses the scipy KD-tree implementation mentioned by #tcaswell above.
import numpy as np
from scipy import spatial
import networkx as nx
import matplotlib.pyplot as plt
nnodes = 100
r = 0.15
positions = np.random.rand(nnodes,2)
kdtree = spatial.KDTree(positions)
pairs = kdtree.query_pairs(r)
G = nx.Graph()
G.add_nodes_from(range(nnodes))
G.add_edges_from(list(pairs))
pos = dict(zip(range(nnodes),positions))
nx.draw(G,pos)
plt.show()
Related
I am trying to triangulate a large amount of massive data using Delaunay to scipy.spatial for triangulation and networkx to get the node adjacency relations. My code works very well on small data sets but when I try to introduce volumes of about 2 miollion points I always get the following error:
raise NetworkXError(f"The node {n} is not in the graph.") from e
NetworkXError: The node 1 is not in the graph.
It seems like my graph store the first node and nothing more. When I did my research I found that networkx is well adapted to massive data
Here is my code :
import numpy as np
import networkx as nx
import scipy.spatial
points = np.genfromtxt('las1.xyz', delimiter = ';')
xy= points[:,0:2]
z= points[:,2]
delTri = scipy.spatial.Delaunay(xy)
edges = set()
for n in range(delTri.nsimplex):
edge = sorted([delTri.vertices[n,0], delTri.vertices[n,1]])
edges.add((edge[0], edge[1]))
edge = sorted([delTri.vertices[n,0], delTri.vertices[n,2]])
edges.add((edge[0], edge[1]))
edge = sorted([delTri.vertices[n,1], delTri.vertices[n,2]])
edges.add((edge[0], edge[1]))
pts_neigh = {}
graph = nx.Graph(list(edges))
for i in range(len(xy)):
pts_neigh[i] = list(graph.neighbors(i))
I still get the edges list from my networkx graph but it seems like it fails at the level of constructing the nodes.
I will be so grateful for your help.
Although it's possible to instantiate graph with specific data, the syntax can be a bit complex. An easier option is to explicitly add edges from a list:
graph = nx.Graph()
graph.add_edges_from(list(edges))
I am trying to create a directed network with more than 5000 nodes. The edges between the nodes are based on the difference in a certain value assigned to each node; if the difference in values between node pairs is less than a threshold, there is an edge. I generate an adjacency matrix and want to check if the directed graph is weakly connected, and also compute Page rank. Currently, I use the code below to generate the graph and it takes me 78s and occupies nearly 7GB memory. I want to know if there is a more efficient (time and memory) way of constructing and evaluating large networks in Python.
%reset -f
!pip install faiss-gpu
import faiss
import numpy as np
import torch
import random
import networkx as nx
import time
device='cuda'
res = faiss.StandardGpuResources()
start=time.time()
# Total Nodes
N = 5000
# Mean
mu = 0.5*np.pi
# Variance
var = np.pi/18
# Maximum degree of each node
max_degree = 1000
# Threshold
value_thres = np.pi/6
# Placeholders
Values = torch.zeros((N,1),dtype=torch.double,device='cuda')
Matrixs = torch.zeros((2,N,max_degree),dtype=torch.double,device='cuda')
Adj_Matrix = torch.zeros((N,N),dtype=torch.long,device='cuda')
#Generate a directed network with N nodes whose connectivity is based on values
start_network=time.time()
Values[:,0] = torch.normal(mu,var,(N,))
# Find neighbors upto max_degree
# Pytorch to numpy
Current = np.float32(Values[:,0].cpu().detach().numpy())
index_flat = faiss.IndexFlatL2(Current[:,None].shape[1])
gpu_index_flat = faiss.index_cpu_to_gpu(res, 0, index_flat)
gpu_index_flat.add(Current[:,None])
m, n = gpu_index_flat.search(Current[:,None], max_degree)
# Indices of possible neighbors
Matrixs[1] = torch.from_numpy(n).long()
# Value Separation
Matrixs[0] = torch.squeeze(torch.cdist(Values[:,0][:,None][:,None],Values[:,0][:,None][Matrixs[1].long()],p=2))<value_thres
# Construct Adjacency Matrix
Adj_Matrix[Matrixs[1].long()] = 1
Adj_Matrix-=torch.eye(N,dtype=torch.long,device='cuda')
G = nx.from_numpy_matrix(Adj_Matrix.cpu().detach().numpy())
end=time.time()
print('Network Creation Time',end-start_network)
print('Total Time',end-start-start_network)
From the snippet in the question, it's hard to isolate the time/memory of networkx (I don't have 'cuda' on my machine, so unable to replicate). However, the following code runs for about 36 seconds:
import networkx as nx
import numpy as np
A = np.random.randint(2, size=(5000, 5000))
G = nx.from_numpy_matrix(A) # about 36 seconds
There could be scope for generating a faster algorithm by writing a custom low-level graph constructor, but it's unlikely that it will have memory advantages.
I need a graph with N clusters, that somewhat represents the structure of social networks. I planned to go about this by creating N graphs with barabasi albert structure, and then connecting them by a single node.
import networkx as nx
a = nx.barabasi_albert_graph(10,2)
b = nx.barabasi_albert_graph(10,2)
nx.draw(a)
nx.draw(b)
what I want is them connected like this:
But I cannot see or find any simple way of doing this, are there any networkX functionality that can do just this?
Joining two graphs by edge is really simple:
import matplotlib.pyplot as plt
import networkx as nx
a = nx.barabasi_albert_graph(10,2)
b = nx.barabasi_albert_graph(10,2)
c = nx.union(a,b, rename=('a-', 'b-'))
c.add_edge('a-0', 'b-0')
nx.draw_networkx(c,with_labels=True,node_size=500)
plt.show()
And if you want to merge graphs on a common node (this is stated in your question contrary to the title), you can do this:
import matplotlib.pyplot as plt
import networkx as nx
a = nx.barabasi_albert_graph(10,2)
b = nx.barabasi_albert_graph(10,2)
a= nx.relabel_nodes(a, { n: str(n) if n==0 else 'a-'+str(n) for n in a.nodes })
b= nx.relabel_nodes(b, { n: str(n) if n==0 else 'b-'+str(n) for n in b.nodes })
c = nx.compose(a,b)
nx.draw_networkx(c,with_labels=True,node_size=500)
plt.show()
How can I draw a graph with it's communities using python networkx like this image :
image url
The documentation for networkx.draw_networkx_nodes and networkx.draw_networkx_edges explains how to set the node and edge colors. The patches bounding the communities can be made by finding the positions of the nodes for each community and then drawing a patch (e.g. matplotlib.patches.Circle) that contains all positions (and then some).
The hard bit is the graph layout / setting the node positions.
AFAIK, there is no routine in networkx to achieve the desired graph layout "out of the box". What you want to do is the following:
Position the communities with respect to each other: create a new, weighted graph, where each node corresponds to a community, and the weights correspond to the number of edges between communities. Get a decent layout with your favourite graph layout algorithm (e.g.spring_layout).
Position the nodes within each community: for each community, create a new graph. Find a layout for the subgraph.
Combine node positions in 1) and 3). E.g. scale community positions calculated in 1) by a factor of 10; add those values to the positions of all nodes (as computed in 2)) within that community.
I have been wanting to implement this for a while. I might do it later today or over the weekend.
EDIT:
Voila. Now you just need to draw your favourite patch around (behind) the nodes.
import numpy as np
import matplotlib.pyplot as plt
import networkx as nx
def community_layout(g, partition):
"""
Compute the layout for a modular graph.
Arguments:
----------
g -- networkx.Graph or networkx.DiGraph instance
graph to plot
partition -- dict mapping int node -> int community
graph partitions
Returns:
--------
pos -- dict mapping int node -> (float x, float y)
node positions
"""
pos_communities = _position_communities(g, partition, scale=3.)
pos_nodes = _position_nodes(g, partition, scale=1.)
# combine positions
pos = dict()
for node in g.nodes():
pos[node] = pos_communities[node] + pos_nodes[node]
return pos
def _position_communities(g, partition, **kwargs):
# create a weighted graph, in which each node corresponds to a community,
# and each edge weight to the number of edges between communities
between_community_edges = _find_between_community_edges(g, partition)
communities = set(partition.values())
hypergraph = nx.DiGraph()
hypergraph.add_nodes_from(communities)
for (ci, cj), edges in between_community_edges.items():
hypergraph.add_edge(ci, cj, weight=len(edges))
# find layout for communities
pos_communities = nx.spring_layout(hypergraph, **kwargs)
# set node positions to position of community
pos = dict()
for node, community in partition.items():
pos[node] = pos_communities[community]
return pos
def _find_between_community_edges(g, partition):
edges = dict()
for (ni, nj) in g.edges():
ci = partition[ni]
cj = partition[nj]
if ci != cj:
try:
edges[(ci, cj)] += [(ni, nj)]
except KeyError:
edges[(ci, cj)] = [(ni, nj)]
return edges
def _position_nodes(g, partition, **kwargs):
"""
Positions nodes within communities.
"""
communities = dict()
for node, community in partition.items():
try:
communities[community] += [node]
except KeyError:
communities[community] = [node]
pos = dict()
for ci, nodes in communities.items():
subgraph = g.subgraph(nodes)
pos_subgraph = nx.spring_layout(subgraph, **kwargs)
pos.update(pos_subgraph)
return pos
def test():
# to install networkx 2.0 compatible version of python-louvain use:
# pip install -U git+https://github.com/taynaud/python-louvain.git#networkx2
from community import community_louvain
g = nx.karate_club_graph()
partition = community_louvain.best_partition(g)
pos = community_layout(g, partition)
nx.draw(g, pos, node_color=list(partition.values())); plt.show()
return
Addendum
Although the general idea is sound, my old implementation above has a few issues. Most importantly, the implementation doesn't work very well for unevenly sized communities. Specifically, _position_communities gives each community the same amount of real estate on the canvas. If some of the communities are much larger than others, these communities end up being compressed into the same amount of space as the small communities. Obviously, this does not reflect the structure of the graph very well.
I have written a library for visualizing networks, which is called netgraph. It includes an improved version of the community layout routine outlined above, which also considers the sizes of the communities when arranging them. It is fully compatible with networkx and igraph Graph objects, so it should be easy and fast to make great looking graphs (at least that is the idea).
import matplotlib.pyplot as plt
import networkx as nx
# installation easiest via pip:
# pip install netgraph
from netgraph import Graph
# create a modular graph
partition_sizes = [10, 20, 30, 40]
g = nx.random_partition_graph(partition_sizes, 0.5, 0.1)
# since we created the graph, we know the best partition:
node_to_community = dict()
node = 0
for community_id, size in enumerate(partition_sizes):
for _ in range(size):
node_to_community[node] = community_id
node += 1
# # alternatively, we can infer the best partition using Louvain:
# from community import community_louvain
# node_to_community = community_louvain.best_partition(g)
community_to_color = {
0 : 'tab:blue',
1 : 'tab:orange',
2 : 'tab:green',
3 : 'tab:red',
}
node_color = {node: community_to_color[community_id] for node, community_id in node_to_community.items()}
Graph(g,
node_color=node_color, node_edge_width=0, edge_alpha=0.1,
node_layout='community', node_layout_kwargs=dict(node_to_community=node_to_community),
edge_layout='bundled', edge_layout_kwargs=dict(k=2000),
)
plt.show()
This question already has answers here:
Bipartite graph in NetworkX
(4 answers)
Closed 7 years ago.
I have an n1-by-n2 bi-adjacency matrix A of a bipartite graph. The matrix A is a scipy.sparse csc matrix. I would like to plot the bipartite graph using A in networkx. Assume that the nodes are colored according to their class labels called node_class. I could do the following:
import networkx as nx
G = nx.from_numpy_matrix(A)
graph_pos = nx.fruchterman_reingold_layout(G)
degree = nx.degree(G)
nx.draw(G, node_color = node_class, with_labels = False, node_size = [v * 35 for v in degree.values()])
The above code works fine for a square dense adjacency matrix. However not for a non-square bi-adjacency matrix A. The error is:
'Adjacency matrix is not square.'
Moreover the matrix A I have is a scipy.sparse matrix` because it is very large and have lots of zeros. So I would want to avoid making an (n1+n2)-by-(n1+n2) adjacency matrix by stacking A and adding zeros.
I checked the documentation of NetworkX for bipartite graphs, it does not mention how to plot bi-partite graph using bi-adjacency matrix, or create a graph using bi-adjacency sparse matrix. If someone could tell me how to plot the bipartite graph, that would be great!
I don't believe there is a NetworkX function that creates a graph from a biadjacency matrix, so you'll have to write your own. (However, they do have a bipartite module you should check out.)
Here's one way to define a function that takes a sparse biadjacency matrix and converts it to a NetworkX graph (see the comments for explanation).
# Input: M scipy.sparse.csc_matrix
# Output: NetworkX Graph
def nx_graph_from_biadjacency_matrix(M):
# Give names to the nodes in the two node sets
U = [ "u{}".format(i) for i in range(M.shape[0]) ]
V = [ "v{}".format(i) for i in range(M.shape[1]) ]
# Create the graph and add each set of nodes
G = nx.Graph()
G.add_nodes_from(U, bipartite=0)
G.add_nodes_from(V, bipartite=1)
# Find the non-zero indices in the biadjacency matrix to connect
# those nodes
G.add_edges_from([ (U[i], V[j]) for i, j in zip(*M.nonzero()) ])
return G
See an example use case below, where I use nx.complete_bipartite_graph to generate a complete graph:
import networkx as nx, numpy as np
from networkx.algorithms import bipartite
from scipy.sparse import csc_matrix
import matplotlib.pyplot as plt
RB = nx.complete_bipartite_graph(3, 2)
A = csc_matrix(bipartite.biadjacency_matrix(RB, row_order=bipartite.sets(RB)[0]))
G = nx_graph_from_biadjacency_matrix(A)
nx.draw_circular(G, node_color = "red", with_labels = True)
plt.show()
And here's the output graph:
Here is a simple example:
import networkx as nx
import matplotlib.pyplot as plt
from networkx.algorithms import matching
%matplotlib inline
ls=[
[0,0,0,1,1],
[1,0,0,0,0],
[1,0,1,0,0],
[0,1,1,0,0],
[1,0,0,0,0]
]
g = nx.Graph()
a=['a'+str(i) for i in range(len(ls))]
b=['b'+str(j) for j in range(len(ls[0]))]
g.add_nodes_from(a,bipartite=0)
g.add_nodes_from(b,bipartite=1)
for i in range(len(ls)):
for j in range(len(ls[i])):
if ls[i][j] != 0:
g.add_edge(a[i], b[j])
pos_a={}
x=0.100
const=0.100
y=1.0
for i in range(len(a)):
pos_a[a[i]]=[x,y-i*const]
xb=0.500
pos_b={}
for i in range(len(b)):
pos_b[b[i]]=[xb,y-i*const]
nx.draw_networkx_nodes(g,pos_a,nodelist=a,node_color='r',node_size=300,alpha=0.8)
nx.draw_networkx_nodes(g,pos_b,nodelist=b,node_color='b',node_size=300,alpha=0.8)
# edges
pos={}
pos.update(pos_a)
pos.update(pos_b)
#nx.draw_networkx_edges(g,pos,edgelist=nx.edges(g),width=1,alpha=0.8,edge_color='g')
nx.draw_networkx_labels(g,pos,font_size=10,font_family='sans-serif')
m=matching.maximal_matching(g)
nx.draw_networkx_edges(g,pos,edgelist=m,width=1,alpha=0.8,edge_color='k')
plt.show()