Drawing massive networkx graph: Array too big - python

I'm trying to draw a networkx graph with weighted edges, but right now I'm having some difficulty.
As the title suggests, this graph is really huge:
Number of Nodes: 103362
Number of Edges: 1419671
And when I try to draw this graph with the following code:
pos = nx.spring_layout(G)
nx.draw(G, node_color='#A0CBE2',edge_color='#BB0000',width=2,edge_cmap=plt.cm.Blues,with_labels=False)
plt.savefig("edge_colormap.png") # save as png
plt.show() # display
(This is just me testing functionality, not my desired end result). I get the error:
ValueError: array is too big.
It's triggered from the spring_layout algorithm. Any idea what's causing this? Even when I use a different pos algorithm I get the same error, how can I avoid it (if I can)?
On another note, I want to colour the edges based on their weight. As you can see there are a lot of edges and probably a wide range of weights, what is the best way to do this?
Thanks for your patience.
EDIT FROM MY COMMENT:
I'm trying to investigate the density of the data I have. Basically I am looking at 50,000 matches each containing 10 players, and whenever two players meet in a game I +1 to the weight of the edge between them. The idea is that my end result will show me the strength of my data set. In my mind I want the heaviest edges at the centre and as we move out from the centre the data is less densely connected.

The problem lies in the spring_layout approach. With this many nodes it will take a while to calculate where all of them should go with respect to one another. With this many nodes I would suggest either figuring out the x,y positions yourself or plot much smaller subgraphs. (<5000 nodes or your computer might be a bit sluggish for a while.
Here is what 1000 nodes from an erdos_renyi_graph (randomly chosen edges) looks like.
It pulled off 2 nodes to highlight.
Next is what 1500 looks like
It got a little more detail. Now with 7-8 interesting nodes.
There isn't much to be gained by so many edges and so many nodes on a graph. And what happens if you don't like the output, you would need to re-run it again.
To get x,y positions of each node take a look at this. in NetworkX show graph with nodes at exact (x,y) position. Result is rotated

Related

Looking for python packages that can calculate nodes/region/edges while respecting X,Y coordinates of nodes

I have a list of cities (nodes) plotted in a 2D plane each given by an X,Y coordinate.
I now want to add roads (edges) to it, but the roads cannot intersect. I want to create the most number of roads possible. By count, not by total length.
In more general graph theory parlance, I think I want the maximum number of edges (or regions?? maybe it's the same thing), where edges do not intersect in 2-dimensions, for a given set of Nodes at X,Y points.
In a brief view of NetworkX, it seems that they generate Graphs by making "nodes" but nodes can be "anywhere" and cannot force nodes to be at a certain location with respect to each other (they have abstracted too far!).
Edit: networkx add_node with specific position
suggests that you can plot them in a given location. #Stef thanks!!
Am i thinking about the problem correctly?
Can I visualize using some python package my Nodes/edges, where this package can automatically calculate the proper edges given a set of nodes?
Is automatically finding the maximum number of non-intersecting edges a thing (and what is this called so I can find out more about it?)
Very possibly similar to this question, but this question wasn't really answered and from 8 years ago (Algorithm for finding minimal cycle basis of planar graph)

Generating a random planar graph in python

I am looking to generate a random planar graph in python with around 20 vertices. I checked out this planar graph generator but two problems emerged:
The algorithm on the aforementioned GitHub project seems a bit too overkill to generate a random planar graph that doesn’t have those many edges
Because it’s meant to generate massive graph, that algorithm is very complex, and therefore also a bit clunky and difficult to use
With that said, is there a simpler way to randomly generate a relatively small planar graph in python?
Create required number of nodes
Assign random x,y locations to the nodes.
WHILE nodes with no connected edges
Select N a random node with no edge
LOOP select M a different node at random
IF edge N-M does NOT intersect previous edges
Add N-M edge to graph
BREAK out of LOOP

Programming Combination Math (Using Python)

Traffic map Image
Traffic map contains straight segments of two types. The ones with arrows can only go one way in the direction of the arrow and those without arrows can go in two directions. Calculate the number of ways to go from A to B without any repeated lines?
With this math problem, how to solve it ? I don't know what to do right now !
It's a graph theory problem, in your problem try to consider every junction as a node and the segments as the edges of the graph, generally your graph is a directed-graph, when the segments that have two directions are just two edges from the same rwo nodes.
The algorithm you need to implement is DFS (Depth First Traversal).
The idea is as following:
Start the DFS traversal from source.
Keep storing the visited vertices in an array or HashMap say ‘path[]’.
If the destination vertex is reached, print contents of path[].
The important thing is to mark current vertices in the path[] as visited also so that the traversal doesn’t go in a cycle.

Tracking cycles while adding random edges to a sparse graph

Scenario: I have a graph, represented as a collection of nodes (0...n). There are no edges in this graph.
To this graph, I connect nodes at random, one at a time. An alternative way of saying this would be that I add random edges to the graph, one at a time.
I do not want to create simple cycles in this graph.
Is there a simple and/or very efficient way to track the creation of cycles as I add random edges? With a graph traversal, it is easy, since we only need to track the two end nodes of a single path. But, with this situation, we have any number of paths that we need to track - and sometimes these paths combine into a larger path, and we need to track that too.
I have tried several approaches, which mostly come down to maintaining a list of "outer nodes" and a set of nodes internal to them, and then when I add an edge going through it and updating it. But, it becomes extremely convoluted, especially if I remove an edge in the graph.
I have attempted to search out algorithms or discussions on this, and I can't really find anything. I know I can do a BFS to check for cycles, but it's so so so horribly inefficient to BFS after every single edge addition.
Possible solution I came up with while in the shower.
What I will do is maintain a list of size n, representing how many times that node has been on an edge.
When I add an edge (i,j), I will increment list[i] and list[j].
If after an edge addition, list[i] > 1, and list[j] > 1, I will do a DFS starting from that edge.
I realized I don't need to BFS, I only need to DFS from the last added edge, and I only need to do it if it at least has potential to be in a cycle (it's nodes show up twice).
I doubt it is optimal.. maybe some kind of list of disjoint sets would be better. But this is way better than anything I was thinking of before.
If you keep track of the connected components of your graph, you can test for every edge you insert whether the involved nodes are already in the same component. If they are, then the edge you are inserting will introduce a cycle to your graph.
Have a look at this post that seems to give some good references on how to do this: https://cstheory.stackexchange.com/questions/2548/is-there-an-online-algorithm-to-keep-track-of-components-in-a-changing-undirecte

Pipeline to create Voronoi Meshes

I would like to implement a Maya plugin (this question is independent from Maya) to create 3D Voronoi patterns, Something like
I just know that I have to start from point sampling (I implemented the adaptive poisson sampling algorithm described in this paper).
I thought that, from those points, I should create the 3D wire of the mesh applying Voronoi but the result was something different from what I expected.
Here are a few example of what I get handling the result i get from scipy.spatial.Voronoi like this (as suggested here):
vor = Voronoi(points)
for vpair in vor.ridge_vertices:
for i in range(len(vpair) - 1):
if all(x >= 0 for x in vpair):
v0 = vor.vertices[vpair[i]]
v1 = vor.vertices[vpair[i+1]]
create_line(v0.tolist(), v1.tolist())
The grey vertices are the sampled points (the original shape was a simple sphere):
Here is a more complex shape (an arm)
I am missing something? Can anyone suggest the proper pipeline and algorithms I have to implement to create such patterns?
I saw your question since you posted it but didn’t have a real answer for you, however as I see you still didn’t get any response I’ll at least write down some ideas from me. Unfortunately it’s still not a full solution for your problem.
For me it seems you’re mixing few separate problems in this question so it would help to break it down to few pieces:
Voronoi diagram:
The diagram is by definition infinite, so when you draw it directly you should expect a similar mess you’ve got on your second image, so this seems fine. I don’t know how the SciPy does that, but the implementation I’ve used flagged some edge ends as ‘infinite’ and provided me the edges direction, so I could clip it at some distance by myself. You’ll need to check the exact data you get from SciPy.
In the 3D world you’ll almost always want to remove such infinite areas to get any meaningful rendering, or at least remove the area that contains your camera.
Points generation:
The Poisson disc is fine as some sample data or for early R&D but it’s also the most boring one :). You’ll need more ways to generate input points.
I tried to imagine the input needed for your ball-like example and I came up with something like this:
Create two spheres of points, with the same center but different radius.
When you create a Voronoi diagram out of it and remove infinite areas you should end up with something like a football ball.
If you created both spheres randomly you’ll get very irregular boundaries of the ‘ball’, but if you scale the points of one sphere, to use for the 2nd one you should get a regular mesh, similar to ball. You can also use similar points, but add some random offset to control the level of surface irregularity.
Get your computed diagram and for each edge create few points along this edge - this will give you small areas building up the edges of bigger areas. Play with random offsets again. Try to ignore edges, that doesn't touch any infinite region to get result similar to your image.
Get the points from both stages and compute the diagram once more.
Mesh generation:
Up to now it didn’t look like your target images. In fact it may be really hard to do it with production quality (for a Maya plugin) but I see some tricks that may help.
What I would try first would be to get all my edges and extrude some circle along them. You may modulate circle size to make it slightly bigger at the ends. Then do Boolean ‘OR’ between all those meshes and some Mesh Smooth at the end.
This way may give you similar results but you’ll need to be careful at mesh intersections, they can get ugly and need some special treatment.

Categories