We Keep Coding

Networkx dfs_edges not functioning as intended by professor

The expected output from dfs_edges(G, 4) is correct, but it's off by the position of one edges and I can't figure out why
Input:
G = nx.Graph()
edges = [(1,2),(1,3),(1,5),(2,5),(2,6),(3,4),(3,5),(4,5),(4,6),(4,7),(5,6)]
root = 4
G.add_edges_from(edges)
printGraph(G)
Expected Output:
DFS traversal from from 4 = [(4, 3), (4, 7), (3, 1), (1, 2), (2, 5), (5, 6)]
My Code:
import networkx as nx
def printGraph(G):
print("DFS traversal from from 4 =", list(nx.dfs_edges(G,4)))
Result from code:
DFS traversal from from 4 = [(4, 3), (3, 1), (1, 2), (2, 5), (5, 6), (4, 7)]
The problem is the position of the (4, 7) edge, while I can manually place it where it should be, but that would mean it wouldn't work on the hidden text cases (Moodle)

This is your graph:
In a depth-first search starting at node 4, I would expect node 7 be be visited either first or last, not right after node 3. The output you're getting is valid for dfs_edges. The expected output is what's incorrect.

The answer got and expected are both correct, turns out he didn't want dfs_edges, he wanted the edges of the dfs tree instead, so this worked:
print("DFS traversal from from 4 =", nx.dfs_tree(G, root)).edges())

Finding a closed path from list of start and end nodes

I have a list of edges (E) of a graph with nodes V = [1,2,3,4,5,6]:
E = [(1,2), (1,5), (2,3), (3,1), (5,6), (6,1)]
where each tuple (a,b) refers to the start & end node of the edge respectively.
If I know the edges form a closed path in graph G, can I recover the path?
Note that E is not the set of all edges of the graph. Its just a set of edges.
In this example, the path would be 1->2->3->1->5->6->1
A naive approach, I can think of is using a tree where I start with a node, say 1, then I look at all tuples that start with 1, here, (1,2) and (1,5). Then I have two branches, and with nodes as 2 & 5, I continue the process till I end at the starting node at a branch.
How to code this efficiently in python?

The networkx package has a function that can generate the desired circuit for you in linear time...
It is possible, that construction of nx.MultiDiGraph() is slower and not such efficient, as desired in question, or usage of external packages for only one function is rather excessive. If it is so, there is another way.
Plan: firstly we will find some way from start_node to start_node, then we will insert all loops, that were not visited yet.
from itertools import chain
from collections import defaultdict, deque
from typing import Tuple, List, Iterable, Iterator, DefaultDict, Deque
def retrieve_closed_path(arcs: List[Tuple[int, int]], start_node: int = 1) -> Iterator[int]:
if not arcs:
return iter([])
# for each node `u` carries queue of its
# neighbours to be visited from node `u`
d: DefaultDict[int, Deque[int]] = defaultdict(deque)
for u, v in arcs:
# deque pop and append complexity is O(1)
d[u].append(v)
def _dfs(node) -> Iterator[int]:
out: Iterator[int] = iter([])
# guarantee, that all queues
# will be emptied at the end
while d[node]:
# chain returns an iterator and helps to
# avoid unnecessary memory reallocations
out = chain([node], _dfs(d[node].pop()), out)
# if we return in this loop from recursive call, then
# `out` already carries some (node, ...) and we need
# only to insert all other loops which start at `node`
return out
return chain(_dfs(start_node), [start_node])
def path_to_string(path: Iterable[int]) -> str:
return '->'.join(str(x) for x in path)
Examples:
E = [(1, 2), (2, 1)]
p = retrieve_closed_path(E, 1)
print(path_to_string(p))
>> 1->2->1
E = [(1, 2), (1, 5), (2, 3), (3, 1), (5, 6), (6, 1)]
p = retrieve_closed_path(E, 1)
print(path_to_string(p))
>> 1->5->6->1->2->3->1
E = [(1, 2), (2, 3), (3, 4), (4, 2), (2, 1)]
p = retrieve_closed_path(E, 1)
print(path_to_string(p))
>> 1->2->3->4->2->1
E = [(5, 1), (1, 5), (5, 2), (2, 5), (5, 1), (1, 4), (4, 5)]
p = retrieve_closed_path(E, 1)
print(path_to_string())
>> 1->4->5->1->5->2->5->1

You're looking for a directed Eulerian circuit in your (sub)graph. An Eulerian circuit is a trail that visits every edge exactly once.
The networkx package has a function that can generate the desired circuit for you in linear time:
import networkx as nx
edges = [(1,2), (1,5), (2,3), (3,1), (5,6), (6,1)]
G = nx.MultiDiGraph()
G.add_edges_from(edges)
# Prints [(1, 5), (5, 6), (6, 1), (1, 2), (2, 3), (3, 1)]
# which matches the desired output (as asked in the comments).
print([edge for edge in nx.algorithms.euler.eulerian_circuit(G)])
The documentation cites a 1973 paper, if you're interested in understanding how the algorithm works. You can also take a look at the source code here. Note that we're working with multigraphs here, since you can have multiple edges that have the same source and destination node. There are probably other implementations floating around on the Internet, but they may or may not work for multigraphs.

Get networkx subgraph containing all nodes in between

I have a networkx DiGraph and I want to extract a subgraph from it by passing in a list of nodes. The subgraph however can contain all nodes that might be in between the nodes that I have passed. I checked nx.subgraph() but it does not work like I intend to. As for a small example:
import networkx as nx
G = nx.DiGraph()
edges = [(7, 4), (3, 8), (3, 2), (3, 0), (3, 1), (7, 5), (7, 6), (7, 8)]
G.add_edges_from(edges)
H = get_subgraph(G, [0,6,7,8])
How can I write the function get_subgraph() so that H has the edges [(3, 8), (3, 0), (7, 6), (7, 8)]?
The subgraph I need is such that it contains all the nodes that are in the ougoing and incoming paths between the nodes that I pass in the get_subgraph()function.

A way to do this could be to find the longest path length between the specified set of nodes, and then find the corresponding induced subgraph containing all nodes in the path. However, being a directed graph, there will be no direct path between say nodes 3 and 7. So we need to find the paths in an undirected copy of the graph. Let's set up the problem:
G = nx.DiGraph()
edges = [(7, 4), (3, 8), (3, 2), (3, 0), (3, 1), (7, 5), (7, 6), (7, 8)]
G.add_edges_from(edges)
plt.figure(figsize=(10,6))
pos = nx.spring_layout(G, scale=20, k=3/np.sqrt(G.order()))
nx.draw(G, pos, node_color='lightblue',
with_labels=True,
node_size=1500,
arrowsize=20)
Now we ca obtain and undirected copy of the graph with nx.to_undirected and find all nx.shortest_path_length for the specified nodes:
from itertools import combinations
H = nx.to_undirected(G)
nodelist = [0,6,7,8]
paths = {}
for nodes in combinations(nodelist, r=2):
paths[nodes] = nx.shortest_path_length(H, *nodes)
print(paths)
# {(0, 6): 4, (0, 7): 3, (0, 8): 2, (6, 7): 1, (6, 8): 2, (7, 8): 1}
We can find the longest path in the undirected graph with:
max_path = max(paths.items(), key=lambda x: x[1])[0]
longest_induced_path = nx.shortest_path(H, *max_path)
And the corresponding induced subgraph can be obtained with Graph.subgraph:
sG = nx.subgraph(G, longest_induced_path)
pos = nx.spring_layout(sG, scale=20, k=3/np.sqrt(G.order()))
nx.draw(sG, pos, node_color='lightblue',
with_labels=True,
node_size=1500,
arrowsize=20)

i understand this from question:
you need all nodes in a path but provide some nodes of that path and algorithm should give all nodes of that path and then you can pass that nodes to a graph and make a new graph.
it should be what you want:
1. you must iterate over all pairs of nodes with this method:
from itertools import combinations
b= combinations('ABCD', 2)
print(list(b)) --> [('A', 'B'), ('A', 'C'), ('A', 'D'), ('B', 'C'), ('B', 'D'), ('C', 'D')]
you must get all pathes with this:
https://networkx.github.io/documentation/stable/reference/algorithms/simple_paths.html
you must select path with maximum nodes and that is your solution.

How to merge values from dictionary on different keys while iterating through it ? Finite element mesh merge algorithm

I am working on my phd and I am stuck on this step. The problem consists of implementing a finite element mesh merging algorithm and maybe my solution is not the best, so if you think of a better one I am open to suggestions.
Regarding the problem: I have a finite element mesh, which is composed of QUAD elements (squares with 4 nodes) and TRIA elements (triangles with 3 nodes). These elements are connected on edges, an edge is defined by 2 nodes (edge=[node1,node2]). I have a list of edges that I do not want to merge, but for the rest of the edges I want the program to merge the elements with the common edge.
As a simple example: assume I have 4 elements A,B,C and D (QUAD elms, defined by 4 nodes). The mesh looks something like this
1--------------2----------------3
| | |
| A | B |
| | |
4--------------5----------------6
| | |
| C | D |
| | |
7--------------8----------------9
These elements are defined in a dictionary:
mesh_dict={'A': [1,2,5,4], 'B':[2,3,6,5], 'C':[4,5,8,7],'D':[5,6,9,8]}
I also have a dictionary for the node position with values for X,Y,Z coordinates. Let's say I want to merge on edge [4,5] and [5,6].
My solution is the following: I start iterating through the elements in mesh_dict, I find the neighbors of the element with a function get_elm_neighbors(element), I check the angle between elements with function check_angle(elm1,elm2,angle) (I need the angle between elements to be below a certain threshold), than I check for which edge should be merged by get_edge_not_bar(), than I have a function which updates the nodes for the first element to complete the merging.
for e in mesh_dict:
if e not in delete_keys:
neighbors=get_elm_neighbors(e)
for key,value in neighbors.items():
check = check_angle(e,key,0.5)
if check:
nodes = get_edge_not_bar(value)
if nodes:
new_values=merge_elms(e,key,nodes)
d = {e: new_values}
mesh_dict_merged.update(d)
mesh_dict.update(d)
delete_keys.append(key)
My problem is that I need to delete the elements that remain after the merging. For example in the above case I start on element A and I merge on the edge [4,5], after that the elm A definition will be 'A':[1,2,8,7], then I need to delete elm C and proceed with the iteration.
My solution was to create a duplicate dictionary mesh_dict_merge in which I update the values for the elements and then delete the ones that I don't want to while iterating through the original dict but taking into consideration the deleted elements (deleted_keys list) to not go through them
I guess my question is if there is a way to iterate through the dictionary, update values and delete keys while doing so ? Or if there is a better solution to approach this problem, maybe iterate through nodes instead of elements ?
EDIT: changed 'A': [1,2,4,5] to 'A': [1,2,5,4]

It can be done updating the elements on-the-fly. But I should not recommend it because your algorithm will depend on the order you iterate the elements, and may be not deterministic. This mean that two meshes with identical geometry and topology could give different results depending on the labels you use.
The recommendation is :
Compute all dihedral angles in your mesh. Store those that are under your merge threshold.
Find the minimum angle and merge the two elements that share that edge.
Update the dihedral angles around the new element. This include removing angles from elements that have merged, and optionally include new angles for the new element.
Repeat from step 2 until every angle is over the threshold, or until the number of elements is the desired.
The optional part in step 3 allows to determine the aggressiveness of your method. Sometimes it is better not to include new angles and repeat several times the complete process to avoid focus the reduction too much in a zone.

I thought about how to find adjacent elements by finding elements that shared the same edge - but I had to have edges as a pair of end indices in sorted order.
I could then work out touches (should work for triangle elements too).
I introduce dont_merge as a set of ordered edge indices that cannot be merged away then merge into merged_ordered_edges and finally convert back to the mesh format of your original with edges going around each element.
I have commented out a call to check_angle(name1, name2) which you would have to add in. I assume that the check would succeed every time by the comment.
# -*- coding: utf-8 -*-
"""
Finite element mesh merge algorithm
https://stackoverflow.com/questions/59079755/how-to-merge-values-from-dictionary-on-different-keys-while-iterating-through-it
Created on Thu Nov 28 21:59:07 2019
#author: Paddy3118
"""
#%%
mesh_dict={'A': [1,2,5,4], 'B':[2,3,6,5], 'C':[4,5,8,7],'D':[5,6,9,8]}
#
ordered_edges = {k: {tuple(sorted(endpoints))
for endpoints in zip(v, v[1:] + v[:1])}
for k, v in mesh_dict.items()}
# = {'A': {(1, 2), (1, 4), (2, 5), (4, 5)},
# 'B': {(2, 3), (2, 5), (3, 6), (5, 6)},
# 'C': {(4, 5), (4, 7), (5, 8), (7, 8)},
# 'D': {(5, 6), (5, 8), (6, 9), (8, 9)}}
#%%
from collections import defaultdict
touching = defaultdict(list)
for name, edges in ordered_edges.items():
for edge in edges:
touching[edge].append(name)
touches = {edge: names
for edge, names in touching.items()
if len(names) > 1}
# = {(2, 5): ['A', 'B'],
# (4, 5): ['A', 'C'],
# (5, 6): ['B', 'D'],
# (5, 8): ['C', 'D']}
#%%
dont_merge = set([(4, 5), (23, 24)])
for edge, (name1, name2) in touches.items():
if (edge not in dont_merge
and ordered_edges[name1] and ordered_edges[name2]
#and check_angle(name1, name2)
):
# merge
ordered_edges[name1].update(ordered_edges[name2])
ordered_edges[name1].discard(edge) # that edge is merged away
ordered_edges[name2] = set() # gone
merged_ordered_edges = {}
for name, edges in ordered_edges.items():
if edges:
merged_ordered_edges[name] = sorted(edges)
edges.clear() # Only one name of shared object used
# = {'A': [(1, 2), (1, 4), (2, 3), (3, 6), (4, 5), (5, 6)],
# 'C': [(4, 5), (4, 7), (5, 6), (6, 9), (7, 8), (8, 9)]}
## You would then need a routine to change the ordered edges format
## back to your initial mesh_dict format that goes around the periphery
## (Or would you)?
#%%
def ordered_to_periphery(edges):
"""
In [124]: ordered_to_periphery([(1, 2), (1, 4), (2, 3), (3, 6), (4, 5), (5, 8), (6, 9), (8, 9)])
Out[124]: [(1, 2), (2, 3), (3, 6), (6, 9), (9, 8), (8, 5), (5, 4), (4, 1)]
"""
p = [edges.pop(0)] if edges else []
last = p[-1][-1] if p else None
while edges:
for n, (i, j) in enumerate(edges):
if i == last:
p.append((i, j))
last = j
edges.pop(n)
break
elif j == last:
p.append((j, i))
last = i
edges.pop(n)
break
return p
#%%
merged_mesh = {name: ordered_to_periphery(edges)
for name, edges in merged_ordered_edges.items()}
# = {'A': [(1, 2), (2, 3), (3, 6), (6, 5), (5, 4), (4, 1)],
# 'C': [(4, 5), (5, 6), (6, 9), (9, 8), (8, 7), (7, 4)]}
P.S. Any chance of a mention if you use this?

Strange interferences bewteen Heapq module and dictionary

On one hand, I have a grid defaultdict that stores the neighboring nodes of each node on a grid and its weight (all 1 in the example below).
node (w nbr_node)
grid = { 0: [(1, -5), (1, -4), (1, -3), (1, -1), (1, 1), (1, 3), (1, 4), (1, 5)],
1: [(1, -4), (1, -3), (1, -2), (1, 0), (1, 2), (1, 4), (1, 5), (1, 6)],
2: [(1, -3), (1, -2), (1, -1), (1, 1), (1, 3), (1, 5), (1, 6), (1, 7)],
3: [(1, -2), (1, -1), (1, 0), (1, 2), (1, 4), (1, 6), (1, 7), (1, 8)],
...
}
On the other, I have a Djisktra function that computes the shortest path between 2 nodes on this grid. The algorithm uses the heapq module and works perfectly fine.
import heapq
def Dijkstra(s, e, grid): #startpoint, endpoint, grid
visited = set()
distances = {s: 0}
p = {}
queue = [(0, s)]
while queue != []:
weight, node = heappop(queue)
if node in visited:
continue
visited.add(node)
for n_weight, n_node in grid[node]:
if n_node in visited:
continue
total = weight + n_weight
if n_node not in distances or distances[n_node] > total:
distances[n_node] = total
heappush(queue, (total, n_node))
p[n_node] = node
Problem: when calling the Djikstra function multiple times, heappush is... adding new keys in the grid dictionary for no reason !
Here is a MCVE:
from collections import defaultdict
# Creating the dictionnary
grid = defaultdict(list)
N = 4
kernel = (-N-1, -N, -N+1, -1, 1, N-1, N, N+1)
for i in range(N*N):
for n in kernel:
if i > N and i < (N*N) - 1 - N and (i%N) > 0 and (i%N) < N - 1:
grid[i].append((1, i+n))
# Calling Djikstra multiple times
keys = [*range(N*N)]
while keys:
k1, k2 = random.sample(keys, 2)
Dijkstra(k1, k2, grid)
keys.remove(k1)
keys.remove(k2)
The original grid defaultdict:
dict_keys([5, 6, 9, 10])
...and after calling the Djikstra function multiple times:
dict_keys([5, 6, 9, 10, 4, 0, 1, 2, 8, 3, 7, 11, 12, 13, 14, 15])
When calling the Djikstra function multiple times without heappush (just commenting heappush at the end):
dict_keys([5, 6, 9, 10])
Question:
How can I avoid this strange behavior ?
Please note that I'm using Python 2.7 and can't use numpy.

I could reproduce and fix. The problem is in the way you are building grid: it contains values that are not in keys from -4 to 0 and from 16 to 20 in the example. So you push those inexistant nodes on the head, and later pop them.
And you end in executing for n_weight, n_node in grid[node]: where node does not (still) exists in grid. As grid is a defaultdict, a new node is automatically inserted with an empty list as value.
The fix is trivial (at least for the example data): it is enough to ensure that all nodes added as value is grid exist as key with a modulo:
for i in range(N*N):
for n in kernel:
grid[i].append((1, (i+n + N + 1)%(N*N)))
But even for real data it should not be very hard to ensure that all nodes existing in grid values also exist in keys...
BTW, if grid had been a simple dict the error would have been immediate with a KeyError on grid[node].