We Keep Coding

Load a Graph from .osm file using Osmnx/Python

I want to load a graph from XML, i.e. .osm file, using Osmnx Python library.
The .osm file contains roads not connected each other, for example only highway=primary and highway=primary_link of a country's region.
I use the parameter retain_all to avoid discarding all the roads, since
retain_all: if True, return the entire graph even if it is not connected. otherwise, retain only the largest weakly connected component.
I use this instruction:
graph = ox.graph_from_xml('temp.osm', retain_all=True)
But I get the following error
AttributeError: 'float' object has no attribute 'deg2rad'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "D:\code.py", line 37, in <module>
graph = ox.graph_from_xml('temp.osm', retain_all=True)
File "D:\Python\lib\site-packages\osmnx\graph.py", line 518, in graph_from_xml
G = _create_graph(response_jsons, bidirectional=bidirectional, retain_all=retain_all)
File "D:\Python\lib\site-packages\osmnx\graph.py", line 587, in _create_graph
G = distance.add_edge_lengths(G)
File "D:\Python\lib\site-packages\osmnx\distance.py", line 154, in add_edge_lengths
dists = great_circle_vec(c[:, 0], c[:, 1], c[:, 2], c[:, 3]).round(precision)
File "D:\Python\lib\site-packages\osmnx\distance.py", line 60, in great_circle_vec
y1 = np.deg2rad(lat1)
TypeError: loop of ufunc does not support argument 0 of type float which has no callable deg2rad method
If I remove retain_all parameter, of course, the error does not occurr but the graph will contain only one primary road.
How can I keep all the roads even if not connected in the map?

I forgot to post my solution. I solved using another Python library, called Pyrosm:
osm = OSM('temp.pbf')
nodes, edges = osm.get_network(nodes=True, network_type='driving')
graph = osm.to_graph(nodes, edges, graph_type='networkx', retain_all=True)

How to plot frequency spectrum graph by reading my wav file with FFT

import matplotlib.pyplot as plt
from scipy.fftpack import fft
from scipy.io import wavfile # get the api
fs, data = wavfile.read('output.wav') # load the data
a = data.T[0] # this is a two channel soundtrack, I get the first track
b = [(ele/2**8.)*2-1 for ele in a] # this is 8-bit track, b is now normalized on [-1,1)
c = fft(b) # calculate fourier transform (complex numbers list)
d = len(c)/2 # you only need half of the fft list (real signal symmetry)
plt.plot(abs(c[:(d-1)]), 'r')
plt.show()
I GOT ERROR
C:\Users\isaco\PycharmProjects\test1\venv\Scripts\python.exe
C:/Users/isaco/PycharmProjects/test1/main.py
Traceback (most recent call last):
File "C:\Users\isaco\PycharmProjects\test1\main.py", line 9, in
plt.plot(abs(c[:(d-1)]), 'r')
TypeError: slice indices must be integers or None or have an __index__ method
Process finished with exit code 1
2. ON MY OTHER COMPUTER
I tried running this code the same way on my other computer. I got a different error on the other computer.
C:\Users\isaco\PycharmProjects\pythonProject1\venv\Scripts\python.exe
C:/Users/isaco/PycharmProjects/pythonProject1/main.py
Traceback (most recent call last):
File "C:\Users\isaco\PycharmProjects\pythonProject1\main.py", line 7, in
b = [(ele/2**8.)*2-1 for ele in a] # this is 8-bit track, b is now normalized on [-1,1)
TypeError: 'numpy.int16' object is not iterable
Process finished with exit code 1
I'm new to Python. Why am I getting two different errors? I am in the learning process. I'm doing a project on Digital Signal Processing.
After reading the output.wav file where I recorded my voice, I need to calculate it with FFT and then plot the frequency spectrum on the graph.
I couldn't find the source of the error. How can I fix ? Could you help ?

Adjusting the width of edges in a python graphviz graph

I am trying to visualize a transition probability matrix for a finite Markov chain using the python interface to graphviz. I want the states of the Markov chain to be nodes in the graph, and I want the edges of the graph to have width proportional to the conditional probability of a transition between states. I.e. I want thick edges drawn for edges with big weights and skinny ones for edges with small weights.
The discussion at (directed weighted graph from pandas dataframe)
is similar to what I want, but it would present transition probability information as textual labels rather than by edge width, which would lead to an unhelpful and difficult-to-read graph.
I am happy to consider tools other than graphviz for this task.
Here is the class I'm trying to build:
import graphviz
import matplotlib.pyplot as plt
import numpy as np
class MarkovViz:
"""
Visualize the transition probability matrix of a Markov chain as a directed
graph, where the width of an edge is proportional to the transition
probability between two states.
"""
def __init__(self, transition_probability_matrix=None):
self._graph = None
if transition_probability_matrix is not None:
self.build_from_matrix(transition_probability_matrix)
def build_from_matrix(self, trans, labels=None):
"""
Args:
trans: A pd.DataFrame or 2D np.array. A square matrix containing the
conditional probabability of a transition from the level
represented by the row to the level represented by the column.
Each row sums to 1.
labels: A list-like sequence of labels to use for the rows and
columns of 'trans'. If trans is a pd.DataFrame or similar then
this entry can be None and labels will be taken from the column
names of 'trans'.
Effects:
self._graph is created as a directed graph, and populated with nodes
and edges, with edge weights taken from 'trans'.
"""
if labels is None and hasattr(trans, "columns"):
labels = list(trans.columns)
index = list(trans.index)
if labels != index:
raise Exception("Mismatch between index and columns of "
"the transition probability matrix.")
trans = trans.values
trans = np.array(trans)
self._graph = graphviz.Digraph("MyGraph")
dim = trans.shape[0]
if trans.shape[1] != dim:
raise Exception("Matrix must be symmetric")
for i in range(dim):
for j in range(dim):
if trans[i, j] > 0:
self._graph.edge(labels[i], labels[j], weight=trans[i, j])
def plot(self, ax: plt.Axes):
self._graph.view()
I would initialize an example object using a data frame that looks something like
foo bar baz
foo 0.5 0.5 0
bar 0.0 0.0 1
baz 1.0 0.0 0
I'm running into the following error
File "<stdin>", line 1, in <module>
File "/.../markov/markovviz.py", line 16, in __init__
self.build_from_matrix(transition_probability_matrix)
File "/.../markov/markovviz.py", line 53, in build_from_matrix
self._graph.edge(labels[i], labels[j], weight=trans[i, j])
File "/.../graphviz/dot.py", line 153, in edge
attr_list = self._attr_list(label, attrs, _attributes)
File "/.../graphviz/lang.py", line 139, in attr_list
content = a_list(label, kwargs, attributes)
File "/.../graphviz/lang.py", line 112, in a_list
for k, v in tools.mapping_items(kwargs) if v is not None]
File "/.../graphviz/lang.py", line 112, in <listcomp>
for k, v in tools.mapping_items(kwargs) if v is not None]
File ".../graphviz/lang.py", line 73, in quote
if is_html_string(identifier) and not isinstance(identifier, NoHtml):
TypeError: cannot use a string pattern on a bytes-like object
which says to me that the only allowable attributes for an edge are strings or bytes. My questions:
Is it even possible to show the graph I'm trying to build in the python interface to graphviz?
If so, how do I associate numeric weights with the edges?
Once I have the weights attached to the edges, how do I draw the graph?

Your problems stems from the line:
self._graph.edge(labels[i], labels[j], weight=trans[i, j])
The problem here is that dot attributes can only be string values, whereas looking at the rest of your code, it looks as if trans[i, j] will probably return a float value.
The simplest solution is probably to just call str():
self._graph.edge(labels[i], labels[j], weight=str(trans[i, j]))
Here's a test that reproduces the problem and the solution:
>>> import graphviz
>>> g = graphviz.Digraph()
>>> g.edge('a', 'b', weight=1.5)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/lars/.local/share/virtualenvs/python-LD_ZK5QN/lib/python3.9/site-packages/graphviz/dot.py", line 153, in edge
attr_list = self._attr_list(label, attrs, _attributes)
File "/home/lars/.local/share/virtualenvs/python-LD_ZK5QN/lib/python3.9/site-packages/graphviz/lang.py", line 139, in attr_list
content = a_list(label, kwargs, attributes)
File "/home/lars/.local/share/virtualenvs/python-LD_ZK5QN/lib/python3.9/site-packages/graphviz/lang.py", line 111, in a_list
items = [f'{quote(k)}={quote(v)}'
File "/home/lars/.local/share/virtualenvs/python-LD_ZK5QN/lib/python3.9/site-packages/graphviz/lang.py", line 111, in <listcomp>
items = [f'{quote(k)}={quote(v)}'
File "/home/lars/.local/share/virtualenvs/python-LD_ZK5QN/lib/python3.9/site-packages/graphviz/lang.py", line 73, in quote
if is_html_string(identifier) and not isinstance(identifier, NoHtml):
TypeError: expected string or bytes-like object
>>> g.edge('a', 'b', weight=str(1.5))
>>> print(g)
digraph {
a -> b [weight=1.5]
}
>>>
Once I have the weights attached to the edges, how do I draw the graph?
Take a look at the render and view methods:
>>> help(g.render)
render(filename=None, directory=None, view=False, cleanup=False, format=None, renderer=None, formatter=None, quiet=False, quiet_view=False) method of graphviz.dot.Digraph instance
Save the source to file and render with the Graphviz engine.
[...]
>>> help(g.view)
view(filename=None, directory=None, cleanup=False, quiet=False, quiet_view=False) method of graphviz.dot.Digraph instance
Save the source to file, open the rendered result in a viewer.
[...]

Random deletion of edges in graph

I am trying to delete edges in a random process in a graph as a function of p where p is from 0 to 1. In the first iteration 0.1 or 10% of the nodes are deleted randomly from the graph. In the second iteration 20% of the remaining edges are deleted and so on.
My error is occurring when the edges which have been deleted come up in the random function again.
My attempt:
import networkx as nx
import random
import numpy as np
graph = nx.fast_gnp_random_graph(20,0.3)
p_values = [0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9]
for i in p_values:
print(i)
for i in p_values:
array=[]
n=nx.number_of_edges(graph)
edges = list(graph.edges)
no_edges_del = int(n*i)
print(no_edges_del)
for j in range(no_edges_del):
chosen_edge = random.choice(edges)
print(chosen_edge)
print(chosen_edge[0])
graph.remove_edge(chosen_edge[0], chosen_edge[1])
GC = nx.number_of_nodes(max(nx.connected_component_subgraphs(graph), key=len))
array.append(GC/n)
error-
Traceback (most recent call last):
File "1.py", line 26, in <module>
graph.remove_edge(chosen_edge[0], chosen_edge[1])
File "D:\anaconda\lib\site-packages\networkx\classes\graph.py", line 1011, in remove_edge
raise NetworkXError("The edge %s-%s is not in the graph" % (u, v))
networkx.exception.NetworkXError: The edge 14-15 is not in the graph

You get the set of edges before the for loop starts. You need to remove the edges from this set as they are removed from the graph so that they aren't chosen again during a later iteration.
Alternatively, get the set of edges from the graph on each iteration just before you choose the one to remove.

A solution may be the following.
At each iteration you consider the current percentage p and remove p*number_of_remaining_edges edges.
import random
import networkx as nx
g = nx.fast_gnp_random_graph(20,0.3)
p_values = [0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9]
for p in p_values:
g.remove_edges_from(random.sample(g.edges(),k=int(p*g.number_of_edges())))

PYMC MAP Fit problems

I use PyMC to implement a multinomial-dirichlet pair. I want to MAP the model for all the instances that we have.
The issue I face is that once MAP.fit() the prior distribution is changed. Thus, for every new instance, I need to have a new prior distribution, which should be fine. However, I keep seeing this error:
Traceback (most recent call last):
File "/Users/xingweiy/Project/StarRating/TimePlot/BayesianPrediction/DiricheletMultinomialStarRating.py", line 41, in <module>
prediction = predict.predict(input,prior)
File "/Users/xingweiy/Project/StarRating/TimePlot/BayesianPrediction/predict.py", line 12, in predict
likelihood = pm.Categorical('rating',prior,value = exp_data,observed = True)
File "/Library/Python/2.7/site-packages/pymc-2.3.4-py2.7-macosx-10.9-intel.egg/pymc/distributions.py", line 3170, in __init__
verbose=verbose, **kwds)
File "/Library/Python/2.7/site-packages/pymc-2.3.4-py2.7-macosx-10.9-intel.egg/pymc/PyMCObjects.py", line 772, in __init__
if not isinstance(self.logp, float):
File "/Library/Python/2.7/site-packages/pymc-2.3.4-py2.7-macosx-10.9-intel.egg/pymc/PyMCObjects.py", line 929, in get_logp
raise ZeroProbability(self.errmsg)
pymc.Node.ZeroProbability: Stochastic rating's value is outside its support,
or it forbids its parents' current values.
Here is the code:
alpha= np.array([0.1,0.1,0.1,0.1,0.1])
prior = pm.Dirichlet('prior',alpha)
exp_data = np.array(input)
likelihood = pm.Categorical('rating',prior,value = exp_data,observed = True)
MaximumPosterior = inf.inference(prior, likelihood, exp_data)
def inference(prior,likelihood,observation):
model = Model({'likelihood':likelihood,'prior':prior})
M = MAP(model)
M.fit()
result = M.prior.value
result = np.append(result,1- np.sum(M.prior.value))
return result
I think it is a bug of pymc package. Is there any way to do MAP without changing the prior distribution?
Thanks
The answer in the link below solved my issue:
https://groups.google.com/forum/#!topic/pymc/uYQSGW4acf8

Basically, the dirichlet distribution generates probability that is close to 0.
The link below solved my issue:
https://groups.google.com/forum/#!topic/pymc/uYQSGW4acf8