I have been trying to generate a code in python that automatically creates a graph from unknown amount of data point in an excel spread sheet.
The data in the excel sheet does not always have the same ending point so I need the code to automatically read how many data points are in the excel file and graph it automatically. It also has to be able to handle large numbers of data point like into the 4000's. And also find the average, max and min.
Sample of the excel data (I only need it to read the first two columns which will be the x and y)
Would anyone know how to recreate this flow chart in networkx? I know I can manually add these nodes and add the edges, but I wanted to write some code that could be applicable for other data types like CSV or EXCEl.
My main idea was to convert this flow into Excel and then have the program read that file and create nodes and edge.
I do have something that is able to read the file and then create nodes using that file. However, I cannot get the edges and the specificities of the flow correct in network.
If anyone could help that would be great. Thanks
This is the flow I wish to recreate
I would try loading the csv into a pandas dataframe. Networkx can be very finicky on how the incoming data needs to be formatted. So I would import it to the dataframe the have networkx read it out of there.
Now I am using the data which is stored in my local influxDB. I have import the data into Chronograf to visualize the data. From Chronograf, I export the data into python and use python to plot and save the plot into images.
However, all the methods I used are all done by manually. Is there any way that can help me to complete all those methods for example like periodically or automatically update the data and plot it?
I have an excel file including some charts. The file is big and complicated and The data is coming from an external source, it means that the data is not included in the excel file discretely, and the only way to see the data is to extract it from the charts. I know how to do this in Excel, but I am looking for a way to read the data from the charts in python(Instead of reading the tabular data).
Is there way to save a "Figure" in matplotlib to a file such that if you later wanted to modify the Figure, e.g. change data points, resize the figure, etc. you could load up the file in a new python script and do that?
Right now I save the majority of my plots as Pdfs, but that doesn't allow me to make edits later on. I have to go dig up my old source code and data files. I've lost track of the number of times I've lost the plot-generating code and have to essentially reproduce it all from scratch.
It would be nice if I could just save a plot as a self-contained data file, like Photoshop does with its .psd files, so that I can just load it up directly, type "object.plot()", and not have to worry about external dependencies. Does such a format exist, or if not is there any way I could achieve this?
There is a method of saving the plotted object called pickling. I don't have much experience with it but it should allow you to save the plot to a file using
fig = plt.figure
pl.dump(fig, file('file_name.pickle','w'))
and using
fig = pl.load(open('file_name.pickle','rb'))
fig.show()
to load the saved graph.
Matplotlib warns that, "Pickle files are not designed for long term storage, are unsupported when restoring a pickle saved in another matplotlib version". To be safe, I would just save the array containing the data to the plot to either a .csv or .txt file, and keep this file in a folder with the python file to plot the graph. This way you will always be able to plot your data (no matter the version of matplotlib you are using). You will also have the data and code in the same place, and you can easily read the data from the .csv or .txt file, save it to arrays, and graph it using
file = open("file_name.txt", "r")
if file.mode == 'r':
data = f.read().splitlines()
data_array1 = data[0].split(",")
data_array2 = data[1].split(",")
p, = plt.plot(data_array1, data_array2)
I also suggest uploading your python files along with your .csv or .txt files to Github.
If you would like to read more about pickling in matplotlib I suggest reading the two pages linked below.
(1) Pickle figures from matplotlib
and (2) https://matplotlib.org/3.1.3/users/prev_whats_new/whats_new_1.2.html#figures-are-picklable