We Keep Coding

Gnuplot: Plotting one series per line of input values - python

I have a CSV file that looks like this:
fft_0,fft_1,fft_2,fft_3,fft_4,fft_5,fft_6,fft_7,fft_8,fft_9,fft_10,fft_11,fft_12,fft_13,fft_14,fft_15,fft_16,fft_17,fft_18,fft_19,fft_20,fft_21,fft_22,fft_23,fft_24,fft_25,fft_26,fft_27,fft_28,fft_29,fft_30,fft_31,fft_32,fft_33,fft_34,fft_35,fft_36,fft_37,fft_38,fft_39,fft_40,fft_41,fft_42,fft_43,fft_44,fft_45,fft_46,fft_47,fft_48,fft_49,fft_50,fft_51,fft_52,fft_53,fft_54,fft_55,fft_56,fft_57,fft_58,fft_59,fft_60,fft_61,fft_62,fft_63,fft_64,fft_65,fft_66,fft_67,fft_68,fft_69,fft_70,fft_71,fft_72,fft_73,fft_74,fft_75,fft_76,fft_77,fft_78,fft_79,fft_80,fft_81,fft_82,fft_83,fft_84,fft_85,fft_86,fft_87,fft_88,fft_89,fft_90,fft_91,fft_92,fft_93,fft_94,fft_95,fft_96,fft_97,fft_98,fft_99,fft_100,fft_101,fft_102,fft_103,fft_104,fft_105,fft_106,fft_107,fft_108,fft_109,fft_110,fft_111,fft_112,fft_113,fft_114,fft_115,fft_116,fft_117,fft_118,fft_119,fft_120,fft_121,fft_122,fft_123,fft_124,fft_125,fft_126,fft_127,fft_128,fft_129,fft_130,fft_131,fft_132,fft_133,fft_134,fft_135,fft_136,fft_137,fft_138,fft_139,fft_140,fft_141,fft_142,fft_143,fft_144,fft_145,fft_146,fft_147,fft_148,fft_149,fft_150,fft_151,fft_152,fft_153,fft_154,fft_155,fft_156,fft_157,fft_158,fft_159,fft_160,fft_161,fft_162,fft_163,fft_164,fft_165,fft_166,fft_167,fft_168,fft_169,fft_170,fft_171,fft_172,fft_173,fft_174,fft_175,fft_176,fft_177,fft_178,fft_179,fft_180,fft_181,fft_182,fft_183,fft_184,fft_185,fft_186,fft_187,fft_188,fft_189,fft_190,fft_191,fft_192,fft_193,fft_194,fft_195,fft_196,fft_197,fft_198,fft_199,fft_200,fft_201,fft_202,fft_203,fft_204,fft_205,fft_206,fft_207,fft_208,fft_209,fft_210,fft_211,fft_212,fft_213,fft_214,fft_215,fft_216,fft_217,fft_218,fft_219,fft_220,fft_221,fft_222,fft_223,fft_224,fft_225,fft_226,fft_227,fft_228,fft_229,fft_230,fft_231,fft_232,fft_233,fft_234,fft_235,fft_236,fft_237,fft_238,fft_239,fft_240,fft_241,fft_242,fft_243,fft_244,fft_245,fft_246,fft_247,fft_248,fft_249,fft_250,fft_251,fft_252,fft_253,fft_254,fft_255
0.0,31.751653360516546,6.275520840013165,14.595752130320351,3.7868790683610323,23.453145120657318,17.968676508902426,13.077879510660095,5.524149879619814,9.159839191815607,3.4879570496415435,5.789890348672875,6.37158005487321,3.53593114100446,9.005729135469483,11.511943709884944,4.877113072123605,3.980766752139159,11.482684912439783,17.220737110365697,27.088212459502966,110.93476064590726,59.78887389689556,32.51075628874421,24.591418574709603,4.640853279203423,7.448545851889065,9.188366734225628,11.366826382304204,10.077018109797907,3.1841494684815577,5.190248731077729,3.4161263016208325,2.2026391207219995,14.97641334282317,5.673077411085896,12.884780698455764,6.998674823994672,7.342744118836382,3.949308507257039,4.395757811042787,16.103476980952966,28.2719870806011,10.88302198678709,2.685111390491833,10.686339936971915,6.408929713964431,12.069758412421345,15.72375095954655,4.119204604927627,14.874704697329062,11.498505785311869,11.117260027598117,9.494678844281257,5.266405910530414,10.915082550855102,8.417975130307454,18.801404884876543,29.51008177200432,10.843737620239258,20.365544144100863,49.61885082721084,24.743136974252824,95.12429979320565,216.99226248669268,49.341356893894115,18.766444144153137,51.565370474695136,43.7242611005549,54.48857810233984,43.66891218332103,26.853261639029913,6.866934892914652,11.299141562323118,5.318490604101198,31.524180684221047,21.059671101201328,11.17069564618422,17.446945303923478,15.79632859533589,25.8754797065767,25.202369290835293,40.374088558894044,26.000283711509223,55.62708932638661,66.77073066677998,71.71203146112138,48.980985792557924,14.286410652210266,4.601609981437789,15.070118260346232,23.802325341146616,8.327118721719636,11.442341360890286,22.05004217818876,11.104160787103668,12.484928716546818,9.739491305620902,5.5297569624177,7.37163250954064,35.75106868880271,7.399083813594336,26.329051989363144,16.913714138751928,22.74781071883487,20.59299921308038,41.09358072023589,81.47189388863296,61.740342954926696,12.309218911196457,10.244104070553457,4.305193519431507,4.316878112694274,15.939544124902632,12.56060454938032,6.747750800215833,10.325389983324909,14.998130859829269,28.18890104826635,11.0605430438416,8.185204031195694,26.87475337528722,12.662790798826446,41.035417238947375,65.0858268884065,65.54994660181138,21.677593542254662,149.35573051267886,212.42969462503808,74.30926892134302,24.507354570739256,40.76411049649439,35.10651624387605,36.09565222913914,23.539710351057625,25.542632631949893,4.102760694986416,11.980098370080432,13.609502549213738,21.93156128168799,10.081283648530139,15.769674404840714,5.192044221522882,11.771446279635393,12.094956753577113,9.917024863836527,1.2572695424313944,3.722053458526404,8.795658724551126,21.56071410293817,28.922799068534882,13.483122896667437,10.395965791400695,23.505506841117732,8.485560877844723,2.356254230310154,28.165902806102135,12.980298399346841,16.336712296798527,6.730154151533989,16.677864587364514,3.3409877465665216,7.077281578595023,6.446743875035899,19.33833179858605,11.2040774924139,23.43426023135171,21.06169641831372,8.485140511390588,9.068503134834177,3.670077295125947,19.14430606857007,32.42559040891663,28.48925355507031,7.279904785720349,25.259393926407167,20.742537308065206,24.09984129072362,11.414295313454046,3.6413514677095136,41.19050815699398,9.206157892358178,19.755370447153105,36.994783243438796,32.80074469851852,35.87411321258091,11.503107316858005,74.10255021280251,67.24504666591955,104.16768866937738,95.33294914217088,151.9884214593015,136.32292370844223,216.3859464252064,123.41220158493012,59.26669418804178,80.98912876710403,146.95475095348888,46.520877172048024,40.11129083574055,31.4062153719097,51.2525380944314,19.618477161063097,24.64912362290385,39.665889309349865,4.5420695812167695,12.352393331711527,12.913093028159468,5.7475173407962785,32.16753002725559,1.7358492153434433,8.723950884616885,12.467271027329899,14.083802770107315,17.92250369303905,7.86710602026345,7.794809991411701,13.58518272133987,6.524921604389587,15.979388080578659,16.559110564691427,20.67916970007909,18.224245061592946,7.096787929904062,8.373793561958955,6.55812817381509,13.905987460060079,20.595153639426073,8.807333502576107,13.82796348336836,2.18887726814401,6.318371489436107,2.993337999476443,18.749653399598177,6.345591182705673,9.761309360285061,30.96989586919761,8.11119534172852,10.866890070470923,11.517755176146323,27.4609347160214,16.135144173727756,10.6448246437425,4.154792885980834,17.62396431228251,9.509942088880527,15.555429074090473,15.031228264687387,0.9801466560635561,0.7645844300931341,14.154728612808947,12.552555345029779,32.44796170517449,19.530604457901777,11.95976683879656,46.42256522056603
0.0,19.16432992306051,19.616997783340636,24.44119947886979,4.201389789467854,21.74285992321492,21.832776487663676,5.487401249578947,11.746651370331632,1.9601369533755433,5.622561176369094,4.96631977374031,10.097588355385781,5.08483320481312,9.445414620036079,14.222808225039186,2.696213466317052,7.638899760644791,15.041153329998348,10.391749304040763,32.3476575096254,108.99204640400545,63.57497610505674,22.696101813324205,7.872359102700095,21.52130460513689,10.111810799980553,12.724059804396923,19.009158214375578,12.65449693937476,9.132000916109536,6.326763343031359,4.879855924465264,3.871345327977384,10.924451311265537,7.589696386926958,8.18649996093532,11.551503954959513,11.064148190492709,6.176531003718973,15.62762103722472,11.29474719219796,15.477100373471712,34.31169456951676,4.869630599520906,15.140471298507281,8.259616875565543,7.215380685669043,10.46794882830988,19.43560849652611,4.529173954916338,4.35512822937366,2.895187451236368,12.790221568530207,15.09029777868783,17.31295887292244,6.845725037238285,11.696984535700611,29.19412768846885,10.816150558470914,17.795519392349565,18.95627545053494,19.39179148839016,51.25978750420467,258.9220546686427,26.60895810146767,60.12526975188942,57.965437788392094,56.17033578719,61.73076599694442,42.65192391821165,11.56253870818451,25.30486166396842,14.689367444907104,17.68946470063505,12.000531269987114,22.046788191423992,10.461190794614817,15.381278597253058,20.56941785579303,11.378832916602038,17.079026604217585,27.64811379662995,15.945260538344455,17.85973583587812,91.07244530248128,38.79336310420707,9.977752149534174,4.141735910598209,12.882679361205826,7.206277012438345,10.771526867887964,5.877615826490578,12.242065277916515,13.334594521312354,4.29007685290975,7.68709934271143,18.61924420361514,8.74553477917526,8.73628302646769,20.257192090857302,28.872335112983553,4.536051852152141,30.617387864205337,17.858689013107693,41.73630055399866,58.39281594824643,87.24012438534487,31.066544056426327,41.13149447740131,32.42269508871998,26.269987698517536,11.740763622173773,11.004766324349793,18.09863936075447,9.819248943124844,12.893499378743552,10.563553823125307,39.29361373727987,12.346894479852338,8.459770692508902,57.310236078713004,33.1311985608261,28.907897941787073,70.32655132146188,84.24779353195314,61.25485848237276,84.84845471667461,288.2138613792481,109.16667530743895,77.57183940606394,87.3425139617006,68.57701371566155,45.68772118776169,50.0797905239745,33.09749205216209,18.007490834401867,16.334260550527407,31.433003074249346,9.246945751725423,7.074363738698788,4.307268210922044,19.74271610979451,12.91162042059288,21.84937055988338,32.36398718338645,7.281588590668135,10.08373606540601,24.552336456417798,14.7860174786099,21.65070646215015,7.715805417675984,10.173892601354071,10.838760273837256,29.862805175973264,20.040881398274966,26.198025016148872,7.413741554718165,17.48934776894029,6.937450524150569,12.17991859460801,11.285099309978527,16.819830852613755,8.77192614757165,12.64143434466125,25.385516815011027,7.3675506195003875,12.129509801209169,28.94513170225861,20.395460170229377,22.488513131279987,21.01810252560465,35.136025982361815,10.139560341414699,3.555043053681657,31.343112942532223,14.592550641227064,17.64468740002523,26.880409076658076,22.11422795022885,27.4977856180402,30.104920205536807,39.53169334361884,11.71294285354865,23.425805677846594,67.79219601206428,42.27107919700437,64.88620076244453,117.15891789296428,99.78292099303975,48.71538731259637,90.15956877488314,95.77784729934608,101.95350156422501,96.21863000138868,142.43852409191666,151.1858655495999,120.64765073899484,79.83194264924788,65.59195836122893,31.013619590967267,23.420785189214566,28.712244731781265,24.5375312236021,37.231275049400224,22.30698063637999,3.466152702405225,21.930586433683153,12.983286870554023,33.3275146767313,16.019629125908292,13.374161083992483,14.140770556831407,22.17215831856053,11.313270491326389,15.68024673516814,12.986483554144087,23.331903515521233,3.864860667678062,13.76293655930224,29.7546331148781,10.197145120256085,9.949295198338554,23.03483024580453,7.954340759704595,5.206921160805673,12.442782717659927,15.289085530882318,5.497348053429443,6.255156597717796,8.342835500469203,14.66372085309629,16.322270250668957,8.194736256097809,24.397351809392624,17.884522130787275,7.68573327606993,8.180457570504249,33.303256527001146,5.769925562021384,15.466629545524382,12.651255525791012,15.995160619641469,12.922309957994937,6.556673050829525,6.941908198528731,4.617550389093913,27.335731696306702,5.814856112558841,18.49864018853029,13.96291681057734,35.903064134866256,38.73619761839379,32.56594885594187,54.105909842976,30.470991913000084
[...]
I want to be able to plot a new series per line (legend not necessary). The x-values should just start at 0 and increment from there, the y-values are given by the numbers in the file. The result should be something like this:
This was made in Excel, and if it weren't so labourous, this would be fine.
Note that answers using python/matplotlib are also welcomed.
Thanks a lot!
EDIT: This is as far as I've come, based on the answer provided in gnuplot: How to plot each line of a data file as Y and incremental X and other places. So far, there's for some reason no output file written, although it's definitely doing something (because the last command takes a few seconds to finish).
set term png
set output "fft.png"
set datafile sep ','
plot for [i=1:255] 'fft.csv' using (i):(column(i)) with lines
EDIT 2: I now have this, which is closer, but still not what I want. Each line in the input file should be a series. It seems now each column is a series.
set term png
set output "fft.png"
set datafile sep ','
unset key
plot for [i=1:255] 'fft.csv' using (i):(column(i)) with lines

Right, I think I solved it (using the answer from https://stackoverflow.com/a/8716495/666259 and the answer from Christoph):
set terminal png size 1920,1080
set datafile sep ","
unset key
set xtics 0,5,255
set title "mytitle"
set autoscale cbfix
plot "fft.csv" matrix using 1:3:2 with lines palette
This gives me something like this:

In order to have every line with a different color, use either palette or linecolor variable:
set datafile separator ','
set autoscale cbfix
plot 'file.csv' matrix every :::1 using 1:3:2 with lines palette
This uses the row number to color the lines based on the current palette. This is very useful if you have many lines to draw. set autoscale cbfix prevents the color range to be extended to the next major tics.
For only a few lines, you could also use
plot 'file.csv' matrix every :::1 using 1:3:2 linecolor variable with lines
This uses the row number as index for the linetype.

Related

I have been trying to make a figure using plotly that combines multiple figures together. In order to do this, I have been trying to use the make_subplots function, but I have found it very difficult to have the plots added in such a way that they are properly formatted. I can currently make singular plots (as seen directly below):
However, whenever I try to combine these singular plots using make_subplots, I end up with this:
This figure has the subplots set up completely wrong, since I need each of the four subplots to contain data pertaining to the four methods (A, B, C, and D). In other words, I would like to have four subplots that look like my singular plot example above.
I have set up the code in the following way:
for sequence in sequences:
#process for making sequence profile is done here
sequence_df = pd.DataFrame(sequence_profile)
row_number=1
grand_figure = make_subplots(rows=4, cols=1)
#there are four groups per sequence, so the grand figure should have four subplots in total
for group in sequence_df["group"].unique():
figure_df_group = sequence_df[(sequence_df["group"]==group)]
figure_df_group.sort_values("sample", ascending=True, inplace=True)
figure = px.line(figure_df_group, x = figure_df_group["sample"], y = figure_df_group["intensity"], color= figure_df_group["method"])
figure.update_xaxes(title= "sample")
figure.update_traces(mode='markers+lines')
#note: the next line fails, since data must be extracted from the figure, hence why it is commented out
#grand_figure.append_trace(figure, row = row_number, col=1)
figure.update_layout(title_text="{} Profile Plot".format(sequence))
grand_figure.append_trace(figure.data[0], row = row_number, col=1)
row_number+=1
figure.write_image(os.path.join(output_directory+"{}_profile_plot_subplots_in_{}.jpg".format(sequence, group)))
grand_figure.write_image(os.path.join(output_directory+"grand_figure_{}_profile_plot_subplots.jpg".format(sequence)))
I have tried following directions (like for example, here: ValueError: Invalid element(s) received for the 'data' property) but I was unable to get my figures added as is as subplots. At first it seemed like I needed to use the graph object (go) module in plotly (https://plotly.com/python/subplots/), but I would really like to keep the formatting/design of my current singular plot. I just want the plots to be conglomerated in groups of four. However, when I try to add the subplots like I currently do, I need to use the data property of the figure, which causes the design of my scatter plot to be completely messed up. Any help for how I can ameliorate this problem would be great.

Ok, so I found a solution here. Rather than using the make_subplots function, I just instead exported all the figures onto an .html file (Plotly saving multiple plots into a single html) and then converted it into an image (HTML to IMAGE using Python). This isn't exactly the approach I would have preferred to have, but it does work.
UPDATE
I have found that plotly express offers another solution, as the px.line object has the parameter of facet that allows one to set up multiple subplots within their plot. My code is set up like this, and is different from the code above in that the dataframe does not need to be iterated in a for loop based on its groups:
sequence_df = pd.DataFrame(sequence_profile)
figure = px.line(sequence_df, x = sequence_df["sample"], y = sequence_df["intensity"], color= sequence_df["method"], facet_col= sequence_df["group"])
Although it still needs more formatting, my plot now looks like this, which is works much better for my purposes:

Working with Jupyter Lab!
I loaded a simple ASCII file (as I've done 100s of times before...), with three columns and 2000+ lines. I did it with
with open(file) as f:
d = f.readlines()
and also with NumPy d = np.loadtxt(file, delimiter=',') to see if something would change.
All values in the 2nd column (Latitude) are -32. and something and all values in the 3rd column (Longitude) are -52. and something. However, de variation of the data is about the 5th decimal number... (I think this is making things weird!)
When I printed the data on the screen, it seems ok! But when I try to plot them, I got pretty weird stuff... the numbers in the X and Y axis are nonsense, especially the one scaling the x-axis. 1e-5-5.2103e1
I open the data in a spreadsheet (Libre Calc), and there the plot seems alright. Then I save it with another name and tried to load in the Jupyter again and got the same weird result.
I also tried it also using a different computer... same result!
Tried a script using Atom... same result!
Can someone give a clue about what is going on?
The file is shared at:
https://drive.google.com/file/d/1eDwlijQ7y3KoIRafoE00eqK3UYsIcMvf/view?usp=sharing
First lines o the file...
9738,-32.13689233,-52.10339483
9739,-32.13689233,-52.10339483
9740,-32.13689233,-52.10339483
9741,-32.13689233,-52.10339483
9742,-32.13689233,-52.10339483
9743,-32.13689233,-52.10339483
9744,-32.13689233,-52.10339483
9745,-32.13689233,-52.10339483
9746,-32.13689233,-52.10339483
9747,-32.13689233,-52.10339483
9748,-32.13689233,-52.10339483
9749,-32.13689433,-52.10339417
9750,-32.13689433,-52.10339417
9751,-32.13689433,-52.10339417
9752,-32.13689433,-52.10339417
9753,-32.13689433,-52.10339417
9754,-32.13689433,-52.10339417
9755,-32.13689433,-52.10339417
9756,-32.13689433,-52.10339417
9757,-32.13689433,-52.10339417
9758,-32.13689433,-52.10339417
9759,-32.13688733,-52.10339367

It looks like your plot is correct, even though the scaling of the axes is quite strange (likely due to the very small range). I plotted the same data using Altair, which was able to handle the axes ranges much better:
import pandas as pd
import altair as alt
with open("lat_long.csv", newline="") as f:
frame = pd.read_csv(f, delimiter=",", header=None, names=["index", "longitude", "latitude"])
alt.Chart(frame).mark_circle(size=10).encode(
alt.X('latitude',
scale=alt.Scale(zero=False)
),
alt.Y('longitude',
scale=alt.Scale(zero=False)
),
).interactive()
The result matches yours, except for the more sensible axes:
My guess is that the pandas plotter is just doing something strange with displaying the axes, but the data is being read in correctly.

This is how matplotlib displays very large and small tick markers. For example, to read the y-axis you take that number at the top and add each tick marker. I'll admit, the number on the x-axis is weird, but you can see a recognizable prefix in there: -5.2103e1. The 1e-5 might just be signifying the decimal place the ticks start at. Looking through the file, the plot makes sense, and the labels do as well.

In matplotlib I wish to know the cleanest and most robust means of overlaying labels onto an axis. This is probably best demonstrated with an example:
While normal axis labels/ticks are placed every 5.00 units additional labels without ticks have been overlayed onto the axis (this can be seen at 1113.75 which partially covers 1114.00 and 1105.00 which is covered entirely). The labels also have the same font and size as their normal, ticked, counterparts with the background (if any) going right up to the axis (as a tick mark would).
What is the simplest way of obtaining this effect in matplotlib?
Edit
Following on from #Ken's suggestion I have managed to obtain the effect for an existing tick/label by using ax.yaxis.get_ticklines and ax.yaxis.get_ticklabels to both remove the tick marker and change the background/font/zorder of a label. However, I am unsure how best to add a new tick/label to an axis.
In other words I am looking for a function add_tick(ax.yaxis, loc) that adds a tick at location loc and returns the tickline and ticklabel objects for me to operate on.

I haven't ever tried to do that, but I think that the Artist tutorial might be helpful for you. In particular, the last section has the following code:
for line in ax1.yaxis.get_ticklines():
# line is a Line2D instance
line.set_color('green')
line.set_markersize(25)
line.set_markeredgewidth(3)
I think that using something like line.set_markersize(0) might make the markers have size zero. The difficult part might be finding the ones that need that done. It is possible that the line.xdata or line.ydata arrays might contain enough information to isolate the ones you need. Of course, if you are manually adding the tick marks, it is possible that as you do that the instance gets returned, so you can just modify them as you create them.

The best solution I have been able to devise:
# main: axis; olocs: locations list; ocols: location colours
def overlay_labels(main, olocs, ocols):
# Append the overlay labels as ticks
main.yaxis.set_ticks(np.append(main.yaxis.get_ticklocs(), olocs))
# Perform generic formatting to /all/ ticks
# [...]
labels = reversed(main.yaxis.get_ticklabels())
markers = reversed(main.yaxis.get_ticklines()[1::2]) # RHS ticks only
glines = reversed(main.yaxis.get_gridlines())
rocols = reversed(ocols)
# Suitably format each overlay tick (colours and lines)
for label,marker,grid,colour in izip(labels, markers, glines, rocols):
label.set_color('white')
label.set_backgroundcolor(colour)
marker.set_visible(False)
grid.set_visible(False)
It is not particularly elegant but does appear to work.

If I understand correctly, the function:
matplotlib.pyplot.plot(x, y)
plots len(x)-1 separate line segments - one going from (x[0], y[0]) to (x[1],y[1]), one going from (x[1],y[1]) to (x[2], y[2]), etc. In my application, I want to display a curve consisting of a series of line segments connecting data points in this way, but there is an extra piece of data (z) associated with the transition between each of these data points, that I want to represent by the color of the line segment. Clearly one way of doing this is the following:
for i in range(len(x)-1)):
matplotlib.pyplot.plot(x[i:i+2],y[i:i+2], color=z[i])
but is there a way to do it that doesn't involve a separate call to matplotlib.pyplot.plot for each line segment?

You should be able to use matplotlib.collections.LineCollection which accepts a colors parameter (sequence of RGBA tuples).

If you implement a colorbar, something similar to this, then you should be able to do it in one line. The trick may be to pick a colorscale that changes on the order of your line segments.

How can I use pyplot.clabel to attach the file names to the lines being plotted?
plt.clabel(data) line gives the error

Helou,
plt.clabel is only used with contour lines, according to documentation: "adds labels to line contours in cs, where cs is a ContourSet object returned by contour (axes_api). Please check contour example:

You may use plt.annotate or plt.text.
And, as an aside, 1) you probably want to use different variables for the file names and numpy arrays you're loading your data into (what is data in data=plb.loadtxt(data)),
2) you probably want to move the label positioning into the loop (in your code, what is data in the plt.clabel(data)).