first of all: I'm completely new to python.
I'm trying to visualize some measured data. Each entry has a quadrant, number and sector. The original data lies in a .xlsx file. I've managed to use a .pivot_table to sort the data according to its sector. Due to overlapping, number and quadrant also have to be indexed. Now I want to plot it as a bar chart, where the bars are grouped by sector and the colors represent the quadrant.
But because number also has to be indexed, it shows up in the bar chart as a separate group. There should only be three groups, 0, i and a.
MWE:
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
d = {'quadrant': ["0","0","0","0","0","0","I","I","I","I","I","I","I","I","I","I","I","I","II","II","II","II","II","II","II","II","II","II","II","II","III","III","III","III","III","III","III","III","III","III","III","III","IV","IV","IV","IV","IV","IV","IV","IV","IV","IV","IV","IV"], 'sector': [0,"0","0","0","0","0","a","a","a","a","a","a","i","i","i","i","i","i","a","a","a","a","a","a","i","i","i","i","i","i","a","a","a","a","a","a","i","i","i","i","i","i","a","a","a","a","a","a","i","i","i","i","i","i"], 'number': [1,2,3,4,5,6,1,2,3,4,5,6,1,2,3,4,5,6,1,2,3,4,5,6,1,2,3,4,5,6,1,2,3,4,5,6,1,2,3,4,5,6,1,2,3,4,5,6,1,2,3,4,5,6], 'Rz_m': [67.90,44.17,44.30,63.43,49.87,39.33,61.17,69.37,66.20,44.20,64.77,39.93,44.33,50.97,55.90,51.33,58.23,44.53,50.03,47.40,58.67,71.57,57.60,70.77,63.93,47.37,46.90,34.73,41.27,48.23,58.30,47.07,50.53,51.20,32.67,50.37,37.50,55.50,41.20,48.07,56.80,49.77,40.87,44.43,44.00,60.03,63.73,72.80,51.60,45.53,60.27,71.00,59.63,48.70]}
df = pd.DataFrame(data=d)
B = df.pivot_table(index=['sector','number', 'quadrant'])
B.unstack().plot.bar(y='Rz_m')
The data viz ecosystem in Python is pretty diverse and there are multiple libraries you can use to produce the same chart. Matplotlib is a very powerful library, but it's also quite low-level, meaning you often have to do a lot of preparatory work before getting to the chart, so usually you'll find people use seaborn for static visualisations, especially if there is a scientific element to them (it has built-in support for things like error bars, etc.)
Out of the box, it has a lot of chart types to support exploratory data analysis and is built on top of matplotlib. For your example, if I understood it right, it would be as simple as:
import seaborn as sns
sns.catplot(x="sector", y="Rz_m", hue="quadrant", data=df, ci=None,
height=6, kind="bar", palette="muted")
And the output would look like this:
Note that in your example, you missed out "" for one of the zeroes and 0 and "0" are plotted as separate columns. If you're using seaborn, you don't need to pivot the data, just feed it the df as you've defined it.
For interactive visualisations (with tooltips, zoom, pan, etc.), you can also check out bokeh.
There is an interesting wrinkle to this - how to center the nested bars on the label. By default the bars are drawn with center alignment which works fine for an odd number of columns. However, for an even number, you'd want them to be centered on the right edge. You can make a small alteration in the source code categorical.py, lines beginning 1642 like so:
# Draw the bars
offpos = barpos + self.hue_offsets[j]
barfunc(offpos, self.statistic[:, j], -self.nested_width,
color=self.colors[j], align="edge",
label=hue_level, **kws)
Save the .png and then change it back, but it's not ideal. Probably worth flagging up to the library maintainers.
I have a CSV file that looks like this:
fft_0,fft_1,fft_2,fft_3,fft_4,fft_5,fft_6,fft_7,fft_8,fft_9,fft_10,fft_11,fft_12,fft_13,fft_14,fft_15,fft_16,fft_17,fft_18,fft_19,fft_20,fft_21,fft_22,fft_23,fft_24,fft_25,fft_26,fft_27,fft_28,fft_29,fft_30,fft_31,fft_32,fft_33,fft_34,fft_35,fft_36,fft_37,fft_38,fft_39,fft_40,fft_41,fft_42,fft_43,fft_44,fft_45,fft_46,fft_47,fft_48,fft_49,fft_50,fft_51,fft_52,fft_53,fft_54,fft_55,fft_56,fft_57,fft_58,fft_59,fft_60,fft_61,fft_62,fft_63,fft_64,fft_65,fft_66,fft_67,fft_68,fft_69,fft_70,fft_71,fft_72,fft_73,fft_74,fft_75,fft_76,fft_77,fft_78,fft_79,fft_80,fft_81,fft_82,fft_83,fft_84,fft_85,fft_86,fft_87,fft_88,fft_89,fft_90,fft_91,fft_92,fft_93,fft_94,fft_95,fft_96,fft_97,fft_98,fft_99,fft_100,fft_101,fft_102,fft_103,fft_104,fft_105,fft_106,fft_107,fft_108,fft_109,fft_110,fft_111,fft_112,fft_113,fft_114,fft_115,fft_116,fft_117,fft_118,fft_119,fft_120,fft_121,fft_122,fft_123,fft_124,fft_125,fft_126,fft_127,fft_128,fft_129,fft_130,fft_131,fft_132,fft_133,fft_134,fft_135,fft_136,fft_137,fft_138,fft_139,fft_140,fft_141,fft_142,fft_143,fft_144,fft_145,fft_146,fft_147,fft_148,fft_149,fft_150,fft_151,fft_152,fft_153,fft_154,fft_155,fft_156,fft_157,fft_158,fft_159,fft_160,fft_161,fft_162,fft_163,fft_164,fft_165,fft_166,fft_167,fft_168,fft_169,fft_170,fft_171,fft_172,fft_173,fft_174,fft_175,fft_176,fft_177,fft_178,fft_179,fft_180,fft_181,fft_182,fft_183,fft_184,fft_185,fft_186,fft_187,fft_188,fft_189,fft_190,fft_191,fft_192,fft_193,fft_194,fft_195,fft_196,fft_197,fft_198,fft_199,fft_200,fft_201,fft_202,fft_203,fft_204,fft_205,fft_206,fft_207,fft_208,fft_209,fft_210,fft_211,fft_212,fft_213,fft_214,fft_215,fft_216,fft_217,fft_218,fft_219,fft_220,fft_221,fft_222,fft_223,fft_224,fft_225,fft_226,fft_227,fft_228,fft_229,fft_230,fft_231,fft_232,fft_233,fft_234,fft_235,fft_236,fft_237,fft_238,fft_239,fft_240,fft_241,fft_242,fft_243,fft_244,fft_245,fft_246,fft_247,fft_248,fft_249,fft_250,fft_251,fft_252,fft_253,fft_254,fft_255
0.0,31.751653360516546,6.275520840013165,14.595752130320351,3.7868790683610323,23.453145120657318,17.968676508902426,13.077879510660095,5.524149879619814,9.159839191815607,3.4879570496415435,5.789890348672875,6.37158005487321,3.53593114100446,9.005729135469483,11.511943709884944,4.877113072123605,3.980766752139159,11.482684912439783,17.220737110365697,27.088212459502966,110.93476064590726,59.78887389689556,32.51075628874421,24.591418574709603,4.640853279203423,7.448545851889065,9.188366734225628,11.366826382304204,10.077018109797907,3.1841494684815577,5.190248731077729,3.4161263016208325,2.2026391207219995,14.97641334282317,5.673077411085896,12.884780698455764,6.998674823994672,7.342744118836382,3.949308507257039,4.395757811042787,16.103476980952966,28.2719870806011,10.88302198678709,2.685111390491833,10.686339936971915,6.408929713964431,12.069758412421345,15.72375095954655,4.119204604927627,14.874704697329062,11.498505785311869,11.117260027598117,9.494678844281257,5.266405910530414,10.915082550855102,8.417975130307454,18.801404884876543,29.51008177200432,10.843737620239258,20.365544144100863,49.61885082721084,24.743136974252824,95.12429979320565,216.99226248669268,49.341356893894115,18.766444144153137,51.565370474695136,43.7242611005549,54.48857810233984,43.66891218332103,26.853261639029913,6.866934892914652,11.299141562323118,5.318490604101198,31.524180684221047,21.059671101201328,11.17069564618422,17.446945303923478,15.79632859533589,25.8754797065767,25.202369290835293,40.374088558894044,26.000283711509223,55.62708932638661,66.77073066677998,71.71203146112138,48.980985792557924,14.286410652210266,4.601609981437789,15.070118260346232,23.802325341146616,8.327118721719636,11.442341360890286,22.05004217818876,11.104160787103668,12.484928716546818,9.739491305620902,5.5297569624177,7.37163250954064,35.75106868880271,7.399083813594336,26.329051989363144,16.913714138751928,22.74781071883487,20.59299921308038,41.09358072023589,81.47189388863296,61.740342954926696,12.309218911196457,10.244104070553457,4.305193519431507,4.316878112694274,15.939544124902632,12.56060454938032,6.747750800215833,10.325389983324909,14.998130859829269,28.18890104826635,11.0605430438416,8.185204031195694,26.87475337528722,12.662790798826446,41.035417238947375,65.0858268884065,65.54994660181138,21.677593542254662,149.35573051267886,212.42969462503808,74.30926892134302,24.507354570739256,40.76411049649439,35.10651624387605,36.09565222913914,23.539710351057625,25.542632631949893,4.102760694986416,11.980098370080432,13.609502549213738,21.93156128168799,10.081283648530139,15.769674404840714,5.192044221522882,11.771446279635393,12.094956753577113,9.917024863836527,1.2572695424313944,3.722053458526404,8.795658724551126,21.56071410293817,28.922799068534882,13.483122896667437,10.395965791400695,23.505506841117732,8.485560877844723,2.356254230310154,28.165902806102135,12.980298399346841,16.336712296798527,6.730154151533989,16.677864587364514,3.3409877465665216,7.077281578595023,6.446743875035899,19.33833179858605,11.2040774924139,23.43426023135171,21.06169641831372,8.485140511390588,9.068503134834177,3.670077295125947,19.14430606857007,32.42559040891663,28.48925355507031,7.279904785720349,25.259393926407167,20.742537308065206,24.09984129072362,11.414295313454046,3.6413514677095136,41.19050815699398,9.206157892358178,19.755370447153105,36.994783243438796,32.80074469851852,35.87411321258091,11.503107316858005,74.10255021280251,67.24504666591955,104.16768866937738,95.33294914217088,151.9884214593015,136.32292370844223,216.3859464252064,123.41220158493012,59.26669418804178,80.98912876710403,146.95475095348888,46.520877172048024,40.11129083574055,31.4062153719097,51.2525380944314,19.618477161063097,24.64912362290385,39.665889309349865,4.5420695812167695,12.352393331711527,12.913093028159468,5.7475173407962785,32.16753002725559,1.7358492153434433,8.723950884616885,12.467271027329899,14.083802770107315,17.92250369303905,7.86710602026345,7.794809991411701,13.58518272133987,6.524921604389587,15.979388080578659,16.559110564691427,20.67916970007909,18.224245061592946,7.096787929904062,8.373793561958955,6.55812817381509,13.905987460060079,20.595153639426073,8.807333502576107,13.82796348336836,2.18887726814401,6.318371489436107,2.993337999476443,18.749653399598177,6.345591182705673,9.761309360285061,30.96989586919761,8.11119534172852,10.866890070470923,11.517755176146323,27.4609347160214,16.135144173727756,10.6448246437425,4.154792885980834,17.62396431228251,9.509942088880527,15.555429074090473,15.031228264687387,0.9801466560635561,0.7645844300931341,14.154728612808947,12.552555345029779,32.44796170517449,19.530604457901777,11.95976683879656,46.42256522056603
0.0,19.16432992306051,19.616997783340636,24.44119947886979,4.201389789467854,21.74285992321492,21.832776487663676,5.487401249578947,11.746651370331632,1.9601369533755433,5.622561176369094,4.96631977374031,10.097588355385781,5.08483320481312,9.445414620036079,14.222808225039186,2.696213466317052,7.638899760644791,15.041153329998348,10.391749304040763,32.3476575096254,108.99204640400545,63.57497610505674,22.696101813324205,7.872359102700095,21.52130460513689,10.111810799980553,12.724059804396923,19.009158214375578,12.65449693937476,9.132000916109536,6.326763343031359,4.879855924465264,3.871345327977384,10.924451311265537,7.589696386926958,8.18649996093532,11.551503954959513,11.064148190492709,6.176531003718973,15.62762103722472,11.29474719219796,15.477100373471712,34.31169456951676,4.869630599520906,15.140471298507281,8.259616875565543,7.215380685669043,10.46794882830988,19.43560849652611,4.529173954916338,4.35512822937366,2.895187451236368,12.790221568530207,15.09029777868783,17.31295887292244,6.845725037238285,11.696984535700611,29.19412768846885,10.816150558470914,17.795519392349565,18.95627545053494,19.39179148839016,51.25978750420467,258.9220546686427,26.60895810146767,60.12526975188942,57.965437788392094,56.17033578719,61.73076599694442,42.65192391821165,11.56253870818451,25.30486166396842,14.689367444907104,17.68946470063505,12.000531269987114,22.046788191423992,10.461190794614817,15.381278597253058,20.56941785579303,11.378832916602038,17.079026604217585,27.64811379662995,15.945260538344455,17.85973583587812,91.07244530248128,38.79336310420707,9.977752149534174,4.141735910598209,12.882679361205826,7.206277012438345,10.771526867887964,5.877615826490578,12.242065277916515,13.334594521312354,4.29007685290975,7.68709934271143,18.61924420361514,8.74553477917526,8.73628302646769,20.257192090857302,28.872335112983553,4.536051852152141,30.617387864205337,17.858689013107693,41.73630055399866,58.39281594824643,87.24012438534487,31.066544056426327,41.13149447740131,32.42269508871998,26.269987698517536,11.740763622173773,11.004766324349793,18.09863936075447,9.819248943124844,12.893499378743552,10.563553823125307,39.29361373727987,12.346894479852338,8.459770692508902,57.310236078713004,33.1311985608261,28.907897941787073,70.32655132146188,84.24779353195314,61.25485848237276,84.84845471667461,288.2138613792481,109.16667530743895,77.57183940606394,87.3425139617006,68.57701371566155,45.68772118776169,50.0797905239745,33.09749205216209,18.007490834401867,16.334260550527407,31.433003074249346,9.246945751725423,7.074363738698788,4.307268210922044,19.74271610979451,12.91162042059288,21.84937055988338,32.36398718338645,7.281588590668135,10.08373606540601,24.552336456417798,14.7860174786099,21.65070646215015,7.715805417675984,10.173892601354071,10.838760273837256,29.862805175973264,20.040881398274966,26.198025016148872,7.413741554718165,17.48934776894029,6.937450524150569,12.17991859460801,11.285099309978527,16.819830852613755,8.77192614757165,12.64143434466125,25.385516815011027,7.3675506195003875,12.129509801209169,28.94513170225861,20.395460170229377,22.488513131279987,21.01810252560465,35.136025982361815,10.139560341414699,3.555043053681657,31.343112942532223,14.592550641227064,17.64468740002523,26.880409076658076,22.11422795022885,27.4977856180402,30.104920205536807,39.53169334361884,11.71294285354865,23.425805677846594,67.79219601206428,42.27107919700437,64.88620076244453,117.15891789296428,99.78292099303975,48.71538731259637,90.15956877488314,95.77784729934608,101.95350156422501,96.21863000138868,142.43852409191666,151.1858655495999,120.64765073899484,79.83194264924788,65.59195836122893,31.013619590967267,23.420785189214566,28.712244731781265,24.5375312236021,37.231275049400224,22.30698063637999,3.466152702405225,21.930586433683153,12.983286870554023,33.3275146767313,16.019629125908292,13.374161083992483,14.140770556831407,22.17215831856053,11.313270491326389,15.68024673516814,12.986483554144087,23.331903515521233,3.864860667678062,13.76293655930224,29.7546331148781,10.197145120256085,9.949295198338554,23.03483024580453,7.954340759704595,5.206921160805673,12.442782717659927,15.289085530882318,5.497348053429443,6.255156597717796,8.342835500469203,14.66372085309629,16.322270250668957,8.194736256097809,24.397351809392624,17.884522130787275,7.68573327606993,8.180457570504249,33.303256527001146,5.769925562021384,15.466629545524382,12.651255525791012,15.995160619641469,12.922309957994937,6.556673050829525,6.941908198528731,4.617550389093913,27.335731696306702,5.814856112558841,18.49864018853029,13.96291681057734,35.903064134866256,38.73619761839379,32.56594885594187,54.105909842976,30.470991913000084
[...]
I want to be able to plot a new series per line (legend not necessary). The x-values should just start at 0 and increment from there, the y-values are given by the numbers in the file. The result should be something like this:
This was made in Excel, and if it weren't so labourous, this would be fine.
Note that answers using python/matplotlib are also welcomed.
Thanks a lot!
EDIT: This is as far as I've come, based on the answer provided in gnuplot: How to plot each line of a data file as Y and incremental X and other places. So far, there's for some reason no output file written, although it's definitely doing something (because the last command takes a few seconds to finish).
set term png
set output "fft.png"
set datafile sep ','
plot for [i=1:255] 'fft.csv' using (i):(column(i)) with lines
EDIT 2: I now have this, which is closer, but still not what I want. Each line in the input file should be a series. It seems now each column is a series.
set term png
set output "fft.png"
set datafile sep ','
unset key
plot for [i=1:255] 'fft.csv' using (i):(column(i)) with lines
Right, I think I solved it (using the answer from https://stackoverflow.com/a/8716495/666259 and the answer from Christoph):
set terminal png size 1920,1080
set datafile sep ","
unset key
set xtics 0,5,255
set title "mytitle"
set autoscale cbfix
plot "fft.csv" matrix using 1:3:2 with lines palette
This gives me something like this:
In order to have every line with a different color, use either palette or linecolor variable:
set datafile separator ','
set autoscale cbfix
plot 'file.csv' matrix every :::1 using 1:3:2 with lines palette
This uses the row number to color the lines based on the current palette. This is very useful if you have many lines to draw. set autoscale cbfix prevents the color range to be extended to the next major tics.
For only a few lines, you could also use
plot 'file.csv' matrix every :::1 using 1:3:2 linecolor variable with lines
This uses the row number as index for the linetype.