How to create 2-3 hrs from multiple columns - python

I’ve a df with multiple time series looks like this
df_observ
id
Date
start time
Action 1 time
end of action2
end Time
Observ1 time
observ2 time
observ1 value
observ2 value
indv1
3-2017
00:00:00
02:40:00
04:25:00
04:38:00
00:05:00
01:45:00
57
111
indv2
11-6-2019
00:00:00
00:46:00
02:16:00
02:40:00
01:01:00
02:37:00
68
113
indv2
13-4-2017
00:00:00
02:22:00
04:25:00
04:38:00
00:05:00
03:10:06
82
125
indv3
23-5-2022
00:00:00
01:34:00
02:22:00
03:34:00
02:24:00
03:25:00
67
101
indv4
8-11-2021
00:00:00
00:05:00
03:16:00
03:52:14
01:01:00
02:11:00
63
108
all-time series are subtracted from the start time. is there a way to plot obsev value changes in different time points ?
Thanks!

Related

why I take this plot with matplotlib.pyplot when add date too x axis

First data frame:
date time open high low close volume avg
0 2021-05-23 00:00:00 37458.51 38270.64 31111.01 34655.25 217136.046593 NaN
1 2021-05-24 00:00:00 34681.44 39920.00 34031.00 38796.29 161630.893971 NaN
2 2021-05-25 00:00:00 38810.99 39791.77 36419.62 38324.72 111996.228404 NaN
3 2021-05-26 00:00:00 38324.72 40841.00 37800.44 39241.91 104780.773396 NaN
4 2021-05-27 00:00:00 39241.92 40411.14 37134.27 38529.98 86547.158794 NaN
5 2021-05-28 00:00:00 38529.99 38877.83 34684.00 35663.49 135377.629720 NaN
6 2021-05-29 00:00:00 35661.79 37338.58 33632.76 34605.15 112663.092689 NaN
7 2021-05-30 00:00:00 34605.15 36488.00 33379.00 35641.27 73535.386967 NaN
8 2021-05-31 00:00:00 35641.26 37499.00 34153.84 37253.81 94160.735289 NaN
9 2021-01-06 00:00:00 37253.82 37894.81 35666.00 36693.09 81234.663770 NaN
10 2021-02-06 00:00:00 36694.85 38225.00 35920.00 37568.68 67587.372495 NaN
11 2021-03-06 00:00:00 37568.68 39476.00 37170.00 39246.79 75889.106011 NaN
12 2021-04-06 00:00:00 39246.78 39289.07 35555.15 36829.00 91317.799245 NaN
13 2021-05-06 00:00:00 36829.15 37925.00 34800.00 35513.20 70459.621490 NaN
14 2021-06-06 00:00:00 35516.07 36480.00 35222.00 35796.31 47650.206637 NaN
15 2021-07-06 00:00:00 35796.31 36900.00 33300.00 33552.79 77574.952573 NaN
16 2021-08-06 00:00:00 33556.96 34068.01 31000.00 33380.81 123251.189037 NaN
17 2021-09-06 00:00:00 33380.80 37534.79 32396.82 37388.05 136607.597517 NaN
18 2021-10-06 00:00:00 37388.05 38491.00 35782.00 36675.72 109527.284943 NaN
19 2021-11-06 00:00:00 36677.83 37680.40 35936.77 37331.98 78466.005300 NaN
20 2021-12-06 00:00:00 37331.98 37463.63 34600.36 35546.11 87717.549990 NaN
21 2021-06-13 00:00:00 35546.12 39380.00 34757.00 39020.57 86921.025555 NaN
22 2021-06-14 00:00:00 39020.56 41064.05 38730.00 40516.29 108522.391949 NaN
23 2021-06-15 00:00:00 40516.28 41330.00 39506.40 40144.04 80679.622838 NaN
24 2021-06-16 00:00:00 40143.80 40527.14 38116.01 38349.01 87771.976937 NaN
25 2021-06-17 00:00:00 38349.00 39559.88 37365.00 38092.97 79541.307119 NaN
26 2021-06-18 00:00:00 38092.97 38202.84 35129.29 35819.84 95228.042935 NaN
27 2021-06-19 00:00:00 35820.48 36457.00 34803.52 35483.72 68712.449461 NaN
28 2021-06-20 00:00:00 35483.72 36137.72 33336.00 35600.16 89878.170850 NaN
29 2021-06-21 00:00:00 35600.17 35750.00 31251.23 31608.93 168778.873159 NaN
30 2021-06-22 00:00:00 31614.12 33298.78 28805.00 32509.56 204208.179762 NaN
31 2021-06-23 00:00:00 32509.56 34881.00 31683.00 33678.07 126966.100563 NaN
32 2021-06-24 00:00:00 33675.07 35298.00 32286.57 34663.09 86625.804260 NaN
33 2021-06-25 00:00:00 34663.08 35500.00 31275.00 31584.45 116061.130356 NaN
34 2021-06-26 00:00:00 31576.09 32730.00 30151.00 32283.65 107820.375287 NaN
35 2021-06-27 00:00:00 32283.65 34749.00 31973.45 34700.34 96613.244211 NaN
36 2021-06-28 00:00:00 34702.49 35297.71 33862.72 34494.89 82222.267819 NaN
37 2021-06-29 00:00:00 34494.89 36600.00 34225.43 35911.73 90788.796220 NaN
38 2021-06-30 00:00:00 35911.72 36100.00 34017.55 35045.00 77152.197634 NaN
39 2021-01-07 00:00:00 35045.00 35057.57 32711.00 33504.69 71708.266112 15.362372
40 2021-02-07 00:00:00 33502.33 33977.04 32699.00 33786.55 56172.181378 15.386331
41 2021-03-07 00:00:00 33786.54 34945.61 33316.73 34669.13 43044.578641 15.154877
42 2021-04-07 00:00:00 34669.12 35967.85 34357.15 35286.51 43703.475789 14.677524
43 2021-05-07 00:00:00 35288.13 35293.78 33125.55 33690.14 64123.874245 14.486827
44 2021-06-07 00:00:00 33690.15 35118.88 33532.00 34220.01 58210.596349 14.305665
45 2021-07-07 00:00:00 34220.02 35059.09 33777.77 33862.12 53807.521675 14.133561
46 2021-08-07 00:00:00 33862.11 33929.64 32077.00 32875.71 70136.480320 14.336865
47 2021-09-07 00:00:00 32875.71 34100.00 32261.07 33815.81 47153.939899 14.479159
48 2021-10-07 00:00:00 33815.81 34262.00 33004.78 33502.87 34761.175468 14.564313
49 2021-11-07 00:00:00 33502.87 34666.00 33306.47 34258.99 31572.647448 14.517866
50 2021-12-07 00:00:00 34259.00 34678.43 32658.34 33086.63 48181.403762 14.627892
51 2021-07-13 00:00:00 33086.94 33340.00 32202.25 32729.77 41126.361008 14.839689
52 2021-07-14 00:00:00 32729.12 33114.03 31550.00 32820.02 46777.823484 15.192346
53 2021-07-15 00:00:00 32820.03 33185.25 31133.00 31880.00 51639.576353 15.623083
54 2021-07-16 00:00:00 31874.49 32249.18 31020.00 31383.87 48499.864154 16.058731
55 2021-07-17 00:00:00 31383.86 31955.92 31164.31 31520.07 34012.242132 16.472596
56 2021-07-18 00:00:00 31520.07 32435.00 31108.97 31778.56 35923.716186 16.669426
57 2021-07-19 00:00:00 31778.57 31899.00 30407.44 30839.65 47340.468499 17.041150
58 2021-07-20 00:00:00 30839.65 31063.07 29278.00 29790.35 61034.049017 17.671053
59 2021-07-21 00:00:00 29790.34 32858.00 29482.61 32144.51 82796.265128 17.564616
60 2021-07-22 00:00:00 32144.51 32591.35 31708.00 32287.83 46148.092433 17.463500
61 2021-07-23 00:00:00 32287.58 33650.00 31924.32 33634.09 50112.863626 16.984139
62 2021-07-24 00:00:00 33634.10 34500.00 33401.14 34258.14 47977.550138 16.242346
63 2021-07-25 00:00:00 34261.51 35398.00 33851.12 35381.02 47852.928313 15.607586
64 2021-07-26 00:00:00 35381.02 40550.00 35205.78 37237.60 152452.512724 16.219395
65 2021-07-27 00:00:00 37241.33 39542.61 36383.00 39457.87 88397.267015 16.800613
66 2021-07-28 00:00:00 39456.61 40900.00 38772.00 40019.56 101344.528441 17.599907
67 2021-07-29 00:00:00 40019.57 40640.00 39200.00 40016.48 53998.439283 18.359237
68 2021-07-30 00:00:00 40018.49 42316.71 38313.23 42206.37 73602.784805 19.368676
69 2021-07-31 00:00:00 42206.36 42448.00 41000.15 41461.83 44849.791012 20.349200
70 2021-01-08 00:00:00 41461.84 42599.00 39422.01 39845.44 53953.186326 20.714136
71 2021-02-08 00:00:00 39850.27 40480.01 38690.00 39147.82 50837.351954 20.816480
72 2021-03-08 00:00:00 39146.86 39780.00 37642.03 38207.05 57117.435853 20.578895
73 2021-04-08 00:00:00 38207.04 39969.66 37508.56 39723.18 52329.352430 20.396351
74 2021-05-08 00:00:00 39723.17 41350.00 37332.70 40862.46 84343.755621 20.526294
75 2021-06-08 00:00:00 40862.46 43392.43 39853.86 42836.87 75753.941347 21.042989
76 2021-07-08 00:00:00 42836.87 44700.00 42446.41 44572.54 73396.740808 21.756471
77 2021-08-08 00:00:00 44572.54 45310.00 43261.00 43794.37 69329.092698 22.533424
78 2021-09-08 00:00:00 43794.36 46454.15 42779.00 46253.40 74587.884845 23.450453
79 2021-10-08 00:00:00 46248.87 46700.00 44589.46 45584.99 53814.643421 24.359303
80 2021-11-08 00:00:00 45585.00 46743.47 45341.14 45511.00 52734.901977 25.229618
81 2021-12-08 00:00:00 45510.67 46218.12 43770.00 44399.00 55266.108781 25.471002
82 2021-08-13 00:00:00 44400.06 47886.00 44217.39 47800.00 48239.370431 25.995794
83 2021-08-14 00:00:00 47799.99 48144.00 45971.03 47068.51 46114.359022 26.537795
84 2021-08-15 00:00:00 47068.50 47372.27 45500.00 46973.82 42110.711334 26.878796
85 2021-08-16 00:00:00 46973.82 48053.83 45660.00 45901.29 52480.574014 27.326937
86 2021-08-17 00:00:00 45901.30 47160.00 44376.00 44695.95 57039.341629 27.285215
87 2021-08-18 00:00:00 44695.95 46000.00 44203.28 44705.29 54099.415985 27.184539
88 2021-08-19 00:00:00 44699.37 47033.00 43927.70 46760.62 53411.753920 27.302916
89 2021-08-20 00:00:00 46760.62 49382.99 46622.99 49322.47 56850.352228 27.840242
90 2021-08-21 00:00:00 49322.47 49757.04 48222.00 48821.87 46745.136584 28.412062
91 2021-08-22 00:00:00 48821.88 49500.00 48050.00 49239.22 37007.887795 28.889153
92 2021-08-23 00:00:00 49239.22 50500.00 49029.00 49488.85 52462.541954 29.512800
93 2021-08-24 00:00:00 49488.85 49860.00 47600.00 47674.01 51014.594748 29.565824
94 2021-08-25 00:00:00 47674.01 49264.30 47126.28 48973.32 44655.830342 29.446836
95 2021-08-26 00:00:00 48973.32 49352.84 46250.00 46843.87 49371.277774 29.028026
96 2021-08-27 00:00:00 46843.86 49149.93 46348.00 49069.90 42068.104965 28.630156
97 2021-08-28 00:00:00 49069.90 49299.00 48346.88 48895.35 26681.063786 28.287626
98 2021-08-29 00:00:00 48895.35 49632.27 47762.54 48767.83 32652.283473 27.744622
99 2021-08-30 00:00:00 48767.84 48888.61 46853.00 46982.91 40288.350830 26.903998
100 2021-08-31 00:00:00 46982.91 48246.11 46700.00 47100.89 48645.527370 26.051605
101 2021-01-09 00:00:00 47100.89 49156.00 46512.00 48810.52 49904.655280 25.499838
102 2021-02-09 00:00:00 48810.51 50450.13 48584.06 49246.64 54410.770538 25.311075
103 2021-03-09 00:00:00 49246.63 51000.00 48316.84 49999.14 59025.644157 25.265214
104 2021-04-09 00:00:00 49998.00 50535.69 49370.00 49915.64 34664.659590 25.221647
105 2021-05-09 00:00:00 49917.54 51900.00 49450.00 51756.88 40544.835873 25.504286
106 2021-06-09 00:00:00 51756.88 52780.00 50969.33 52663.90 49249.667081 25.962876
107 2021-07-09 00:00:00 52666.20 52920.00 42843.05 46863.73 123048.802719 25.276717
108 2021-08-09 00:00:00 46868.57 47340.99 44412.02 46048.31 65069.315200 24.624866
109 2021-09-09 00:00:00 46048.31 47399.97 45513.08 46395.14 50651.660020 23.989928
110 2021-10-09 00:00:00 46395.14 47033.00 44132.29 44850.91 49048.266180 23.670387
111 2021-11-09 00:00:00 44842.20 45987.93 44722.22 45173.69 30440.408100 23.366822
112 2021-12-09 00:00:00 45173.68 46460.00 44742.06 46025.24 32094.280520 22.938381
113 2021-09-13 00:00:00 46025.23 46880.00 43370.00 44940.73 65429.150560 22.820722
114 2021-09-14 00:00:00 44940.72 47250.00 44594.44 47111.52 44855.850990 22.594896
115 2021-09-15 00:00:00 47103.28 48500.00 46682.32 48121.41 43204.711740 22.007531
116 2021-09-16 00:00:00 48121.40 48557.00 47021.10 47737.82 40725.088950 21.432816
117 2021-09-17 00:00:00 47737.81 48150.00 46699.56 47299.98 34461.927760 20.965565
118 2021-09-18 00:00:00 47299.98 48843.20 47035.56 48292.74 30906.470380 20.306487
119 2021-09-19 00:00:00 48292.75 48372.83 46829.18 47241.75 29847.243490 19.735184
120 2021-09-20 00:00:00 47241.75 47347.25 42500.00 43015.62 78003.524443 20.139851
121 2021-09-21 00:00:00 43016.64 43639.00 39600.00 40734.38 84534.080485 20.985744
122 2021-09-22 00:00:00 40734.09 44000.55 40565.39 43543.61 58349.055420 21.676235
123 2021-09-23 00:00:00 43546.37 44978.00 43069.09 44865.26 48699.576550 22.029837
124 2021-09-24 00:00:00 44865.26 45200.00 40675.00 42810.57 84113.426292 22.735109
125 2021-09-25 00:00:00 42810.58 42966.84 41646.28 42670.64 33594.571890 23.405118
126 2021-09-26 00:00:00 42670.63 43950.00 40750.00 43160.90 49879.997650 23.734984
127 2021-09-27 00:00:00 43160.90 44350.00 42098.00 42147.35 39776.843830 23.925323
128 2021-09-28 00:00:00 42147.35 42787.38 40888.00 41026.54 43372.262400 24.312088
129 2021-09-29 00:00:00 41025.01 42590.00 40753.88 41524.28 33511.534870 24.702028
130 2021-09-30 00:00:00 41524.29 44141.37 41410.17 43824.10 46381.227810 24.581907
131 2021-01-10 00:00:00 43820.01 48495.00 43283.03 48141.61 66244.874920 23.367632
132 2021-02-10 00:00:00 48141.60 48336.59 47430.18 47634.90 30508.981310 22.214071
133 2021-03-10 00:00:00 47634.89 49228.08 47088.00 48200.01 30825.056010 21.285226
134 2021-04-10 00:00:00 48200.01 49536.12 46891.00 49224.94 46796.493720 20.470586
135 2021-05-10 00:00:00 49224.93 51886.30 49022.40 51471.99 52125.667930 20.178783
136 2021-06-10 00:00:00 51471.99 55750.00 50382.41 55315.00 79877.545181 20.539207
137 2021-07-10 00:00:00 55315.00 55332.31 53357.00 53785.22 54917.377660 20.881611
138 2021-08-10 00:00:00 53785.22 56100.00 53617.61 53951.43 46160.257850 21.322501
139 2021-09-10 00:00:00 53955.67 55489.00 53661.67 54949.72 55177.080130 21.741347
140 2021-10-10 00:00:00 54949.72 56561.31 54080.00 54659.00 89237.836128 22.304343
141 2021-11-10 00:00:00 54659.01 57839.04 54415.06 57471.35 52933.165751 23.025557
142 2021-12-10 00:00:00 57471.35 57680.00 53879.00 55996.93 53471.285500 23.546775
143 2021-10-13 00:00:00 55996.91 57777.00 54167.19 57367.00 55808.444920 24.057061
144 2021-10-14 00:00:00 57370.83 58532.54 56818.05 57347.94 43053.336781 24.660876
145 2021-10-15 00:00:00 57347.94 62933.00 56850.00 61672.42 82512.908022 25.811065
146 2021-10-16 00:00:00 61672.42 62378.42 60150.00 60875.57 35467.880960 26.903744
147 2021-10-17 00:00:00 60875.57 61718.39 58963.00 61528.33 39099.241240 27.563757
148 2021-10-18 00:00:00 61528.32 62695.78 59844.45 62009.84 51798.448440 28.318027
149 2021-10-19 00:00:00 62005.60 64486.00 61322.22 64280.59 53628.107744 29.251726
150 2021-10-20 00:00:00 64280.59 67000.00 63481.40 66001.41 51428.934856 30.405550
151 2021-10-21 00:00:00 66001.40 66639.74 62000.00 62193.15 68538.645370 31.054053
152 2021-10-22 00:00:00 62193.15 63732.39 60000.00 60688.22 52119.358860 31.117531
153 2021-10-23 00:00:00 60688.23 61747.64 59562.15 61286.75 27626.936780 31.062358
154 2021-10-24 00:00:00 61286.75 61500.00 59510.63 60852.22 31226.576760 30.995921
155 2021-10-25 00:00:00 60852.22 63710.63 60650.00 63078.78 36853.838060 31.244720
156 2021-10-26 00:00:00 63078.78 63293.48 59817.55 60328.81 40217.500830 31.249961
157 2021-10-27 00:00:00 60328.81 61496.00 58000.00 58413.44 62124.490160 30.779004
158 2021-10-28 00:00:00 58413.44 62499.00 57820.00 60575.89 61056.353010 30.489479
159 2021-10-29 00:00:00 60575.90 62980.00 60174.81 62253.71 43973.904140 30.289382
160 2021-10-30 00:00:00 62253.70 62359.25 60673.00 61859.19 31478.125660 30.099291
161 2021-10-31 00:00:00 61859.19 62405.30 59945.36 61299.80 39267.637940 29.713720
162 2021-01-11 00:00:00 61299.81 62437.74 59405.00 60911.11 44687.666720 29.196216
163 2021-02-11 00:00:00 60911.12 64270.00 60624.68 63219.99 46368.284100 29.031364
164 2021-03-11 00:00:00 63220.57 63500.00 60382.76 62896.48 43336.090490 28.804634
165 2021-04-11 00:00:00 62896.49 63086.31 60677.01 61395.01 35930.933140 28.589242
166 2021-05-11 00:00:00 61395.01 62595.72 60721.00 60937.12 31604.487490 28.384619
167 2021-06-11 00:00:00 60940.18 61560.49 60050.00 61470.61 25590.574080 27.973716
168 2021-07-11 00:00:00 61470.62 63286.35 61322.78 63273.59 25515.688300 27.926901
169 2021-08-11 00:00:00 63273.58 67789.00 63273.58 67525.83 54442.094554 28.579845
170 2021-09-11 00:00:00 67525.82 68524.25 66222.40 66947.66 44661.378068 29.294016
171 2021-10-11 00:00:00 66947.67 69000.00 62822.90 64882.43 65171.504046 29.014734
172 2021-11-11 00:00:00 64882.42 65600.07 64100.00 64774.26 37237.980580 28.749416
173 2021-12-11 00:00:00 64774.25 65450.70 62278.00 64122.23 44490.108160 28.041179
174 2021-11-13 00:00:00 64122.22 65000.00 63360.22 64380.00 22504.973830 27.368353
175 2021-11-14 00:00:00 64380.01 65550.51 63576.27 65519.10 25705.073470 26.832078
176 2021-11-15 00:00:00 65519.11 66401.82 63400.00 63606.74 37829.371240 26.479925
177 2021-11-16 00:00:00 63606.73 63617.31 58574.07 60058.87 77455.156090 25.267463
178 2021-11-17 00:00:00 60058.87 60840.23 58373.00 60344.87 46289.384910 24.154719
179 2021-11-18 00:00:00 60344.86 60976.00 56474.26 56891.62 62146.999310 23.454728
180 2021-11-19 00:00:00 56891.62 58320.00 55600.00 58052.24 50715.887260 22.944550
181 2021-11-20 00:00:00 58057.10 59845.00 57353.00 59707.51 33811.590100 22.122892
182 2021-11-21 00:00:00 59707.52 60029.76 58486.65 58622.02 31902.227850 21.302202
183 2021-11-22 00:00:00 58617.70 59444.00 55610.00 56247.18 51724.320470 21.040602
184 2021-11-23 00:00:00 56243.83 58009.99 55317.00 57541.27 49917.850170 20.840946
185 2021-11-24 00:00:00 57541.26 57735.00 55837.00 57138.29 39612.049640 20.651273
186 2021-11-25 00:00:00 57138.29 59398.90 57000.00 58960.36 42153.515220 20.071560
187 2021-11-26 00:00:00 58960.37 59150.00 53500.00 53726.53 65927.870660 20.117912
188 2021-11-27 00:00:00 53723.72 55280.00 53610.00 54721.03 29716.999570 20.161946
189 2021-11-28 00:00:00 54716.47 57445.05 53256.64 57274.88 36163.713700 19.704241
190 2021-11-29 00:00:00 57274.89 58865.97 56666.67 57776.25 40125.280090 18.969898
191 2021-11-30 00:00:00 57776.25 59176.99 55875.55 56950.56 49161.051940 18.417868
192 2021-01-12 00:00:00 56950.56 59053.55 56458.01 57184.07 44956.636560 17.893439
193 2021-02-12 00:00:00 57184.07 57375.47 55777.77 56480.34 37574.059760 17.525876
194 2021-03-12 00:00:00 56484.26 57600.00 51680.00 53601.05 58927.690270 17.858850
195 2021-04-12 00:00:00 53601.05 53859.10 42000.30 49152.47 114203.373748 19.217441
196 2021-05-12 00:00:00 49152.46 49699.05 47727.21 49396.33 45580.820120 20.508102
197 2021-06-12 00:00:00 49396.32 50891.11 47100.00 50441.92 58571.215750 21.472003
198 2021-07-12 00:00:00 50441.91 51936.33 50039.74 50588.95 38253.468770 22.161968
199 2021-08-12 00:00:00 50588.95 51200.00 48600.00 50471.19 38425.924660 22.962218
200 2021-09-12 00:00:00 50471.19 50797.76 47320.00 47545.59 37692.686650 23.846688
201 2021-10-12 00:00:00 47535.90 50125.00 46852.00 47140.54 44233.573910 24.732127
202 2021-11-12 00:00:00 47140.54 49485.71 46751.00 49389.99 28889.193580 25.583369
203 2021-12-12 00:00:00 49389.99 50777.00 48638.00 50053.90 26017.934210 26.077754
204 2021-12-13 00:00:00 50053.90 50189.97 45672.75 46702.75 50869.520930 26.859770
205 2021-12-14 00:00:00 46702.76 48700.41 46290.00 48343.28 39955.984450 27.602685
206 2021-12-15 00:00:00 48336.95 49500.00 46547.00 48864.98 51629.181000 28.109255
207 2021-12-16 00:00:00 48864.98 49436.43 47511.00 47632.38 31949.867390 28.590496
208 2021-12-17 00:00:00 47632.38 47995.96 45456.00 46131.20 43104.488700 29.278437
209 2021-12-18 00:00:00 46133.83 47392.37 45500.00 46834.48 25020.052710 29.931981
210 2021-12-19 00:00:00 46834.47 48300.01 46406.91 46681.23 29305.706650 30.303705
211 2021-12-20 00:00:00 46681.24 47537.57 45558.85 46914.16 35848.506090 30.761072
212 2021-12-21 00:00:00 46914.17 49328.96 46630.00 48889.88 37713.929240 30.715132
213 2021-12-22 00:00:00 48887.59 49576.13 48421.87 48588.16 27004.202200 30.607162
214 2021-12-23 00:00:00 48588.17 51375.00 47920.42 50838.81 35192.540460 30.051098
215 2021-12-24 00:00:00 50838.82 51810.00 50384.43 50820.00 31661.949460 29.417439
When run below code is well. But I need date in x axis
test['avg'].plot(legend=True,figsize=(12,5))
plt.grid(True)
plt.xlabel('ADX')
plt.ylabel('date')
plt.title('ADX indicator')
plt.gcf().autofmt_xdate()
plt.show()
Correct plot:
But when I chose date for x axis, I take a bad plot. Code is below:
df.set_index('date',drop=True, inplace=True)
Modified data
test['avg'].plot(legend=True,figsize=(12,5))
plt.grid(True)
plt.xlabel('ADX')
plt.ylabel('date')
plt.title('ADX indicator')
plt.gcf().autofmt_xdate()
plt.show()
Bad plot:
and also why I take NaN value for ADX in TA-lib
Can you help me with this problem?
It does appear to be the problem of the source file. The column names are not tab separated. Once this is fixed, the plotting works fine.
The NaN issue is also the source file; the average was not calculated for the first several rows.
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
test = pd.read_csv(r"modified_data.dat", sep='\t')
test.set_index('date')
date = test['date']
avg = test['avg']
fig, ax = plt.subplots(figsize=(20,10))
ax.plot(date, avg)
ax.tick_params(rotation=30, width = 2)
plt.xticks(np.arange(0, len(date)+1, 5))
ax.set_xticks
Output looks like this:

How to get odd hours to even hours in pandas dataframe?

I have such a dataframe with "normal" steps of two hours between the timestamps. But sometimes there are unfortunately gaps within my data. Because of that I would like to round timestamps with odd hours (01:00, 03:00 etc.) to even hours (02:00, 04:00 etc.). Time is my index column.
My dataframe looks like this:
Time Values
2021-10-24 22:00:00 2
2021-10-25 00:00:00 4
2021-10-25 02:00:00 78
2021-10-25 05:00:00 90
2021-10-25 07:00:00 1
How can I get a dataframe like this?
Time Values
2021-10-24 22:00:00 2
2021-10-25 00:00:00 4
2021-10-25 02:00:00 78
2021-10-25 06:00:00 90
2021-10-25 08:00:00 1
Use DateTimeIndex.floor or DateTimeIndex.ceil with a frequency string 2H depending if you want to down or upsample.
df.index = df.index.ceil('2H')
>>> df
Values
Time
2021-10-24 22:00:00 2
2021-10-25 00:00:00 4
2021-10-25 02:00:00 78
2021-10-25 06:00:00 90
2021-10-25 08:00:00 1
If "Time" is a column (and not the index), you can use dt.ceil:
df["Time"] = df["Time"].dt.ceil("2H")
>>> df
Time Values
0 2021-10-24 22:00:00 2
1 2021-10-25 00:00:00 4
2 2021-10-25 02:00:00 78
3 2021-10-25 06:00:00 90
4 2021-10-25 08:00:00 2
Alternatively, if you want to ensure that the data contains every 2-hour interval, you could resample:
df = df.resample("2H", on="Time", closed="right").sum()
>>> df
Values
Time
2021-10-24 22:00:00 2
2021-10-25 00:00:00 4
2021-10-25 02:00:00 78
2021-10-25 04:00:00 0
2021-10-25 06:00:00 90
2021-10-25 08:00:00 2

Pandas (Python): How to apply values to similar row?

sorry for the badly phrased question, currently only the first hour is updated with holiday.
e.g.
2013-01-01 00:00:00 - New Years Day
2013-01-01 00:00:00 - None
2013-01-01 00:00:00 - None
I would like to apply similar holidays to the same date using Pandas (Python).
What would be the most efficient method to apply the holiday to the same dates, there are a number of other holidays to apply as well?
Thank you in advance!
Screenshot of CSV in question
Using a library called holidays together with pandas apply could be a great solution to your problem. Here is a short contained example example
import pandas as pd
import holidays
us_holidays = holidays.UnitedStates()
# Create a sample DataFrame. You can just use your own
data = pd.DataFrame(pd.date_range('2020-01-01', '2020-01-30'), columns=['date'])
data['holiday'] = data['date'].apply(lambda x: us_holidays.get(x))
print(data)
Output
date holiday
0 2020-01-01 New Year's Day
1 2020-01-02 None
2 2020-01-03 None
3 2020-01-04 None
4 2020-01-05 None
5 2020-01-06 None
6 2020-01-07 None
7 2020-01-08 None
8 2020-01-09 None
9 2020-01-10 None
10 2020-01-11 None
11 2020-01-12 None
12 2020-01-13 None
13 2020-01-14 None
14 2020-01-15 None
15 2020-01-16 None
16 2020-01-17 None
17 2020-01-18 None
18 2020-01-19 None
19 2020-01-20 Martin Luther King, Jr. Day
20 2020-01-21 None
21 2020-01-22 None
22 2020-01-23 None
23 2020-01-24 None
24 2020-01-25 None
25 2020-01-26 None
26 2020-01-27 None
27 2020-01-28 None
28 2020-01-29 None
29 2020-01-30 None
IIUC, you have only the first hour of a day listed with a holiday. Here is a small sample of a dataframe with two months of data and three holidays on three separate days.
import pandas as pd
import numpy as np
df = pd.DataFrame({'temp':np.random.randint(50,110, 60*24)}, index=pd.date_range('2013-01-01', periods=(60*24), freq='H'))
df['Holiday'] = np.nan
df.loc['2013-01-01 00:00:00', 'Holiday'] = 'New Years Day'
df.loc['2013-02-02 00:00:00', 'Holiday'] = 'Groundhog Day'
df.loc['2013-02-14 00:00:00', 'Holiday'] = "Valentine's Day"
Now, let's use groupby with day from DatetimeIndex and ffill:
df['Holiday'] = df.groupby(df.index.day)['Holiday'].ffill()
Let's look at a few records:
print(df.head(40))
print(df['2013-02-02'])
print(df['2013-02-13':'2013-02-15'])
Output:
temp Holiday
2013-01-01 00:00:00 51 New Years Day
2013-01-01 01:00:00 71 New Years Day
2013-01-01 02:00:00 61 New Years Day
2013-01-01 03:00:00 90 New Years Day
2013-01-01 04:00:00 77 New Years Day
2013-01-01 05:00:00 69 New Years Day
2013-01-01 06:00:00 50 New Years Day
2013-01-01 07:00:00 99 New Years Day
2013-01-01 08:00:00 86 New Years Day
2013-01-01 09:00:00 72 New Years Day
2013-01-01 10:00:00 89 New Years Day
2013-01-01 11:00:00 62 New Years Day
2013-01-01 12:00:00 53 New Years Day
2013-01-01 13:00:00 91 New Years Day
2013-01-01 14:00:00 51 New Years Day
2013-01-01 15:00:00 93 New Years Day
2013-01-01 16:00:00 97 New Years Day
2013-01-01 17:00:00 83 New Years Day
2013-01-01 18:00:00 87 New Years Day
2013-01-01 19:00:00 58 New Years Day
2013-01-01 20:00:00 84 New Years Day
2013-01-01 21:00:00 92 New Years Day
2013-01-01 22:00:00 106 New Years Day
2013-01-01 23:00:00 104 New Years Day
2013-01-02 00:00:00 78 NaN
2013-01-02 01:00:00 104 NaN
2013-01-02 02:00:00 96 NaN
2013-01-02 03:00:00 103 NaN
2013-01-02 04:00:00 60 NaN
2013-01-02 05:00:00 87 NaN
2013-01-02 06:00:00 108 NaN
2013-01-02 07:00:00 85 NaN
2013-01-02 08:00:00 67 NaN
2013-01-02 09:00:00 61 NaN
2013-01-02 10:00:00 91 NaN
2013-01-02 11:00:00 79 NaN
2013-01-02 12:00:00 99 NaN
2013-01-02 13:00:00 82 NaN
2013-01-02 14:00:00 75 NaN
2013-01-02 15:00:00 90 NaN
temp Holiday
2013-02-02 00:00:00 82 Groundhog Day
2013-02-02 01:00:00 58 Groundhog Day
2013-02-02 02:00:00 102 Groundhog Day
2013-02-02 03:00:00 90 Groundhog Day
2013-02-02 04:00:00 79 Groundhog Day
2013-02-02 05:00:00 50 Groundhog Day
2013-02-02 06:00:00 50 Groundhog Day
2013-02-02 07:00:00 83 Groundhog Day
2013-02-02 08:00:00 80 Groundhog Day
2013-02-02 09:00:00 50 Groundhog Day
2013-02-02 10:00:00 52 Groundhog Day
2013-02-02 11:00:00 69 Groundhog Day
2013-02-02 12:00:00 100 Groundhog Day
2013-02-02 13:00:00 61 Groundhog Day
2013-02-02 14:00:00 62 Groundhog Day
2013-02-02 15:00:00 76 Groundhog Day
2013-02-02 16:00:00 83 Groundhog Day
2013-02-02 17:00:00 109 Groundhog Day
2013-02-02 18:00:00 109 Groundhog Day
2013-02-02 19:00:00 81 Groundhog Day
2013-02-02 20:00:00 52 Groundhog Day
2013-02-02 21:00:00 108 Groundhog Day
2013-02-02 22:00:00 68 Groundhog Day
2013-02-02 23:00:00 75 Groundhog Day
temp Holiday
2013-02-13 00:00:00 93 NaN
2013-02-13 01:00:00 93 NaN
2013-02-13 02:00:00 74 NaN
2013-02-13 03:00:00 97 NaN
2013-02-13 04:00:00 58 NaN
2013-02-13 05:00:00 103 NaN
2013-02-13 06:00:00 79 NaN
2013-02-13 07:00:00 65 NaN
2013-02-13 08:00:00 72 NaN
2013-02-13 09:00:00 100 NaN
2013-02-13 10:00:00 66 NaN
2013-02-13 11:00:00 60 NaN
2013-02-13 12:00:00 95 NaN
2013-02-13 13:00:00 51 NaN
2013-02-13 14:00:00 71 NaN
2013-02-13 15:00:00 58 NaN
2013-02-13 16:00:00 58 NaN
2013-02-13 17:00:00 98 NaN
2013-02-13 18:00:00 61 NaN
2013-02-13 19:00:00 63 NaN
2013-02-13 20:00:00 57 NaN
2013-02-13 21:00:00 102 NaN
2013-02-13 22:00:00 69 NaN
2013-02-13 23:00:00 86 NaN
2013-02-14 00:00:00 94 Valentine's Day
2013-02-14 01:00:00 64 Valentine's Day
2013-02-14 02:00:00 62 Valentine's Day
2013-02-14 03:00:00 59 Valentine's Day
2013-02-14 04:00:00 93 Valentine's Day
2013-02-14 05:00:00 99 Valentine's Day
2013-02-14 06:00:00 64 Valentine's Day
2013-02-14 07:00:00 80 Valentine's Day
2013-02-14 08:00:00 89 Valentine's Day
2013-02-14 09:00:00 96 Valentine's Day
2013-02-14 10:00:00 60 Valentine's Day
2013-02-14 11:00:00 76 Valentine's Day
2013-02-14 12:00:00 82 Valentine's Day
2013-02-14 13:00:00 65 Valentine's Day
2013-02-14 14:00:00 90 Valentine's Day
2013-02-14 15:00:00 62 Valentine's Day
2013-02-14 16:00:00 64 Valentine's Day
2013-02-14 17:00:00 98 Valentine's Day
2013-02-14 18:00:00 52 Valentine's Day
2013-02-14 19:00:00 72 Valentine's Day
2013-02-14 20:00:00 108 Valentine's Day
2013-02-14 21:00:00 85 Valentine's Day
2013-02-14 22:00:00 87 Valentine's Day
2013-02-14 23:00:00 62 Valentine's Day
2013-02-15 00:00:00 106 NaN
2013-02-15 01:00:00 82 NaN
2013-02-15 02:00:00 77 NaN
2013-02-15 03:00:00 52 NaN
2013-02-15 04:00:00 94 NaN
2013-02-15 05:00:00 71 NaN
2013-02-15 06:00:00 95 NaN
2013-02-15 07:00:00 96 NaN
2013-02-15 08:00:00 71 NaN
2013-02-15 09:00:00 69 NaN
2013-02-15 10:00:00 85 NaN
2013-02-15 11:00:00 92 NaN
2013-02-15 12:00:00 106 NaN
2013-02-15 13:00:00 77 NaN
2013-02-15 14:00:00 65 NaN
2013-02-15 15:00:00 104 NaN
2013-02-15 16:00:00 98 NaN
2013-02-15 17:00:00 107 NaN
2013-02-15 18:00:00 106 NaN
2013-02-15 19:00:00 67 NaN
2013-02-15 20:00:00 59 NaN
2013-02-15 21:00:00 81 NaN
2013-02-15 22:00:00 56 NaN
2013-02-15 23:00:00 75 NaN
Note: In this dataframe your datetime column is in the index.
You can try using the apply method: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.apply.html
The input to this is the function you want to be applied to each row. And in this case "axis" should be zero so that it is applied to each row.

Group data by time of the day

I have a dataframe with datetime index:df.head(6)
NUMBERES PRICE
DEAL_TIME
2015-03-02 12:40:03 5 25
2015-03-04 14:52:57 7 23
2015-03-03 08:10:09 10 43
2015-03-02 20:18:24 5 37
2015-03-05 07:50:55 4 61
2015-03-02 09:08:17 1 17
The dataframe includes the data of one week. Now I need to count the time period of the day. If time period is 1 hour, I know the following method would work:
df_grouped = df.groupby(df.index.hour).count()
But I don't know how to do when the time period is half hour. How can I realize it?
UPDATE:
I was told that this question is similar to How to group DataFrame by a period of time?
But I had tried the methods mentioned. Maybe it's my fault that I didn't say it clearly. 'DEAL_TIME' ranges from '2015-03-02 00:00:00' to '2015-03-08 23:59:59'. If I use pd.TimeGrouper(freq='30Min') or resample(), the time periods would range from '2015-03-02 00:30' to '2015-03-08 23:30'. But what I want is a series like below:
COUNT
DEAL_TIME
00:00:00 53
00:30:00 49
01:00:00 31
01:30:00 22
02:00:00 1
02:30:00 24
03:00:00 27
03:30:00 41
04:00:00 41
04:30:00 76
05:00:00 33
05:30:00 16
06:00:00 15
06:30:00 4
07:00:00 60
07:30:00 85
08:00:00 3
08:30:00 37
09:00:00 18
09:30:00 29
10:00:00 31
10:30:00 67
11:00:00 35
11:30:00 60
12:00:00 95
12:30:00 37
13:00:00 30
13:30:00 62
14:00:00 58
14:30:00 44
15:00:00 45
15:30:00 35
16:00:00 94
16:30:00 56
17:00:00 64
17:30:00 43
18:00:00 60
18:30:00 52
19:00:00 14
19:30:00 9
20:00:00 31
20:30:00 71
21:00:00 21
21:30:00 32
22:00:00 61
22:30:00 35
23:00:00 14
23:30:00 21
In other words, the time period should be irrelevant to the date.
You need a 30-minute time grouper for this:
grouper = pd.TimeGrouper(freq="30T")
You also need to remove the 'date' part from the index:
df.index = df.reset_index()['index'].apply(lambda x: x - pd.Timestamp(x.date()))
Now, you can group by time alone:
df.groupby(grouper).count()
You can find somewhat obscure TimeGrouper documentation here: pandas resample documentation (it's actually resample documentation, but both features use the same rules).
In pandas, the most common way to group by time is to use the
.resample() function.
In v0.18.0 this function is two-stage.
This means that df.resample('M') creates an object to which we can
apply other functions (mean, count, sum, etc.)
The code snippet will be like,
df.resample('M').count()
You can refer here for example.

Pandas Dataframe.resample('MS')

I wonder if somebody can help me understand where I may be going wrong. I have a dataframe which contains data (part numbers,sales per month, year) with missing months and I am trying to use dataframe.resample('MS').asfreq() to identify the missing months and insert them into my list. I have performed a groupby previously to ensure that data is collected together prior to the resample.
my code is:
df2=df1[['Part_ID','Order_Qty','Extended_Price','Month','Month1','Year']]
df2['Month1'] = pd.to_datetime(df2.Month.astype(str) +'-01-'+ df2.Year.astype(str))
df2=df2.set_index(pd.DatetimeIndex(df2['Month1']))
df2=df2.groupby([df2['Part_ID'],df2['Year']])
df2=df2['Month1'].resample('MS').asfreq()
df3=df2.to_frame()
print(df3)
df4=df3.reset_index()
After execution some months have been added, but some are missing. Can anybody please explain why?
the output after resample is:
Part_ID Year Month1
08095601/2 2014 2014-07-01 2014-07-01
2014-08-01 2014-08-01
2014-09-01 2014-09-01
2014-10-01 2014-10-01
2014-11-01 2014-11-01
2015 2015-01-01 2015-01-01
2015-02-01 2015-02-01
2015-03-01 2015-03-01
2015-04-01 2015-04-01
2015-05-01 2015-05-01
2015-06-01 2015-06-01
08095601/5 2014 2014-07-01 2014-07-01
2014-08-01 2014-08-01
2014-09-01 2014-09-01
...
ZZSSL 2007 2007-10-01 2007-10-01
2007-11-01 NaT
2007-12-01 2007-12-01
2008 2008-01-01 2008-01-01
2008-02-01 2008-02-01
2008-03-01 2008-03-01
2008-04-01 2008-04-01
2008-05-01 2008-05-01
2008-06-01 2008-06-01
2008-07-01 NaT
2008-08-01 NaT
2008-09-01 2008-09-01
2008-10-01 2008-10-01
2009 2009-01-01 2009-01-01
2009-02-01 2009-02-01
2009-03-01 2009-03-01
2009-04-01 2009-04-01
2009-05-01 2009-05-01
2009-06-01 2009-06-01
2009-07-01 2009-07-01
bracket 2014 2014-07-01 2014-07-01
2014-08-01 2014-08-01
2014-09-01 2014-09-01
2014-10-01 2014-10-01
2014-11-01 2014-11-01
2014-12-01 2014-12-01
2015 2015-01-01 2015-01-01
2015-02-01 NaT
2015-03-01 NaT
2015-04-01 2015-04-01
as you can see for part 08095601/2 in 2014 there is no month 12 and for part ZZSSL there is no month 11 OR 12. bracket has correctly inserted months 2 & 3 in 2015
Any pointers please.

Categories