We Keep Coding

Data Sorting and filtering (From an Excel File)

Please refer to the Data Tables Below...
[Input table]
Input Table
TABLE: Joint Reactions
Joint
LC
F1
F2
F3
M1
M2
M3
Text
Text
KN
KN
KN
KN-m
KN-m
KN-m
1
SF-DL
5.97
4.517
169.179
-2.167
3.249
-0.0042
1
OTH-DL
1.268
0.667
20.592
-0.3083
0.7476
0.0253
1
LL
1.488
1.3
16.341
-0.5738
0.7014
-0.0307
3
SF-DL
5.97
-4.517
169.179
2.167
3.249
0.0042
3
OTH-DL
1.268
-0.667
20.592
0.3083
0.7476
-0.0253
3
LL
1.488
-1.3
16.341
0.5738
0.7014
0.0307
5
SF-DL
1.024
6.039
201.94
-2.1923
0.7081
0.0165
5
OTH-DL
0.373
0.669
31.867
-0.3234
0.2567
0.0234
5
LL
0.2
7.297
28.116
-2.5342
0.1201
-0.021
7
SF-DL
1.024
-6.039
201.94
2.1923
0.7081
-0.0165
7
OTH-DL
0.373
-0.669
31.867
0.3234
0.2567
-0.0234
7
LL
0.2
-7.297
28.116
2.5342
0.1201
0.021
[output table in excel]
Output Table formulae results
TABLE: Joint Reactions
Joint
LC
F1
F2
F3
M1
M2
M3
Text
Text
KN
KN
KN
KN-m
KN-m
KN-m
1
SF-DL
5.97
4.517
169.179
-2.167
3.249
-0.0042
3
SF-DL
5.97
-4.517
169.179
2.167
3.249
0.0042
5
SF-DL
1.024
6.039
201.94
-2.1923
0.7081
0.0165
7
SF-DL
1.024
-6.039
201.94
2.1923
0.7081
-0.0165
1
OTH-DL
1.268
0.667
20.592
-0.3083
0.7476
0.0253
3
OTH-DL
1.268
-0.667
20.592
0.3083
0.7476
-0.0253
5
OTH-DL
0.373
0.669
31.867
-0.3234
0.2567
0.0234
7
OTH-DL
0.373
-0.669
31.867
0.3234
0.2567
-0.0234
1
LL
1.488
1.3
16.341
-0.5738
0.7014
-0.0307
3
LL
1.488
-1.3
16.341
0.5738
0.7014
0.0307
5
LL
0.2
7.297
28.116
-2.5342
0.1201
-0.021
7
LL
0.2
-7.297
28.116
2.5342
0.1201
0.021
I want to sort and filter The Input Table and put the data in output table format.
I have an excel sheet prepared by my friend....He has done Sorting and Filtering using some formulae and functions....refer output table in excel.
I am new to python...and want to do the same thing in python...
extracting data from excel and saving a data frame in python
filtering and sorting as per output table in python
exporting the output table to excel.
Please help me in this regard.
I am unable to come up with a solution since I am new to python

matplotlib for stock data analysis plot not correct

I'm using matplotlib to draw trendance line for stock data.
import pandas as pd
import matplotlib.pyplot as plt
A = pd.read_csv('daily/A.csv', index_col=[0])
print(A)
AAL = pd.read_csv('daily/AAL.csv', index_col=[0])
print(AAL)
A['Close'].plot()
AAL['Close'].plot()
plt.show()
then result is:
High Low Open Close Volume Adj Close
Date
1999-11-18 35.77 28.61 32.55 31.47 62546300.0 27.01
1999-11-19 30.76 28.48 30.71 28.88 15234100.0 24.79
1999-11-22 31.47 28.66 29.55 31.47 6577800.0 27.01
1999-11-23 31.21 28.61 30.40 28.61 5975600.0 24.56
1999-11-24 30.00 28.61 28.70 29.37 4843200.0 25.21
... ... ... ... ... ... ...
2020-06-24 89.08 86.32 89.08 86.56 1806600.0 86.38
2020-06-25 87.35 84.80 86.43 87.26 1350100.0 87.08
2020-06-26 87.56 85.52 87.23 85.90 2225800.0 85.72
2020-06-29 87.36 86.11 86.56 87.29 1302500.0 87.29
2020-06-30 88.88 87.24 87.33 88.37 1428931.0 88.37
[5186 rows x 6 columns]
High Low Open Close Volume Adj Close
Date
2005-09-27 21.40 19.10 21.05 19.30 961200.0 18.19
2005-09-28 20.53 19.20 19.30 20.50 5747900.0 19.33
2005-09-29 20.58 20.10 20.40 20.21 1078200.0 19.05
2005-09-30 21.05 20.18 20.26 21.01 3123300.0 19.81
2005-10-03 21.75 20.90 20.90 21.50 1057900.0 20.27
... ... ... ... ... ... ...
2020-06-24 13.90 12.83 13.59 13.04 140975500.0 13.04
2020-06-25 13.24 12.18 12.53 13.17 117383400.0 13.17
2020-06-26 13.29 12.13 13.20 12.38 108813000.0 12.38
2020-06-29 13.51 12.02 12.57 13.32 114650300.0 13.32
2020-06-30 13.48 12.88 13.10 13.07 68669742.0 13.07
[3715 rows x 6 columns]
yes, the begin of 2 stocks is different, the end date is same.
so i get the plot is like this:
stockplot
this is not normal like others.
so, who could give me any advice, to draw a normal trendance line for 2 stocks?

You can try for making two different plots with same limits and then put one over the another for comparison.

Stooq API for non-US stock market data

I have been trying to complete current data for several stocks from WSE (Warsaw Stock Exchange). I am focused on stooq as yahoo doesn't cover polish stock market.
Once I try the code from pandas-datareader for stooq, I see the below error:
"StooqDailyReader request returned no data;
check URL for invalid inputs: https://stooq.com/q/d/l/"
Under this link csv file telling me the ticker provided is wrong, but the same ticker works well on stooq website directly.
Do you know what may be wrong there?
import pandas_datareader.data as web
prices = web.DataReader('KGH', 'stooq')
print(prices)

Do you want something like this?
import quandl
mydata = quandl.get("WSE/WIG20TR")
print(mydata)
That, produces the following.
Open High Low Close % Change Turnover (1000s)
Date
2016-10-03 2911.76 2919.84 2911.76 2919.84 0.93 401910.61
2016-10-04 2940.43 2968.24 2940.43 2968.24 1.66 707134.92
2016-10-05 2970.79 2982.08 2970.79 2982.08 0.47 713899.62
2016-10-06 2972.15 2981.52 2972.15 2981.52 -0.02 522210.82
2016-10-07 2972.68 2974.87 2964.30 2964.30 -0.58 456252.80
2016-10-10 2977.32 2988.68 2977.32 2988.39 0.81 472514.88
2016-10-11 2987.31 2987.31 2973.30 2973.30 -0.50 457603.07
2016-10-12 2969.48 2979.04 2969.48 2979.04 0.19 447665.23
2016-10-13 2954.50 2954.50 2923.40 2923.40 -1.87 720287.00
2016-10-14 2937.36 2937.36 2910.02 2910.02 -0.46 498163.77
2016-10-17 2919.28 2919.28 2900.82 2900.82 -0.32 621362.01
2016-10-18 2922.67 2922.67 2901.12 2912.68 0.41 548288.70
2016-10-19 2919.90 2956.99 2919.90 2952.13 1.35 676115.64
2016-10-20 2963.88 2965.74 2959.68 2959.68 0.26 440663.31
2016-10-21 2975.40 2975.40 2965.20 2965.20 0.19 580519.75
2016-10-24 2991.65 3017.78 2991.65 3016.87 1.74 470453.44
2016-10-25 3042.78 3042.78 3018.19 3021.62 0.16 549692.97
2016-10-26 3016.35 3016.35 3004.60 3011.71 -0.33 503003.10
2016-10-27 3035.19 3038.10 3034.83 3038.10 0.88 699662.42
2016-10-28 3031.84 3076.10 3031.84 3076.10 1.25 532073.92
2016-10-31 3085.46 3085.46 3070.81 3070.81 -0.17 681466.77
2016-11-02 3026.11 3026.11 2987.83 2987.83 -2.70 928538.72
2016-11-03 2998.86 2998.86 2990.79 2990.79 0.10 656489.77
2016-11-04 2975.22 2980.33 2975.22 2975.30 -0.52 385598.43
2016-11-07 3005.59 3005.59 2981.35 2981.35 0.20 484758.97
2016-11-08 3013.01 3017.61 3008.44 3017.61 1.22 445944.20
2016-11-09 2988.07 3030.20 2988.07 3030.20 0.42 759370.28
2016-11-10 3084.03 3084.03 3040.50 3040.50 0.34 1199406.31
2016-11-14 3031.86 3031.86 2967.93 2967.93 -2.39 972806.47
2016-11-15 2989.57 2989.57 2964.69 2968.20 0.01 604846.49
... ... ... ... ... ...
2019-05-31 3908.15 3969.56 3893.10 3969.56 0.80 816765.70
2019-06-03 3960.51 3984.63 3949.97 3966.72 -0.07 532820.53
2019-06-04 3965.50 3974.06 3946.34 3959.69 -0.18 675309.03
2019-06-05 3967.09 3970.77 3935.23 3941.11 -0.47 655383.13
2019-06-06 3943.94 4027.33 3941.65 4007.89 1.69 995862.14
2019-06-07 4011.61 4050.60 4011.61 4042.75 0.87 616987.47
2019-06-10 4062.75 4068.64 4029.45 4046.13 0.08 651731.30
2019-06-11 4049.73 4078.42 4039.98 4069.34 0.57 1015106.95
2019-06-12 4052.77 4062.90 4013.92 4046.65 -0.56 769784.26
2019-06-13 4040.89 4091.55 4040.53 4077.50 0.76 811131.81
2019-06-14 4075.39 4075.39 4049.28 4053.14 -0.60 620163.19
2019-06-17 4058.67 4062.29 4026.71 4037.13 -0.40 426200.13
2019-06-18 4036.36 4123.23 4030.44 4123.23 2.13 949164.98
2019-06-19 4124.82 4124.82 4108.94 4113.58 -0.23 593252.42
2019-06-21 4124.86 4155.76 4085.95 4093.37 -0.49 1697240.27
2019-06-24 4113.01 4136.59 4099.92 4133.72 0.99 521246.30
2019-06-25 4123.47 4128.56 4073.93 4084.48 -1.19 692376.39
2019-06-26 4095.28 4109.06 4081.32 4109.06 0.60 635235.61
2019-06-27 4119.07 4160.37 4119.07 4140.91 0.78 682904.34
2019-06-28 4141.09 4144.04 4125.12 4132.41 -0.21 592757.96
2019-07-01 4195.52 4195.52 4130.92 4136.08 0.09 527728.12
2019-07-02 4152.31 4156.24 4101.91 4156.24 0.49 719797.10
2019-07-03 4145.92 4170.18 4141.94 4164.17 0.19 670327.79
2019-07-04 4165.05 4184.44 4152.69 4183.57 0.47 490173.23
2019-07-05 4186.73 4186.73 4145.94 4157.86 -0.61 516459.10
2019-07-08 4140.77 4167.74 4131.49 4152.64 -0.13 598552.93
2019-07-09 4148.48 4148.48 4109.20 4125.50 -0.65 679278.48
2019-07-10 4124.36 4174.02 4113.97 4125.75 0.01 764583.50
2019-07-11 4146.63 4167.49 4121.76 4133.21 0.18 598836.12
2019-07-12 4144.03 4145.02 4129.71 4130.38 -0.07 535157.11
[691 rows x 6 columns]
https://www.quandl.com/data/WSE-Warsaw-Stock-Exchange-GPW?keyword=KGH

How to Loop actions between CSV header and Dictionary Keys

I have a dictionary with set of Keys & Values and also have CSV file with columns have the same header name of the dictionary Keys.
i need to apply common function which is if the value of the row of the header that has the name of the key is less than the value defined in the dictionary to write true in a new column in the data set.
the problem is i need to make a loop for all columns to match the header name with the Key name and apply the function according to the value of the key on all column rows.
Below code if i did it with out loop , but if the dictionary increased it will be not efficient way.
Code:
dict = {'RRC_SR%' : 99, 'RAB_SR%' : 97, 'UL_UE_Throughput' : 2, 'PS_CDR%' : 0.15, 'DL_UE_Throughput' : 17,
'INTRA HO SR %' : 99}
kpi = pd.read_csv('KPIs for Triggering.csv')
kpi['RRC_SR Result'] = kpi.loc[:, 'RRC_SR%'].apply(lambda x:x>dict['RRC_SR%'])
kpi['RAB_SR Result'] = kpi.loc[:, 'RAB_SR%'].apply(lambda x:x>dict['RAB_SR%'])
kpi['UL_UE_Throughput Result'] = kpi.loc[:, 'UL_UE_Throughput'].apply(lambda x:x>dict['UL_UE_Throughput'])
kpi['PS_CDR Result'] = kpi.loc[:, 'PS_CDR%'].apply(lambda x:x>dict['PS_CDR%'])
kpi['DL_UE_Throughput Result'] = kpi.loc[:, 'DL_UE_Throughput'].apply(lambda x:x>dict['DL_UE_Throughput'])
kpi['INTRA HO SR Result'] = kpi.loc[:, 'INTRA HO SR %'].apply(lambda x:x>dict['INTRA HO SR %'])
Time Object DL_UE_Throughput UL_UE_Throughput PS_CDR% RAB_SR% RRC_SR% INTRA HO SR %
0:00 A_1 4.76 1.04 0.17 99.88 99.98 98.22
0:00 B_2 8.1 1.04 0.15 99.92 99.99 97.8
0:00 C_3 5.63 0.72 0.11 99.94 99.98 96.17
0:00 D_4 8.65 1.06 0.25 99.75 99.95 99.51
0:00 E_5 10.21 0.59 0.37 99.67 99.97 99.18
0:00 F_6 7.46 0.34 0.38 99.3 99.95 99.56
0:00 G_7 10.08 2.31 0.6 99.38 99.73 93.63
0:00 H_8 10.29 1.29 0.84 99.5 99.84 99.22
0:00 I_9 10.19 4.76 0.92 99.26 99.75 97.66
0:00 J_10 7.45 8.6 0.85 99.29 99.72 98.55

How to plot numbers from an array as annotation using matplotlib?

I am trying to produce the map on basemap using vales extracted from meteorological data. Sample code is:-
y=[2.56422, 3.77284,3.52623,3.51468,3.02199]
z=[0.15, 0.3, 0.45, 0.6, 0.75]
n=[58,651,393,203,123]
fig, ax = plt.subplots()
ax.scatter(z, y)
for i, txt in enumerate(n):
ax.annotate(txt, (z[i],y[i]))
The data I am using is a numpy array. I dont know how to loop through each array to plot the kind of map similar to above. I would like to plot only values (ie. no countour or contourf).
Initially I was trying to plot float values using pylab.plot function. However, it retured with error
ValueError: third arg must be a format string
Then I tried to convert this numpy array to string and then plot with this command:-
temperature = np.array2string(data, precision=2)
and the print statement looks like a modified string:-
print temperature
[[ 19.69 21.09 21.57 21.45 20.59 20.53 20.93 20.63 20.64 21.26
21.29 20.63 20.98 21.01 20.84 20.81 20.55 20.33 20.52 20.23
19.84]
[ 20.77 21.35 20.81 20.64 20.9 20.78 20.79 23.57 20.11 21.07
21.06 21.33 21.48 21.18 21.4 21.09 20.5 20.31 20.12 19.8
19.97]
[ 21.51 21.23 20.55 20.08 20.05 20.78 21.17 24.77 21.17 20.95
21.43 21.47 21.46 21.77 21.69 21.13 20.47 20.04 20.08 20.37
20.14]
[ 21.29 21.1 20.63 20.32 20.22 20.37 24.4 23.82 22.23 21.03
22.11 22.62 22.71 22.37 21.73 21.35 21.03 20.67 20.58 20.89
20.93]
[ 21.24 21.04 20.68 20.56 20.76 20.91 24.26 23.75 23.28 21.26
21.48 22. 21.94 21.78 21.36 21.14 20.96 20.92 21.1 21.19
21.31]
[ 20.83 20.88 20.6 20.87 21.01 21.91 22.33 22.21 21.74 20.66
20.76 20.73 21.04 21.09 20.83 20.7 20.72 20.71 21.23 21.04
20.73]
[ 20.32 20.41 20.19 20.05 20.68 22.17 21.82 20.67 19.85 19.02
18.91 19.6 20.15 20.64 20.64 20.09 19.81 19.76 19.9 19.94
19.46]
[ 19.68 20.37 20.56 20.68 20.93 21.28 21.24 20.33 20.7 20.
18.72 18.94 19.56 19.57 19.83 19.74 19.17 18.53 18.1 18.72
19.12]
[ 18.88 19.71 20.77 20.81 20.32 21.58 20.96 21.33 21.2 20.17
19.95 22.05 19.72 19.85 19.3 18.75 18.69 18.44 17.57 17.2
18.22]
[ 19.11 19.19 20.13 20.78 21.25 21.98 21.15 20.96 20.66 20.14
20.51 21.92 20.36 20.27 19. 18.22 17.81 17.58 17.16 16.67
17.46]
[ 18.5 19.28 19.57 20.01 21.16 21.01 21.06 20.93 20.62 19.89
20.3 20.7 19.7 19.76 18.24 17. 16.36 16.63 17.62 17.32
17.38]
[ 17.6 18.33 20.27 19.97 20.63 20.51 21.09 21.39 20.81 19.55
20. 18.3 17.32 18.24 17.57 17.15 16.42 15.76 16.14 16.45
21.95]
[ 17.04 17.55 18.16 18.32 21.23 20.5 20.41 19.82 20.7 20.55
20.41 18.47 18.05 17.63 17.11 15.6 16.02 15.46 14.29 13.88
23.04]]
Finally, I get this error when I tried to plot the above value on a map with this line
pylab.plot(x, y, temperature)
'Unrecognized character %c in format string' % c)
ValueError: Unrecognized character [ in format string
Problem seems to be with nparray to string conversion.
Any help to solve this issue is appreciated.

Your original solution with ax.annotate is perfectly fine for your more general solution. The only thing to change is that in case of 2d arrays, you need to flatten them before looping over them using np.ravel() (which is also a method of the ndarray class).
However, in your specific case you can spare explicit indexing and the use of ravel() by broadcasting the three arrays you need to plot:
import numpy as np
import matplotlib.pyplot as plt
# generate some dummy data
rng = np.random.default_rng()
z, y = np.mgrid[:3, :3]
n = rng.integers(low=50, high=500, size=z.shape)
fig, ax = plt.subplots()
ax.scatter(z, y)
for zz, yy, txt in np.broadcast(z, y, n):
ax.annotate(txt, (zz, yy))
Note that the result of np.broadcast is the same as if we'd used zip(z.ravel(), y.ravel(), n.ravel()).