We Keep Coding

Why is my PySpark function2 giving error and function 1 working fine, logically they both are doing the same thing ? can someone help me understand?

I am trying to write 2 functions to convert string data in RDD to float format and then finding the average sepal length for iris dataset. Out of the 2 functions one is working fine but 2nd one is giving error. Can someone help me understand what mistake am i making here
is_float = lambda x: x.replace('.','',1).isdigit() and "." in x
def getSapellen(str2):
if isinstance(str2, float):
return str2
attlist=str2.split(",")
if is_float(attlist[0]):
return float(attlist[0])
else:
return 0.0
SepalLenAvg=irisRDD.reduce(lambda x,y: getSapellen(x) + getSapellen(y)) \
/(irisRDD.count()-1)
print(SepalLenAvg)
The above chunk of code is working. I am not able to figure out the mistake in below part
def getSapellen2(str2):
if ( str2.find("Sepal") != -1):
return str2
attlist=str2.split(",")
if isinstance(attlist[0],str):
return float(attlist[0])
else:
return 0.0
SepalLenAvg=irisRDD.reduce(lambda x,y: getSapellen2(x)+ getSapellen2(y)) \
/(irisRDD.count()-1)
print(SepalLenAvg)
On running the second method I am getting following error
TypeError: can only concatenate str (not "float") to str

This error means you are trying to add together string and float - the only place where you are adding things in the code is the lambda applied to whole irisRdd.
That means in at least one instance, calling getSapellen2(x)+ getSapellen2(y) causes str to be returned by one call and float by other.
if you look at first if statement, there is
return str2 - which is returning string, while all other conditions return numbers

That's mean this condition isinstance(str2, float) of getSapellen never true, while this condition str2.find("Sepal") != -1 from getSapellen2 is true at least once. Therefore, type of str2 is definitely not float, it's string, you might want to cast it to float or doing something else and returns float value instead.

Calculating the Downside Deviation of an Array of Returns

I'm trying to calculate the downside deviation of an array of returns using the code below:
def downside_deviation(arr):
downside_returns = 0
arr.loc[arr < 0, 'downside_returns'] = arr
down_stdev = downside_returns**2
arraysize = downside_returns.count()
down_stdev = downside_returns.sum()/arraysize
down_stdev = np.sqrt(down_stdev)*np.sqrt(12)
return down_stdev
But I keep encountering the and AttributeError as below:
AttributeError: 'float' object has no attribute 'loc'
I'm wondering if anyone could me on this error as nothing I have tried has worked so far.
Thanks a million for the help in advance!

It seems like the arr variable should a Pandas DataFrame, but you passed the float object for the arr variable. So, it raises the AttributeError: 'float' object has no attribute 'loc'.
Additionally, I see this arr.loc[arr < 0, 'downside_returns'] = arr might raise the next error if your arr is actually a Pandas DataFrame. To use it correctly, you may need to read more in its documentation (https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.loc.html) - for example, df.loc[df['shield'] > 6, ['max_speed']].

You're passing a float into the function, but it expects it to be a type that has a .loc attribute.
Python is "duck typed". You can pass anything you want as the argument, but when it comes time to use the members, if they are not present you will get an error like this.

lambda function error: TypeError: 'function' object is not iterable

I'm trying to execute the following python notebook:
Them I came to the part where I have to use the function 'compute_cluster_similarities'.
def compute_cluster_similarities(kwds, kwd2id, vectors, lbl2centroid):
kwd2cluster_sims = dict()
for kwd in kwds:
ix = kwd2id[kwd]
nvec = vectors[ix]
sims = []
for lbl, centroid in lbl2centroid.items():
cosine_sim = np.inner(nvec, centroid)
sims.append((lbl,cosine_sim))
sims = sorted(sims, key = lambda lbl,sim: -sim)
kwd2cluster_sims[kwd] = sims
if len(kwd2cluster_sims) % 1000 == 0:
print("%i computed out of %i" % (len(kwd2cluster_sims), len(all_kwds)))
return kwd2cluster_sims
And its returning the error:
TypeError: () missing 1 required positional argument: 'sim'
First of all, I'm still trying to understand this part of the lambda code. I learned what is the objective of the lambda instruction but couldn't understand what is the point of this line of code, like.. is it (implicity) returning 2 values (sims, key)?? What is being sorted?
I think that this error is ocurring due the Python 3, but even if this was executed on Python 2, it doesn't make sense to me.
It's very confusing for me... How can I solve this error? And please, give me some explanation about what it's going on, not just the fix.
EDIT:
I'm using pdb library to debug the code and I realized that this error was being returned by the 'sorted()' function. The original error is:
*** TypeError: 'function' object is not iterable
What I did:
cosine_sim = np.inner(nvec, centroid)
sims.append((lbl,cosine_sim))
import pdb ; pdb.set_trace();
sims = sorted(sims, key = lambda lbl,sim: -sim)
and them at the Pdb mode:
(Pdb) sims, key = lambda lbl,sim: -sim
*** TypeError: 'function' object is not iterable

The function in the key parameter of sorted is fetched with the elements of the list, therefore it accepts only one parameter.
If you substitute:
key = lambda lbl,sim: -sim
with:
key=lambda x: -x[1]
It should work as you expect.
Refer to the documentation for more explanation of how to use sorted and the key parameter.

How do you make a function that divides fractions into simplest form python

I am taking a class and i'm confused. It would really help if you could guide me through the proccess of this and tell me what I am doing wrong. I have an error that has to do with the parentheses since theres nothing in them. I am a newbie so i'm sorry.
def FractionDivider(a,b,c,d):
n = ()
d = ()
n2 = ()
d2 = ()
print int(float(n)/d), int(float(n2)/d2)
return float (n)/d / (n2)/d2

Your function is taking in arguments a, b, c, and d, but you're not using them anywhere. You're instead defining four new variables. Try:
def FractionDivider(n, d, n2, d2):
and get rid of your empty parentheses bits, see if that does what you are trying to do.

you cannot declare a variable as you are doing n = () and then try to assign an integer or string to it.
n=() does not mean:
n equals nothing at the moment but i will assign a variable shortly.
() ---> Tuples https://docs.python.org/3/tutorial/datastructures.html
They are two examples of sequence data types (see Sequence Types —
list, tuple, range). Since Python is an evolving language, other
sequence data types may be added. There is also another standard
sequence data type: the tuple.
so within your function, if you want you varialbes to be assigned what is passed as an argument
for Ex:
def FractionDivider(a,b,c,d):
n = a
d = b
n2 = c
d2 = d
consider reading more on tuples from the above link

n=() is a valid python statement and there is no issue with that. However n=() is evaluating n to an empty tuple(). I believe that what you are trying to do is as follows.
def FractionDivider(a,b,c,d):
'''
Divides a fraction by another fraction...
'''
n = a #setting each individual parameter to a new name.
d = b #creating a pointer is often useful in order to preserve original data
n2 = c #but it is however not necessary in this function
d2 = d
return (float(n)/d) / (float(n2)/d2) #we return our math, Also order of operations exists here '''1/2/3/4 != (1/2)/(3/4)'''
print FractionDivider(1, 2, 3, 4) #here we print the result of our function call.
#indentation is extremely important in Python
here is a simpiler way of writing the same function
def FractionDivider_2(n,d,n2,d2):
return (float(n)/d) / (float(n2)/d2)
print FractionDivider_2(1,2,3,4)

"Function object is unsubscriptable" in basic integer to string mapping function

I'm trying to write a function to return the word string of any number less than 1000.
Everytime I run my code at the interactive prompt it appears to work without issue but when I try to import wordify and run it with a test number higher than 20 it fails as "TypeError: 'function' object is unsubscriptable".
Based on the error message, it seems the issue is when it tries to index numString (for example trying to extract the number 4 out of the test case of n = 24) and the compiler thinks numString is a function instead of a string. since the first line of the function is me defining numString as a string of the variable n, I'm not really sure why that is.
Any help in getting around this error, or even just help in explaining why I'm seeing it, would be awesome.
def wordify(n):
# Convert n to a string to parse out ones, tens and hundreds later.
numString = str(n)
# N less than 20 is hard-coded.
if n < 21:
return numToWordMap(n)
# N between 21 and 99 parses ones and tens then concatenates.
elif n < 100:
onesNum = numString[-1]
ones = numToWordMap(int(onesNum))
tensNum = numString[-2]
tens = numToWordMap(int(tensNum)*10)
return tens+ones
else:
# TODO
pass
def numToWordMap(num):
mapping = {
0:"",
1:"one",
2:"two",
3:"three",
4:"four",
5:"five",
6:"six",
7:"seven",
8:"eight",
9:"nine",
10:"ten",
11:"eleven",
12:"twelve",
13:"thirteen",
14:"fourteen",
15:"fifteen",
16:"sixteen",
17:"seventeen",
18:"eighteen",
19:"nineteen",
20:"twenty",
30:"thirty",
40:"fourty",
50:"fifty",
60:"sixty",
70:"seventy",
80:"eighty",
90:"ninety",
100:"onehundred",
200:"twohundred",
300:"threehundred",
400:"fourhundred",
500:"fivehundred",
600:"sixhundred",
700:"sevenhundred",
800:"eighthundred",
900:"ninehundred",
}
return mapping[num]
if __name__ == '__main__':
pass

The error means that a function was used where there should have been a list, like this:
def foo(): pass
foo[3]
You must have changed some code.
By the way, wordify(40) returned "fourty". I spell it "forty"
And you have no entry for zero

In case someone looks here and has the same problem I had, you can also get a pointer to a function object if the wrong variable name is returned. For example, if you have function like this:
def foo():
my_return_val = 0
return return_val
my_val = foo()
then my_val will be a pointer to a function object which is another cause to "TypeError: 'function' object is unsubscriptable" if my_val is treated like a list or array when it really is a function object.
The solution? Simple! Fix the variable name in foo that is returned to my_return_val.