We Keep Coding

Getting a Different Sort Order Between "SPSS sort cases by" and Python "sort_values(by=[])

I am attempting to convert some code from SPSS into Python. In the code, the SPSS "sort cases by" command is resulting in a different sorted order than the Pandas "df.sort_values(by=[]) command. For reference, here is the code in the two programs:
SPSS
GET FILE='C:\Data\sorttest.sav'.
sort cases by variable1.
dataset name sorttest.
execute.
Python
import pandas as pd
df_sorttest = pd.read_spss('C:\\Data\\sorttest.sav')
df_sorttest = df_sorttest.sort_values(by=['variable1'])
I assume this is because they are using different sorting algorithms, but I'm not sure how to fix it so I can get the same results in Python.

Thanks to It_is_Chris for the recommendation to specify the algorithm. I set it to kind='mergesort' and it got the correct order.

Python: how to import Excel cell's displayed (formatted) value instead of real value

For example, a cell has a real value of 1.96. The cell is formatted with rounding so that it displays 2 in Excel.
Another example, a cell has a value of 5, but it is formatted to display currency, so $5.
There could be any type of formatting. So I cannot know advance what formatting I am looking for.
How do I import these displayed values with Python? I've looked into Pandas and openpyxl but couldn't find a way.
I can do it semi-manually by creating a custom VBA module. following Duncan O'Donnell's answer (the last one) here: https://superuser.com/questions/678934/how-can-i-get-the-displayed-value-of-a-cell-in-ms-excel-for-text-that-was-conv
But I need to fully automate and do it in Python. Any help is appreciated, thanks!

Openpyxl has a feature called number_format which could come in handy. I basically played with the output of dir to get to this.
#let's say we have a value in cell C2 which is 1.96 but formatted to 2.0
print(ws['C2'].number_format)
# #
The # indicates it is formatted as an integer.
# Create a conditional:
if fmt=="#":
r = round(ws['C2'].value)
print(r)
2.0
It's a bit of a hack, but it should help with your use case.
For the dollar part, I will refer you to this stack overflow post, as I believe it captures your use case quite well.

I was able to solve this with VBA... so I guess there is no easy way with Python after all.

How to find the best combination of two random parameters

Sorry if this is a general question I'm a bit of a beginner and I'm looking for a simple and uncomplicated library or way to do the following
I have some code that eventually depends on two parameters eg:
param1 = 12
param2 = 5
and at the end I get a variable that changes depending on these parameters
score = 5031
I want an easy way to loop over the code with randomized parameters until it finds the combination that gives the highest score

you can get it by looping for every index in the list and choosing the max out of it with max built in function

Is there a way to name a range within excel using xlwings API? Trying to use string formulas that reference excel defined ranges

I am trying to use xlwing's sheetname.range("Some object defined range").formula = "=Some excel formula" function, but I am not sure about how I can reference an object defined range as a string within the formula above.
I was able to define the object range as follows:
X = sht0.range('C10').expand('down').value
For all values within this range, I would like to perform a calcuation using an excel formula and I would like to use the following function.
sht0.range('F1').formula = "=SUM(X)"
It's a simple Sum function that could work if I just use the sum(X) function in python and use the result in the equation above without creating a string, but what if I would like to use any other built in excel functions instead?
One potential work around is using the xw.api syntax to create a "Named Range" in excel using the pywin32 syntax, but I can't seem to figure out what the syntax conversion would be.
As an example, in VBA it's:
Range("D7").Select
ActiveWorkbook.Names.Add Name:="SomeNameforaRange", RefersToR1C1:="=Sheet1!R7C4"
I tried the following in xlwings for python along with a few other attempts in the syntax:
sht0.range('C10').expand('down').api.Names = "SomeNameforaRange"
...but that doesn't work. It would just freeze my notebook or gives me an error.
Sorry if this question is very basic, I am a bit new to Python and xlwings, but I appreciate any help you may be to give.

computing Std deviation using RDD v/s SparkSQL in Python

I am pretty new to the world of spark ( and to an extend even Python , but better). I am trying to compute the standard deviation and had used the following code. The first using SparkSQL and the code is as follows:
sqlsd=spark.sql("SELECT STDDEV(temperature) as stdtemp from
washing").first().stdtemp
print(sqlsd)
The above works fine ( I think) and it gives the result as 6.070
Now when I try to do this using RDD with the following code:-
def sdTemperature(df,spark):
n=float(df.count())
m=meanTemperature(df,spark)
df=df.fillna({'_id':0,'_rev':0,'count':0,'flowrate':0,'fluidlevel':0,
'frequency':0,'hardness':0,'speed':0,'temperature':0,'ts':0,'voltage':0})
rddT=df.rdd.map(lambda r: r.temperature)
c=rddT.count()
s=rddT.map(lambda x: pow(x-m,2)).sum()
print(n,c,s)
sd=sqrt(s/c)
return sd
when I run the above code, I get a different result. the value I get is 53.195
what I am in doing wrong?. All I am trying to do above is to compute the std deviation for a spark dataframe column temperature and use lambda.
thanks in advance for help ..

thanks to Zero323 who gave me the clue. I skipped the null values . the modified code is as follows:-
df2=df.na.drop(subset=["temperature"])
rddT=df2.rdd.map(lambda r: r.temperature)
c=rddT.count()
s=rddT.map(lambda x: pow(x-m,2)).sum()
sd=math.sqrt(s/c)
return(sd)

There are two types of standard deviations - please refer to this : https://math.stackexchange.com/questions/15098/sample-standard-deviation-vs-population-standard-deviation
Similar question -
Calculate the standard deviation of grouped data in a Spark DataFrame
The stddev() in Hive is a pointer to the stddev_samp(). The stddev_pop() is what you are looking for(inferred from the 2nd part of you code). So your sql query should be select stddev_pop(temperature) as stdtemp from washing