We Keep Coding

Converting String to Integer fails

I am trying to convert a string column of a csv file into an integer or float type using pyspark. Everytime I convert it, the output of the conversion is "null".
When I try to check if the string contains a number it says "false.
How can I convert the string?
THis is my try to solve this.Shoprt pic of the data
`w=weather.withColumn("Temperature",col("Temperature").cast('int'))
w.printSchema()
`

The issue is the value in your "Temperature" column is 26 °F, this value can of course not be casted to an int, because only the first 2 characters of the string are integers. You need a way to retrieve only the first integer characters of this string. I'd say there are 2 options, splitting the value on " " and get the first item or use regex.
For this example I'm going to use the splitting, while not the most robust, you can try the regex out yourself.
from pyspark.sql import functions as F
w = weather.withColumn("Temperature",F.split(col("Temperature"), " ").getItem(0).cast('int'))

How to generate a string consisting of 16 digits but between every 4 numbers there is a hyphen(-)?

What kind of technique would allow me to generate a string in Python similar to this output:
1234-1234-1234-1234

Python has a built in wrap function, made to split strings like this. In conjunction with string.join, you can rejoin the split string with your character of choice:
from textwrap import wrap
s = '1234567890abcdef'
print('-'.join(wrap(s, 4)))
>>> 1234-5678-90ab-cdef
The wrap function takes your string, and the number of characters to split on (in this case 4).
The result from this is used in '-'.join to join each element together with dashes, which gives the result you were looking for.
Note: if you're starting with a number instead of a string, you can easily convert it using str():
s = str(1234567890123456)
print('-'.join(wrap(s, 4)))
>>> 1234-5678-9012-3456

Python3 most efficient String to Float conversion

I extract data from scrapy .
There is a string representing a float ' 0,18' .
What is the most efficient way to convert a String into a float ?
Right now, I convert like this. There are space characters to remove. Comma is replaced by dot.
>>> num = ' 0,18'
>>> float(num.replace(' ','').replace(',','.'))
0.18
I believe my method is far from efficient in time complexity when dealing with tons of data.

You may drop the whitespace stripping. float will eat up whitespace:
>>> float(' 0.18')
0.18

This is okay but if you look at how this is processed, at a high level, there are three function calls, every time:
Replace the empty space with nothing
Replace the comma with a dot
Convert string to float
To simply reduce code, you can get rid of step 1. And just replace the comma with the dot and then convert the string to float.

String splitting in python by finding non-zero character

I want to do the following split:
input: 0x0000007c9226fc output: 7c9226fc
input: 0x000000007c90e8ab output: 7c90e8ab
input: 0x000000007c9220fc output: 7c9220fc
I use the following line of code to do this but it does not work!
split = element.rpartition('0')
I got these outputs which are wrong!
input: 0x000000007c90e8ab output: e8ab
input: 0x000000007c9220fc output: fc
what is the fastest way to do this kind of split?
The only idea for me right now is to make a loop and perform checking but it is a little time consuming.
I should mention that the number of zeros in input is not fixed.

Each string can be converted to an integer using int() with a base of 16. Then convert back to a string.
for s in '0x000000007c9226fc', '0x000000007c90e8ab', '0x000000007c9220fc':
print '%x' % int(s, 16)
Output
7c9226fc
7c90e8ab
7c9220fc

input[2:].lstrip('0')
That should do it. The [2:] skips over the leading 0x (which I assume is always there), then the lstrip('0') removes all the zeros from the left side.
In fact, we can use lstrip ability to remove more than one leading character to simplify:
input.lstrip('x0')

format is handy for this:
>>> print '{:x}'.format(0x000000007c90e8ab)
7c90e8ab
>>> print '{:x}'.format(0x000000007c9220fc)
7c9220fc

In this particular case you can just do
your_input[10:]
You'll most likely want to properly parse this; your idea of splitting on separation of non-zero does not seem safe at all.
Seems to be the XY problem.

If the number of characters in a string is constant then you can use
the following code.
input = "0x000000007c9226fc"
output = input[10:]
Documentation
Also, since you are using rpartitionwhich is defined as
str.rpartition(sep)
Split the string at the last occurrence of sep, and return a 3-tuple containing the part before the separator, the separator itself, and the part after the separator. If the separator is not found, return a 3-tuple containing two empty strings, followed by the string itself.
Since your input can have multiple 0's, and rpartition only splits the last occurrence this a malfunction in your code.

Regular expression for 0x00000 or its type is (0x[0]+) and than replace it with space.
import re
st="0x000007c922433434000fc"
reg='(0x[0]+)'
rep=re.sub(reg, '',st)
print rep

python: regular expressions, how to match a string of undefind length which has a structure and finishes with a specific group

I need to create a regexp to match strings like this 999-123-222-...-22
The string can be finished by &Ns=(any number) or without this... So valid strings for me are
999-123-222-...-22
999-123-222-...-22&Ns=12
999-123-222-...-22&Ns=12
And following are not valid:
999-123-222-...-22&N=1
I have tried testing it several hours already... But did not manage to solve, really need some help

Not sure if you want to literally match 999-123-22-...-22 or if that can be any sequence of numbers/dashes. Here are two different regexes:
/^[\d-]+(&Ns=\d+)?$/
/^999-123-222-\.\.\.-22(&Ns=\d+)?$/
The key idea is the (&Ns=\d+)?$ part, which matches an optional &Ns=<digits>, and is anchored to the end of the string with $.

If you just want to allow strings 999-123-222-...-22 and 999-123-222-...-22&Ns=12 you better use a string function.
If you want to allow any numbers between - you can use the regex:
^(\d+-){3}[.]{3}-\d+(&Ns=\d+)?$
If the numbers must be of only 3 digits and the last number of only 2 digits you can use:
^(\d{3}-){3}[.]{3}-\d{2}(&Ns=\d{2})?$

This looks like a phone number and extension information..
Why not make things simpler for yourself (and anyone who has to read this later) and split the input rather than use a complicated regex?
s = '999-123-222-...-22&Ns=12'
parts = s.split('&Ns=') # splits on Ns and removes it
If the piece before the "&" is a phone number, you could do another split and get the area code etc into separate fields, like so:
phone_parts = parts[0].split('-') # breaks up the digit string and removes the '-'
area_code = phone_parts[0]
The portion found after the the optional '&Ns=' can be checked to see if it is numeric with the string method isdigit, which will return true if all characters in the string are digits and there is at least one character, false otherwise.
if len(parts) > 1:
extra_digits_ok = parts[1].isdigit()