I am new to Python, I am calling an external service and printing the data which is basically byte literal array.
results = q.sync('([] string 2#.z.d; `a`b)')
print(results)
[(b'2018.06.15', b'a') (b'2018.06.15', b'b')]
To Display it without the b, I am looping through the elements and decoding the elements but it messes up the whole structure.
for x in results:
for y in x:
print(y.decode())
2018.06.15
a
2018.06.15
b
Is there a way to covert the full byte literal array to string array (either of the following) or do I need to write a concatenate function to stitch it back?
('2018.06.15', 'a') ('2018.06.15', 'b')
(2018.06.15,a) (2018.06.15,b)
something like the following (though I want to avoid this approach )
for x in results:
s=""
for y in x:
s+="," +y.decode()
print(s)
,2018.06.15,a
,2018.06.15,b
Following the previous answer, your command should be as follows:
This code will result in a list of tuples.
[tuple(x.decode() for x in item) for item in result]
The following code will return tuples:
for item in result:
t = ()
for x in item:
t = t + (x.decode(),)
print(t)
You can do it in one line, which gives you back a list of decoded tuples.
[tuple(i.decode() for i in y) for x in result for y in x]
Related
I am currently working with JSON and Python and I have a problem.
When I write:
x = {}
x['red'] = {'name': "red"}
y = {}
y['red'] = {'p': 1}
z = x['red']['name'], y['red']['p']
print(z)
I get back:
('red', 1)
But I want it like:
red1
Without using
print(x['red']['name'], y['red']['p'])
Thanks for your help :)
When we resolve the variables in the line
z = x['red']['name'], y['red']['p']
we get this:
z = "red", 1
This is, in Python the same as writing:
z = ("red", 1)
This line defines a data-type called a "tuple". It is similar to a list. When you use print to write out the value of this variable, Python formats this as such and adds the parens.
If you want the string "red1" as output, you need to do some minor string processing. In your tuple, your first item is a string, the next is an integer. Those two are not directly concatenable in Python using +. You either need to convert (cast) the number first, or use a string formatting function:
Example 1 - Using str() to cast the number to string
z = x['red']['name'] + str(y['red']['p'])
Example 2 - Using simple string formatting
z = '%s%s' % (x['red']['name'], y['red']['p'])
Example 3 - Using f-strings
z = f"{x['red']['name']}{y['red']['p']}"
Just concatenate the string, easy:
z = y['red']['name'] + str(z['red']['p'])
print(z)
You need to call str() around z['red']['p'] because it's an integer. By converting it to a string, you can then concatenate the two strings into one string
When you type z = x['red']['name'], y['red']['p'] Python automatically takes it as tuple.
But you want this to be treated as string and get concatenated results.
for this you may use,
z = str(x['red']['name']) + str(y['red']['p'])
print(''.join(z)) does the trick.
Cheers
Demo: https://repl.it/repls/MinorRoyalTrace
Edit: use print(''.join(str(element) for element in z)) instead for handle str+ int
I need to pass coordinates in an url but I need to convert the rdd to a string and separate with a semicolon.
all_coord_iso_rdd.take(4)
[(-73.57534790039062, 45.5311393737793),
(-73.574951171875, 45.529457092285156),
(-73.5749282836914, 45.52922821044922),
(-73.57501220703125, 45.52901077270508)]
type(all_coord_iso_rdd)
pyspark.rdd.PipelinedRDD
Results lookin for:
"-73.57534790039062,45.5311393737793;-73.574951171875,45.529457092285156,
-73.5749282836914,45.52922821044922;-73.57501220703125,45.52901077270508"
The form of my URL should be as follows:
http://127.0.0.1/match/v1/driving/-73.57534790039062,45.5311393737793; -73.574951171875,45.529457092285156,-73.5749282836914,45.52922821044922;-73.57501220703125,45.52901077270508
From the snippet you posted all_coord_iso_rdd is an rdd, where each row is a tuple(float, float). Calling take(n) returns n records from the rdd.
x = all_coord_iso_rdd.take(4)
print(x)
#[(-73.57534790039062, 45.5311393737793),
# (-73.574951171875, 45.529457092285156),
# (-73.5749282836914, 45.52922821044922),
# (-73.57501220703125, 45.52901077270508)]
The value returned is simply a list of tuples of floating point numbers. To convert it into the desired format, we can use str.join inside of a list comprehension.
First, you need to convert the floats to str and then we can join the values in each tuple using a ",". We use map(str, ...) to map each value to a str.
This yields:
print([",".join(map(str, item)) for item in x])
#['-73.5753479004,45.5311393738',
# '-73.5749511719,45.5294570923',
# '-73.5749282837,45.5292282104',
# '-73.575012207,45.5290107727']
Finally join the resultant list using ";" to get your desired output.
print(";".join([",".join(map(str, item)) for item in x]))
Here is a pure spark way of doing the same (may be useful for larger
rdds/different use cases):
list=[(-73.57534790039062, 45.5311393737793),(-73.574951171875, 45.529457092285156),\
(-73.5749282836914, 45.52922821044922),(-73.57501220703125, 45.52901077270508)]
rdd=sc.parallelize(list)
rdd.map(lambda row: ",".join([str(elt) for elt in row]))\
.reduce(lambda x,y: ";".join([x,y]))
I am trying to extract data from a .txt file (Aperture, ShutterSpeed) and then apply some calculations with a function I already created, and have the results plotted in a certain way.
What is the best way to do it in Python?
I have already read the data:
#!/usr/bin/python
import math
filename='/home/stagiaire/Bureau/derr.txt'
with open(filename) as f:
data = f.read()
data = data.split('\n')
Fnumber = [row.split(' ')[0] for row in data]
ShutterSpeed = [row.split(' ')[1] for row in data]
But it is not accepted in my function that calculates the Lux value,
def Calcul_Lux(Fnumber,ShutterSpeed):
x=math.pow(Fnumber,2)/(ShutterSpeed)
IL=math.log(x,2)
lux = math.pow(2,IL) * 2.5
return lux
The idea is to have my data as a list I guess (two columns Fnumber and ShutterSpeed), but I don't know how to do it.
Input from string to float
When reading the data, it will be read as a string, this means that the values you obtain in Fnumber and ShutterSpeed are also of type string. As the Calcul_Lux function expects numbers, you need to typecast the string values as numbers.
To typecast a string into a floating point number you can use float(<object>), if you prefer to have an integer you can use int(<object>)
To solve your problem you can change the assignment of Fnumber and ShutterSpeed like this:
Fnumber = [float(row.split(' ')[0]) for row in data]
ShutterSpeed = [float(row.split(' ')[1]) for row in data]
Call function in one go
To call the function for each value and return it as a list, we first need to pack the Fnumber and ShutterSpeed into a single list of tuples. We can do this using the zip() function:
tuples = zip(Fnumber, ShutterSpeed)
Next we can call the function Calcul_Lux for each item in the list using the map() function, which basically executes the given lambda function on each item in the list:
output = map(lambda tup: Calcul_Lux(tup[0], tup[1]), tuples)
This will return all you values as a list.
Plotting values
To plot the output and compare them with a constant line we can use the matplotlib library. We can use its plot() function to plot our output as points and we can draw a comparison line using the axhline() function. In the end the code to plot would look something like this.
import matplotlib.pyplot as plt # To import the matplotlib library
plt.plot(range(len(output)), output, 'ro') # To draw the points from the output
plt.axhline(y=48000) # To draw the 48.000 comparison line
plt.show() # To show the plot
The ValueError: ordinal must be >= 1 is caused because we only provided the y values, without any x values. To plot the output according to index we can change the plt.plot(output, 'ro') to include an x value like this:
plt.plot(range(len(output)), output, 'ro')
with open(filename) as f:
pairs = ((float(x) for x in line.split()) for line in f)
luxes = [Calcul_Lux(fnum, speed) for fnum, speed in pairs]
Here's how I would do it. pairs is a generator that yields a generator of floats for each line. The list comprehension accepts that generator, pulls out the two values and passes them to Calcul_Lux
Your Calcul_Lux funtion is expecting to be passed two values (f-number and shutterspeed), but you have your values in two lists. You can zip the lists together, which looks like this:
>>> one = [1,2,3]
>>> two = ['a', 'b', 'c']
>>> zip(one, two)
[(1, 'a'), (2, 'b'), (3, 'c')]
And then pass the values one at a time to your function, so:
for fn, ss in zip(Fnumber, ShutterSpeed):
Calcul_Lux(fn,ss)
You may be better off extracting the data like this:
import math
def Calcul_Lux(Fnumber,ShutterSpeed):
x=math.pow(Fnumber,2)/(ShutterSpeed)
IL=math.log(x,2)
lux = math.pow(2,IL) * 2.5
return lux
filename='/home/stagiaire/Bureau/derr.txt'
with open(filename) as f:
data = f.read()
data = data.split('\n')
cameraData = [(float(row.split(' ')[0]),float(row.split(' ')[1])) for row in data]
luxes=[Calcul_Lux(Fnumber,ShutterSpeed) for Fnumber,ShutterSpeed in cameraData]
I am having issues with creating a function that takes a list of tuples and then returns one string which is the first character of each tuple. Below is my current code, but nothing it happening, I do not get a syntax error. Any help would be appreciated.
lst_of_tups = ([('hello', 'all'), ('music', 'playing'), ('celebration', 'station')])
def build_string(lst_of_tups):
final_str = ""
for tup in list_of_tups:
for item in tup:
final_str = final_str + item[0]
return final_str
print build_string
**** expected output: hampcs****
those string manipulation functions are error-prone: they define lots of variables, can return within inner loops, have unexpected side-effects...
Once you're used to list comprehensions, you can create such programs easily & with great execution performance (string concatenation is slow). One way:
def build_string(lst_of_tups):
return "".join([x[0] for y in lst_of_tups for x in y])
basically, it's just 2 loops (flattening the data) within a list comprehension to extract each first character from every string, joined together using str.join to rebuild the string.
Once you reach a return statement in a function, the function ends for good. The line
print build_string
cannot be reached. (Another problem is that the name build_string is not defined.)
Use your function like this:
result = build_string(lst_of_tups) # calls your function and puts the return value in the result variable
print result # print the result
Of course, the intermediary variable result is not necessary, you could just issue print build_string(lst_of_tups) as well.
So this is what I'm trying to do:
input: ABCDEFG
Desired output:
***DEFG
A***EFG
AB***FG
ABC***G
ABCD***
and this is the code I wrote:
def loop(input):
output = input
for index in range(0, len(input)-3): #column length
output[index:index +2] = '***'
output[:index] = input[:index]
output[index+4:] = input[index+4:]
print output + '\n'
But I get the error: TypeError: 'str' object does not support item assignment
You cannot modify the contents of a string, you can only create a new string with the changes. So instead of the function above you'd want something like this
def loop(input):
for index in range(0, len(input)-3): #column length
output = input[:index] + '***' + input[index+4:]
print output
Strings are immutable. You can not change the characters in a string, but have to create a new string. If you want to use item assignment, you can transform it into a list, manipulate the list, then join it back to a string.
def loop(s):
for index in range(0, len(s) - 2):
output = list(s) # create list from string
output[index:index+3] = list('***') # replace sublist
print(''.join(output)) # join list to string and print
Or, just create a new string from slices of the old string combined with '***':
output = s[:index] + "***" + s[index+3:] # create new string directly
print(output) # print string
Also note that there seemed to be a few off-by-one errors in your code, and you should not use input as a variable name, as it shadows the builtin function of the same name.
In Python, strings are immutable - once they're created they can't be changed. That means that unlike a list you cannot assign to an index to change the string.
string = "Hello World"
string[0] # => "H" - getting is OK
string[0] = "J" # !!! ERROR !!! Can't assign to the string
In your case, I would make output a list: output = list(input) and then turn it back into a string when you're finished: return "".join(output)
In python you can't assign values to specific indexes in a string array, you instead will probably want to you concatenation. Something like:
for index in range(0, len(input)-3):
output = input[:index]
output += "***"
output += input[index+4:]
You're going to want to watch the bounds though. Right now at the end of the loop index+4 will be too large and cause an error.
strings are immutable so don't support assignment like a list, you could use str.join concatenating slices of your string together creating a new string each iteration:
def loop(inp):
return "\n".join([inp[:i]+"***"+inp[i+3:] for i in range(len(inp)-2)])
inp[:i] will get the first slice which for the first iteration will be an empty string then moving another character across your string each iteration, the inp[i+3:] will get a slice starting from the current index i plus three indexes over also moving across the string one char at a time, you then just need to concat both slices to your *** string.
In [3]: print(loop("ABCDEFG"))
***DEFG
A***EFG
AB***FG
ABC***G
ABCD***