Im currently running some correlation in test in python. I get the results I need but they're in x.01234567890 and I'm trying to format them to output them into percentage.
this is reading from a csv.
results = (
{
clean_df['NZDUSD_Close'].corr(clean_df['USDJPY_Close']),
clean_df['NZDUSD_Close'].corr(clean_df['EURGBP_Close']),
....
eclean_df['AUDUSD_Close'].corr(clean_df['EURUSD_Close']),
clean_df['GBPUSD_Close'].corr(clean_df['EURUSD_Close'])
}
)
print(results)
The above works. There are 15 results and returns all in floats, but I get string errors when I formatted this as a tuple I decided to make it a dictionary.
I've tried the following for formatting the output:
print (f"{results:.2%}")
This works for a single variable but not the list.
for key in results.keys():
print(key, "{:.2f}".format(results[key]))
If the error your encountering is:
"AttributeError: 'set' object has no attribute 'keys'"
Then it's because your dictionary is inside a tuple and a tuple has no "keys" attribute. You would be best off if you used a tuple:
results = (...)
and printing it like:
print(f"{results:.2%}")
Doesn't work because python doesn't know to iterate over all the objects in the result.
Another way of printing the tuple is:
for val in results:
print("{:.2%}".format(val))
This will iterate over every value in results and print it in a 2 point percentage format
Related
I have put together a script to output all fields from a json file.
I have running into issues getting it to print each finding from highsev and specifically the id field from each finding.
Here is the code:
json_object=json.load(f)
vulnsBySeverity["HIGHSEVERITY"] = []
for result in json_object['results']:
vulnsBySeverity["HIGHSEVERITY"] = [vuln for vuln in result['vulnerabilities'] if vuln["severity"] == "high"]
highsev=vulnsBySeverity["HIGHSEVERITY"]
print("HIGH VULNS were")
for finding in highsev:
print(highsev[finding]['id'])
Its complaining :
TypeError: list indices must be integers or slices, not dict
I think I understand what its saying that finding call in the print() has to be a int or slice im just wondering how i can do that, as finding contains this:
{'id': 'CVE-2020-15999'}
Im simply just trying to loop through highsev and output the value of id in a loop.
What might i be doing wrong here? can someone please help
You're already looping through highsev with your for finding in highsev:
If your highsev list contains dictionaries like this {'id': 'CVE-2020-15999'}, you can do this:
for finding in highsev:
print(finding['id'])
You can change last line to:
print(finding['id'])
As you are looping through highsev.
I am running an API to grab some information from a website where I am storing the information in a list '[]'. How can I run a for loop through this information to:
1) Iterate through the list of objects in a for loop (specifically comparing one objects text
2) If one value of the object equals a 1 word, save whole object into a new list
I have tried running a for loop through the list/objects but get the error ''list' object is not callable'
tree = Et.fromstring(response.content)
for child in tree.findall('meterConsumption'):
for audit in child.findall('audit'):
for creator in audit.findall('createdBy'):
for ID in child.findall('id'):
print ('Entry ID: ',ID.text)
for use in child.findall('usage'):
print ('Use: ',use.text)
for cost in child.findall('cost'):
print ('Cost: ',cost.text)
for startdate in child.findall('startDate'):
print ('Startdate: ',startdate.text)
for enddate in child.findall('endDate'):
print ('Enddate: ',enddate.text)
#save object to list
allentries.append(Entry(ID.text,
use.text,
cost.text,
startdate.text,
enddate.text,
creator.text))
for x in allentries():
print (x.entryid)
I am looking to get a list of all key value pairs in the object. For example it would like:
Id[1], use[1], cost[1], startdate[1], enddate[1], creator[1]
Id[2], use[2], cost[2], startdate[2], enddate[2], creator[2]
Id[3], use[3], cost[3], startdate[3], enddate[3], creator[3]
The say from this, if creator == "human".append to all info from this object to a new object list
Triple for loops followed a for loop for child.findall('id') will result in a compile-time error called as identation error.
for child in tree.findall('meterConsumption'):
for audit in child.findall('audit'):
for creator in audit.findall('createdBy'):
#identation error
for ID in child.findall('id'):
print ('Entry ID: ', ID.text)
List object is not callable means u trying to call list-objects.
allentries is a list & u are trying to call a list by using ().
Remove this ().
You'll get the 'list' object is not callable error when you try to call a list like you would call a function. That's happening in your second last line:
for x in allentries():
print (x.entryid)
Since allentries is your list, tacking on a () at the end of it is the syntax for "calling" it, which doesn't make sense for objects that aren't functions. So the correct syntax is:
for x in allentries:
print (x.entryid)
Per your second question, I'd suggest looking into pandas DataFrames as handy ways to work with tabular data in Python. Since child.findall() gives you a list of objects that you're extracting text from with .text, you can pass a dictionary of lists to the DataFrame constructor like so:
import pandas as pd
# Initialize empty dataframe
allentries = pd.DataFrame()
for child in tree.findall('meterConsumption'):
for audit in child.findall('audit'):
for creator in audit.findall('createdBy'):
# Also touched up your indentation
for ID in child.findall('id'):
print ('Entry ID: ',ID.text)
for use in child.findall('usage'):
print ('Use: ',use.text)
for cost in child.findall('cost'):
print ('Cost: ',cost.text)
for startdate in child.findall('startDate'):
print ('Startdate: ',startdate.text)
for enddate in child.findall('endDate'):
print ('Enddate: ',enddate.text)
# Use list comprehensions to extract text attributes
allentries.append({
'ID': [ID.text for ID in child.findall('id')],
'use': [use.text for use in child.findall('use')],
'cost': [cost.text for cost in child.findall('cost')],
'startdate': [startdate.text for startdate in child.findall('startDate')],
'enddate': [enddate.text for enddate in child.findall('endDate')]
})
# Display the final dataframe
print(allentries)
So in my python script I have the following dictionary, except it's listed in string form:
{'MSVCRT.dll': ['atoi'], 'KERNEL32.DLL': ['VirtualFree', 'ExitProcess', 'VirtualProtect', 'LoadLibraryA', 'VirtualAlloc', 'GetProcAddress'], 'SHLWAPI.dll': ['PathFileExistsA'], 'USER32.dll': ['wsprintfA']}
I however would like to have this code as a dictionary of lists, as it clearly is. I tried the following code in orderto attempt to convert the string:
try:
dictimports = ast.literal_eval(stris)
print(dictimports)
except:
print("dict convert failed")
However it hits the except everytime :(
So to reiterate, I would like the keys to be say 'KERNEL32.DLL', and then those keys to have the list as the contents of the values, so have a list with the values ['VirtualFree', 'ExitProcess', 'VirtualProtect', 'LoadLibraryA', 'VirtualAlloc', 'GetProcAddress'] in this instance.
stris = {'MSVCRT.dll': ['atoi'], 'KERNEL32.DLL': ['VirtualFree', 'ExitProcess', 'VirtualProtect', 'LoadLibraryA', 'VirtualAlloc', 'GetProcAddress'], 'SHLWAPI.dll': ['PathFileExistsA'], 'USER32.dll': ['wsprintfA']}
stris is a dictionary. what seems to be the problem?
type(stris)
dict
stris.keys()
dict_keys(['MSVCRT.dll', 'KERNEL32.DLL', 'SHLWAPI.dll', 'USER32.dll'])
if your stris is a string - in which case you'd have
stris = "{'MSVCRT.dll': ['atoi'], 'KERNEL32.DLL': ['VirtualFree', 'ExitProcess', 'VirtualProtect', 'LoadLibraryA', 'VirtualAlloc', 'GetProcAddress'], 'SHLWAPI.dll': ['PathFileExistsA'], 'USER32.dll': ['wsprintfA']}"
and you will convert it to a dict
ast.literal_eval(stris)
{'MSVCRT.dll': ['atoi'], 'KERNEL32.DLL': ['VirtualFree','ExitProcess','VirtualProtect','LoadLibraryA','VirtualAlloc',
'GetProcAddress'],'SHLWAPI.dll': ['PathFileExistsA'],'USER32.dll':['wsprintfA']}
You could use eval() to convert the string to a dict.
The expression argument is parsed and evaluated as a Python expression
eval(stris) will execute the executions given as string an in your case return the parsed dictionary.
But be aware of this: Using python's eval() vs. ast.literal_eval()?
I'm new to Spark. I have a dataframe that contains the results of some analysis. I converted that dataframe into JSON so I could display it in a Flask App:
results = result.toJSON().collect()
An example entry in my json file is below. I then tried to run a for loop in order to get specific results:
{"userId":"1","systemId":"30","title":"interest"}
for i in results:
print i["userId"]
This doesn't work at all and I get errors such as: Python (json) : TypeError: expected string or buffer
I used json.dumps and json.loads and still nothing - I keep on getting errors such as string indices must be integers, as well as the above error.
I then tried this:
print i[0]
This gave me the first character in the json "{" instead of the first line. I don't really know what to do, can anyone tell me where I'm going wrong?
Many Thanks.
If the result of result.toJSON().collect() is a JSON encoded string, then you would use json.loads() to convert it to a dict. The issue you're running into is that when you iterate a dict with a for loop, you're given the keys of the dict. In your for loop, you're treating the key as if it's a dict, when in fact it is just a string. Try this:
# toJSON() turns each row of the DataFrame into a JSON string
# calling first() on the result will fetch the first row.
results = json.loads(result.toJSON().first())
for key in results:
print results[key]
# To decode the entire DataFrame iterate over the result
# of toJSON()
def print_rows(row):
data = json.loads(row)
for key in data:
print "{key}:{value}".format(key=key, value=data[key])
results = result.toJSON()
results.foreach(print_rows)
EDIT: The issue is that collect returns a list, not a dict. I've updated the code. Always read the docs.
collect() Return a list that contains all of the elements in this RDD.
Note This method should only be used if the resulting array is
expected to be small, as all the data is loaded into the driver’s
memory.
EDIT2: I can't emphasize enough, always read the docs.
EDIT3: Look here.
import json
>>> df = sqlContext.read.table("n1")
>>> df.show()
+-----+-------+----+---------------+-------+----+
| c1| c2| c3| c4| c5| c6|
+-----+-------+----+---------------+-------+----+
|00001|Content| 1|Content-article| |2018|
|00002|Content|null|Content-article|Content|2015|
+-----+-------+----+---------------+-------+----+
>>> results = df.toJSON().map(lambda j: json.loads(j)).collect()
>>> for i in results: print i["c1"], i["c6"]
...
00001 2018
00002 2015
Here is what worked for me:
df_json = df.toJSON()
for row in df_json.collect():
#json string
print(row)
#json object
line = json.loads(row)
print(line[some_key])
Keep in mind that using .collect() is not advisable, since it collects the distributed data frames, and defeats the purpose of using data frames.
To get an array of python dicts:
results = df.toJSON().map(json.loads).collect()
To get an array of JSON strings:
results = df.toJSON().collect()
To get a JSON string (i.e. a JSON string of an array):
results = df.toPandas().to_json(orient='records')
and using that to get an array of Python dicts:
results = json.loads(df.toPandas().to_json(orient='records'))
When using the cx_oracle module in python , when i run a query and print the output, it gives it as a tuple object. However I want to have the output as a dictionary object having it neatly :
{,'' , ,'' , ,'' , }
is there a quick thing I can do for this ?
I figured out a method to do this. Below is the relevant code snippet:
column_name = [column1,column2,column3]
for row in cur:
list_row = list(row) # convert the tuple row to a list type.
final = zip(column_names,list_row) # zip the two lists.
dict_final = dict(final) # Convert the resulting final list to a dictionary.
I don't know if there is a better way to do this, but this is what I came up with.