I have two functions and I'm using similar parameters but for one they work as expected and for the other, they do not:
from fastapi import FastAPI, Header, HTTPException, Body
#app.post("/portfolio/analytics/carbon-footprint", dependencies=[Depends(api_counter)])
async def getPortfolioCarbonFootprint(
tickers: list = Body(...),
func: str = Body(...),
http_client: aiohttp.ClientSession = fastapi.Depends(http_client)
):
print(tickers)
return res
#historical prices
#app.post("/portfolio/analytics/historicalprices", dependencies=[Depends(api_counter)])
async def getPortfolioHistoricalPrices(
tickers: list = Body(...),
http_client: aiohttp.ClientSession = fastapi.Depends(http_client)
):
print(tickers)
jsonResults = await getHistoricalPrices(tickers)
return jsonResults
For both I send it this json:
{"tickers" : [ "IBM.US", "MSFT.US"]}
the first function works perfectly. The second returns this error:
{
"detail": [
{
"loc": [
"body"
],
"msg": "value is not a valid list",
"type": "type_error.list"
}
]
}
Here's where it gets weird. If I send this:
[ "IBM.US", "MSFT.US"]
then it works as expected.
So function 1, works fine. Function 2 is copied from function 1 and it doesn't accept tickers as input but sending it a raw list works.
The difference between the two functions is the number of parameters to be filled in by the user. In the first function you have tickers and func, and in the second one you only have tickers.
From the FastAPI documentation:
But if you have only a single item body parameter from a Pydantic model Item.
By default, FastAPI will then expect its body directly.
But if you want it to expect a JSON with a key item and inside of it the model contents, you can use the special Body parameter embed
So in the second function, if you want to have a key, you must write:
tickers: list = Body(..., embed=True)
Related
I have a JSON object in S3 which follows this structure:
<code> : {
<client>: <value>
}
For example,
{
"code_abc": {
"client_1": 1,
"client_2": 10
},
"code_def": {
"client_2": 40,
"client_3": 50,
"client_5": 100
},
...
}
I am trying to retrieve the numerical value with an S3 Select query, where the "code" and the "client" are populated dynamically with each query.
So far I have tried:
sql_exp = f"SELECT * from s3object[*][*] s where s.{proc}.{client_name} IS NOT NULL"
sql_exp = f"SELECT * from s3object s where s.{proc}[*].{client_name}[*] IS NOT NULL"
as well as without the asterisk inside the square brackets, but nothing works, I get ClientError: An error occurred (ParseUnexpectedToken) when calling the SelectObjectContent operation: Unexpected token found LITERAL:UNKNOWN at line 1, column X (depending on the length of the query string)
Within the function defining the object, I have:
resp = s3.select_object_content(
Bucket=<bucket>,
Key=<filename>,
ExpressionType="SQL",
Expression=sql_exp,
InputSerialization={'JSON': {"Type": "Document"}},
OutputSerialization={"JSON": {}},
)
Is there something off in the way I define the object serialization? How can I fix the query so I can retrieve the desired numerical value on the fly when I provide ”code” and “client”?
I did some tinkering based on the documentation, and it works!
I need to access the single event in the EventStream (resp) as follows:
event_stream = resp['Payload']
# unpack successful query response
for event in event_stream:
if "Records" in event:
output_str = event["Records"]["Payload"].decode("utf-8") # bytes to string
output_dict = json.loads(output_str) # string to dict
Now the correct SQL expression is:
sql_exp= f"SELECT s['{code}']['{client}'] FROM S3Object s"
where I have gotten (dynamically) my values for code and client beforehand.
For example, based on the dummy JSON structure above, if code = "code_abc" and client = "client_2", I want this S3 Select query to return the value 10.
The f-string resolves to sql_exp = "SELECT s['code_abc']['client_2'] FROM S3Object s", and when we call resp, we retrieve output_dict = {'client_2': 10} (Not sure if there is a clear way to get the value by itself without the client key, this is how it looks like in the documentation as well).
So, the final step is to retrieve value = output_dict['client_2'], which in our case is equal to 10.
Here is the JSON response I get from an API request:
{
"associates": [
{
"name":"DOE",
"fname":"John",
"direct_shares":50,
"direct_shares_details":{
"shares_PP":25,
"shares_NP":25
},
"indirect_shares":50,
"indirect_shares_details": {
"first_type": {
"shares_PP": 25,
"shares_NP": 0
},
"second_type": {
"shares_PP": 25,
"shares_NP": 0
}
}
}
]
}
However, in some occasions, some values will be equal to None. In that case, I handle it in my function for all the values that I know will be integers. But it doesn't work in this scenario for the nested keys inside indirect_shares_details:
{
"associates": [
{
"name":"DOE",
"fname":"John",
"direct_shares":50,
"direct_shares_details":{
"shares_PP":25,
"shares_NP":25
},
"indirect_shares":None,
"indirect_shares_details": None
}
}
]
}
So when I run my function to get the API values and put them in a custom dict, I get an error because the keys are simply inexistant in the response.
def get_shares_data(response):
associate_from_api = []
for i in response["associates"]:
associate_data = {
"PM_shares": round(company["Shares"], 2),
"full_name": i["name"] + " " + ["fname"]
"details": {
"shares_in_PM": i["direct_shares"],
"shares_PP_in_PM": i["direct_shares_details"]["shares_PP"],
"shares_NP_in_PM": i["direct_shares_details"]["shares_NP"],
"shares_directe": i["indirect_shares"],
"shares_indir_PP_1": i["indirect_shares_details"]["first_type"]["shares_PP"],
"shares_indir_NP_1": i["indirect_shares_details"]["first_type"]["shares_NP"],
"shares_indir_PP_2": i["indirect_shares_details"]["second_type"]["shares_PP"],
"shares_indir_NP_2": i["indirect_shares_details"]["second_type"]["shares_NP"],
}
}
for key,value in associate_data["details"].items():
if value != None:
associate_data["details"][key] = value * associate_data["PM_shares"] / 100
else:
associate_data["calculs"][key] = 0.0
associate_from_api.append(associate_data)
return associate_from_api
I've tried conditioning the access of the nested keys only if the parent key wasn't equal to None but I ended up declaring 3 different dictionaries inside if/else conditions and it turned into a mess, is there an efficient way to achieve this?
You can try accessing the values using dict.get('key') instead of accessing them directly, as in dict['key'].
Using the first approach, you will get None instead of KeyError if the key is not there.
EDIT: tested using the dictionary from the question:
You can try pydantic
Install pydantic
pip install pydantic
# OR
conda install pydantic -c conda-forge
Define some models based on your response structure
from pydantic import BaseModel
from typing import List, Optional
# There are some common fields in your json response.
# So you can put them together.
class ShareDetail(BaseModel):
shares_PP: int
shares_NP: int
class IndirectSharesDetails(BaseModel):
first_type: ShareDetail
second_type: ShareDetail
class Associate(BaseModel):
name: str
fname: str
direct_shares: int
direct_shares_details: ShareDetail
indirect_shares: int = 0 # Sets a default value for this field.
indirect_shares_details: Optional[IndirectSharesDetails] = None
class ResponseModel(BaseModel):
associates: List[Associate]
use ResponseModel.parse_xxx functions to parse response.
Here I use parse_file funtion, you can also use parse_json function
See: https://pydantic-docs.helpmanual.io/usage/models/#helper-functions
def main():
res = ResponseModel.parse_file("./NullResponse.json",
content_type="application/json")
print(res.dict())
if __name__ == "__main__":
main()
Then the response can be successfully parsed. And it automatically validates the input.
I'm parsing a json file that looks like:
my_json:
"DataChangedEntry": {
"CurrentValue": {
"RefId": {
"Value": "aaaaaaa"
So to get "Value" it looks like:
my_json["DataChangedEntry"]["CurrentValue"]["RefId"]["Value"]
I want to send it to a try/except function (because I have a lot of fields to get) but I don't know how to send the json object over.
I've tried:
get_value = my_function(my_json, ["DataChangedEntry"]["CurrentValue"]["RefId"]["Value"])
But I get error:
TypeError: list indices must be integers or slices, not str
The my_function is just
def my_function(json_prefix, json_field):
try:
value = json_prefix[json_field]
return value
except:
logging.exception('Exception: ')
You have to pass each key as a separate argument (or as a list of separate arguments).
def my_function(obj, *fields):
for f in fields:
try:
obj = obj[f]
except KeyError:
logging.exception("Exceptoin: ")
return
return obj
my_function(my_json, "DataChangedEntry", "CurrentValue", ...)
I confess that this idea of sending this data separately to a function is quite different.
But to directly return the value aaaaaaa:
Send the first argument as the JSON object value
Send the second argument as the JSON object name string
Send third argument as key list
Then you can use eval() to convert the union of strings into code:
def my_function(json_full, json_prefix, json_field):
my_json = json_full
my_json_str = json_prefix
key_field = '["' + '"]["'.join(json_field) + '"]'
try:
value = eval(f'{json_prefix}{key_field}')
return value
except Exception as e:
return e
def main():
my_json = {
"DataChangedEntry": {
"CurrentValue": {
"RefId": {
"Value": "aaaaaaa"
},
},
},
}
get_value = my_function(my_json, 'my_json', ["DataChangedEntry","CurrentValue","RefId","Value"])
print(get_value)
if __name__ == "__main__":
main()
Output:
aaaaaaa
I have a method which takes one argument, and post to a host.
def fts(searchstring):
search_string="\""+searchstring+"\""
headers = {'Content-Type': 'application/json',}
data = '{ "explain": true,"fields": ["*"],"highlight": {},"query": { "query": '+search_string+'},"size":0}'
response = requests.post('hostname', headers=headers, data=data, auth=('uname', 'password'))
if response.status_code != 200:
raise Exception("{} - {}".format(response.status_code, response.text))
print(response.json())
I want to run this method in several processes at a time with different arguments. Is this possible? I tried to use multiprocessing but couldn't able to get it.
Thanks
Code pool.map(fts, searchstring) will run simultaneously but it will treat searchstring as list of chars and send every single char to different fts
You need list with all strings
all_results = pool.map(fts, [searchstring1, searchstring2, searchstring3, ...])
BTW: If you will need to send more arguments then you will need list with tuples or sublists.
all_results = pool.map(fts, [(searchstring1, param1), (searchstring2, param2), ...])
and define function as
def fts(args):
searchstring, param = args
def enumerate_all_config_objects(baseDN):
url = 'https://www.AwebsiteThatIWontProvide.com'
payload={"ObjectDN":baseDN,"Pattern":"*aws*","Pattern":"*jkb*"}
r = requests.post(url, verify='/PathToKeyForVerification/', headers=headers,data=json.dumps(payload))
response = r.json()
status = r.status_code
print " "
print "Returned status code:", status
print " "
return response ['Objects'][0]['GUID']
Output:
Returned status code: 200
{11871545-8c5b-4c3c-9609-7372fae1add5}
Process finished with exit code 0
I am trying to return ONLY the "GUID" information from a json request. This works (the 1187154...), as I enter values into the index between ['objects'] and ['guid'], each value is successfully produced from the list. My problem is, even though I am printing out the actual response to verify the output is correct, the final script should not require anything being dumped to a CSV file. I have to perform everything in memory. The next function that I need to create will use the returned GUID values and query the server with those values.
How do I get the items in the list to display from the output of enumerate_all_config_objects? I would like to print them to troubleshoot initially. Then I will comment out this feature and have the second function pull each value from that list and use it.
Two problems:
Print out the list which will always have an unknown number of entries.
Create another function to reference / use the values from that list.
The list is populated correctly, I've verified this. I just don't know how to access it or print it.
If I understood correctly, you are looking to do something like:
def enumerate_all_config_objects(baseDN):
url = 'https://www.AwebsiteThatIWontProvide.com'
payload = {"ObjectDN": baseDN, "Pattern": "*aws*", "Pattern": "*jkb*"}
r = requests.post(url, verify='/PathToKeyForVerification/', headers=headers,data=json.dumps(payload))
response = r.json()
status = r.status_code
return map(lambda x: x["GUID"] , response['Objects'])
def use_guids(guid_list):
#do_stuff, for example, to show the guids:
for guid in guid_list:
print(guid)
use_guids(enumerate_all_config_objects(baseDN=<replaceWithYourParameter>))
Edit : To clear out the questions from your comment, I decided to mock the call to the API which you said already works
def enumerate_all_config_objects():
foo = {"GUID" : 1}
bar = {"GUID" : 2}
baz = {"GUID" : 3}
response = {"Objects": [foo, bar, baz] }
mapped = list(map(lambda x: x["GUID"] , response['Objects']))
return map(lambda x: x["GUID"] , response['Objects'])
def use_guids(guid_list):
#do_stuff, for example, to show the guids:
for guid in guid_list:
print(guid)
use_guids(enumerate_all_config_objects())
prints out
1
2
3
When you want to use the value computed from a function, you need to use the return keyword.
For example return map(lambda x: x["GUID"] , response['Objects']), in this new example would return a map object containing [1, 2, 3]. This return value can then be used such as in my first example by passing it to another function.
In the example just above, the list is passed to the use_guids function, which prints the contents of the list.
Edit 2 : If you really insist on calling a function that handles one GUID, you can do that in this way:
def enumerate_all_config_objects(baseDN):
url = 'https://www.AwebsiteThatIWontProvide.com'
payload = {"ObjectDN": baseDN, "Pattern": "*aws*", "Pattern": "*jkb*"}
r = requests.post(url, verify='/PathToKeyForVerification/', headers=headers,data=json.dumps(payload))
response = r.json()
status = r.status_code
for obj in response['Objects']:
use_guid(obj["GUID"])
def use_guid(guid):
print(guid)
# Do some other stuff.
# ...
enumerate_all_config_objects(baseDN=<replaceWithYourParameter>)