I am finding that I keep getting errors when accessing objects that are nest in three other objects.
The following is an example of one of the entities returned when accessing the API:
entity {
id: "000069R"
trip_update {
trip {
trip_id: "064650_R..S"
route_id: "R"
start_time: "10:46:30"
start_date: "20220902"
nyct_trip_descriptor {
train_id: "1R 1046+ CTL/95S"
direction: SOUTH
}
}
stop_time_update {
stop_id: "G08S"
arrival {
time: 1662129990
}
departure {
time: 1662129990
}
nyct_stop_time_update {
scheduled_track: "D1"
actual_track: "D1"
}
}
}
This is my code:
import gtfs_realtime_pb2
import urllib.request
trainsets=[
['a','c','e'],
['b','d','f', 'm'],
['g'],
['j','z'],
['n','q','r', 'w'],
['l'],
['1','2','3','4','5','6','7']
]
set_number=0
# get the train from user and find the corresponding trainset
char = input("What train are you looking for? ")
char = char.lower()
for sets in trainsets:
if char in sets:
print('Found it!')
set_number = trainsets.index(sets)
# form string from the values in specified list
desired_trainset= ''.join(trainsets[set_number])
# Communicate with the api
feed = gtfs_realtime_pb2.FeedMessage()
response = urllib.request.Request(f'https://api-endpoint.mta.info/Dataservice/mtagtfsfeeds/nyct%2Fgtfs-{desired_trainset}')
response.add_header("x-api-key", '<API KEY HERE>')
feed.ParseFromString(urllib.request.urlopen(response).read())
# access specific info from the api
for entity in feed.entity:
if entity.HasField('trip_update'):
#! ERROR: AttributeError: nyct_trip_descriptor
print(entity.trip_update.trip.nyct_trip_descriptor.direction)
#! ERROR: AttributeError: 'google._upb._message.RepeatedCompositeContainer' object has no attribute 'arrival'
print(entity.trip_update.stop_time_update.arrival.time)
The last two print statements are where I keep getting an error. I can access start_time and start_date
or stop_id from the entity just fine, but I keep getting an error when trying to access objects inside other objects.
Using:
Python: 3.10.6
Windows: 11
I've asked a couple of my friends and they don't see the error.
Thank you for helping.
EDIT:
Someone requested to print the entity before the error. This is one of the responses I get back from the data when I do the following code:
print(entity.trip_update.trip)
#response I get back
trip_id: "006600_R..N"
route_id: "R"
start_time: "01:06:00"
start_date: "20220903"
nyct_trip_descriptor {
train_id: "1R 0106 95S/WHL"
}
Someone commented that the nyct_trip_descriptor is an optional field, and after rereading the documentation i confirmed that it is optional but I'm not sure how to proceed further and whether or not that is the reason why I'm getting this error. Thank you for helping.
Edit:
I've done the following to return a default value when 'nyct_trip_descriptor' is missing:
nyct_trip_descriptor = getattr(entity.trip_update.trip, 'nyct_trip_descriptor', 'not found')
print(nyct_trip_descriptor)
This didn't work since all the entities I got from the API just stated 'not found', but when printing just the entity 'nyct_trip_descriptor' is present as seen in the first code snippet.
Related
Here is my sample code
import boto3
import os
ENV = "dev"
DB = "http://awsservice.com"
REGION = "us-east-1"
TABLE = "traffic-count"
def main():
os.environ["AWS_PROFILE"] = ENV
client = boto3.resource("dynamodb", endpoint_url=DB, region_name=REGION)
kwargs = {'Key': {'id': 'D-D0000012345-P-1'},
'UpdateExpression': 'ADD #count.#car :delta \n SET #parentKey = :parent_key, #objectKey = :object_key',
'ExpressionAttributeValues': {':delta': 1, ':parent_key': 'District-D0000012345', ':object_key': 'Street-1'},
'ExpressionAttributeNames': {'#car': 'car', '#count': 'count', '#parentKey': 'parentKey', '#objectKey': 'objectKey'}}
client.Table(TABLE).update_item(**kwargs)
if __name__ == "__main__":
main()
What I want to achieve is this:
With a single API call (in this update_item), I want to be able to
If the item does not exit. create an item with a map count and initialise it with {'car': 1} and set the fields parent_key and object_key.
or
If the item already exists, update the field to {'car': 2} (if the original count is 1)
Previously, if I did not use a map, I can successfully update with this expression,
SET #count = if_not_exist(#count, :zero) + :delta,
#parentKey = :parent_key, #objectKey = :object_key
However I am getting this error:
botocore.exceptions.ClientError: An error occurred
(ValidationException) when calling the UpdateItem operation: The
document path provided in the update expression is invalid for update
Which document path is causing the problem? How can I fix it?
For those who landed on this page with similar error:
The document path provided in the update expression is invalid for update
The reason may be:
for the item on which the operation is being performed,
this attribute (count, for example) is not yet set.
Considering the sample code from question,
The exception could be coming from all those items where count is empty or not set. So the update query doesn't know on which map the new value(s) (car, for example) needs to be set or updated.
In the question, it was possible for the OP in the beginning because, the attribute is not a map and the process is simply setting the value to count as is. It's not trying to access a key of an unknown map to set the value.
This can be handled by catching the exception. For example:
from botocore.exceptions import ClientError
...
try:
response = table.update_item(
Key={
"pk": pk
},
UpdateExpression="set count.car = :c,
ExpressionAttributeValues={
':c': "some car"
},
ReturnValues="UPDATED_NEW"
)
except ClientError as e:
if e.response['Error']['Code'] == 'ValidationException':
response = table.update_item(
Key={
"pk": pk
},
UpdateExpression="set count = :count",
ExpressionAttributeValues={
':count': {
':c': "some car"
}
},
ReturnValues="UPDATED_NEW"
)
def api_all():
return jsonify(data)
#app.route('/api/', methods=['GET'])
def api_id():
if('id' in request.args and 'key' in request.args):
id = int(request.args['id'])
key = str(request.args['key'])
else:
return "Error: Missing argument."
results = []
for user in data["users"]: # Error from here
print(user)
if(user['id'] == id and user['key'] == key):
results.append(user)
return jsonify(results)
app.run()
There is my main code.
"users": [
{
"id":"0",
"name":"Jane",
"balance":"100$",
"key":"byt3dsz69pl0hmb"
},
{
"id":"1",
"name":"John",
"balance":"100$",
"key":"z0apdio4bvn549e"
}
]
}
Here is data.json or, the data variable. I cannot get this to work, I get the following error:
TypeError: '_io.TextIOWrapper' object is not subscriptable
I am building a basic api to get account balances, this is juts for educational purposes so it does not have to be secure. Any help would be greatly appreciated I have used flask before but I can never get my head around how JSON works with python, I find it quite hard to understand.
That code snippet converts id from the request to an int, but the value of the id key in the JSON is a string. Since
>>> 1 == "1"
False
>>>
you'll to make the types match before you can compare them.
I'm requesting some time entries for users with the Clockify API(). For some reason, I am receiving some responses which include entries without an end-time. I noticed that, the unexpectedly returned entries belong to currently running time entires... However, I did not specify/use the 'in-progress' parameter... What is happening here?
Here is my code:
def fetch_users_time_entries(users):
API_URL = "https://api.clockify.me/api/v1"
for user in users:
url = "{}/workspaces/{}/user/{}/time-entries?hydrated=true&page-size=1000&start=2019-08-05T00:00:01Z".format(API_URL, WORKSPACE_ID, user['clockify_id'])
time_entries = requests.get(url, headers=HEADER)
for time_entry in time_entries.json():
Here is a sample of an unexpected "end" value:
{
'id':'SECRET',
'description':'',
'tags':[
{
'id':'SECRET',
'name':'CERTI',
'workspaceId':'SECRET'
}
],
'user':None,
'billable':True,
'task':{
'id':'SECRET',
'name':'Etapa: Execução e Controle',
'projectId':'SECRET',
'assigneeId':'',
'estimate':'PT0S',
'status':'ACTIVE'
},
'project':{
'id':'SECRET',
'name':'C105',
'hourlyRate':{
'amount':0,
'currency':'USD'
},
'clientId':'SECRET',
'workspaceId':'SECRET',
'billable':True,
'memberships':[
{
'userId':'SECRET',
'hourlyRate':None,
'targetId':'SECRET',
'membershipType':'PROJECT',
'membershipStatus':'ACTIVE'
}
],
'color':'#8bc34a',
'estimate':{
'estimate':'PT0S',
'type':'AUTO'
},
'archived':False,
'duration':'PT25H20M12S',
'clientName':'NEO',
'public':True
},
'timeInterval':{
'start':'2019-08-22T18:55:55Z',
'end':None,
'duration':None
},
'workspaceId':'SECRET',
'totalBillable':None,
'hourlyRate':None,
'isLocked':False,
'userId':'SECRET',
'projectId':'SECRET'
}
I was only expecting time entries that were completed. Any suggestions?
UPDATE (10/16/19):
Another follow-up. They just send me an e-mail saying they fixed the problem. Putting the parameter "in-progress" to false will return only completed time entries. #matthew-e-miller it would be nice to add this to the answer. – Lukas Belck 5 hours ago
Okay, so I finally had a chance to reproduce the problem and it seems... There is not an end-time filter. They have misleadingly provided a start and end parameter, but these both filter on start-time.
The start and end parameters work like this:
The in-progress works as described in the doc, but it doesn't work for your application.
Answer:
I think your best bet is to request all the time entries, place them into a dict/list, and then use your python script to remove elements with "'end: 'None'".
import requests
import json
headers = {"content-type": "application/json", "X-Api-Key": "your api key""}
workspaceId = "your workspace id"
userId = "your user id"
params = {'start': '2019-08-28T11:10:32.998Z', 'end': '2019-08-29T02:05:02Z', 'in-progress': 'true'}
API_URL = "https://api.clockify.me/api/v1/workspaces/{workspaceId}/user/{userId}/time-entries"
print(API_URL)
result_one = requests.get(API_URL, headers=headers, params=params)
print(result_one)
List = json.loads(result_one.text)
for entry in List:
if entry.get("timeInterval")['end'] == None:
List.remove(entry)
print(List)
Output:
List containing only entries which do not have timeInterval.end == 'None'.
Here is time spent on this answer-edit:
I'm trying to create an if statement based on the value from an API I'm using. This API contains a status code value, "status". if this is 404 (or others) I want to return an error, else carry on.
An example of the JSON:
{
"data": {
"test_index": {
"test_a": [...], // 429 items
"test_b": [...] // 182 items
}
},
"status": 200
}
When running the code below:
import json
import urllib.request as ur
API = ur.urlopen('https://example.com')
data = json.loads(API.read())
if data['status'][0] == 404:
print("404")
else:
print("Not 404")
I get the following error:
TypeError: 'int' object is not subscriptable
Which is referring to line 7, the if statement.
How do I convert this JSON value to something I can work with?
data['status'] is an integer, and you cannot subscript an integer with an index, like you do for a list, Change your code as follows.
if data['status'] == 404:
print("404")
else:
print("Not 404")
I have a file which was exported from BIND containing TSIG values for about 500 domain names. I need to repurpose the data into JSON for a REST API query. The BIND data is formatted like so:
// secondary-example.com.
key "2000000000000.key." {
algorithm hmac-md5;
secret "ahashedvalue=";
};
zone "secondary-example.com." {
type slave;
file "sec/secondary-example.com.";
allow-transfer { 1.1.1.1;
1.1.2.2;
};
also-notify { 1.1.1.1;
2.2.2.2;
};
masters {
1.2.3.4 key 2000000000000.key.;
};
};
From this I need to extract the key, zone and secret. Here's an example API request.
{
"properties":{
"name":"secondary-example.com.",
"accountName":"example",
"type":"SECONDARY"
},
"secondaryCreateInfo":{
"primaryNameServers":{
"nameServerIpList":{
"nameServerIp1":{
"ip":"1.2.3.4",
"tsigKey":"2000000000000.key.",
"tsigKeyValue":"ahashedvalue="
}
}
}
}
}
I'm having difficulty crafting a regular expression appropriate for the scenario. I'm looking construct the JSON in a python script and send the request through Postman.
I spent a couple days reading up on regex and figured out a solution. So, each of those "zones" began with a comment... e.g. "secondary-example.com"... and each set of BIND info was 17 lines long exactly. This solution is hackey and always assumes data is correct, but it managed to work.
Separate the zones into chunks of text.
zones = []
cur_zone = ''
f = open(bind_file).readlines()
for line in f:
if line[0:2] == '//':
zones.append(cur_zone)
cur_zone = ''
else:
cur_zone = cur_zone + line
zones.pop(0) # Drop the first list item, it's empty
Iterate through those chunks and match the needed parameters.
for z in zones:
z_lines = z.splitlines()
# Regex patterns to make the required parameters
key = re.findall('\"(.*)\"', z_lines[0])[0]
secret = re.findall('\"(.*)\"', z_lines[2])[0]
name = re.findall('\"(.*)\"', z_lines[5])[0]
master = re.findall('\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}', z_lines[15])[0]