We Keep Coding

Can't update record in DynamoDB

Really new to python and coding in general - would really help if someone could point me in the right direction with some code.
So to start off with I am making a proof of concept for a car park running AWS Rekognition and I need some help with updating the database. As you can see with the code below it inputs the reg_plate, entry_time and exit_time into the database all okay. But, what I am trying to do is when Rekognition is invoked a second time with the same reg_plate it updates the exit_time only for the current record in the database.
import boto3
import time
def detect_text(photo, bucket):
client=boto3.client('rekognition')
response=client.detect_text(Image={'S3Object':{'Bucket':bucket,'Name':photo}})
textDetections=response['TextDetections']
for text in textDetections:
if text['Type'] == 'LINE':
return text['DetectedText']
return False
def main(event, context):
bucket=''
photo='regtest.jpg'
text_detected=detect_text(photo,bucket)
if (text_detected == False):
exit()
print("Text detected: " + str(text_detected))
entry_time = str(int(time.time()))
dynamodb = boto3.client('dynamodb')
table_name = 'Customer_Plate_DB'
item = {
'reg_plate': {
'S': str(text_detected)
},
'entry_time': {
'S': entry_time
},
'exit_time': {
'S': str(0)
}
}
dynamodb.put_item(TableName=table_name, Item=item)
Tried various if statements but with no luck as whenever I try it just keeps making new records in the database and the exit_time is never updated.

In DynamoDB a PutItem will overwrite/insert data, so its not what you need if you are wanting to update a single attribute. You will need to use UpdateItem:
response = dynamodb.update_item(
TableName=table_name,
Key={
'reg_plate': {'S': str(text_detected)},
'entry_time': {'S': entry_time}
},
UpdateExpression="SET #t = :t",
ExpressionAttributeValues={ ":t": str(0) },
ExpressionAttributeNames={"#t":"exit_time"},
ConditionExpression='attribute_exists(reg_plate)'
)
In your question you do not tell me what your partition and/or sort keys are, so here I assume you have a partition key only called pk, which you can change to suit your needs.
In the above example, for item where pk = '123' I set exit_time to a value only if the item already exists in your table (Condition Expression).

I have an Error with python flask cause of an API result (probably cause of my list) and my Database

I use flask, an api and SQLAlchemy with SQLite.
I begin in python and flask and i have problem with the list.
My application work, now i try a news functions.
I need to know if my json informations are in my db.
The function find_current_project_team() get information in the API.
def find_current_project_team():
headers = {"Authorization" : "bearer "+session['token_info']['access_token']}
user = requests.get("https://my.api.com/users/xxxx/", headers = headers)
user = user.json()
ids = [x['id'] for x in user]
return(ids)
I use ids = [x['id'] for x in user] (is the same that) :
ids = []
for x in user:
ids.append(x['id'])
To get ids information. Ids information are id in the api, and i need it.
I have this result :
[2766233, 2766237, 2766256]
I want to check the values ONE by One in my database.
If the values doesn't exist, i want to add it.
If one or all values exists, I want to check and return "impossible sorry, the ids already exists".
For that I write a new function:
def test():
test = find_current_project_team()
for find_team in test:
find_team_db = User.query.filter_by(
login=session['login'], project_session=test
).first()
I have absolutely no idea to how check values one by one.
If someone can help me, thanks you :)
Actually I have this error :
sqlalchemy.exc.InterfaceError: (InterfaceError) Error binding
parameter 1 - probably unsupported type. 'SELECT user.id AS user_id,
user.login AS user_login, user.project_session AS user_project_session
\nFROM user \nWHERE user.login = ? AND user.project_session = ?\n
LIMIT ? OFFSET ?' ('my_tab_login', [2766233, 2766237, 2766256], 1, 0)

It looks to me like you are passing the list directly into the database query:
def test():
test = find_current_project_team()
for find_team in test:
find_team_db = User.query.filter_by(login=session['login'], project_session=test).first()
Instead, you should pass in the ID only:
def test():
test = find_current_project_team()
for find_team in test:
find_team_db = User.query.filter_by(login=session['login'], project_session=find_team).first()
Asides that, I think you can do better with the naming conventions though:
def test():
project_teams = find_current_project_team()
for project_team in project_teams:
project_team_result = User.query.filter_by(login=session['login'], project_session=project_team).first()

All works thanks
My code :
project_teams = find_current_project_team()
for project_team in project_teams:
project_team_result = User.query.filter_by(project_session=project_team).first()
print(project_team_result)
if project_team_result is not None:
print("not none")
else:
project_team_result = User(login=session['login'], project_session=project_team)
db.session.add(project_team_result)
db.session.commit()

Python Boto3 - Unable to list Instances without tags

Here I'm writing a python program to list all the ec2 instances without the tag "Owner". Where I'm stuck right now is if the program encounters any Instance already having the "Owner" tag then it exits without checking further Instances. What I want is to skip those instances and continue to check other instances.
Here is my Code:
import boto3
client = boto3.client('ec2',region_name='ap-south-1')
a = client.describe_instances()
count=0
y="Owner"
for i in a['Reservations']:
for j in i['Instances']:
for k in range(len(j['Tags'])):
if y == j['Tags'][k]['Key']:
count +=1
else:
continue
if count==0:
print "The following Instance does not have Owner Tag: "+j['InstanceId']
Here is how the json dict object would look like:
{
'Reservations': [{
'Instances': [{
.
.
.
.
'Tags':[
{
'Value': 'sample1',
'Key': 'Name'
},
{
'Value': 'user',
'Key': 'Owner'
}
]
}
}
}

Your continue is causing a few problems.
Here's an alternate version:
import boto3
key_to_find = 'Owner'
client = boto3.client('ec2')
response = client.describe_instances()
for reservation in response['Reservations']:
for instance in reservation['Instances']:
if key_to_find not in [tag['Key'] for tag in instance['Tags']]:
print (key_to_find + ' tag not found for Instance ' + instance['InstanceId'])

I noticed a couple of things in your code that if you did them slightly different maybe it would be easier to control the program's flow. For starters, use more descriptive variable names, if you have a list of instances for example, a good name would simply be instances. Here is an example of how you could do this:
import boto3
client = boto3.client('ec2',region_name='ap-south-1')
data = client.describe_instances()
count=0
reservations = data["Reservations"]
for reservation in reservations:
instances = reservation["Instances"]
for instance in instances:
tags = instance["Tags"]
instanceId = instance['InstanceId']
hasOwner = len([tag for tag in tags if tag["Key"] == "Owner"]) > 0
print "The following Instance does not have Owner Tag: " + instanceId
Notice how I also added some spacing to increase readability.
Lastly, if you have a string, namely "Owner", it would be silly to put thin into a variable called owner, since that would only be confusing if it were ever changed.

You are exiting the loop after finding the first tag that does not match. Get all the tag and then check. Try this:
for i in a['Reservations']:
for j in i['Instances']: # For each instance
keys = [tag['Key'].upper() for tag in j['Tags']] # Collect all tags
if 'OWNER' not in keys: # Case insensitive check of Owner tag
print "The following Instance does not have Owner Tag: "+j['InstanceId']

having just written almost exactly this there are few areas that were over looked here. In no particular order:
The examples given will only cover one region.
The examples will only find instance that are missing an owner tag, There is the case where the Tag exists, but the Value is empty
In the case of newly launched instance, with no Tags added, the answer examples will error out.
Below answers the question, and covers the points above.
#Open an initial client to get a list of existing regions
client = boto3.client('ec2', region_name='us-east-1')
# Get a list of all existing regions
zones = client.describe_regions()
# Loop through all regions as anyone can launch in any region though we all
# tend to work in us-east-1
for region in zones["Regions"]:
#reset a dict used for tracking to make sure we don't carry data forward
bt_missing = {}
region_called = region["RegionName"]
# open a new client as we move through each region
ec2 = boto3.client('ec2', region_name=region_called)
# Get a list of running instances in the region
response = ec2.describe_instances()
# Loop though all instances looking for instances with No tags, Missing Billing Owner tags,
# or empty Billing Owner tags... adding the instance-id to a list
for resv in response["Reservations"]:
for inst in resv["Instances"]:
if "Tags" not in [x for x in inst]:
print(f"The instance: {inst['InstanceId']} has no tags at all, if your's please update: {region_called}")
continue
if "Billing_Owner" not in [t['Key'] for t in inst["Tags"]]:
bt_missing.update({inst["InstanceId"]: region_called})
for resv in response["Reservations"]:
for inst in resv["Instances"]:
if "Tags" not in [x for x in inst]:
continue
for tag in inst["Tags"]:
if tag['Key'] == "Billing_Owner" and not tag['Value']:
bt_missing.update({inst["InstanceId"]: region_called})
# Take the list of all instance-ids in all regions that have blank, or missing
# billing owner tags or no tags at all, and retrieve the Name tag for that instance
# and put that into a dict -- instanceid: name-region format
# reset inst to not carry data foward
inst = ""
for inst in bt_missing:
report_on = ec2.describe_tags(
Filters=[
{
'Name': 'resource-id',
'Values': [inst],
},
],
)
for tag in report_on["Tags"]:
if tag["Key"] == "Name":
instname = tag["Value"]
message = f"The instance: {inst} has no Billing Owner tag, if yours, please update: Name: {instname}, region: {bt_missing[inst]}"
print(message)

Sqlalchemy not committing object changes to postgres DB

I am using the following passage of code:
#app.route('/budget_item/<int:budget_id>/edit', methods=['GET', 'POST'])
def budget_item_edit(budget_id):
budget_item = session.query(Budget).filter_by(id=budget_id).one()
print "Start EDIT sequence"
# Return form data from HTML initial load form
elif request.method == 'POST':
budget_amount_reallocated_total = budget_item.budget_amount_reallocated_total
#ORIGINAL BUDGET
if request.form['transaction_type'] == 'Original Budget':
#amount
if request.form['amount'] == "":
amount = 0
else:
amount = float(str(request.form['amount']))
budget_item = Budget(
#created_date = "",
budget_transaction_type = request.form['transaction_type'],
budget_line = request.form['budget_line'],
amount = amount,
description = request.form['description']
#date_received = request.form['date_received']
)
try:
count = 1
while count < 10000:
count += 1
#budget_line
setattr(budget_item,'budget_line'+str(count),request.form['budget_line'+str(count)])
#amount
setattr(budget_item,'amount'+str(count),float(request.form['amount'+str(count)]))
budget_amount_reallocated_total += float(request.form['amount'+str(count)])
setattr(budget_item, 'budget_amount_reallocated_total', budget_amount_reallocated_total)
#description
setattr(budget_item,'description'+str(count), request.form['description'+str(count)])
#date_received
setattr(budget_item,'date_received'+str(count),request.form['date_received'+str(count)])
session.commit()
except:
session.commit()
return redirect(url_for('budget_master'))
else:
print "I'm done! This is not a post request"
This block of code is setup to pass data from an HTML via a POST request an then update a corresponding object in the Postgres DB. I can confirm that the object queried from the DB "budget_item" is being updated by settattr. At the end of the passage, I use commit() to update the object; however, the database doesn't reflect the changes. Just to test to make sure things are flowing, I've tried session.add(budget_item) followed by session.commit() to make sure the connect to the DB is OK. That works. How do i update this budget_item object into the database? Any help is much appreciated.

i think that a simple
budget_item.budget_amount_reallocated_total = budget_amount_reallocated_total
session.add(budget_item)
session.commit()
is the right way to do it

To answer your question, to update the budget_item that already exists in the database you need to update the Budget instance that you retrieved from the database, i.e.
budget_item = session.query(Budget).filter_by(id=budget_id).one()
not the one that you have newly created with:
budget_item = Budget(...)
Here the first budget_item represents the row in the database, so this is the one to update. To that end you can replace the code that creates the second Budget instance with this:
budget_item.budget_transaction_type = request.form['transaction_type']
budget_item.budget_line = request.form['budget_line']
budget_item.amount = amount
budget_item.description = request.form['description']
Once you have finished updating the Budget instance you can call session.commit() to flush it to the database.
As mentioned in my comment to your question, it appears that you are trying to add a large number of additional attributes to budget_item all of which will be ignored by sqlalchemy unless they are defined in the mapping between the Budget instance and the Budget table.

Mongoengine, retriving only some of a MapField

For Example.. In Mongodb..
> db.test.findOne({}, {'mapField.FREE':1})
{
"_id" : ObjectId("4fb7b248c450190a2000006a"),
"mapField" : {
"BOXFLUX" : {
"a" : "f",
}
}
}
The 'mapField' field is made of MapField of Mongoengine.
and 'mapField' field has a log of key and data.. but I just retrieved only 'BOXFLUX'..
this query is not working in MongoEngine....
for example..
BoxfluxDocument.objects( ~~ querying ~~ ).only('mapField.BOXFLUX')
AS you can see..
only('mapField.BOXFLUX') or only only('mapField__BOXFLUX') does not work.
it retrieves all 'mapField' data, including 'BOXFLUX' one..
How can I retrieve only a field of MapField???

I see there is a ticket for this: https://github.com/hmarr/mongoengine/issues/508
Works for me heres an example test case:
def test_only_with_mapfields(self):
class BlogPost(Document):
content = StringField()
author = MapField(field=StringField())
BlogPost.drop_collection()
post = BlogPost(content='Had a good coffee today...',
author={'name': "Ross", "age": "20"}).save()
obj = BlogPost.objects.only('author__name',).get()
self.assertEquals(obj.author['name'], "Ross")
self.assertEquals(obj.author.get("age", None), None)

Try this:
query = BlogPost.objects({your: query})
if name:
query = query.only('author__'+name)
else:
query = query.only('author')

I found my fault! I used only twice.
For example:
BlogPost.objects.only('author').only('author__name')
I spent a whole day finding out what is wrong with Mongoengine.
So my wrong conclusion was:
BlogPost.objects()._collection.find_one(~~ filtering query ~~, {'author.'+ name:1})
But as you know it's a just raw data not a mongoengine query.
After this code, I cannot run any mongoengine methods.
In my case, I should have to query depending on some conditions.
so it will be great that 'only' method overwrites 'only' methods written before.. In my humble opinion.
I hope this feature would be integrated with next version. Right now, I have to code duplicate code:
not this code:
query = BlogPost.objects()
query( query~~).only('author')
if name:
query = query.only('author__'+name)
This code:
query = BlogPost.objects()
query( query~~).only('author')
if name:
query = BlogPost.objects().only('author__'+name)
So I think the second one looks dirtier than first one.
of course, the first code shows you all the data
using only('author') not only('author__name')