I'm writing a python program to save and retrieve a customer data in cloud datastore. My entity looks like below:
entity.update({
'customerId': args['customerId'],
'name': args['name'],
'email': args['email'],
'city': args['city'],
'mobile': args['mobile']
})
datastore_client.put(entity)
I'm successfully saving the data. Now, I want to retrieve a random email id from the a record. I have written the below code:
def get_customer():
query = datastore_client.query(kind='CustomerKind')
results = list(query.fetch())
chosen_customer = random.choice(results)
print(chosen_customer)
But instead getting only one random email id, I'm getting the entire row like this:
<Entity('CustomerKind', 6206716152643584) {'customerId': '103', 'city': 'bhubaneswar', 'name': 'Amit', 'email': 'amit#gmail.com', 'mobile': '7879546732'}>
Can anyone suggest how can I get only 'email': 'amit#gmail.com' ? I'm new to datastore.
When using
query = datastore_client.query(kind='CustomerKind')
results = list(query.fetch())
you are retrieving all the properties from all the entities that will be returned.
Instead, you can use a projection query, which allows you to retrieve only the specified properties from the entities:
query = client.query(kind="CustomerKind")
query.projection = ["email"]
results = list(query.fetch())
Using projection queries is recommended for cases like this, in which you only need some properties as they reduce cost and latency.
Related
Im trying to add a movie to my mongo DB table but it is only included the last value could someone tell me where I am going wring the code I am using in my python file is below:
#app.route('/insert_movie', methods=['POST'])
def insert_movie():
movie_added = {'title': request.form.get('title')}
{'imdb.rating': request.form.get('imdb.rating')}
{'tomato.rating': request.form.get('tomato.rating')}
{'year': request.form.get('year')}
{'runtime': request.form.get('runtime')}
{'actors': request.form.get('actors')}
{'director': request.form.get('director')}
{'plot': request.form.get('plot')}
mongo.db.movie_information.insert_one(movie_added)
return redirect(url_for('get_movies'))
Try putting all of the fields into a single object (see below). That way, when you insert one movie, that movie contains all of the fields from the one object that you sent.
movie_added = {'title': request.form.get('title'),
'imdb.rating': request.form.get('imdb.rating'),
'tomato.rating': request.form.get('tomato.rating'),
'year': request.form.get('year'),
'runtime': request.form.get('runtime'),
'actors': request.form.get('actors'),
'director': request.form.get('director'),
'plot': request.form.get('plot')}
mongo.db.movie_information.insert_one(movie_added)
return redirect(url_for('get_movies'))`
I got a list in Python with Twitter user information and exported it with Pandas to an Excel file.
One row is one Twitter user with nearly all information of the user (name, #-tag, location etc.)
Here is my code to create the list and fill it with the user data:
def get_usernames(userids, api):
fullusers = []
u_count = len(userids)
try:
for i in range(int(u_count/100) + 1):
end_loc = min((i + 1) * 100, u_count)
fullusers.extend(
api.lookup_users(user_ids=userids[i * 100:end_loc])
)
print('\n' + 'Done! We found ' + str(len(fullusers)) + ' follower in total for this account.' + '\n')
return fullusers
except:
import traceback
traceback.print_exc()
print ('Something went wrong, quitting...')
The only problem is that every row is in JSON object and therefore one long comma-seperated string. I would like to create headers (no problem with Pandas) and only write parts of the string (i.e. ID or name) to colums.
Here is an example of a row from my output.xlsx:
User(_api=<tweepy.api.API object at 0x16898928>, _json={'id': 12345, 'id_str': '12345', 'name': 'Jane Doe', 'screen_name': 'jdoe', 'location': 'Nirvana, NI', 'description': 'Just some random descrition')
I have two ideas, but I don't know how to realize them due to my lack of skills and experience with Python.
Create a loop which saves certain parts ('id','name' etc.) from the JSON-string in colums.
Cut off the User(_api=<tweepy.api. API object at 0x16898928>, _json={ at the beginning and ) at the end, so that I may export they file as CSV.
Could anyone help me out with one of my two solutions or suggest a "simple" way to do this?
fyi: I want to do this to gather data for my thesis.
Try the python json library:
import json
jsonstring = "{'id': 12345, 'id_str': '12345', 'name': 'Jane Doe', 'screen_name': 'jdoe', 'location': 'Nirvana, NI', 'description': 'Just some random descrition')"
jsondict = json.loads(jsonstring)
# type(jsondict) == dictionary
Now you can just extract the data you want from it:
id = jsondict["id"]
name = jsondict["name"]
newdict = {"id":id,"name":name}
I'm trying to get some POST requests using Postman to MongoDb and everything works well.
Following is the code:
def add_npo():
add_new_npo = mongo.db.npos
name = request.json['name']
description = request.json['description']
category = request.json['category']
status = request.json["status"]
npo_id = add_new_npo.insert({'name': name, 'description':
description, 'category': category,'status': status})
new_npo = add_new_npo.find_one({'_id': npo_id })
output = {'name': new_npo["name"], 'description':
new_npo["description"], 'category': new_npo["category"], 'status':
new_npo["status"]}
return jsonify({'result' : output})
But how can I add new fields on the document without assigning them in advance?
You can add new fields on the document by update operation.
For example,
db.col.update({your filter}, {"$set": {"newField": newFieldValue}})
I am trying to get a list of ids from the most recent posts on my profile so I can perform operations on each post individually elsewhere but am having issues isolating the ids from the rest of the post data. It says that the posts object is a dictionary but when I try to iterate through it using posts['id'] I only get back myID, and when I use for p in posts posts[p] I get back everything. I am relatively new to python and am having some major issues, how can I get a list of just the ids?
import facebook
import json
token = {'****'}
graph = facebook.GraphAPI(token)
page_ids = []
myID = graph.request('/me?fields=id')
print(myID['id'])
posts = graph.get_object(id=myID['id'],fields='posts.fields(object_id)')
print(posts)
{'object_id': '503243983830603', 'id': '2894021623958853_2890876384273377'}, {'object_id': '402662493728056', 'id': '2894021623958853_2890866360941046'}, {'id': '2894021623958853_2890842810943401'}], 'paging': {'previous': 'https://graph.facebook.com/v4.0/2894021623958853/posts?fields=object_id&since=1569687697&access_token=EAAKsaVifZCCYBAE7ofO8qRSheQsBhi2mYrZB39wzfhCZAJ2ejGoyZAi8hdKZAoEjIWUEk7Y1Y8nCb9yrU17JEXf2jGMF7E4SVpneE3EYbCV2zDRb1K8ZCkcY5tQP00DALPWYXNLimGxyGsugwK5GPOZC19suEItAszJM4RHFaTUMMZBXiyiHH0mnzpdUP0vl2MklfCct4mk6EwZDZD&limit=25&__paging_token=enc_AdCi4Pnb01TJi24NGPgqiXLRi3AuoPtBkgJfx43aXRQwPPzkfcJ0BHQuqNXebzs5Vm01uEHcSrvAtTiYcTKvJu5raYCKn4pftMqeyD60hDz0PgZDZD&__previous=1', 'next': 'https://graph.facebook.com/v4.0/2894021623958853/posts?fields=object_id&access_token=EAAKsaVifZCCYBAE7ofO8qRSheQsBhi2mYrZB39wzfhCZAJ2ejGoyZAi8hdKZAoEjIWUEk7Y1Y8nCb9yrU17JEXf2jGMF7E4SVpneE3EYbCV2zDRb1K8ZCkcY5tQP00DALPWYXNLimGxyGsugwK5GPOZC19suEItAszJM4RHFaTUMMZBXiyiHH0mnzpdUP0vl2MklfCct4mk6EwZDZD&limit=25&until=1569590626&__paging_token=enc_AdDfKEIsYp9uDTOhVdXvJ2KKarEfKqyiV3stKSk3ZBPh5rNWDDihjnBZC29jw2xQF1fmjViUe18SIlW8CW4CVV5sFsEtxdSFOnQZA63RNgWrk4ZAqQZDZD'}}, 'id': '2894021623958853'}
I'm trying to place an order using V2 of the HITBTC API (docs here). I'm trying to place an order via a POST request, and everything is fine authorization wise, but upon placing the order, the following function returns what the server is sending back, which is the following JSON:
{'error': {'code': 2001, 'message': 'Symbol not found', 'description': 'Try get /api/2/public/symbol, to get list of all available symbols.'}}
My issue arises with the fact that I'm passing the pair I wish to order in the format that's specified by this call for the symbols, which returns JSON like the following:
{"id":"NOAHBTC","baseCurrency":"NOAH","quoteCurrency":"BTC","quantityIncrement":"1000","tickSize":"0.000000001","takeLiquidityRate":"0.001","provideLiquidityRate":"-0.0001","feeCurrency":"BTC"}
I'm passing a string formatted exactly as 'id' is formatted.
def HITBTCorder(pair, side, quantity, price, session):
'''
Creates an order on HITBTC, returns status (filled or not filled)
Side: 'buy' or 'sell'
'''
orderData = json.dumps({'symbol': pair, 'side': side, 'quantity': quantity, 'price': price})
print(orderData)
response = session.post('https://api.hitbtc.com/api/2/order', data = orderData)
responseDict = json.loads(response.text)
return responseDict
The code I'm running looks like this:
session = requests.session()
session.auth = ('APIPUBLIC', 'APISECRET')
response = trade.HITBTCorder("NOAHBTC", 'buy', 1000, tickers.HITBTCprice("NOAHBTC"), session)
Any idea how to get this working?
You may replace
orderData = json.dumps({'symbol': pair, 'side': side, 'quantity': quantity, 'price': price})
to:
orderData = json.dumps({'symbol': pair.lower(), 'side': side, 'quantity': quantity, 'price': price})
because symbol is required to be sent as lowercase.
Data needs to be URL encoded in request body (quantity=1&symbol=ETHBTC...) not JSON to be accepted by server, hope it helps :)
import urllib.parse as parse;
data = parse.urlencode(yourparamsasdict);