I'm writing a small web app with the Pyramid framework (which is awesome) and I'm having trouble updating a document. I have no problems querying for a document with the same id- I just can't seem to update it. Here is the code I'm using:
for key, value in request.POST.iteritems():
if value:
to_insert[key] = value
if "_id" in request.POST:
try:
_id = ObjectId(request.matchdict['id'])
except InvalidId:
return Response("Error generating id")
request.db['concerts'].update(
{ '_id': _id },
{ "$set": to_insert },
upsert=False
)
If I do:
request.db['concerts'].find( {'_id' : _id }
I find the document that I wish to update, so it doesn't seem to be an issue of the id not being there. It just doesn't seem to commit anything to the database. The only issue I could see is that I update the entire document- I don't check the fields beforehand. Is this the proper way to go about this? I didn't see much in the documentation about updating an indiscriminate number of fields.
Related
I am quite new to django rest framework and I would like to discuss what is the best way to implement an API endpoints I am aiming to have.
I have following JSON example:
[
{
'top_lvl_key1': {
id: 1
'some_key1': "value"
'some_key2' : "value"
}
},
{
'top_lvl_key2': {
id: 1
'some_key1': "value"
'some_key2' : "value"
}
},
{
'top_lvl_key1': {
id: 2
'some_key1': "value"
'some_key3' : "value"
}
},
{
'top_lvl_key3': {
id: 1
'some_key1': "value"
'some_key2' : "value"
}
}
]
and I want to have 3 endpoints
[POST] /insert inserting and parsing the entire JSON to store it into models
[GET] /detail/<top_lvl_key>/ Display all record for specfici top_lvl_key
[GET] /detail//<top_lvl_key>/id/ Display a record for specific top_lvl_key and specific id (only 1 record)
As you can see, I need to store each type of top_lvl_key to different model so I can retrieve for GET requests. However I am not sure how should I got about serializing the JSON, since the dictionaries of the top_lvl_keys can differ. For example top_lvl_key_1 can sometimes have some_key_1 and some_key_2, but sometimes some_key_1 and some_key_2. So far I have been implementing the models with blank=True, null=True, but I am not sure this is best solution as there a lot of attributes which can be NULL now. Is there a better way to go about this?
Thanks a lot in advance
I'm trying to update contacts using Python and Microsoft Graph within a profile page I've created using Django.
I can access contacts and get all the data I need, however I can't work out how to update the fields.
The only information I can find is on the Graph website, however I can't work out how to translate this into usable code:
PATCH PATCH https://graph.microsoft.com/beta/me/contacts/
Content-type: application/json
{
"title": "Mr",
"givenName": "Steve"
}
I assume there is a way to just put this together as a simple link but I cannot work it out. I've tried the following:
PATCH https://graph.microsoft.com/beta/me/contacts/{id}/title/Mr
PATCH https://graph.microsoft.com/beta/me/contacts/{id}/title:Mr
PATCH https://graph.microsoft.com/beta/me/contacts/{id}/title/$value==Mr
but they all produce errors
There are no tutorials for doing this with Python on the Microsoft site and it's proving very difficult to find any info on it. So hopefully someone can help out.
Cheers!
!!!!!!!!!!!!! UPDATE !!!!!!!!!!!!!!!!!!
Here is my current code which still sadly does nothing:
In my views.py:
def profile(request):
if request.session['has_id']==False:
contact_id = request.session['contact_id'] = request.POST.get('edit')
request.session['has_id'] = True
else:
contact_id = request.session['contact_id']
context = ct.profile(request,id=request.session.get('contact_id'),init=initialize_context,get_tok=get_token)
if request.method=="PATCH":
ct.update(contact_id,'title',request.PATCH.get('title'))
return render(request, 'tutorial/profile.html', context)
and my updater:
def update(id,key,value):
url = '{}/me/contacts/{}'.format(graph_url,id)
payload = {key : value}
head = {
"Content-type" : "application/json",
}
requests.patch(url=url,data=payload,headers=head)
Finally worked it out, I thought I'd tried something like this yesterday but apparently not.
Here's how to do it!
views.py
def profile(request):
if request.session['has_id']==False:
contact_id = request.session['contact_id'] = request.POST.get('edit')
request.session['has_id'] = True
else:
contact_id = request.session['contact_id']
context = ct.profile(request,id=request.session.get('contact_id'),init=initialize_context,get_tok=get_token)
if request.method=="PATCH":
ct.update(contact_id,'title',request.PATCH.get('title'))
return render(request, 'tutorial/profile.html', context)
contacts_helper.py:
def update(token,id,key,value):
graph_client = OAuth2Session(token=token)
url = '{}/me/contacts/{}'.format(graph_url,id)
payload = {key : value}
graph_client.patch(url,data=json.dumps(payload),headers={'Content-type': 'application/json'})
Obviously if you're looking at this you've probably already set up auth_helper.py and graph_helper.py but if you haven't then you should head over the Microsoft Graph website and follow these instructions:
https://developer.microsoft.com/en-us/graph/get-started/python
Toolchain/frameworks
I'm using django==2.1.3 and python-cloudant==2.1.3 and running CouchDB ver. 2.2.0, and pretty much doing all of my setup/configuration through Fauxton. I like to think that I know my way around python/django in general, and I'm testing this approach in a small little project to see how it works
Problem Description
Suppose I have a fairly simple CRUD application with just 1 model:
class Asset(models.Model):
asset_id = models.CharField(max_length=32)
asset_name = models.CharField(max_length=32)
and I have a view that I use to create the asset
class CreateAssetView(views.View):
def get(self, request, *args, **kwargs):
#some code here
def post(self, request, *args, **kwargs):
#some code here|
#log request data into database
client = CouchDB('myusername', 'mypassword', url='http://127.0.0.1:5984', connect=True)
db = client['assets']
log_data = {'view_name': self.view_name, 'post_data': post_data,'user': request.user.username,
'time': str(timezone.now())}
db.create_document(log_data)
return render(...)
I understand that I should be doing the logging portion using a middleware (which I plan to) and probably just use django's CreateView in that case, I'm doing this approach for now just during early development.
What I'm having a problem wrapping my head around is creating a user with myusername and mypassword that has the permissions to:
Write new documents
Read old documents
not edit already created documents
I could even settle for 1 and 3 only (and only use admin to read). I spent a little bit of time playing around with Fauxton's interface for permissions, but I can only basically create a user and assign a role (couldn't even get to assigning a password :/)
clarification
The Asset is not a CouchDB document, that's a normal SQL model, I only want to dump the logs with post data to CouchDB
Any help/gudiance/documentation pointers would be really appreciated
Overview
Couchdb has one overriding level of administrator setup in configuration instead of setup in the _users database and assigned the _admin permission to prevent any possibility of being locked out.
Each individual database then has a rough level security policy of 2 levels:
admins
members
specified via:
names
roles
making 4 fields.
These levels control access slightly differently for the 2 types of documents a db can contain:
id: _design/* - Design documents can contain functions that will be executed in some context
id: other - Normal documents are just normal data
Both levels of database access have read access to all documents in the database, but admins have write access to _design documents. Write access to normal documents is usually given to all users granted any access to the db, but can be limited by validate design documents.
To Summarize
The process for setting a unique security policy up is:
Experience the provided validate design document as a consumer while setting up _users.
Setup a new database and its basic security giving your users member access.
Add a design doc to the new database with a validate function that restricts member write access.
1 Setting up _users entries
Add a role to a user
As an admin add role: ["logger"] to a user's doc and save it, note that this must be done by admin due to this part of the default _users design document:
// DB: _users doc: _design/_auth
function(newDoc, oldDoc, userCtx, secObj) {
..
if (oldRoles.length !== newRoles.length) {
throw({forbidden: 'Only _admin may edit roles'});
}
Change the password for the user.
Either the admin or the user can change their password by setting password:"mynewpassword" in their document (which couchdb will transform into a hashed/salted password during the save process). This works for a user since they can add/modify fields aside from their name and roles, as long as a user is editing their own doc:
// DB: _users doc: _design/_auth
function(newDoc, oldDoc, userCtx, secObj) {
..
if (userCtx.name !== newDoc.name) {
throw({
forbidden: 'You may only update your own user document.'
});
}
// then checks that they don't modify roles
You could repeat this process with a user you assign an adminlogger role to create a delegated administrator that you assign permissions to can reconfigure a database or you can continue to use the couchdb admin with its _admin role for all administration.
2 Setup a new database and its basic security
Create a db named logger
assign the logger a security policy:
{
"admins": {
"names": [
],
"roles": [
"adminlogger"
]
},
"members": {
"names": [
],
"roles": [
"logger"
]
}
}
3. Create a new validate design document in the new db
As either _admin user or a user with the adminlogger role create a new validate design doc by copying the _users design document, removing the _rev and modifying the function:
// DB: logger doc: _design/auth
function(newDoc, oldDoc, userCtx, secObj) {
// Don't let non-admins write a pre-existing document:
if (!is_server_or_database_admin()) {
if (!!oldDoc) {
throw({
forbidden: 'You may not update existing documents.'
});
}
}
// Where the function to define admins can be copied verbatim from the doc:
var is_server_or_database_admin = function(userCtx, secObj) {
// see if the user is a server admin
if(userCtx.roles.indexOf('_admin') !== -1) {
return true; // a server admin
}
// see if the user a database admin specified by name
if(secObj && secObj.admins && secObj.admins.names) {
if(secObj.admins.names.indexOf(userCtx.name) !== -1) {
return true; // database admin
}
}
// see if the user a database admin specified by role
if(secObj && secObj.admins && secObj.admins.roles) {
var db_roles = secObj.admins.roles;
for(var idx = 0; idx < userCtx.roles.length; idx++) {
var user_role = userCtx.roles[idx];
if(db_roles.indexOf(user_role) !== -1) {
return true; // role matches!
}
}
}
return false; // default to no admin
}
}
If you followed these steps then the user you gave the logger role in step 1 can run your code to write new documents only in logger database configured in steps 2 and 3.
I have built a multi-page survey application using a Django 1.6.2 SessionWizardView but I am having some trouble with how the data submitted through the survey form is being saved in my MySQL database django_db. That is I simply do not recognize or understand what is being saved. This is my first time building an application like this or even working with a databases so please forgive my ignorance.
My problem is that when I look at my database via phpMyAdmin nothing is recognizable as being from my application, So I have no idea where to start.
I believe the data from my SessionWizardView should get stored in the django_session table but when I inspect it it looks like gibberish.
Here is a copy of the session_data- Is this what I am meant to be looking at?
ZTEwNWUzZmI4NjA5MmQ3Nzk2MDQ1MWY3YzE2MjYxZWZjNDJmODQ3Yjp7ImVpZ2h0X2ltYWdlIjoiUDFEOS5qcGciLCJwYXRoX3R3b19pbWFnZXMiOltdLCJwYXRoX29uZV9pbWFnZXMiOltdLCJ0aGlyZF9pbWFnZSI6IlA5RDguanBnIiwiaW1hZ2VzIjpbXSwid2l6YXJkX3N1cnZleV93aXphcmRfb25lIjp7InN0ZXBfZmlsZXMiOnt9LCJzdGVwIjpudWxsLCJleHRyYV9kYXRhIjp7fSwic3RlcF9kYXRhIjp7fX0sInNpeHRoX2ltYWdlIjoiUDVENC5qcGciLCJmb3VydGhfaW1hZ2UiOiJQNEQzLmpwZyIsImZpZnRoX2ltYWdlIjoiUDJEMS5qcGciLCJ3aXphcmRfc3VydmV5X3dpemFyZF90d28iOnsic3RlcF9maWxlcyI6e30sInN0ZXAiOm51bGwsImV4dHJhX2RhdGEiOnt9LCJzdGVwX2RhdGEiOnt9fSwicGF0aF90aHJlZV9pbWFnZXMiOlsiUDNEMS5qcGciLCJQNEQyLmpwZyIsIlA1RDMuanBnIiwiUDZENC5qcGciLCJQN0Q1LmpwZyIsIlA4RDYuanBnIiwiUDlENy5qcGciLCJQMUQ4LmpwZyIsIlAyRDkuanBnIl0sImluc3RydWN0aW9uX3Rhc2tfb25lX2ltYWdlcyI6WyJJVDFBLmpwZyIsIklUMUIuanBnIiwiSVQxQy5qcGciXSwiZmlyc3RfaW1hZ2UiOiJQNkQ1LmpwZyIsInNlY29uZF9pbWFnZSI6IlA4RDcuanBnIiwic2V2ZW50aF9pbWFnZSI6IlAzRDIuanBnIiwic2xpZGVyX0RWX3ZhbHVlcyI6W10sImluc3RydWN0aW9uX3Rhc2tfdHdvX2ltYWdlcyI6WyJJVDJBLmpwZyIsIklUMkIuanBnIiwiSVQyQy5qcGciXSwibmludGhfaW1hZ2UiOiJQN0Q2LmpwZyIsIndpemFyZF9zdXJ2ZXlfd2l6YXJkX3RocmVlIjp7InN0ZXBfZmlsZXMiOnsiMSI6e30sIjAiOnt9LCIyIjp7fX0sInN0ZXAiOiIzIiwiZXh0cmFfZGF0YSI6e30sInN0ZXBfZGF0YSI6eyIxIjp7IjEtbm90aGluZyI6WyIiXSwic2xpZGVyX3ZhbHVlIjpbIisxMDAiXSwiY3NyZm1pZGRsZXdhcmV0b2tlbiI6WyIwblNZWTFuZkZ6QzdBb0R0UFlwZkpyc1NySGhGRWhONyJdLCJzdWJtaXQiOlsiTmV4dCJdLCJzdXJ2ZXlfd2l6YXJkX3RocmVlLWN1cnJlbnRfc3RlcCI6WyIxIl19LCIwIjp7IjAtbm90aGluZyI6WyIiXSwiY3NyZm1pZGRsZXdhcmV0b2tlbiI6WyIwblNZWTFuZkZ6QzdBb0R0UFlwZkpyc1NySGhGRWhONyJdLCJzdXJ2ZXlfd2l6YXJkX3RocmVlLWN1cnJlbnRfc3RlcCI6WyIwIl0sInN1Ym1pdCI6WyJOZXh0Il0sInNsaWRlcl92YWx1ZSI6WyIrMTAwIl19LCIyIjp7ImNzcmZtaWRkbGV3YXJldG9rZW4iOlsiMG5TWVkxbmZGekM3QW9EdFBZcGZKcnNTckhoRkVoTjciXSwiMi1ub3RoaW5nIjpbIiJdLCJzdXJ2ZXlfd2l6YXJkX3RocmVlLWN1cnJlbnRfc3RlcCI6WyIyIl0sInN1Ym1pdCI6WyJOZXh0Il19fX19
Questions:
Can someone take a look at my process/code below and tell me if I am
missing a step, or have not included something in my code?
Am I missing something at the MySQL end? Should I be creating
specific tables or somehow customizing it so that it stores the data
in a useable/readable format?
Am I even looking in the right place for completed survey
application data? Where does a SessionWizardView store/send it?
Once again this is my first database application so thank you for your patients
Process:
I created the database in MySQL (Ver 14.14 Distrib 5.6.20) via
Terminal CREATE database django_db; and the tables in it are
created when I run the command python manage.py syncdb Other than
that I do not touch the database.
I can complete my survey built using the SessionWizardView both on my
local machine and on the public server. No errors and it apepars
everything works fine
I have setup phpMyAdmin and can see the django_db database. However
I don't really know what I am looking at.
Code:
forms.py
For the most part the survey has relatively simple questions such as:
class SurveyFormA(forms.Form):
#When were you born?
birthdate = forms.DateField(widget=extras.SelectDateWidget(years = range(1995, 1900, -1)), label='What is your Date of Birth?', required = False)
#What is your current relationship status?
SINGLE = 'Single'
INARELATIONSHIP = 'In a relationship'
MARRIED = 'Married'
DIVORCED = 'Divorced'
SEPARATED = 'Separated'
WIDOWED = 'Widowed'
RELATIONSHIP = (
("", "----------"),
(SINGLE, "Single"),
(INARELATIONSHIP, "In a relationship"),
(MARRIED, "Married"),
(DIVORCED, "Divorced"),
(SEPARATED, "Separated"),
(WIDOWED, "Widowed"),
)
relationship = forms.ChoiceField(widget=forms.Select(), choices=RELATIONSHIP, initial= "", label='What is your relationship status?', required = False)
class SurveyFormB(forms.Form): #Internet usage questions
(second page questions here)
....
....
These seem to work fine as you can see in the image below
Each of the SessionWizardViews has a get_context_data used for captureing and storing data from one page of the survey form to the next and done method. I am not showing the full get_context_data as it is quite long.
views.py
class SurveyWizardOne(SessionWizardView):
def get_context_data(self, form, **kwargs):
context = super(SurveyWizardOne, self).get_context_data(form, **kwargs)
....
....
def done(self, form_list, **kwargs):
return render(self.request, 'Return_to_AMT.html', {
'form_data': [form.cleaned_data for form in form_list],
})
Database Connection:
My Django site is connected to a MySQL database in settings.py
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'django_db',
'USER': 'root',
'PASSWORD': '************',
'HOST': '127.0.0.1',
#'PORT': '',
}
}
You have to either write the models.py or set a specific database that you have in mind to be the database accessed. Neither of which seem to be done here.
These links from the official documentation should help :
https://docs.djangoproject.com/en/1.6/ref/databases/
https://docs.djangoproject.com/en/1.6/topics/db/
PS - You should consider using Django 1.8 instead. It's even better documented and much easier to work with. Additionally porting your codes from one Django version to the next is really very easy. Hardly a little more than simple copy pasting.
I'm trying to build in a way to handle a large number of posted options, e.g.
my_posted_data = {"item": "value", "item_options":{"a":2, "b":2} }
This would be coming from somewhere else in an api situation where I'm not in control of the environment and it is simulated for now. I'll post that through the requests library; and moving server-side, I try to get this from the route/view in my application. request.form gets read into a variable(form) which is passed to a task_manager queue. In the task I'll try to do:
options = form.get("item_options", None)
options always ends up as NoneType. Why is this not selecting the dict(like) value of {"a": 2, "b": 2}? I guess I'm doing it wrong, but what at this point I am unable to pinpoint.
Based on this scant picture I've provided, how do I post and and retrieve nested values with Flask request in the most effective way?
EDIT: I had to go a different way, using JSON data because I realized for the situation that is best, the form is more for user input from an html page, and this is something that requires a different approach.
By using Flask-WTF with Field Enclosures in WTForms, you can easily handle nested post data.
class ListField(Field):
def process_formdata(self, valuelist):
self.data = valuelist
class ItemsForm(Form):
a = StringField()
b = StringField()
class TestForm(FlaskForm):
item = StringField()
product = ListField(FormField(ItemsForm))
Since FormField add default prefix, the JSON will looks like
{
"item": "value",
"product": {
"product-a": "string",
"product-b": "string",
}
}