I've read mongoengine documentation about switching collection to save document. And test this code and it worked successfully:
from mongoengine.context_managers import switch_db
class Group(Document):
name = StringField()
Group(name="test").save() # Saves in the default db
with switch_collection(Group, 'group2000') as Group:
Group(name="hello Group 2000 collection!").save() # Saves in group2000 collection
But the problem is when I want to find saved document in switch collection switch_collection doesn't work at all.
with switch_collection(Group, 'group2000') as GroupT:
GroupT.objects.get(name="hello Group 2000 collection!") # Finds in group2000 collection
As of mongoengine==0.10.0 mongoengine.context_managers.switch_collection(cls, collection_name)
used as "with switch_collection(Group, 'group1') as Group:" in the example
doesn't work inside functions. It gives unboundlocalerror. A simple get around with existing resources is :
To get:
new_group = Group.switch_collection(Group(),'group1')
from mongoengine.queryset import QuerySet
new_objects = QuerySet(Group,new_group._get_collection())
Use new_objects.all() to get all objects etc.
To save:
group_obj = Group()
group_obj.switch_collection('group2')
group_obj.save()
Although Prachetos Sadhukhan answer works for me, I prefer to get the collection directly, not relying on private _get_collection method:
from mongoengine import connection
new_group_collection = connection.get_db()['group1']
from mongoengine.queryset import QuerySet
new_objects = QuerySet(Group, new_group_collection)
Related
I'm trying to work a bit with pymongo, and I currently have a database that I need to look inside, and if the document with a specific field exists, then the document should be updated.
First I created a entry by running this a few times:
import pymongo
client = pymongo.MongoClient()
mydb = client["mydb"]
data = {'name': "john"}
mycol = mydb['something']
mycol.insert_one(data)
Which works the way I want it to.
Now, I need to check whether or not an entry exists where name = "john".
I followed this tutorial, which basically just shows this snippet:
db.student.find({name:{$exists:true}})
I tried to implement this, so it now looks like this:
import pymongo
from pymongo import cursor
client = pymongo.MongoClient()
mydb = client["mydb"]
print(mydb.something.find({"name":{"john"}}))
and this just returns <pymongo.cursor.Cursor object at 0x7fbf266239a0>
which I don't really know what to do with.
I also looked at some similar questions here, and found some suggestions for something like this:
print(mydb.values.find({"name" : "john"}).limit(1).explain())
But this just gives me a long json-looking string, which by the way doesnt change if I put other things in for "john".
So how do I check whether a document where "name" = "john" exists? and perhaps also then edit the document?
EDIT
I now tried the following solution:
import pymongo
from pymongo import cursor
client = pymongo.MongoClient()
mydb = client["mydb"]
mycol = mydb['something']
name = "john"
print(mycol.find_one({name:{"$exists":True}}))
But it only prints me None
Change find() to find_one(), or if you're expecting more than one result, iterate the cursor using a for loop:
print(db.student.find_one({'name':{'$exists': True}}))
or
for student in db.student.find({'name': {'$exists': True}}):
print(student)
I have Django app which creates collections in MongoDB automatically. But when I tried to integrate the delete functionality, collections that are created with delete functionality are not deleted. Collections that are automatically created are edited successfully. This method is called in another file, with all parameters.
An interesting thing to note is when I manually tried to delete via python shell it worked. I won't be deleting the collections which are not required anymore.
import pymongo
from .databaseconnection import retrndb #credentials from another file all admin rights are given
mydb = retrndb()
class Delete():
def DeleteData(postid,name):
PostID = postid
tbl = name + 'Database'
liketbl = PostID + 'Likes'
likecol = mydb[liketbl]
pcol = mydb[tbl]
col = mydb['fpost']
post = {"post_id":PostID}
ppost = {"PostID":PostID}
result1 = mydb.commentcol.drop() #this doesn't work
result2 = mydb.likecol.drop() #this doesn't work
print(result1,'\n',result2) #returns none for both
try:
col.delete_one(post) #this works
pcol.delete_one(ppost) #this works
return False
except Exception as e:
return e
Any solutions, I have been trying to solve this thing for a week.
Should I change the database engine as Django doesn't support NoSQL natively. Although I have written whole custom scripts that do CRUD using pymongo.
I have two tables Document and Picture. The relationship is that one document can have several pictures. What should happen is that once a document is uploaded to the PostgreSQL, the document should be downloaded and transformed into a jpeg, and then uploaded to the Picture table.
I'm using sqlalchemy and flask in my application.
So far I tried using events to trigger a method after insert. Unfortunately, I am receiving the error sqlalchemy.exc.ResourceClosedError: This transaction is closed when I commit.
The code:
from app.models.ex_model import Document, Picture
from pdf2image import convert_from_bytes
from sqlalchemy import event
import io
import ipdb
from app.core.app_setup import db
#event.listens_for(Document, 'after_insert')
def receive_after_insert(mapper, connection, target):
document = target.document
images = convert_from_bytes(document, fmt='jpeg')
images_bytes = map(lambda img: convert_to_byte(img), images)
map(lambda img_byte: upload_picture(img_byte, target.id, ), images_bytes)
db.session.commit()
def convert_img_to_byte(img):
img_byte = io.BytesIO()
img.save(img_byte, format='jpeg')
img_byte = img_byte.getvalue()
return img_byte
def upload_picture(img_byte, document_id):
picture = Picture(picture=img_byte, document_id=document_id)
db.session.add(picture)
As Session.add method states:
Its state will be persisted to the database on the next flush
operation.Repeated calls to add() will be ignored.
So your add calls should be followed by session.flush() call.
...
def upload_picture(img_byte, document_id):
picture = Picture(picture=img_byte, document_id=document_id)
db.session.add(picture)
db.session.flush()
Furthermore I would pay attention to performance of inserting records. There's a good article for this point in official docs: https://docs.sqlalchemy.org/en/13/faq/performance.html#i-m-inserting-400-000-rows-with-the-orm-and-it-s-really-slow
As such the current approach is not the fastest one, therefore I would choose either sqlalchemy_orm_bulk_insert or sqlalchemy_core approach.
Inside the expenses collection I have this Json:
{
"_id" : ObjectId("5ad0870d2602ff20497b71b8"),
"Hotel" : {}
}
I want to insert a document or another object if possible inside Hotel using Python.
My Python code:
from pymongo import MongoClient
client = MongoClient('localhost', 27017)
db = client['db']
collection_expenses = db ['expenses']
#insert
d = int(input('Insert how many days did you stay?: '))
founded_expenses = collection_expenses.insert_one({'days':d})
The code above inserts the document inside the collection. What should I change to add the days inside de Hotel object?
Thanks in advance.
Instead of using insert_one, you may want to take a look to the save method, which is a little bit more permissive.
Admitting your document is already created in the collection:
[...]
expenses = db['expenses']
# Find your document
expense = expense.find_one({})
expense["Hotel"] = { "days": d }
# This will either update or save as a new document the expense dict,
# depending on whether or not it already has an _id parameter
expenses.save(expense)
Knowing that find_one will return you None if no such document exist, you may want to upsert a document. You can thus easily do so with save.
I am trying to use endpoints to update some JSON values in my datastore. I have the following Datastore in GAE...
class UsersList(ndb.Model):
UserID = ndb.StringProperty(required=True)
ArticlesRead = ndb.JsonProperty()
ArticlesPush = ndb.JsonProperty()
In general what I am trying to do with the API is have the method take in a UserID and a list of articles read (with an article being represented by a dictionary holding an ID and a boolean field saying whether or not the user liked the article). My messages (centered on this logic) are the following...
class UserID(messages.Message):
id = messages.StringField(1, required=True)
class Articles(messages.Message):
id = messages.StringField(1, required=True)
userLiked = messages.BooleanField(2, required=True)
class UserIDAndArticles(messages.Message):
id = messages.StringField(1, required=True)
items = messages.MessageField(Articles, 2, repeated=True)
class ArticleList(messages.Message):
items = messages.MessageField(Articles, 1, repeated=True)
And my API/Endpoint method that is trying to do this update is the following...
#endpoints.method(UserIDAndArticles, ArticleList,
name='user.update',
path='update',
http_method='GET')
def get_update(self, request):
userID = request.id
articleList = request.items
queryResult = UsersList.query(UsersList.UserID == userID)
currentList = []
#This query always returns only one result back, and this for loop is the only way
# I could figure out how to access the query results.
for thing in queryResult:
currentList = json.loads(thing.ArticlesRead)
for item in articleList:
currentList.append(item)
for blah in queryResult:
blah.ArticlesRead = json.dumps(currentList)
blah.put()
for thisThing in queryResult:
pushList = json.loads(thisThing.ArticlesPush)
return ArticleList(items = pushList)
I am having two problems with this code. The first is that I can't seem to figure out (using the localhost Google APIs Explorer) how to send a list of articles to the endpoints method using my UserIDAndArticles class. Is it possible to have a messages.MessageField() as an input to an endpoint method?
The other problem is that I am getting an error on the 'blah.ArticlesRead = json.dumps(currentList)' line. When I try to run this method with some random inputs, I get the following error...
TypeError: <Articles
id: u'hi'
userLiked: False> is not JSON serializable
I know that I have to make my own JSON encoder to get around this, but I'm not sure what the format of the incoming request.items is like and how I should encode it.
I am new to GAE and endpoints (as well as this kind of server side programming in general), so please bear with me. And thanks so much in advance for the help.
A couple things:
http_method should definitely be POST, or better yet PATCH because you're not overwriting all existing values but only modifying a list, i.e. patching.
you don't need json.loads and json.dumps, NDB does it automatically for you.
you're mixing Endpoints messages and NDB model properties.
Here's the method body I came up with:
# get UsersList entity and raise an exception if none found.
uid = request.id
userlist = UsersList.query(UsersList.UserID == uid).get()
if userlist is None:
raise endpoints.NotFoundException('List for user ID %s not found' % uid)
# update user's read articles list, which is actually a dict.
for item in request.items:
userslist.ArticlesRead[item.id] = item.userLiked
userslist.put()
# assuming userlist.ArticlesPush is actually a list of article IDs.
pushItems = [Article(id=id) for id in userlist.ArticlesPush]
return ArticleList(items=pushItems)
Also, you should probably wrap this method in a transaction.