I'm using the Evernote API for Python to create an app that allows the user to create and update notes, but I'm having trouble understanding how to efficiently update Evernote resources. This mainly occurs when I'm converting from HTML to ENML (Evernote Markup Language), where I'm creating resources from img tags (right now I'm only considering image resources).
My question is this: how can I tell, given HTML, if a note's resources needs to be updated? I've considered comparing the image data to all of the current resources' data, but that seems really slow. Right now I just make a new resource for each img tag.
Some helpful resources I've found include the Evernote resources guide and this sample code in the Evernote SDK. Any advice is appreciated.
The best way would be a comparison of the MD5 hash of the file. Evernote notes track resources by their MD5 hash.
To see the MD5 hash of the file attached to an Evernote note, just look at the ENML elements labeled "en-media", the form of the tags can be seen below:
<en-media type="mime-type" hash="md5-of-file" />
Where mime-type is the file type and md5-of-file is the MD5 hash of the file. To get the ENML of a note call getNote (documentation here) and make sure to specify you want the contents. The ENML contents of the note is the value of the content attribute of the object that is returned by getNote (a note object).
While hashes can be expensive MD5 is relatively quick and it will be quicker to compute the MD5 hash of a file than it will be to wait for the network to download images.
Also, the updateResource method documentation says:
"Submit a set of changes to a resource to the service. This can be
used to update the meta-data about the resource, but cannot be used to
change the binary contents of the resource (including the length and
hash). These cannot be changed directly without creating a new
resource and removing the old one via updateNote."
So the only way to "update" a resource is to remove the old resource from the note and create a new one in its place. You can do this by removing the resource by remove the Resource Object from the list contained in the resources attribute of the note in question. To add a new note simple add a new resource object to the same list.
Related
I'm trying to set up a cloud function that performs user authentication. I need it to open an object stored in a Cloud Storage bucket, read its content and verify if username and password match those coming from the HTTP request.
In some cases the function needs to add a user: it should retrieve the content of the .json file stored in the bucket, add a username:password pair and save the content in the same object.
Basically, it has to modify the content of the object.
I can't find the way o do it using the Cloud Storage Python client library. None of the tutorials listed in the GitHub pages mentions anything like "modify a file" or similar concepts (at least in their short descriptions).
I also looked for a method to perform this operation in the Blob class source code, but I couldn't find it.
Am I missing something? This looks to me as a very common operation, one that should have a very straightforward method, like blob.modify(new_content).
I have to confess that I am completely new to GCP, so there is probably an obvious reason behind this (or maybe I just missed it).
Thank you in advance!
Cloud Storage is a blob storage and you can only read, write and delete the object. You can't update the content (only the metadata) and can't move/rename a file (move and rename operation perform a copy (create a new object) followed by a delete (of the old object)).
In addition, the directories don't exist, all the file are put at the root level of the bucket. The file name contains the path from the root to the leaf. The / is only a human representation for the folders (and the UI use that representation), but the directories are only virtual.
Finally, you can't search on a file suffix, only per prefix of the file name (including the full path from the root path /)
In summary, it's not a file system, it's a blob storage. Change your design or your file storage option.
I am working on a project that scan qrcodes on a product e.g Cup, and verify whether the product is the database or not, I have built the scanning app using Flutter, now my problem is creating QRCODES each having a unique identity or a specified URL that is linked to the database.
I have tried to search on the internet how I can do this but I am not finding useful answers any provided on this will be highly appreciated thanks in advance!
If you want to use Phyton, I can recommend this library for QR Code creation and rendering.
I'm not sure if you can connect Firebase with a Phyton app, it would probably easier to write a pure Angular/React/Vue web app.
Regarding the unique IDs. I'm not sure who is scanning the items. It might be a good idea to generate a URL QR Code, so if people scan the code with a different app, they get redirected to your website (maybe displaying help information how to scan the code).
For example: https://example.com/help/product-scan?pid=<the actual product id>
You can use any ID that can be stored in a database, e.g.:
A human-readable string identifier, e.g. octocat_cup
Numeric value, like an autogenerated key from a SQL database
UUID v4 (great for distributed systems and offline usage)
The question is simply as stated in the title. But the reason is probably important for a good answer.
I need to generate static html files at runtime. Generating the files is not the problem. Using jinja2 and webapp2 I can painlessly generate dynamic html files at runtime and serve them to my users. My problem is that generating these files is expensive. So I want to be able to save them as static html files; so that, when user tries to access them, I just serve the static html if it exists. If it does not exist, then I can create the file and then serve it.
Again I am already using jinja2 to create my string (i.e. file content). So my question is, how do I save the file to a path that my app.yaml file can map to?
Also if you are thinking memcache I am already using that. It is not sufficient storage. So while it’s a buffer, it’s not as good as having static html files.
Some back story:
If I can generate the files in localhost, that'd be even better. The thing is the site has a huge number of pages. But the structure is common to all the pages. So we store the relevant data in the datastore and use jinja2 to inflate the different pages. But because of heavy usage, memcache is not able to keep up. So now it's looking more cost-effective to create static html pages from the data stored in the datastore. Creating those pages by hand is obscene and error prone. So if we could generate the html pages as usual (i.e. jinja2 template and datastore) and then have an automated system for creating the html pages, it would be awesome. We could do the app.yaml part manually.
So after I do
template = jinja_environment.get_template(‘template.html')
content = template.render(template_values)
What do I do to save the file to say ./relevant/path/filename.html
IfI understand your question properly, the best way to do this would be to store your static files in Google Cloud Storage (GCS). Easy to read and write to GCS. And easy to delete the file when you no longer need it.
You can't write to files in App Engine -- see https://cloud.google.com/appengine/kb/java?csw=1#writefile for explanation.
Why don't you store the post-rendered page content back in datastore? I'm assuming the "expensive" part of the approach you're using now is the building of the templated content from the data. Once you've done that, instead of writing it off to local files, you could either put it as an additional property on the original data entity or you could create an entirely new model just for the rendered pages.
Essentially this would be a durable datastore implementation instead of memcache, if the issue is that pages are getting evicted too often from memcache.
A single fetch (directly from the key, since you can use keynames that you can create based on the page you're fetching, so you don't have to even do a query) is lightweight. Then, if you're still using memcache, then you'd check memcache first; if not there, you fetch the entity from the datastore and check to see if the rendered property contains data. If not, you use the data in the entity to render it with Jinja, store it back off in the rendered property, put the entity, store it off in memcache, and return the rendered content back to the client.
I'm using django-mongodb-engine for a project I'm working on and the documentation advises users to use the GridFS storage system for storing blobs as opposed to using the filesystem method. Obviously, this is one reason why we choose to use mongodb in the first place. One issue though, the documentation is sparse to say the least. In the docs they mention to use the GridFSField as your blob model field. One problem... where is the GridFSField?
class Better(models.Model):
blob = GridFSField()
iPython/Django shell:
from django_mongodb_engine.storage import GridFSStorage
#... define the class/exec
/usr/local/lib/python2.7/dist-packages/django/core/management/commands/shell.pyc in Better()
1 class Better(models.Model):
----> 2 blob = GridFSField()
3
NameError: name 'GridFSField' is not defined
Um, okay Django! Where is it defined then?!
This is not really a specific answer to your question as that answer is going to most likely be about your setup configuration. But as you seem to be going through documentation examples and therefore evaluating I thought it would be worthwhile to provide some points on using GridFS.
The intention of GridFS is not to be a way of storing "blobs" or replacing usage of the "filesystem method" as you (or the ODM documentation) are stating. Yes it can be used that way, but the sole reason for it's existence is to overcome the 16MB limitation MongoDB has on BSON document storage.
There is a common misconception that GridFS is a "feature" of MongoDB, yet it is actually a specification implemented on the driver side, for dealing with chunking large document content. There is no magic that occurs on the server side at all, as far as internal operations to MongoDB are concerned, this is just another BSON document with fields and data.
What the driver implementation is doing is breaking the content up into smaller chunks and distributing the content over several documents in a collection. Likewise when reading the content, there are methods provided to follow and fetch the various documents that make up the total content. In a nutshell, reading and writing using the GridFS methods results in multiple calls over the wire to MongoDB.
With that in mind, if your content is actually going to always be a size under 16MB then you are probably better off just using your encoded binary data within a single document, as updates will be atomic and the result will be faster reads from a single read operation per document.
So if you must have documents over 16MB in size, then use GridFS. If not just encode the content into a normal document field as that is all GridFS is doing anyway.
For more information, please read the FAQ:
http://docs.mongodb.org/manual/faq/developers/#when-should-i-use-gridfs
You can find GridFSField under django_mongodb_engine.fields
i.e.
from django_mongodb_engine.fields import GridFSField
from django.db import models
class Image(models.Model):
blob = GridFSField()
This seems so basic - I must be missing something.
I am trying to download my entities, update a few properties, and upload the entities. I'm using the Django nonrel & appengine projects, so all the entities are stored as id rather than name.
I can download the entities to csv fine, but when I upload (via appcfg.py upload_data ...), the keys come in as name=... rather than id=...
In the config file, I added -
import_transform: transform.create_foreign_key('auth_user', key_is_id=True)
to see if this would, as the documentation for transform states, "convert the key into an integer to be used as an id." With this import_transform, I get this error -
ErrorOnTransform: Numeric keys are not supported on input at this time.
Any ideas?
As the error message indicates, overwriting entities with numeric IDs isn't currently supported. You may be able to work around it by providing a post-upload function that recreates the entity with the relevant key, but I'd suggest stepping back and analyzing why you're doing this - why not just update the entities in-place on App Engine, or use remote_api to do this? Doing a bulk download and upload seems a cumbersome way to handle it.