I am trying to overwrite an image in my Cloud Storage over the Python API, but after I overwrite it and refresh (and delete browsercache) the Cloud Webpage or the public link the image is still the same, even the next day but sometimes it gets randomly updated to the new image!
Edit: The Metadata get updated, but not the filesize-info and it still shows the old image in the Cloud-Webpage and at the public url.
What I am expecting is that if I am uploading a file to Cloud Storage via a API that I can download the new file from the public link a short time afterwards instead of the old image.
I expected to be able to define the cache behaviour with the Cache-Control File-directive (Edit: it is propably not an issue about caching because even the next day the image stays the old one).
This is my code:
blob = bucket.blob(name)
blob.cache_control = "no-store"
blob.upload_from_filename(name)
I tried:
Deleting the old image over the Cloud-Webpage and then after a few
seconds upload the new image with the same name via Python: It works!
I can download the new image from the public link and see it in the
Cloud-Webpage. Edit: It seems to work only some times!
Deleting the Image with Python and directly afterwards upload the new
image via Python: Not working. While it is deleted the public link
doesnt show it. But after I uploaded the new one the public link
shows the old one again.
I read that the standard cache settings of public bucket files is
"public, max-age=3600". So I used the Cache-Control Directive and set
it to "no-store" or "public, age=0". Then I confirmed these
Cache-Control settings are reflected in the headers in the browser
debug console. But still the old image is loading anytime.
I changed the bucket type to regional instead of multi-region. Even after deleting the bucket, recreating it and moving the data inside it again the old image is still showing up!
Any tip is highly appreciated!
I made it work!
It was propably not related to Google Cloud Storage.
But if someone might did the same mistake as I:
I used Django's FilesSystemStorage-Class and saved the new file with the same name as the old one in the /temp directory, assuming that the old one will be overriden if it still exists. But instead it gives the new file another name. And later I upload the old file with blob.upload_from_filename(name)
Thats why all the things happend so randomly.
Thanks to all who thought about solving this!
Related
In the google appengine datastore, there is a BlobKey (labled as csv). The key is in the following format: encoded_gs_file:we1o5o7klkllfekomvcndhs345uh5pl31l. I would like to provide a download button to save this information.
The web app is being run using dev_appserver.py and uses python 2.7 (Django) as the backend. Currently, a button exists, but when clicking on it, it returns a 404 error. The download link that the button provides is:
https://localhost:8080/data?key=encoded_gs_file:dwndjndwamwljioihkm
The interesting thing is that, when i go to localhost:8000/datastore and then navigate to the Blobstore Viewer, i can navigate to Entity Kind: GsFileInfo, and there is a key generated. The problem is that it gives a 404.
The other blob item next to is a png and if i use the key for that png, i get a download. What is wrong with the key, how can i get the write key from the datastore.
I'm still scratching my head in regards to your issue, but something that seems suspect is that size is None and finalized is False for your CSV file.
I looked back at your previous question (google cloud: downloading as an CSV) and noticed that you're not calling .close() on your GCS/blobstore file after you finish writing to it.
Maybe that will fix this.
I'm developing my Django website since about 2 months and I begin to get a good global result with my own functions.
But, now I have to start a very hard part (to my mind) and I need some advices, ideas before to do that.
My Django website creates some PDF files from HTML templates with Django variables. Up to now, I'm saving PDF files directly on my Desktop (in a specific folder) but it's completely unsecured.
So, I installed another web application which is named LogicalDoc in order to save PDF file directly on this application. PDF files are created and sent to LogicalDoc.
LogicalDoc owns 2 API : SOAP and REST (http://wiki.logicaldoc.com/rest/#/) and I know that Django could communicate with REST method.
I'm reading this part of Django documentation too in order to understand How I can process : https://docs.djangoproject.com/en/dev/topics/http/file-uploads/
I made a scheme in order to understand what I'm exposing :
Then, I write a script which makes some things :
When the PDF file is created, I create a folder inside LogicalDoc which takes for example the following name : lastname_firstname_birthday
Two possibilities : If the folder exists,I don't create a new folder, else I create it.
Once it's done, I send the PDF file directly inside the folder by comparing PDF name with folder name to do that
I have some questions about this process :
Firstly, is it possible to make this kind of things ?
Is it hard to do that ?
What kind of advices could you give me ?
Thank you so much !
PS : If you need some part of my script, mainly PDF creating part, I can post it just after my question ;)
An idea is pretty simple, however it always requires some practice.
I strongly advice you to use REST api and forget about SOAP as the only thing it can bring to you - is 'pain' :)
If we check documentation, document/create it gives next information.
Endpoint we have to communicate with.
[protocol]://[server]:[port]/document/create
HTTP method to use - POST
List of parameters to provide with your request: body,
document, content
Even more, you can test API by clicking on "Try it out" button and check requests in "Network" tab of your browser (if you open Developer Tools)
I am not sure what kind of metadata do you have to provide in 'document' parameter but what I know you can easy get an idea of what should be done by testing it and putting XML or JSON data into 'document' parameter.
Content is an array of bytes transferred to the server (which would be your file).
To sum up, a request to 'document/create' uri will be simple
body = { 'headers': {},'object': {},}
document = "<note>data</note>"
content=open('report.xls', 'rb') #r - reading, b - binary
r = requests.post('http://logicaldoc/document/create', body=body, document=document, content=content)
Please keep in mind that file transferring requests take time and sometimes you may get timeout exception. Your code will stop and will be waiting for response, so it may be a good idea to get some practice with asyncio or celery. Just keep in mind those kind of possible issues.
I'd like to use the Dropbox API (with access only to my own account) to generate a link to SomeFile.xlsx that I can put in an email to multiple Dropbox account holders, all of whom are presumed to have access to the file. I'd like for the same link, when clicked on, to talk to Dropbox to figure out where SomeFile.xlsx is on their local filesystem and open that up directly.
In other words, I do NOT want to link to the cloud copy of the file. I want to link to the clicker's locally-synced version of the file.
Does Dropbox have that service and does the API let me consume it? I haven't been able to discover the answer from the documentation yet.
No, Dropbox doesn't have an API like this.
In my GAE Python app, I'm writing code to store images in GCS.
I write the images as follows:
bucket_name = os.environ.get(u'BUCKET_NAME', app_identity.get_default_gcs_bucket_name())
filename = u'/{}/{}/image'.format(bucket_name, str(object.key.id()))
mimetype = self.request.POST[u'image_file'].type
gcs_file = cloudstorage.open(filename, 'w', content_type=mimetype,
options={'x-goog-acl': 'public-read'})
gcs_file.write(self.request.get(u'image_file'))
gcs_file.close()
The first time I use this code to write a particular filename, I can access that file with its filename:
https://storage.googleapis.com/<app>.appspot.com/<id>/image
And I can also click the name "image" on the GCS Storage Browser and see the image.
Yay! It all seems to work.
But when I upload a different image to the same filename, something confusing happens: when I display the filename in the browser, either via an <img> tag or as the URL in a separate browser tab, the old image appears. Yet when I display "image" via the GCS Storage Browser, it shows the new image.
By the way, as an additional data point, although I specify public-read when I open the file for writing, the "shared publicly" column is blank for that file on the GCS Storage Browser page.
I tried deleting the file before the open statement, even though w is supposed to act as an overwrite, but it didn't make any difference.
Can anyone explain how the filename continues to access the old version of the file, even though the GCS Storage Browser shows the new version, and more importantly, what I need to do to make the filename access the new version?
EDIT:
Continuing to research this problem, I found the following statement at https://cloud.google.com/storage/docs/accesscontrol:
If you need to ensure that updates become visible immediately, you should set
a Cache-Control header of "Cache-Control:private, max-age=0, no-transform" on
such objects.
However, I can't see how to do this using Cloudstorage "open" command or in any other way from my Python program. So if this is the solution, can someone tell me how to set the Cache-Control header for these image files I'm creating?
Here is an example open setting cache control:
with gcs.open(new_zf.gcs_filename, 'w', content_type=b'multipart/x-zip',
options={b'x-goog-acl': b'public-read', b'cache-control': b'private, max-age=0, no-cache'}) as nzf:
taken from this respository
I created new directory via 'myimages' under static directory. And whenever user upload files, file get stores in that directory. Once file uploaded I can access that on html using '/static/myimages/imagename.png'. It works fine whenever I am uploading new image. But whenver I try to override image. It shows me old image only.
I manually check in directory. It has new image, but still when I try to access that via browser, it gives me old image.
This sounds like your browser is caching the images. Try setting the cache_max_age=0 on your add_static_view, or viewing the page in your browser's incognito mode where it'll use a different cache.