Ok, I,ve built my first API example using Django. I had this CSV file that I inserted into my database; then I creates my basic API system. Now I can output a JSON with 3 fields: title, image, description (as they were in the CSV). Perfect.
Now my question is pretty generic but conceptually relevant:
The image field is populated with urls but some of them are not correct. I mean not all of them are pointing to jpg, png or any other image file. I want my api system to output an error when accessing a resource that contains a bad url in the image field.
How can I handle this (and eventual other wrong fields)?
Thank you in advance!
Without seeing how you're using the images, it's hard to say how to handle non-existing files. If you want to throw an exception before the image is used, you can check to see if it exists, and throw one if it doesn't like so:
import os
path_to_file = 'my_image.jpg'
if not (os.path.exists(path_to_file) and os.path.isfile(path_to_file)):
# This needs to be OSError (I think) before Python 3.3 (I think)
raise FileExistsError('The file {} does not exist.'.format(path_to_file))
# do stuff with file if it exists
If you make your question more specific I can make this answer more specific.
Related
I'm trying to download a spreadsheet from Google Drive inside a program I'm writing (so the data can be easily updated across all users), but I've run into a few problems:
First, and perhaps foolishly, I'm only wanting to use the basic python distribution, so I'm not requiring people to download multiple modules to run it. The urllib.request module seems to work well enough for basic downloading, specifically the urlopen() function, when I've tested it on normal webpages (more on why I say "normal" below).
Second, most questions and answers on here deal with retrieving a .csv from the spreadsheet. While this might work even better than trying to parse the feeds (and I have actually gotten it to work), using only the basic address means only the first sheet is downloaded, and I need to add a non-obvious gid to get the others. I want to have the program independent of the spreadsheet, so I only have to add new data online and the clients are automatically updated; trying to find a gid programmatically gives me trouble because:
Third, I can't actually get the feeds (interface described here) to be downloaded correctly. That does seem to be the best way to get what I want—download the overview of the entire spreadsheet, and from there obtain the addresses to each sheet—but if I try to send that through urlopen(feed).read() it just returns b''. While I'm not exactly sure what the problem is, I'd guess that the webpage is empty very briefly when it's first loaded, and that's what urlopen() thinks it should be returning. I've included what little code I'm using below, and was hoping someone had a way of working around this. Thanks!
import urllib.request as url
key = '1Eamsi8_3T_a0OfL926OdtJwLoWFrGjl1S2GiUAn75lU'
gid = '1193707515'
# Single sheet in CSV format
# feed = 'https://docs.google.com/spreadsheets/d/' + key + '/export?format=csv&gid=' + gid
# Document feed
feed = 'https://spreadsheets.google.com/feeds/worksheets/' + key + '/private/full'
csv = url.urlopen(feed).read()
(I don't actually mind publishing the key/gid, because I am planning on releasing this if I ever finish it.)
Requres OAuth2 or a password.
If you log out of google and try again with your browser, it fails (It failed when I did logged out). It looks like it requires a google account.
I did have it working with and application password a while ago. But I now use OAuth2. Both are quite a bit of messing about compared to CSV.
This sounds like a perfect use case for a wrapper library i once wrote. Let me know if you find it useful.
I'm looking to store some individual settings to each user's computer. Things like preferences and a license key. From what I know, saving to the registry could be one possibility. However, that won't work on Mac.
One of the easy but not so proper techniques are just saving it to a settings.txt file and reading that on load.
Is there a proper way to save this kind of data? I'm hoping to use my wx app on Windows and Mac.
There is no proper way. Use whatever works best for your particular scenario. Some common ways for storing user data include:
Text files (e.g. Windows INI, cfg files)
binary files (sometimes compressed)
Windows registry
system environment variables
online profiles
There's nothing wrong with using text files. A lot of proper applications uses them exactly for the reason that they are easy to implement, and additionally human readable. The only thing you need to worry about is to make sure you have some form of error handling in place, in case the user decides to replace you config file content with some rubbish.
Take a look at Data Persistence on python docs. One option a you said could be persist them to a simple text file. Or you can save your data using some serialization format as pickle (see previous link) or json but it will be pretty ineficient if you have several keys and values or it will be too complex.
Also, you could save user preferences in an .ini file using python's ConfigParser module as show in this SO answer.
Finally, you can use a database like sqlite3 which is simpler to handle from your code in order to save and retrieve preferences.
I am looking for the method Django uses to generate unique filename when we upload a file.
For example, if I upload a file called test.csv twice in the same directory, the first one will be saved as test.csv and the second file will be saved as test_2.csv. I already tried to find how Django manages that, but I only found django.utils.text.get_valid_filename which could be useful, but that not what I am looking for...
I already saw other topics with random naming solution, that not what I am looking for here :) I really trying to understand how Django manage that problem.
I actually take a closer look with your help and a found something :)
So basically I have to do something like:
from django.core.files.storage import FileSystemStorage
fss = FileSystemStorage()
filepath = fss.get_available_name(filepath)
Thank you all :)
PS: If you are interesting, the comment from django.core.file.storage.FileSystemStorage._save says:
There's a potential race condition between get_available_name and
saving the file; it's possible that two threads might return the
same name, at which point all sorts of fun happens. So we need to
try to create the file, but if it already exists we have to go back
to get_available_name() and try again.
If you see the implementation of class django.core.files.storage.Storage you will know how Django 1.6 manages the file names.
Look into the save method of this class. In this the line
name = self.get_available_name(name)
is doing the trick.
This is the default implementation of getting the new file name before saving the file. If you want to write your own version (like the file should be overridden) then consider writing your own custom storage system
Actually, you were on the right track.
From the docs,
Internally, Django uses a django.core.files.File instance any time it
needs to represent a file.
And also,
Behind the scenes, Django delegates decisions about how and where to
store files to a file storage system
Which means that, when the file is uploaded, using the default storage (FileSystemStorage), Django delegates the naming (or the available name), behind the scene, for the file to the storage, which then uses: get_available_name(name).
So, If you want to change the way files are named when uploaded, you need to add a custom file storage, which would basically only override get_available_name. The documentation on the matter is here.
Hello I'm having problems opening a image file in web2py. I don't know if I'm making just a simple mistake or that it'd more than that. I thought I was doing it right looking at examples from the web and on this site. Heres my code in the model:
db.define_table('image',
Field('picture', 'upload'))
I think the error means that there is a picture in the database but it can't retrieve that image and open it or I be completely wrong please if anyone can spread some light on the solution i will be very thankful
The parameter passed to Image.open, db.image.picture, is a Field object and not the value that was filled into the form. You probably want:
pic = Image.open(os.path.join(request.folder,'uploads',form.vars.picture))
Edit: Updated to use full path to the picture file. However, as pointed out by Anthony in comments below, this will not work with a blob field that was used to store the image in the original question (before it was edited as discussed in comments below). The use of retrieve as described here will provide the file data.
So, i'm trying to create an google app engine (python) app that allows people to share files. I have file uploads working well, but my concern is about checking the file extension and making sure, primarily, that the files are read only, and secondly, that they are of the filetype that is specified. These will not be image files, as a know they are a lot of image resources already. Specifically, .stl mesh files, but i'd like to be able to do this more generally.
I know there are modules that can do this, python-magic seems to be able to do this for example, but i can't seem to find any that i'm able to import without LoadModuleRestricted. I'm considering writing my own parser, but that would be a lot of work for such a common (i'm assuming) issue.
Anyway, i'm totally stumped so this is my first stackoverflow question, so hope i'm doing well etiquette wise. Let me know, and thanks!
It sounds like you want to read the first few bytes of the uploaded file to verify that its signature matches the purported mime type. Assuming that you're uploading to blobstore (i.e., via a url obtained from blobstore.get_upload_url(), then once you're redirected to the upload handler whose path you gave to get_upload_url, you can open blob using a BlobReader, then read and verify the signature.
The Blobstore sample app lays out the framework. You'd glue in code in UploadHandler once you have blob_info (using blob_info.key() to open the blob).