I want to make an own static file view that returns the file defined in the GET request. The file must be in an extra directory. The URL must be like /e?s=NAME_OF_FILE. My problem is, hackers can use this like /e?s=/PATH/TO/DATABASE to get any file from the server. I have already a workaround, but i think there are better solutions.
My code:
path = os.path.abspath(os.path.join(script_path, filename))
if path.startswith(script_path):
# Good
else:
# Bad
This is for "hidden static files", that should not be handled by the server.
What you are doing is of not much help. Some things you could do -
In the webserver turn off directory listing so that the 'hacker' does not get the list of all files in that directory.
Instead of exposing the actual filename to outside world you could take the filename, generate a MD5 has of this filename, store this mapping somewhere in your servers and expose this MD5 as the filename. So it becomes /e?s=MD5_HASH_OF_FILENAME. What this does is make it extremely difficult for the 'hacker' to 'guess' the filename. Brute-force does not help as MD5 are not easy to guess. So in effect, only people who have been some how sent this URL will have access to it.
You can expose this static file viewing API to only authenticated users rather than make it public API. You can use #login_required decorator.
Finally, enable HTTPS on your webserver.
Related
I am very new to twisted. I am serving a directory using twisted library from python. But i dont want to serve every file in that directory. I want to exclude json files or specific files from being served to the web. How can i do this?
The reason i need this is i don't want my software to be reverse engineered and my important files that i dont want to share in that directory to be accessed.
I searched through the internet but could not find something that accomplishes what i want.
My Code:
resource = File(Globals.server_config.FOLDER_DIRECTORY)
factory = Site(resource)
endpoint = endpoints.TCP4ServerEndpoint(reactor, port=Globals.server_config.WEBSERVER_PORT, interface=Globals.server_config.WEBSERVER_IP)
endpoint.listen(factory)
reactor.run(installSignalHandlers=False)
How can i modify this code to exclude certain specific files (like asd.txt) or file types (like json)?
I'm trying to set up a cloud function that performs user authentication. I need it to open an object stored in a Cloud Storage bucket, read its content and verify if username and password match those coming from the HTTP request.
In some cases the function needs to add a user: it should retrieve the content of the .json file stored in the bucket, add a username:password pair and save the content in the same object.
Basically, it has to modify the content of the object.
I can't find the way o do it using the Cloud Storage Python client library. None of the tutorials listed in the GitHub pages mentions anything like "modify a file" or similar concepts (at least in their short descriptions).
I also looked for a method to perform this operation in the Blob class source code, but I couldn't find it.
Am I missing something? This looks to me as a very common operation, one that should have a very straightforward method, like blob.modify(new_content).
I have to confess that I am completely new to GCP, so there is probably an obvious reason behind this (or maybe I just missed it).
Thank you in advance!
Cloud Storage is a blob storage and you can only read, write and delete the object. You can't update the content (only the metadata) and can't move/rename a file (move and rename operation perform a copy (create a new object) followed by a delete (of the old object)).
In addition, the directories don't exist, all the file are put at the root level of the bucket. The file name contains the path from the root to the leaf. The / is only a human representation for the folders (and the UI use that representation), but the directories are only virtual.
Finally, you can't search on a file suffix, only per prefix of the file name (including the full path from the root path /)
In summary, it's not a file system, it's a blob storage. Change your design or your file storage option.
I've built an application following the file upload process (more or less) along the lines of the Flask file upload documentation found here, https://flask.palletsprojects.com/en/1.1.x/patterns/fileuploads/.
In this portion of the code, UPLOAD_FOLDER = '/path/to/the/uploads', this points to one, single directory where file uploads will live. The problem I'm trying to solve is when I deploy my app to a server there will be multiple, simultaneous users. With a single upload directory, users will collide when they upload files with the same names--a situation that will occur in my app.
What I want to do is create a unique temp directory that is unique to each browser session. So, user 1 would have their own unique temp directory and user 2 would have their own unique temp directory and so on.
In this case, I think there would not be any user collision. Can anyone please suggest how I would create such unique temp directories associated with each browser session in the file upload process? Something along the lines of UPLOAD_FOLDER = '/path/to/the/uploads/user1_session', etc for each unique user?
Ok, so lacking further information and any sort of view on what your code/program looks like this is the what I would recommend at the moment.
I am relatively new to programming as well so this might not be the best answer. But in my experience you really,really do not want to be creating multiple directories per user/per session. That is a bad idea. This is where databases comes in handy.
Now in regards to your problem the easiest/fastest way to resolve this issue is to look into how password salt and hashing is done.
Just hash and salt your filenames.
Here is a link that provides a simple yet through explanation on how it is done.
I am trying to serve up some user uploaded files with Flask, and have an odd problem, or at least one that I couldn't turn up any solutions for by searching. I need the files to retain their original filenames after being uploaded, so they will have the same name when the user downloads them. Originally I did not want to deal with databases at all, and solved the problem of filename conflicts by storing each file in a randomly named folder, and just pointing to that location for the download. However, stuff came up later that required me to use a database to store some info about the files, but I still kept my old method of handling filename conflicts. I have a model for my files now and storing the name would be as simple as just adding another field, so that shouldn't be a big problem. I decided, pretty foolishly after I had written the implmentation, on using Amazon S3 to store the files. Apparently S3 does not deal with folders in the way a traditional filesystem does, and I do not want to deal with the surely convoluted task of figuring out how to create folders programatically on S3, and in retrospect, this was a stupid way of dealing with this problem in the first place, when stuff like SQLalchemy exists that makes databases easy as pie. Anyway, I need a way to store multiple files with the same name on s3, without using folders. I thought of just renaming the files with a random UUID after they are uploaded, and then when they are downloaded (the user visits a page and presses a download button so I need not have the filename in the URL), telling the browser to save the file as its original name retrieved from the database. Is there a way to implement this in Python w/Flask? When it is deployed I am planning on having the web server handle the serving of files, will it be possible to do something like this with the server? Or is there a smarter solution?
I'm stupid. Right in the Flask API docs it says you can include the parameter attachment_filename in send_from_directory if it differs from the filename in the filesystem.
So I am trying to port a Python webapp written with Flask to Google App Engine. The app hosts user uploaded files up to 200mb in size, and for non-image files the original name of the file needs to be retained. To prevent filename conflicts, e.g. two people uploading stuff.zip, each containing completely different and unrelated contents, the app creates a UUID folder on the filesystem and stores the file within that, and serves them to users. Google App Engine's Cloud Storage, which I was planning on using to store the user files, by making a bucket - according to their documentation has "no notion of folders". What is the best way to go about getting this same functionality with their system?
The current method, just for demonstration:
# generates a new folder with a shortened UUID name to save files
# other than images to avoid filename conflicts
else:
# if there is a better way of doing this i'm not clever enough
# to figure it out
new_folder_name = shortuuid.uuid()[:9]
os.mkdir(
os.path.join(app.config['FILE_FOLDER'], new_folder_name))
file.save(
os.path.join(os.path.join(app.config['FILE_FOLDER'], new_folder_name), filename))
new_folder_path = os.path.join(
app.config['FILE_FOLDER'], new_folder_name)
return url_for('uploaded_file', new_folder_name=new_folder_name)
From the Google Cloud Storage Client Library Overview documentation:
GCS and "subdirectories"
Google Cloud Storage documentation refers to "subdirectories" and the GCS client library allows you to supply subdirectory delimiters when you create an object. However, GCS does not actually store the objects into any real subdirectory. Instead, the subdirectories are simply part of the object filename. For example, if I have a bucket my_bucket and store the file somewhere/over/the/rainbow.mp3, the file rainbow.mp3 is not really stored in the subdirectory somewhere/over/the/. It is actually a file named somewhere/over/the/rainbow.mp3. Understanding this is important for using listbucket filtering.
While Cloud Storage does not support subdirectories per se, it allows you to use subdirectory delimiters inside filenames. This basically means that the path to your file will still look exactly as if it was inside a subdirectory, even though it is not. This apparently should concern you only when you're iterating over the entire contents of the bucket.
From the Request URIs documentation:
URIs for Standard Requests
For most operations you can use either of the following URLs to access objects:
storage.googleapis.com/<bucket>/<object>
<bucket>.storage.googleapis.com/<object>
This means that the public URL for their example would be http://storage.googleapis.com/my_bucket/somewhere/over/the/rainbow.mp3. Their service would interpret this as bucket=my_bucket and object=somewhere/over/the/rainbow.mp3 (i.e. no notion of subdirectories, just an object name with embedded slashes in it); the browser however will just see the path /my_bucket/somewhere/over/the/rainbow.mp3 and will interpret it as if the filename is rainbow.mp3.