The entire question is pretty much in the title.
The only documentation I can find on the class is the very sparing cgi documentation and it doesn't mention in the least how the class receives the file, how it's stored, what functions it supports etc.
I'm very interested in where the uploaded file is stored. Clearly it's not in memory, since Bottle mentions the FileStorage.read() function is dangerous on large files. If it's placed on the disk, I would like to move it to a permanent location without having to read through it in Python copy it bit by bit to a new location.
But I have no clue where to begin due to the poor documentation of the class. Any ideas?
A little late, but looking into this myself:
It all comes down to the 'make_file' method in cgi.py:
def make_file(self, binary=None):
import tempfile
return tempfile.TemporaryFile("w+b")
The tempfile docs ( http://docs.python.org/2/library/tempfile.html ) identify that the file is created in a default directory chosen from a platform-dependent list, but that the user can control the directory location by setting one of the environment variables: TMPDIR, TEMP or TMP.
Please also note from the documentation:
Under Unix, the directory entry for the file is removed immediately
after the file is created. Other platforms do not support this; your
code should not rely on a temporary file created using this function
having or not having a visible name in the file system.
Hope this helps.
I hope this could help:
http://epydoc.sourceforge.net/stdlib/cgi.FieldStorage-class.html
cgi.FieldStorage() has
a "filename",
a "value" - the file itself,
a "file" and
a "type"
And something else... you could read in the doc.
Here is my code:
f_sd=open(tempfilesd,'r+b')
newdata_sd = cgi.FieldStorage()
newdata_sd.filename='sdfile.jpg'
newdata_sd.name='file'
newdata_sd.file=f_sd
form.vars.file = newdata_sd
The FieldStorage.file attribute is actually not a file but a cStringIO object, which is described as a memory file on the docs: http://docs.python.org/library/stringio.html
Maybe this can help you a bit.
Related
I've served a directory using
python -m http.server
It works well, but it only shows file names. Is it possible to show created/modified dates and file size, like you see in ftp servers?
I looked through the documentation for the module but couldn't find anything related to it.
Thanks!
http.server is meant for dead-simple use cases, and to serve as sample code.1 That's why the docs link right to the source.
That means that, by design, it doesn't have a lot of configuration settings; instead, you configure it by reading the source and choosing what methods you want to override, then building a subclass that does that.
In this case, what you want to override is list_directory. You can see how the base-class version works, and write your own version that does other stuff—either use scandir instead of listdir, or just call stat on each file, and then work out how you want to cram the results into the custom-built HTML.
Since there's little point in doing this except as a learning exercise, I won't give you complete code, but here's a skeleton:
class StattyServer(http.server.HTTPServer):
def list_directory(self, path):
try:
dirents = os.scandir(path)
except OSError:
# blah blah blah
# etc. up to the end of the header-creating bit
for dirent in dirents:
fullname = dirent.path
displayname = linkname = dirent.name
st = dirent.stat()
# pull stuff out of st
# build a table row to append to r
1. Although really, it's sample code for an obsolete and clunky way of building servers, so maybe that should be "to serve as sample code to understand legacy code that you probably won't ever need to look at but just in case…".
Is there any way to read a macOS file alias, modify its contents (particularly the target file path), and write the modified alias back out?
For example, if I have the following directory structure:
./one/file.txt
./two/file.txt
./file_alias
where file_alias resolves to ./one/file.txt. I would like to be able to programmatically, in Python, read ./file_alias, determine its path, change 'one' to 'two', and write the revised alias out, overwriting ./file_alias. Upon completion, file_alias would resolve to ./two/file.txt.
Searching I've found an answer to a related question that suggests it can't be done (#Milliway's answer to [1]), a Carbon module with no substantive documentation and a statement that its functionality has been removed [2], a partially deprecated macostools module that depends on Carbon [3], an equivalent, unanswered question (except a tentative suggestion to use PyObjC) [4], and a recently updated mac_alias package [5], but have not found a way to accomplish the task based on any of these.
The mac_alias package at first seemed interesting, but I have found no way to import the bytes needed to construct an in-memory Alias object from an existing alias file (using bytes from a binary read of the alias file produces errors) and even if I could construct an in-memory Alias record and modify it, there is no way to write it out to disk.
The machine where I want this is running 10.12.x (Sierra) and I am using the built-in python 2.7.10. I find I can actually import Carbon and macostools, and suspect Carbon.File might conceivably provide what I need, but I cannot find any documentation for it. I could upgrade to High Sierra and/or install and use Python 3.x, but those don't seem to be helpful or relevant at this stage.
I realize that the alias also contains an inode, that will be stale after such a change, but thankfully, in part due to a bug I filed and a bit of persistence back when I was with Apple, an alias resolves the path first, only falls back to the inode if the path fails to resolve, and updates the inode if the path does resolve (and the inode has changed).
Any help, suggestions, pointers appreciated.
[1] How to handle OSX Aliases in Python with os.walk()?
[2] https://docs.python.org/2/library/carbon.html
[3] https://docs.python.org/2/library/macostools.html
[4] change an alias target python
[5] https://pypi.python.org/pypi/mac_alias
Solved it, using PyObjC, despite there being almost no documentation for PyObjC. You have to carefully convert ObjectiveC interfaces for NSURL to PyObjC calls, using the techniques described in "An Introduction to PyObjC" found on this site while referring to the NSURL interfaces described here.
Code in #MagerValp's reply to this question helped figure out how to get the target of an alias. I had to work out how to create a new alias with a revised target.
Below is a test program that contains and exercises all the functionality needed. Its setup and use are documented with comments in the code.
I'm a bad person and didn't do doc strings or descriptions of inputs and return values, but I've kept all functions short and single-functioned and hopefully I've named all variables and functions sufficiently clearly that they are not needed. There's an admittedly weird combination of CamelCaps and underscore_separated variable and function names. I normally use CamelCaps for global constants and underscore_separated names for functions and variables, but in this case I wanted to keep the variables and data types referred to in the PyObjC calls, which use camelCaps, unchanged, hence the odd mix.
Be warned, the Mac Finder caches some information about aliases. So if you do a Get Info or a resolve on file_alias immediately after running this program, it will look like it didn't work, even though it did. You have to drag the one folder to the Trash and empty the Trash, and only then will a Get Info or resolve of file_alias show that it does indeed now point to ./two/file.txt. (Grumble, grumble.) Fortunately this will not impact my use of these techniques, nor will it affect most people's use, I suspect. The point of the program will normally be to replace a broken alias with a fixed one, based on the fact that some single, simple thing changed, like the folder name in this example, or the volume name in my real application for this.
Finally, the code:
#!/usr/bin/env python
# fix_alias.py
# A test program to exercise functionality for retargeting a macOS file alias (bookmark).
# Author: Larry Yaeger, 20 Feb 2018
#
# Create a file and directory hierarchy like the following:
#
# one
# file.txt
# two
# file.txt
# file_alias
#
# where one and two are folders, the file.txt files are any files really, and
# file_alias is a Mac file alias that points to ./one/file.txt. Then run this program
# in the same folder as one, two, and file_alias. It will replace file_alias with
# an alias that points to ./two/file.txt.
#
# Note that file_alias is NOT a symbolic link, even though the Mac Finder sometimes
# pretends symbolic links are aliases; they are not.
import os
import string
from Foundation import *
OldFolder = 'one'
NewFolder = 'two'
AliasPath = 'file_alias'
def get_bookmarkData(alias_path):
alias_url = NSURL.fileURLWithPath_(alias_path)
bookmarkData, error = NSURL.bookmarkDataWithContentsOfURL_error_(alias_url, None)
return bookmarkData
def get_target_of_bookmarkData(bookmarkData):
if bookmarkData is None:
return None
options = NSURLBookmarkResolutionWithoutUI | NSURLBookmarkResolutionWithoutMounting
resolved_url, stale, error = \
NSURL.URLByResolvingBookmarkData_options_relativeToURL_bookmarkDataIsStale_error_(
bookmarkData, options, None, None, None)
return resolved_url.path()
def create_bookmarkData(new_path):
new_url = NSURL.fileURLWithPath_(new_path)
options = NSURLBookmarkCreationSuitableForBookmarkFile
new_bookmarkData, error = \
new_url.bookmarkDataWithOptions_includingResourceValuesForKeys_relativeToURL_error_(
options, None, None, None)
return new_bookmarkData
def create_alias(bookmarkData, alias_path):
alias_url = NSURL.fileURLWithPath_(alias_path)
options = NSURLBookmarkCreationSuitableForBookmarkFile
success, error = NSURL.writeBookmarkData_toURL_options_error_(bookmarkData, alias_url, options, None)
return success
def main():
old_bookmarkData = get_bookmarkData(AliasPath)
old_path = get_target_of_bookmarkData(old_bookmarkData)
print old_path
new_path = string.replace(old_path, OldFolder, NewFolder, 1)
new_bookmarkData = create_bookmarkData(new_path)
new_path = get_target_of_bookmarkData(new_bookmarkData)
print new_path
os.remove(AliasPath)
create_alias(new_bookmarkData, AliasPath)
main()
This thread got my interest...
But I don't think it's possible.
Look at this bug report in mac_alias: https://github.com/al45tair/mac_alias/issues/4
it notes that the package handles Alias records not Alias files. The Alias files are a 3rd version which hadn't been reverse engineered yet.
It points to this info on the Alias file: http://indiestack.com/2017/05/resolving-modern-mac-alias-files/
Also this thread on their old bitbucket: https://bitbucket.org/al45tair/mac_alias/issues/3/support-for-version-3-aliases
which points this dead page (thanks, archive.org) https://web.archive.org/web/20170222235430/http://sysforensics.org/2016/08/mac-alias-data-objects/
and info that reading some information is possible via this package: https://pypi.python.org/pypi/plistutils/ which has a bunch of docs on reading alias structures on their github
none of this does what you want though. sorry.
I am trying to call a python function that takes an absolute path as an argument, but the file I want to reference is on the web.
Without cloning the file locally, is there a way I can refer to the file that will make python think the file is local?
In other words, I want to wrap the URL in a variable my_file_path, and have this return True:
os.path.isfile(my_file_path)
Note that I need to fake a file system path, as other calls in the program I am using are expecting a path, and not a file-like object (this includes other functions that call the function I linked)
A really great way to do this is with the requests library. You can get a file-like object using the stream=True option to the get function:
r = requests.get('https://api.github.com/events', stream=True)
loadmat(r.raw, ...)
In the case of needing an actual path, you can use the tempfile module as well:
with tempfile.NamedTemporaryFile() as fd:
r = requests.get('https://api.github.com/events', stream=True)
for chunk in r.iter_content(chunk_size):
fd.write(chunk)
fd.flush()
loadmat(fd.name)
# other code here, where the temp file no longer exists but the data has been read
There is no way to make Python take a URL where it wants a path.
In many cases—like the very function you linked in your question—it actually wants a file-like object, and the object returned by, e.g., urlopen is file-like. But in other cases, that doesn't work.
So, what can you do?
Below the Python level, your operating system may have a way to mount different kinds of remote paths as if they were part of your local filesystem.
At a higher level, write your own wrapper that just downloads the file to a temporary file. That temporary file will, of course, pass the os.path.isfile(my_file_path) test that you wanted, and will work with everything else that needs a file. But it means that you need to keep the two "layers" of your code—the part that wants to deal with URLs, and the part that needs to deal with functions that can only take local files—separate, and write the interface between those layers. On at least some platforms, you can create a temporary file that never gets flushed to disk unless necessary. (You can even create a temporary file that doesn't appear anywhere in the directory tree, but that wouldn't help here, because then you obviously can't pass a pathname around…) So you're not "cloning the file" in any sense that actually matters.
I have a script, that uses a config file called config.py. Actually this is rather a configuration module then. Anyways: the configuration-module contains a lot of parameters and dictionaries and lists of dictionaries and so on.
In the script today it is used like this
import config
def main():
myParameter = config.myParameter
Now I have another application scenario for this script that uses a related config ('config_advanced.py', but the parameters and dictionaries have different values.
My goal is now, to chose the name of the used config-modul as a passed command-line argument:
myScript.py -configuration config_advanced.py
Since the configuration-module is in the same folder than the main script, I guess I have to rename the passed configuration file to 'config.py' first. Afterwards I can perform import config. Otherwise, if I used `import config_advanced, I wouldn't be able to use a call like
config.myParameter
in the main script.
Another possibility could be, to put the configuration-modules in subfolders and keep the name config.py. The passed command-line-argument will have to contain the subfolder then.
Either way I won't be able to perform the import at the top of the main file, since I have to do the argument parsing first. This isn't a technically problem, but someone said that this it at least bad pratice.
What do you think?
What is a better way to do the trick with not much effort?
Thanks a lot
Edit:
One working solution has been
import sys fullpath = "d:\\python\\scripts\\projectA\\configurationFiles\\"
sys.path.append(fullpath)
config = __import__('config_advanced')
Without syspath it does NOT work, so those following tries won't work:
config = __import__('d:\\python\\scripts\\projectA\\configurationFiles\\config_advanced')
config = __import__('d:\\python\\scripts\\projectA\\configurationFiles\\config_advanced.py')
Another possibility that's similar to what you suggest in the question, but which doesn't need you to hide things in subfolders, is to put config_advanced.py and config_plain.py in the same folder as the main script and then dynamically make config.py a link to the actual config file you want to use.
However, martineau's suggestion is much simpler.
OTOH, georg brings up a very valid point, especially if this script isn't just for your own personal use. While using Python itself for the config data is flexible and powerful, it's perhaps a little too powerful. Config data should just be data, not live executable code. If you make a minor mistake when modifying config data you could cause havoc if it's in an executable file. And if a malicious user gets to it, there's no limit to the damage they could cause.
Bad data in a plain old data file will at worst cause a ValueError if it does something weird that your config parsing code isn't suspecting. But bad data in a live Python file could throw all sorts of nasty errors. Or even worse, it could do something evil in complete silence...
In reply to your comments, here's some code to illustrate the first point:
#! /usr/bin/env python
import os
config_file = "config.py"
def link_config(mode):
if os.path.exists(config_file):
os.remove(config_file)
config_name = "config_%s.py" % mode
os.symlink(config_name, config_file)
#.... parse command line to determine config_mode string, then do
link_config(config_mode)
#Now import the newly-linked config file
import config
If config_mode == "plain" the above code will cause config_plain.py to be imported as 'config'
and if config_mode == "advanced" it will cause config_advanced.py to be imported as 'config'
But as I said before, martineau's method is much simpler. And IIRC, os.symlink may not work on non-unix systems.
...
As for your second point, check out the docs for the json module
It might be a very simple question. I was running a python code, and I got an error message as such :
File "/home/mbenchoufi/brisket/../brisket/views.py", line 11, in <module>
from influence.forms import SearchForm
ImportError: No module named forms
The problem is first that I have indeed a file called views.py in /home/myname/brisket/ but I don''t understand the notation : /home/myname/brisket/../brisket/views.py
Do I have a path config problem and what does this notation means ?
Btw, a really weird thing is that I have a file called forms.py, in the influence folder, and in this file a I have a class called SearchForm... How can the error message can be ?
This is not a Python-specific notation, it's a UNIX filesystem notation. .. in a UNIX path means "back up one directory," so for example, in this case, /home/myname/brisket/.. is equivalent to just /home/myname.
The reason Python displays the filename in this way might be that your sys.path has actually has /home/myname/brisket/.. in it for some reason. It's not a problem, since Python will be able to follow the ..s in the path just fine.
What this error message is telling you is that, while processing the file /home/myname/brisket/../brisket/views.py (which is the same file as /home/myname/brisket/views.py) there is a line of code
from influence.forms import SearchForm
which caused an error. Specifically, it's an ImportError, meaning that the file influence/forms.py wasn't found (or could not be read) by Python's import mechanism. You should check the value of sys.path in your Python program to make sure that the parent directory of influence/ is in the list, and make sure that the file is readable. (Also make sure that influence/__init__.py exists, though I'm not sure that particular problem would cause the error you're seeing.)
"/home/myname/brisket/../brisket/views.py"
is equivalent to
"/home/myname/brisket/views.py"
The cause might be an entry in you PYTHONPATH, e.g. like
export PYTHONPATH="$HOME/../brisket:$PYTHONPATH"
http://docs.python.org/using/cmdline.html#envvar-PYTHONPATH
The above approach has the benefit of working for other users, while not requiring an absolute path to /home. Write it like
export PYTHONPATH="/home/brisket:$PYTHONPATH"
to get simpler paths