How would one go about testing a Pylons controller (using Nose Tests) that takes a file upload as a POST parameter?
Like this:
class TestUploadController(TestController):
// ....
def test_upload_files(self):
""" Check that upload of text file works. """
files = [("Filedata", "filename.txt", "contents of the file")]
res = self.app.post("/my/upload/path", upload_files = files)
Uploading file usually requires authenticated user so you may also need to pass "extra_environ" argument to self.app.post() to circumvent that.
See paste.fixture documentation for details on the arguments accepted by self.app.post()
Related
Currently crating inf_conf from entry script (score.py) and environment however, I have a json file that i also want to include in this.
Is there a way i can do this?
I have seen source_directory argument but json file is not in the same folder as score.py file. https://learn.microsoft.com/en-us/python/api/azureml-core/azureml.core.model.inferenceconfig?view=azure-ml-py
inf_conf = InferenceConfig(entry_script="score.py",environment=environment)
Currently, it is required that all the necessary files and objects related to the endpoint be placed in the source_directory:
inference_config = InferenceConfig(
environment=env,
source_directory='./endpoint_source',
entry_script="./score.py",
)
One workaround is to upload your JSON file somewhere else, e.g., on the Blob Storage, and download it in the init() function of your entry script. For example:
score.py:
import requests
def init():
"""
This function is called when the container is initialized/started,
typically after create/update of the deployment.
"""
global model
# things related to initializing the model
model = ...
# download your JSON file
json_file_rul = 'https://sampleazurestorage.blob.core.windows.net/data/my-configs.json'
response = requests.get(json_file_rul)
open('my_configs.json', "wb").write(response.content)
I stuck on the problem of how to organize code / proper way to get xlsx file as output in flask app.
I have a function.py file where for now the xlsx file generates.
The sense is that flask app gets some settings in json format, these settings are processed by function that must return xlsx(?) to app.
The function do some calculations depending on the settings.
The file has the next structure:
def function (settings):
settings=settings
df = pd.read_csv(settings['df'])
from pandas import ExcelWriter
writer = ExcelWriter('file.xlsx')
if settings[somefeature1]==1:
f1=dosmth.to_excel(writer, "feature 1")
if settings[somefeature2]==1:
f2=dosmth.to_excel(writer, "feature 2")
...
writer.save()
But if the file is already generated in function, what should I pass to flask? How the flask app function must look like then (especially in case if I want to return xlsx as json)?
#app.route('/Function', methods = ['POST'])
def Function():
settings = request.get_json(force = True)
return(function(settings)) #???
You should never forget that flask is a framework for creating web applications, a web application is a piece of software that receives a web request and generates a web response.
To make it super simple: your flask function should return something that a common web browser will be able to handle.
In this case your response should return the file but also some metadata to tell to the potential receiver what is inside the response and how to handle it.
I think that something like this could work:
return send_file(filename, mimetype='application/vnd.ms-excel')
I am implementing a way to restrict file upload on Django 1.8 running python 3.4
Basically, I want to check the MIMEType of a file when they upload using mimetype. However, when I manipulate the file name from bad_image.exe to bad_image.exe.jpg, the mimetype is still image/jpeg. This could still result in a malicious attack.
Is there a way to actually implement this? I tried magic too but it still does not work.
You're right, rename a file .exe to .exe.jpg and content_type output (image/jpeg).
But using python-magic if properly check the file as checking their headers and not the extension, so the output was in my test:
PE32 executable (GUI) Intel 80386, for MS Windows
Even so I think the headers may be modified... hope that helps.
EDIT: In my test use cleaning a specific attribute
class UploadFileForm(forms.ModelForm):
class Meta:
model = FileTestUpload
fields = ('title','file')
def clean_file(self):
f = self.cleaned_data.get("file", False)
ftype = magic.from_buffer(f.read()) # InMemoryUploadedFile
print ftype
return f
It is too simple but it was just to test.
I am trying to implement a function in Django to upload an image from a client (an iPhone app) to an Amazon S3 server. The iPhone app sends a HttpRequest (method PUT) with the content of the image in the HTTPBody. For instance, the client PUTs the image to the following URL: http://127.0.0.1:8000/uploadimage/sampleImage.png/
My function in Django looks like this to handle such a PUT request and save the file to S3:
def store_in_s3(filename, content):
conn = S3Connection(settings.ACCESS_KEY, settings.PASS_KEY) # gets access key and pass key from settings.py
bucket = conn.create_bucket("somepicturebucket")
k = Key(bucket)
k.key = filename
mime = mimetypes.guess_type(filename)[0]
k.set_metadata("Content-Type", mime)
k.set_contents_from_string(content)
k.set_acl("public-read")
def upload_raw_data(request, name):
if request.method == 'PUT':
store_in_s3(name,request.raw_post_data)
return HttpResponse('Upload of raw data to S3 successful')
else:
return HttpResponse('Upload not successful')
My problem is how to tell my function the name of the image. In my urls.py I have the following but it won't work:
url(r'^uploadrawdata/(\d+)/', upload_raw_data ),
Now as far as I'm aware, d+ stands for digits, so it's obviously of no use here when I pass the name of a file. However, I was wondering if this is the correct way in the first place. I read this post here and it suggests the following line of code which I don't understand at all:
file_name = path.split("/")[-1:][0]
Also, I have no clue what the rest of the code is all about. I'm a bit new to all of this, so any suggestions of how to simply upload an image would be very welcome. Thanks!
This question is not really about uploading, and the linked answer is irrelevant. If you want to accept a string rather than digits in the URL, in order to pass a filename, you can just use w instead of d in the regex.
Edit to clarify Sorry, didn't realise you were trying to pass a whole file+extension. You probably want this:
r'^uploadrawdata/(.+)/$'
so that it matches any character. You should probably read an introduction to regular expressions, though.
How to restrict the size of file being uploaded.
I am using django 1.1 with apache.
Can I use apache for this and show some html error page if say size is bigger then 100MB.
Thanks.
I mean before uploading the file
On client side it isn't possible...
I suggest to write a custom upload handlers and to override receive_data_chunk.
Example: QuotaUploadHandler
You can do this in javascript in most recent browsers, using the File API: http://www.w3.org/TR/FileAPI/
For example (using jquery):
var TYPES = ['image/jpeg', 'image/jpg', 'image.png'];
var file = $('#my_file_input')[0].files[0];
var size = file.size || file.fileSize;
var type = file.type;
if (size > MAX_BYTES) {
alert('Error: file too large');
} else if (TYPES.indexOf(type) < 0) {
alert('Error: file not a JPG or PNG');
} else {
// proceed with file upload
}
No need for Java or Flash. Of course, you'll still need some sort of checking on the server for users who disable javascript.
apache has a server setting for max file size..(also dont forget max post size). I do not believe apache can show an error page on its own, you can probably use python for that.
unfortunetly I know nothing obout python (yet) so I can't really help you beyond that.
I know php can do that easily so I'm sure there is a method for python.
If you want to get the file size before uploading begins you will need to use Flash or a Java applet.
Writing a custom upload handler is the best approach. I think something like the following would work (untested). It terminates the upload as early as possible.
from django.conf import settings
from django.core.files.uploadhandler import FileUploadHandler, StopUpload
class MaxSizeUploadHandler(FileUploadHandler):
"""
This test upload handler terminates the connection for
files bigger than settings.MAX_UPLOAD_SIZE
"""
def __init__(self, request=None):
super(MaxSizeUploadHandler, self).__init__(request)
def handle_raw_input(self, input_data, META, content_length, boundary, encoding=None):
if content_length > settings.MAX_UPLOAD_SIZE:
raise StopUpload(connection_reset=True)