We Keep Coding

how to convert a form data that contains image files to json file using in python django and i am not using django restframe work just python&django

#csrf_exempt
def registerSeller(request):
print(request.body)
data = json.loads(request.body.decode('utf-8'))
i am new for django and trying to build api with out django restframework and this is the output for request.body but it gives me error
AttributeError: 'bytes' object has no attribute 'read'
request.body
b'-----------------------------344288735834045826241553250616\r\nContent-Disposition: form-data; name="username"\r\n\r\nabe7\r\n-----------------------------344288735834045826241553250616\r\nContent-Disposition: form-data; name="password"\r\n\r\n123\r\n-----------------------------344288735834045826241553250616\r\nContent-Disposition: form-data; name="phonenumber"\r\n\r\n091010101\r\n-----------------------------344288735834045826241553250616\r\nContent-Disposition: form-data; name="email"\r\n\r\nemail#gmial.com\r\n-----------------------------344288735834045826241553250616\r\nContent-Disposition: form-data; name="shop_description"\r\n\r\ndescription new\r\n-----------------------------344288735834045826241553250616\r\nContent-Disposition: form-data; name="shop_name"\r\n\r\nname new\r\n-----------------------------344288735834045826241553250616\r\nContent-Disposition: form-data; name="images"\r\n\r\n[object File],[object File],[object File]\r\n-----------------------------344288735834045826241553250616--\r\n'

You're getting multipart form data, not JSON data. You could access that with request.POST and request.FILES.
If you want something that can accept both that and JSON data, you would need to look at the Content-type request header and parse the data as JSON only if the content type is something like application/json.

python assign literal value of a dictionary to key of another dictionary

I am trying to form a web payload for a particular request body but unable to get it right. What I need is to pass my body data as below
data={'file-data':{"key1": "3","key2": "6","key3": "8"}}
My complete payload request looks like this
payload={url,headers, data={'file-data':{"key1": "3","key2": "6","key3": "8"}},files=files}
However, when I pass this, python tries to parse each individual key value and assigns to the 'file-data' key like this
file-data=key1
file-data=key2
file-data=key3
and so on for as many keys I pass within the nested dictionary. The requirement however, is to pass the entire dictionary as a literal content like this(without splitting the values by each key):
file-data={"key1": "3","key2": "6","key3": "8"}
The intended HTTP trace should thus ideally look like this:
POST /sample_URL/ HTTP/1.1
Host: sample_host.com
Authorization: Basic XYZ=
Cache-Control: no-cache
Content-Type: multipart/form-data; boundary=----UVWXXXX
------WebKitFormBoundaryXYZ
Content-Disposition: form-data; name="file-data"
{"key1": "3","key2": "6","key3":"8" }
------WebKitFormBoundaryMANZXC
Content-Disposition: form-data; name="file"; filename=""
Content-Type:
------WebKitFormBoundaryBNM--
As such, I want to use this as part of a payload for a POST request(using python requests library). Any suggestions are appreciated in advance-
Edit1: To provide more clarity, the API definition is this:
Body
Type: multipart/form-data
Form Parameters
file: required (file)
The file to be uploaded
file-data: (string)
Example:
{
"key1": "3",
"key2": "6",
"key3": "8"
}
The python code snippet I used(after checking suggestions) is this:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import requests
url = "https://sample_url/upload"
filepath='mypath'
filename='logo.png'
f=open(filepath+'\\'+filename)
filedata={'file-data':"{'key1': '3','key2': '6','key3': '8'}"}
base64string = encodestring('%s:%s' % ('user', 'password').replace('\n', '')
headers={'Content-type': 'multipart/form-data','Authorization':'Basic %s' % base64string}
r = requests.post(url=url,headers=headers,data=filedata,files={'file':f})
print r.text
The error I get now is still the same as shown below:
{"statusCode":400,"errorMessages":[{"severity":"ERROR","errorMessage":"An exception has occurred"]
It also says that some entries are either missing or incorrect. Note that I have tried passing the file parameter after opening it in binary mode as well but it throws the same error message
I got the HTTP trace printed out via python too and it looks like this:
send: 'POST sample_url HTTP/1.1
Host: abc.com
Connection: keep-alive
Accept-Encoding: gzip,deflate
Accept: */*
python-requests/2.11.1
Content-type: multipart/form-data
Authorization: Basic ABCDXXX=
Content-Length: 342
--CDXXXXYYYYY
Content-Disposition:form-data; name="file-data"
{\'key1\': \'3\',\'key2\': \'6\'
,\'key3\': \'8\'}
--88cdLMNO999999
Content-Disposition: form-data; name="file";
filename="logo.png"\x89PNG\n\r\n--cbCDEXXXNNNN--

If you want to post JSON with python requests, you should NOT use data but json:
r = requests.post('http://httpbin.org/post', json={"key": "value"})
I can only guess that you are using data because of your example
payload={url,headers, data={'file-data':{"key1": "3","key2": "6","key3": "8"}},files=files}
Whis is not valid python syntax btw.

parse multipart/form-data, received from requests post

I am writing Web Service Client, using requests library. I am getting data in multipart/form-data that contains a file and text-json. I have no idea how to parse it. Is there a proper library to parse multipart/form-data format in python or should I write parser on my own?
my code:
data = {
"prototypeModel" :('prototypeModel', open(prototypeModel, 'rb'), 'application/octet-stream', {'Expires': '0'}),
"mfcc_1" : ('mfcc', open(mfcc_1, 'rb'), 'application/octet-stream', {'Expires': '0'}),
"mfcc_2" : ('mfcc', open(mfcc_2, 'rb'), 'application/octet-stream', {'Expires': '0'}),
"mfcc_3" : ('mfcc', open(mfcc_3, 'rb'), 'application/octet-stream', {'Expires': '0'}),
}
print( '---------------------- start enroll ----------------------')
testEnrollResponse = requests.post(server+sessionID, files = data, json = declaredParameters)
b'\r\n--c00750d1-8ce4-4d29-8390-b50bf02a92cc\r\nContent-Disposition:
form-data; name="playbackHash"\r\nContent-Type:
application/octet-stream\r\n\r\n\x16\x00\x00\x00\x00\x00\x00\x00serialization::archive\n\x00\x04\x08\x04
....
x00\x00R\x94\x9bp\x8c\x00\r\n--c00750d1-8ce4-4d29-8390-b50bf02a92cc\r\nContent-Disposition:
form-data; name="usersMFCC"\r\nContent-Type:
application/octet-stream\r\n\r\n\x16\x00\x00\x00\x00\x00\x00\x00serialization::archive\n\x00\x04\x08\x04\x08\x01\x00\x00\x00\x00\x00\x00\x00\x00\xf8\x16\x00\x00\x00\x00\x00\x00u\xbd\xb4/\xda1\xea\xbf\x0f\xed\xa2<\xc9\xf8\xe7\xbf?\xd5\xf06u\xe7\xf0\xbf\xd4\x8d\xd4\xa1F\xbe\x03#\x85X!\x19\xd8A\x06#\x8co\xf7\r
.....
x80\xd9\x95Yxn\xd0?\r\n--c00750d1-8ce4-4d29-8390-b50bf02a92cc\r\nContent-Disposition:
form-data; name="scoreAndStatus"\r\nContent-Type: application/json;
charset=utf-8\r\n\r\n{"lexLikelihood":1.544479046897232,"overallScore":-nan,"playbackLikelihood":-inf,"status":{"errorCode":0,"errorMessage":""}}\r\n--c00750d1-8ce4-4d29-8390-b50bf02a92cc--\r\n'
I replaced more binary data with " ..... "

If you're receiving a multipart/form-data response, you can parse it using the requests-toolbelt library like so:
$ pip install requests-toolbelt
After installing it
from requests_toolbelt.multipart import decoder
testEnrollResponse = requests.post(...)
multipart_data = decoder.MultipartDecoder.from_response(testEnrollResponse)
for part in multipart_data.parts:
print(part.content) # Alternatively, part.text if you want unicode
print(part.headers)

Code sample for Flask, uses https://github.com/defnull/multipart
import multipart as mp
from multipart import tob
try:
from io import BytesIO
except ImportError:
from StringIO import StringIO as BytesIO
#app.route('/', methods=["GET","POST"])
def index():
...
elif flask.request.method == "POST":
data = flask.request.data
s = data.split("\r")[0][2:]
p = mp.MultipartParser(BytesIO(tob(data)),s)
blob = p.parts()[0].value
f = open("file.bin","wb")
f.write(blob.encode("latin-1"))
f.close()

A working example of parsing multipart data follows. You can try it out at the interactive python prompt.
import email
msg = email.message_from_string('''\
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary=" XXXX"
-- XXXX
Content-Type: text/plain
-- XXXX
Content-Type: text/plain
-- XXXX--
''')
msg.is_multipart()
Once you know its working on your system, you can build your own email message out of the POST data and parse it the same way. If you have the raw post body as a string the rest of the necessary information can be found in the request headers. I added indentation here for clarity, you should not have extraneous indentation in the block string.
epost_data = '''\
MIME-Version: 1.0
Content-Type: %s
%s''' % (self.headers['content-type'], post_data)
msg = email.message_from_string(post_data)
if msg.is_multipart():
for part in msg.get_payload():
name = part.get_param('name', header='content-disposition')
filename = part.get_param('filename', header='content-disposition')
# print 'name %s' % name # "always" there
# print 'filename %s' % filename # only there for files...
payload = part.get_payload(decode=True)
print payload[:100] # output first 100 characters
The first %s will be replaced with the content type, and the second with post_data. You can then write the payload to a file, etc.
Be careful to consider security implications of saving a file. You may not be able to trust the file name posted, it could start with ../../filename.sh for example on some web servers, so if you try to write /my-folder/../../filename.sh the attacker could potentially place a malicious file outside of the location where you are trying to store files. Strong validation of the file being the allowed type before trusting the file itself is also recommended. You do not want to let attackers overwrite any file on your system.

Handling POSTed multipart/form-data file

I'm wondering what is the best way to handle POSTed raw data on the server side.
So I'm using Falconframework and I'm able to receive user submitted file
-----------------------------1209846671886287098156775745
Content-Disposition: form-data; name="qquuid"
d3ad452e-a287-4cb7-ac1f-f0a5cdb54386
-----------------------------1209846671886287098156775745
Content-Disposition: form-data; name="qqfilename"
Screenshot.png
-----------------------------1209846671886287098156775745
Content-Disposition: form-data; name="qqtotalfilesize"
1951677
-----------------------------1209846671886287098156775745
Content-Disposition: form-data; name="qqfile"; filename="Screenshot.png"
Content-Type: image/png
�PNG
.................lots of bites............
Using python and hopefully some other lib i would like to turn it into some sort of file object which i can extract metadata - filename , uuid etc, as well as the file itself.
Which lib should i use?

Here is a middle ware project that looks promising I'm currently trying to implement this myself in a falcon service.
falcon-multipart
I have have pretty good luck as well using cgi.FeildStorage(). As found in the following post.
cgi article
import cgi
def on_post(req, resp):
env = req.env
env.setdefault('QUERY_STRING','')
form = cgi.FieldStorage(fp=req.stream,environ=env)
form['fileinputname'].file
If you are willing to have one non falcon hook here is an example with bottle:
example

Just a very late followup to this old discussion.
As of Falcon 3.0, the framework supports multipart/form-data natively for both WSGI and ASGI applications.

POST request with Multipart/form-data. Content-type not correct

We're trying to write a script with python (using python-requests a.t.m.) to do a POST request to a site where the content has to be MultipartFormData.
When we do this POST request manually (by filling in the form on the site and post), using wireshark, this came up (short version):
Content-Type: multipart/form-data;
Content-Disposition: form-data; name="name"
Data (8 Bytes)
John Doe
When we try to use the python-requests library for achieving the same result, this is sent:
Content-Type: application/x-pandoplugin
Content-Disposition: form-data; name="name"; filename="name"\r\n
Media type: application/x-pandoplugin (12 Bytes)
//and then in this piece is what we posted://
John Doe
The weird thing is that the 'general type' of the packet indeed is multipart/form-data, but the individual item sent (key = 'name', value= 'John Doe') has type application/x-pandoplugin (a random application on my pc I guess).
This is the code used:
response = s.post('http://url.com', files={'name': 'John Doe'})
Is there a way to specify the content-type of the individual items instead of using the headers argument (which only changes the type of the 'whole' packet)?
We think the server doesn't respond correctly due to the fact that it can't understand the content-type we send it.
Little update:
I think the different parts of the multipart content are now identical to the ones sent if I do the POST in the browser, so that's good. Still the server doesn't actually do the changes I send it with the script. The only thing that still is different is the order of the different parts.
For example this is what my browser sends:
Boundary: \r\n------WebKitFormBoundary3eXDYO1lG8Pgxjwj\r\n
Encapsulated multipart part: (text/plain)
Content-Disposition: form-data; name="file"; filename="ex.txt"\r\n
Content-Type: text/plain\r\n\r\n
Line-based text data: text/plain
lore ipsum blabbla
Boundary: \r\n------WebKitFormBoundary3eXDYO1lG8Pgxjwj\r\n
Encapsulated multipart part:
Content-Disposition: form-data; name="seq"\r\n\r\n
Data (2 bytes)
Boundary: \r\n------WebKitFormBoundary3eXDYO1lG8Pgxjwj\r\n
Encapsulated multipart part:
Content-Disposition: form-data; name="name"\r\n\r\n
Data (2 bytes)
And this is what the script (using python-requests) sends:
Boundary: \r\n------WebKitFormBoundary3eXDYO1lG8Pgxjwj\r\n
Encapsulated multipart part:
Content-Disposition: form-data; name="name"\r\n\r\n
Data (2 bytes)
Boundary: \r\n------WebKitFormBoundary3eXDYO1lG8Pgxjwj\r\n
Encapsulated multipart part: (text/plain)
Content-Disposition: form-data; name="file"; filename="ex.txt"\r\n
Content-Type: text/plain\r\n\r\n
Line-based text data: text/plain
lore ipsum blabbla
Boundary: \r\n------WebKitFormBoundary3eXDYO1lG8Pgxjwj\r\n
Encapsulated multipart part:
Content-Disposition: form-data; name="seq"\r\n\r\n
Data (2 bytes)
Could it be possible that the server counts on the order of the parts? According to Multipart upload form: Is order guaranteed?, it apparently is? And if so, is it possible to explicitly force an order using the requests library?
And to make things worse in that case: There is a mixture of a file and just text values.
So forcing an order seems rather difficult. This is the current way I do it:
s.post('http://www.url.com', files=files,data = form_values)
EDIT2:
I did a modification in the requests plugin to make sure the order of the parts is the same as in the original request. This doesn't fix the problem so I guess there is no straightforward solution for my problem. I'll send a mail to the devs of the site and hope they can help me!

your code looks correct.
requests.post('http://url.com', files={'name': 'John Doe'})
... and should send a 'multipart/form-data' Post.
and indeed, I get something like this posted:
Accept-Encoding: gzip, deflate, compress
Connection: close
Accept: */*
Content-Length: 188
Content-Type: multipart/form-data; boundary=032a1ab685934650abbe059cb45d6ff3
User-Agent: python-requests/1.2.3 CPython/2.7.4 Linux/3.8.0-27-generic
--032a1ab685934650abbe059cb45d6ff3
Content-Disposition: form-data; name="name"; filename="name"
Content-Type: application/octet-stream
John Doe
--032a1ab685934650abbe059cb45d6ff3--
I have no idea why you'd get that weird Content-Type header:
Content-Type: application/x-pandoplugin
I would begin by removing Pando Web Plugin from your machine completely, and then try your python-requests code again. (or try from a different machine)

As of today you can do:
response = s.post('http://url.com', files={'name': (filename, contents, content_type)})

Python uses a system-wide configuration file to "guess" the mime-type of a file. If those plugins are registering your file extension with their custom mime-type you'll end up putting that in instead.
The safest approach is make your own mime type guessing that suits the particular server you're sending do, and only use the native python mime type guessing for extensions you didn't think of.
How exactly you specify the content-type manually with python-requests I don't know, but I expect it should be possible.