I want to handle multipart/form-data with python.
The request will look like this.
POST /upload HTTP/1.1
Host: localhost:8000
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:83.0) Gecko/20100101 Firefox/83.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://localhost:8000/uplaod.html
Content-Type: multipart/form-data; boundary=---------------------------13077140074516507283069689500
Content-Length: 20970
Origin: http://localhost:8000
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Cache-Control: max-age=0
-----------------------------13077140074516507283069689500
Content-Disposition: form-data; name="mylogo"; filename="logo.png"
Content-Type: image/png
image_byte_code
-----------------------------13077140074516507283069689500--
This will be received in bytes from client browser.
I need to decode it to string.
But when i decode it the image bytes get destroyed.
Which then when i save it it will not be an image.
I have so far been able to decode it and receive text files.
Using the email library that defaults with python can really help. you just split the header from the data then use this https://julien.danjou.info/handling-multipart-form-data-python/. Make sure you get the boundary data from the headers.
Related
Hi I have a request as so
GET https://dropezy.goadda.in/master/v1/id/products/search?q=buah&storeId=619e02f0f580166d48277fc0&sortBy=relevance HTTP/2.0
accept: application/json
enro-api-key: rhcvwcdd-7aza-zvv7qskyu-4b8jm8kwppkf
cache-control: max-age=31536000
store-id: 619e02f0f580166d48277fc0
user-agent: Mozilla/5.0 (Linux; Android 11; sdk_gphone_x86 Build/RSR1.201013.001; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/83.0.4103.106 Mobile Safari/537.36
content-type: application/json; charset=utf-8
origin: https://www.dropezy.com
x-requested-with: com.enroco.dropezy
sec-fetch-site: cross-site
sec-fetch-mode: cors
sec-fetch-dest: empty
referer: https://www.dropezy.com/id/search/
accept-encoding: gzip, deflate
accept-language: en-US,en;q=0.9
content-length: 0
When I tried using request in python it returns message not found
This is my current code
url = 'https://dropezy.goadda.in/master/v1/id/products/search?q=buah&storeId=619e02f0f580166d48277fc0&sortBy=relevance HTTP/2.0'
accept='application/json'
jsonData = requests.post(url, accept).json()
Can somebody help? How to insert the other components such as user-agent, enro-api-key into requests?
The two requests aren't the same. The first is a GET request, but the second is a POST.
Use requests.get to make a GET request.
If you need to supply headers in a request, yes, you can do that: https://docs.python-requests.org/en/latest/user/quickstart/#custom-headers.
Generally, the docs for Requests are quite good: https://docs.python-requests.org/en/latest/
I need a python script that posts a given Authorization Bearer header to a specific ipand port.
This is what I have so far.
#!/usr/bin/python
import urllib3
import certifi
http = urllib3.PoolManager(ca_certs=certifi.where())
url = 'http://172.10.10.2:3000'
req = http.request('POST', url, fields={'Authorization': 'Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjbWQiOiJscyR7SUZTfS1sYSR7SUZTfS90bXAvbmMifQ.EziCTtJn1PpPXvemJllF36w7ADNkhKiktZ5sv9ADR3o'})
print(req.data.decode('utf-8'))
I currently get an error when running this stating that the Required authorization token is not found
The bearer code is created manually and then input into the script, if there was a way to import that from the site I create it on that would be helpful.
What I need the output to be in this -
GET / HTTP/1.1
Host: 172.10.10.2:3000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: close
Upgrade-Insecure-Requests: 1
Cache-Control: max-age=0
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjbWQiOiJpcCR7SUZTfWEifQ.RkoZinBcg2_5HRGgv1AtErhscIVBRv2hUcGIF08ZlCM
There are similar questions posted, but I still seem to have a problem. I am expecting to receive a registration email after running this. I receive nothing. Two questions. What is wrong? How would I even know if the data was successfully submitted as opposed to the page just loading normally?
serviceurl = 'https://signup.com/'
payload = {'register-fname': 'Peter', 'register-lname': "Parker", 'register-email': 'xyz#email.com', 'register-password': '9dlD313kF'}
r2 = requests.post(serviceurl, data=payload)
print(r2.status_code)
The url for the POST request is actually https://signup.com/api/users, and it returns 200 (in my browser).
You need to replicate what your browser does. This might include certain request headers.
You will want to use your browser's dev tools/network inspector to gather this information.
The information below it from my Firefox on my computer:
Request headers:
Host: signup.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:73.0) Gecko/20100101 Firefox/73.0
Accept: application/json, text/plain, */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Content-Type: application/json;charset=utf-8
Content-Length: 107
Origin: https://signup.com
Connection: keep-alive
Referer: https://signup.com/
Cookie: _vspot_session_id=ce1937cf52382239112bd4b98e0f1bce; G_ENABLED_IDPS=google; _ga=GA1.2.712393353.1584425227; _gid=GA1.2.1095477818.1584425227; __utma=160565439.712393353.1584425227.1584425227.1584425227.1; __utmb=160565439.2.10.1584425227; __utmc=160565439; __utmz=160565439.1584425227.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utmt=1; __qca=P0-1580853344-1584425227133; _gat=1
Pragma: no-cache
Cache-Control: no-cache
Payload:
{"status":true,"code":null,"email":"TestEmail#hotmail.com","user":{"id":20540206,"email":"TestEmail#hotmail.com","name":"TestName TestSurname","hashedpassword":"4ffdbb1c33d14ed2bd02164755c43b4ad8098be2","salt":"700264767700800.7531319164902858","accesskey":"68dd25c3ae0290be69c0b59877636a5bc5190078","isregistered":true,"activationkey":"f1a6732b237379a8a1e6c5d14e58cf4958bf2cea","isactivated":false,"chgpwd":false,"timezone":"","phonenumber":"","zipcode":"","gender":"N","age":-1,"isdeferred":false,"wasdeferred":false,"deferreddate":null,"registerdate":"2020/03/17 06:09:27 +0000","activationdate":null,"addeddate":"2020/03/17 06:09:27 +0000","admin":false,"democount":0,"demodate":null,"invitationsrequest":null,"isvalid":true,"timesinvalidated":0,"invaliddate":null,"subscribe":0,"premium":false,"contributiondate":null,"contributionamount":0,"premiumenddate":null,"promo":"","register_token":"","premiumstartdate":null,"premiumsubscrlength":0,"initial_reg_type":"","retailmenot":null,"sees":null,"created_at":"2020/03/17 06:09:27 +0000","updated_at":"2020/03/17 06:09:27 +0000","first_name":"TestName","last_name":"TestSurname"},"first_name":"TestName","last_name":"TestSurname","mobile_redirect":false}
There's a lot to replicate. Things like the hashed password, salt, dates, etc would have been generated by JavaScript executed by your browser.
Keep in mind, the website owner might not appreciate a bot creating user accounts.
I've made an python server with swagger-codegen. I have one endpoint that receives an file with mutlipart/form-data
And also created an client with go-swagger for testing.
created an file to upload: $ echo "123file content321" > data
and used the client to upload the file to the server. The resulting HTTP request looks like this:
POST /api/order/1/attachment HTTP/1.1
Host: 127.0.0.1:8080
User-Agent: Go-http-client/1.1
Transfer-Encoding: chunked
Accept: application/json
Content-Type: multipart/form-data; boundary=5f3f0ad86e6345b77c869cbe0a5e608f038354cf9ceab74ec2533d7555c0
Accept-Encoding: gzip
ff
--5f3f0ad86e6345b77c869cbe0a5e608f038354cf9ceab74ec2533d7555c0
Content-Disposition: form-data; name="file"; filename="data"
Content-Type: application/octet-stream
123file content321
--5f3f0ad86e6345b77c869cbe0a5e608f038354cf9ceab74ec2533d7555c0--
but the server doesn't accept it and responds:
HTTP/1.0 400 BAD REQUEST
Connection: close
Content-Length: 120
Content-Type: application/problem+json
Date: Fri, 19 May 2017 15:15:44 GMT
Server: Werkzeug/0.12.1 Python/3.6.1
{
"type": "about:blank",
"title": "Bad Request",
"detail": "Missing formdata parameter 'file'",
"status": 400
}
So the request isn't parsed properly. But when I use the swagger-ui, the file is uploaded correctly. Is there problem with the client's request, or the server has a problem?
EDIT: I think that there is missing Content-Lenght or the ff at the beginning of the BODY might not be there
EDIT2: the swagger-ui request:
POST /api/order/1/attachment HTTP/1.1
Host: localhost:8080
Connection: keep-alive
Content-Length: 211
Origin: http://localhost:8080
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36
Content-Type: multipart/form-data; boundary=----WebKitFormBoundarypzmNwrDR7zzpZ7SJ
Accept: application/json
X-Requested-With: XMLHttpRequest
DNT: 1
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.8
------WebKitFormBoundarypzmNwrDR7zzpZ7SJ
Content-Disposition: form-data; name="file"; filename="data"
Content-Type: application/octet-stream
123file content321
------WebKitFormBoundarypzmNwrDR7zzpZ7SJ--
The first request you send is a HTTP/1.1 request using chunked transfer encoding. This means the body is consisting of multiple chunks where each chunk is prefixed by the size in hex followed by \r\n followed by the data and again \r\n. I'm not sure if the ff at the beginning of the body you show really specifies the size of the following data (i.e. 255 bytes). But, the last chunk with a size of 0 is missing, so this request is incomplete. But maybe you just omitted the missing part from this question only.
Apart from that the server is responding with version HTTP/1.0. Chunked transfer encoding is only defined for HTTP/1.1 which means that this request will not be understood by a HTTP/1.0 server. And not even all HTTP/1.1 server will understand chunked transfer encoding in the request even if they should.
The second request you show (created by Chrome) does not use chunked transfer encoding but specifies instead the length of the header using Content-length in the HTTP header. That's the way you should go since this works with all web servers, including HTTP/1.0 servers.
Based on the two requests you have posted I would attempt to set the Content-Length on your go request first and test that. I've run into issues before with the ArangoDB HTTP API not accepting requests without a correct content length value.
If the succeeds then yay.
Otherwise, that ff in your request is the next thing I'd look at getting rid of. But I'd focus on the Content-Length header first.
I want to upload a file to a remote device.
If i look up the connection with wireshark i get this
POST /saveRestore.htm.cgi HTTP/1.1
Host: 10.128.115.214
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:15.0) Gecko/20100101 Firefox/15.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: de-de,de;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Connection: keep-alive
Referer: http://10.128.115.214/saveRestore.htm
Cache-Control: max-age=0
Content-Type: multipart/form-data; boundary=---------------------------961265085509552220604142744
Content-Length: 10708
-----------------------------961265085509552220604142744
Content-Disposition: form-data; name="restore"; filename="config(2).cfg"
Content-Type: application/octet-stream
Now this says that that the browser only accepts text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
If i upload the file with my script it says
--0a7125aebb8845ba8ab9aa21306b01f6
Content-Disposition: form-data; name="restore"; filename="Config.cfg"
Content-Type: text/plain; charset=utf-8
So it's a wrong file type..
so how do i change the content-type of the File ?
My code looks so far as follows:
#!/usr/bin/python
import httplib
import urllib2
from poster.encode import multipart_encode
import poster
from poster.streaminghttp import register_openers
register_openers()
params = {'restore': open("Config.cfg", "rb"), 'upload': 'PC ==>; Unit'}
datagen, headers = multipart_encode(params)
request = urllib2.Request('http://10.128.115.214/saveRestore.htm.cgi', datagen, headers)
u = urllib2.urlopen(request)
print u.read()
In the documentation for poster.encode.MultipartParam it says:
If filetype is set, it is used as the Content-Type for this parameter. If unset it defaults to “text/plain; charset=utf8”
So instead of specifying your parameters like this:
params = {'restore': open("Config.cfg", "rb"), 'upload': 'PC ==>; Unit'}
specify them like this:
params = [MultipartParam('restore', open("Config.cfg", "rb"),
filetype = 'application/octet-stream'),
('upload', 'PC ==>; Unit')]