I am trying to upload a file to a server, using python.
The API of the server has to accept the file, together with some other parameters.
This is why i open the file, and then i create a dictionary, containing the file and the other parameters that the web app accepts, and then encode it and perform a POST request with the item.
This is the code:
from urllib import request
from urllib.parse import urlencode
import json
with open('README.md', 'rb') as f:
upload_credentials = {
"file": f,
"descr": "testing",
"title": "READMEE.md",
"contentType": "text",
"editor": username,
}
url_for_upload = "" #here you place the upload URL
req = request.Request(url_for_upload, method="POST")
form_data = urlencode(upload_credentials)
form_data = form_data.encode()
response = request.urlopen(req, data=form_data)
http_status_code = response.getcode()
content = response.read()
print(http_status_code)
print(content)
However, i get an error in this line: response = request.urlopen(req, data=form_data)
The server responds with a 500 HTTP Status Code.
This is what i get as an error message:
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 500:
500 error codes can mean a lot of things and i do not know how to progress any further, as everything i did seems by the book...
Does anyone have the experience, given the info provided here, to guide me to some potential solution?
EDIT: I am trying to replicate working js code that does the same thing. This is it:
<input type="file" />
<button onclick="upload()">Upload data</button>
<script>
upload = async() =>
{
const fileField = document.querySelector('input[type="file"]');
await uploadDoc(fileField.files[0] );
};
uploadDoc = async( file ) =>
{
let fd = new FormData();
fd.append( 'file', file );
fd.append( 'descr', 'demo_upload' );
fd.append( 'title', name );
fd.append( 'contentType', 'text' );
fd.append( 'editor', user );
let resp = await fetch( url, { method: 'POST', mode: 'cors', body: fd });
};
</script>
The js code is doing a multipart/form-data post request.
I do not believe urllib supports multipart/form-data, you can use request instead.
import requests
with open('README.md', 'rb') as f:
files = {"file": f}
upload_credentials = {
"descr": "testing",
"title": "READMEE.md",
"contentType": "text",
"editor": username,
}
r = requests.post("http://httpbin.org/post", files=files, data=upload_credentials)
print(r.status_code)
print(r.text)
Related
I want to send a python file via http post in dart. I can do it in CURL the following way:
curl -X POST -F 'file=#/home/user/file.py' http://192.168.80.1:9888/dir/file.py
I am also able to do it in python like this:
import requests
url = 'http://192.168.80.1:9888/dir/file.py'
files = {'file': open('file.py', 'rb')}
print(files)
r = requests.post(url, files=files)
But in dart I am not able to send the post. I have tried several methods but is at this on currently:
import 'package:http/http.dart' as http;
import 'dart:io';
void main(List<String> arguments) async {
var response;
var file;
var url = 'http://192.168.80.1:9888/dir/file.py';
file = File('file.py').readAsStringSync();
var files = {'file': file};
response = await http.post(url, body: files);
}
Which result in the following exception: Exception has occurred.
ClientException (Connection closed before full header was received)
I know that the server is working due to CURL and python. How do I mimic the functionality in CURL/python using dart?
I was able to send the python file via a POST using dio.
import 'package:dio/dio.dart' as dio;
Future<dio.FormData> FormData3() async {
return dio.FormData.fromMap({
'file': await dio.MultipartFile.fromFile(
'files/file.py',
filename: 'file.py',
),
});
}
Future<dio.Response> sendFile() async {
dio.Response response;
response = await dio.Dio().post('http://192.168.80.1:9888/dir/file.py',
data: await FormData2(), onSendProgress: (received, total) {
if (total != -1) {
print((received / total * 100).toStringAsFixed(0) + '%');
}
},
options: dio.Options(
method: 'POST',
responseType: dio.ResponseType.plain,
));
return response;
}
void main() async {
response = await sendFile();
}
I have registered the app, added read-write permissions on it, gotten admin consent, and gotten the token. When I make a GET request I get an error.
This my Python code:
import requests, sys, os, json
# Get token
params = {
'client_id': clientID,
'scope': 'https://graph.microsoft.com/.default',
'client_secret': clientSecret,
'grant_type': 'client_credentials'
}
response = requests.post('https://login.microsoftonline.com/{tenant}/oauth2/v2.0/token', data=params)
jresp = response.json()
access_token = jresp['access_token']
# Get item
header = {'Authorization': 'Bearer ' + access_token}
response = requests.get('https://graph.microsoft.com/v1.0/me/drive/root:/Documents/WildRydes.txt', headers = header)
print response
print response.text
print response.content
sys.exit()
This is what is returned:
<Response [500]>
{
"error": {
"code": "BadRequest",
"message": "Unable to retrieve user's mysite URL.",
"innerError": {
"request-id": ,
"date": "2020-04-29T12:42:06"
}
}
}
I have been following the Microsoft Graph API documentation for the past two days straight. They are very good at theory but rubbish at showing concrete examples. Or maybe they're good at hiding where those are.
I am using
Python 3.6.3
Flask==1.0.2
Flask-Cors==3.0.7
Flask-RESTful==0.3.7
for making an API which is used to collect tiff image using post method and save it locally.
api = Api(app)
CORS(app)
class CatchImage(Resource):
#cross_origin(supports_credentials=True)
def post(self):
file = request.files['file']
if file and allowed_file(file.filename):
filename = secure_filename(file.filename)
if which_extenstion(filename) != "tiff":
return json.dumps({
"id": None,
"error": "Unsupported file type"
})
else:
unique_id, folder_path, save_path = get_unique_id(filename)
try:
file.save(save_path)
except FileNotFoundError:
LookupError("no uploads folder")
convert_jpg_save(save_path)
jpg_image_path = get_jpg_image_path(folder_path)
img = [
url_for("send_image", filename=image)
for image in jpg_image_path
]
return jsonify({
"filename ": file.filename,
"id": unique_id,
"urls": img,
"error": None
})
return json.dumps({"error": "no file"})
api.add_resource(CatchImage, '/api/sendimage')
I have tried the API using Postman which is working very fine. But when I try to access the API from Browser I get this
POST http://127.0.0.1:5000/api/sendimage 400 (BAD REQUEST)
The code for the same is below, which generated by Postman.
var form = new FormData();
form.append("file", "/home/blue/Templates/test.tiff");
var settings = {
"async": true,
"crossDomain": true,
"url": "http://127.0.0.1:5000/api/sendimage",
"method": "POST",
"headers": {
"cache-control": "no-cache",
"Postman-Token": "4fdcc138-5567-4c4d-8f7d-8967d45b3c2a"
},
"processData": false,
"contentType": false,
"mimeType": "multipart/form-data",
"data": form
}
$.ajax(settings).done(function (response) {
console.log(response);
});
I do think this has to do with CORS or setting some error which I not able to figure out. I would like to understand what is the probable cause of the problem and it's mostly likely solutions.T hanks in advance for your time – if I’ve missed out anything, over- or under-emphasized a specific point let me know in the comments.
400 (BAD REQUEST) means the data sent is not the data expected by the server. I do not think it is CORS related.
I would recommend to use pdb in your post function to find where the response is thrown.
import pdb; pdb.set_trace()
According to jQuery documentation, you must declare the data type:
$.ajax({
type: 'POST',
url: url,
data: data,
success: success,
dataType: dataType
});
possibly adding this help:
dataType: "json",
References:
Getting 400 bad request error in Jquery Ajax POST
Jquery ajax post request not working
Im using python requests library to make google a drive api request to change permissions of a file, in this case the owner.
Here is what my code looks like
fileId = "123abcEfJl-mNooP45Kl6u" #fake file id
url = https://www.googleapis.com/drive/v2/files/%s/permissions' % fileId
payload = {"role":"owner", "type":"user", "value":"<some_user>#gmail.com"}
headers = {'Authorization': 'Bearer %s'%access_token, 'Content-Type':'application/json'}
permResponse = requests.post(url, data=payload, headers=headers)
print permResponse.text
When I run this, I get the following response:
{
"error": {
"errors": [
{
"domain": "global",
"reason": "parseError",
"message": "Parse Error"
}
],
"code": 400,
"message": "Parse Error"
}
}
I've been following the google docs api for this and have not been able to figure out what im doing incorrectly.
https://developers.google.com/drive/v2/reference/permissions/insert
Even their Try It! section is broken because there isn't an option to add the required 'value' field.
What am I doing incorrectly? Is anyone else running into these issues?
Thanks
I'm using the urllib.request module, and It works fine. This is my code:
key = "?key=" + MY_API_KEY
url_destino = ("https://www.googleapis.com/drive/v2/files/%s/permissions" % source_id)+ key
values = "{"role":"owner", "type":"user", "value":"<some_user>#gmail.com"}"
data = values.encode('utf-8')
request = urllib.request.Request(url_destino, data, method='POST')
request.add_header("Authorization", "Bearer " + token)
request.add_header("Content-Length", len(data))
request.add_header("Content-Type", "application/json")
print(request.header_items()) # for debugging purpouse
f = urllib.request.urlopen(request)
print(f.read())
I've thought to replace the urllib.request by Requests module (it's more clean to work with) in my little library but, now works.
Because I use Python 3 I can't use the google-api-python-client.
I have the following code which retrieves values from a simple 3 input form:
//retrieves data from a form
var $form = $( this ),
prgname= $form.find('input[name="prg"]').val(),
startDate = $("#startdate").datepicker({ dateFormat: 'yy-mm-dd' }).val(),
endDate = $("#enddate").datepicker({ dateFormat: 'yy-mm-dd' }).val();
The following code sends the request to the server:
var request = $.ajax({
url: "/prg/",
type: "post",
data: JSON.stringify({prg: prgname, start:startDate, end:endDate}),
contentType: 'application/json',
dataType: 'json',
success: function() {},
error: function (jqXHR, textStatus, errorThrown){};
on the server end using python and webapp2 Im doing the following, (here is where I am unsure of things)
import json
class PrgHandler(webapp2.RequestHandler):
def post(self):
prg= cgi.escape(self.request.POST['prg'])
start_date = cgi.escape(self.request.POST['start'])
end_date = cgi.escape(self.request.POST['end'])
#some code to write to db here
....
#if successful return a success message
if success:
success_msg = [{'class': 'success', 'msg': 'Successfully saved to the database'}]
else:
success_msg = [{'class': 'error', 'msg': 'Could not saved to the database'}]
data_string = json.dumps(success_msg)
self.response.headers.add_header('content-type', 'application/json', charset='utf-8')
self.response.write(data_string)
When I get the response it is skipping the success function and going directly to the error.
Logging the error values im not getting any thing meaningful:
the error is:
The text status is:error
The jqXHR is:[object Object]
Chrome's console is giving me the error:
Resource interpreted as Document but transferred with MIME type application/json:
I looked that up and the solutions on SO did not work, I think this is an error with the server side code:
self.response.headers.add_header('content-type', 'application/json', charset='utf-8')
If I comment out the above line I get no error in chrome and I just get back the response on a blank page with the correct values in the following format:
[{"msg": "Successfully saved to the database", "class": "success"}]
In the above case it does save to the database so I cannot seem to find anything wrong except for the header and simply don't know how to proceed!
EDIT
The error it seems is from the server side I had removed the following line:
event.preventDefault();
from my script and it caused all the problems now at least Im getting a clear indication of where the problem is. It's from incorrectly getting the posted data, how would I do it the correct way? I tried the following:
json_data = self.request.GET.items()
decoded = json.loads(json_data)
but Im getting a TypeError: expected string or buffer on the following line:
json_data = self.request.GET.items()
Have a look in your debugger. You receive a JSON string in your post (webapp2 multidict). You have to decode this string using json.loads, resulting in a python object.
Here is my jquery code to send and receive json :
function gaeQuery(request) {
var url = "/query";
var payload = {'jsondata' : JSON.stringify(request)};
$.post(
url,
payload,
function(response) {
procesResponse(response);
}, // succes response callback
'json', // response contains JSON content, and will be decoded in a js object
{
contentType: "application/json;charset=utf-8", // send JSON content
timeout: 20000,
tryCount: 0,
retryLimit: 3, // max 3 retries
error: function(xhr, textStatus, errorThrown) { // error handling callback
if (textStatus === 'timeout') {
this.tryCount++;
if (this.tryCount <= this.retryLimit) { //try again until retryLimit
$.ajax(this);
return;
}
alert('We have tried ' + this.retryLimit + ' times and it is still not working. We give in. Sorry.');
return;
}
if (xhr.status === 500) { // internal server error
alert('Oops! There seems to be a server problem, please try again later.');
}
else {
alert('Oops! There was a problem, sorry.'); // something went wrong
}
}
}
);
}
OK so I managed to figure this out and thought I will post the answer that worked for me to help anyone looking for this information because the webapp2 docs are not that helpful when it comes to 'getting' posted json data.
on the client side I did the following:
var request = $.ajax({
url: "/some/url/",
type: "POST",
data: JSON.stringify([{someval: val1, someval2:val2, someval3:val3}]),
contentType: "application/json",
dataType: 'json',
beforeSend: function() {
$('#loading-div').show();
},
complete: function(){
$('#loading-div').hide();
},
success: function(response, textStatus, jqXHR){}
});
The reason I couldnt figure out the problem straight away was because of the following line which I deleted along with some commented out line which prevented the page from redirecting after posting. This was the source of all the weird, unrelated and unhelpful error messages:
event.preventDefault();
on the server side to get the json data being posted to appengine do the following:
jdata = json.loads(cgi.escape(self.request.body))
for vals in jdata:
val1 = vals['someval']
val2 = vals['someval2']
val3 = vals['someval3']
The above was the root of the problem I wasn't doing it right and without the previous line on the client side there was no way to figure it out.
Anyways once you have the data do whatever processing you need to do with it and once you are done and need to send back a json response add the following lines:
//the data would look something like this
data = {'return_value': val1, 'return_value2': val2,
'return_value3': val3, 'return_value4': val4}
return_data = json.dumps(data)
self.response.headers.add_header('content-type', 'application/json', charset='utf-8')
self.response.write(return_data)
Almost forgot on the client side again to access the variables sent back from the server with jquery its very straight forward...do something like:
success: function(response, textStatus, jqXHR){
console.log(response.return_value);
console.log(response.return_value2);
console.log(response.return_value3);
}
Hope this will help someone seeking this information.