I am using Python 2.7. My Rest server side API works fine and I am able to upload a zip file using Postman. I am trying to upload a zip file using Rest client api. I tried requests package, but it is unable to send the files. I am getting an error : missing file argument.
This is my python server side code :
#ns.route('/upload_file', strict_slashes=False)
class Upload(Resource):
#api.expect(upload_parser)
def post(self):
args = upload_parser.parse_args()
file_nameup = args.file.filename
Here is the rest api client code :
import requests
from requests.auth import HTTPBasicAuth
import json
headers={'Username': 'abc#gmail.com', 'apikey':'123-e01b', 'Content-Type':'application/zip'}
f = open('C:/Users/ADMIN/Downloads/abc.zip', 'rb')
files = {"file": f}
resp = requests.post("https://.../analytics/upload_file", files=files, headers=headers )
print resp.text
print "status code " + str(resp.status_code)
if resp.status_code == 200:
print ("Success")
print resp.json()
else:
print ("Failure")
This is my error :
{"message":"Input payload validation failed","errors":{"file":"Missing required
parameter in an uploaded file"}
status code 400
Failure
In postman, I passed a zip file with in body with "file" as key and value as abc.zip file. It worked fine. I tried to use httplib library, but it fails as my post url does not contain port number. This the error with httplib :
python HttpClientEx.py
Traceback (most recent call last):
File "HttpClientEx.py", line 4, in
h = http.client.HTTPConnection(url)
File "c:\python27\Lib\httplib.py", line 736, in init
(self.host, self.port) = self._get_hostport(host, port)
File "c:\python27\Lib\httplib.py", line 777, in _get_hostport
raise InvalidURL("nonnumeric port: '%s'" % host[i+1:])
httplib.InvalidURL: nonnumeric port: '// ....net/analytics/upload_file'
How to invoke rest url post and upload a file using urllib library. Please suggest any other ways to upload file in rest client. Thanks.
I found another duplicate post :
Python Requests - Post a zip file with multipart/form-data
The solution mentioned there did not work. I found that you need to provide the full path of the file, otherwise it will not work.
Use urllib3 module.
https://urllib3.readthedocs.io/en/latest/user-guide.html
Files & binary data
For uploading files using multipart/form-data encoding you can use the same approach as Form data and specify the file field as a tuple of (file_name, file_data):
with open('example.txt') as fp:
file_data = fp.read()
r = http.request(
'POST',
'http://httpbin.org/post',
fields={
'filefield': ('example.txt', file_data),
})
json.loads(r.data.decode('utf-8'))['files']
requests library worked with below changes in my code :
import requests
from requests.auth import HTTPBasicAuth
import json
from pathlib import Path
file_ids = ''
headers={'Username': 'abc#gmail.com', 'apikey':'123-456'}
# Upload file
f = open('C:/Users/ADMIN/Downloads/abc.zip', 'rb')
files = {"file": ("C:/Users/ADMIN/Downloads/abc.zip", f)}
resp = requests.post("https:// ../analytics/upload_file", files=files, headers=headers )
print resp.text
print "status code " + str(resp.status_code)
if resp.status_code == 201:
print ("Success")
data = json.loads(resp.text)
file_ids = data['file_ids']
print file_ids
else:
print ("Failure")
Related
I need to upload a file to the system from the API server. post method, the server accepts an authorization token and a file id. I tried to do this by writing to a file(.write(responce.content)), but the text of the saved file is: "Unable to get a response from the server."
code:
import requests
def test_download():
headers = {"Authorization": "token"}
data = {"fileId": "43232-31621-51923-31515-59031"}
responce = requests.post("url/api/integration/file/download",
headers=headers, json=data)
with open("download_file.txt", "wb") as code: code.write(responce.content)
print(responce.text)
assert responce.status_code == 200
but I need the text of the file from the server. is it possible to download the file in another way? thank you in advance)
I want to download the file, it may be zip/7z. when I used the following code it is giving an error for the 7z file.
import requests, zipfile, StringIO
zip_file_url = "http://www.blog.pythonlibrary.org/wp-content/uploads/2012/06/wxDbViewer.zip"
try:
r = requests.get(zip_file_url, stream=True)
z = zipfile.ZipFile(StringIO.StringIO(r.content))
except requests.exceptions.ConnectionError:
print "Connection refused"
Just ensure the HTTP status code is 200 when requesting the file, and write out the file in binary mode:
import os
import requests
URL = "http://www.blog.pythonlibrary.org/wp-content/uploads/2012/06/wxDbViewer.zip"
filename = os.path.basename(URL)
response = requests.get(URL, stream=True)
if response.status_code == 200:
with open(filename, 'wb') as out:
out.write(response.content)
else:
print('Request failed: %d' % response.status_code)
The downloaded file will then appear in the directory where the script is being run if the request was successful, or an indication the file could not be downloaded.
I need to automate download of reports from serviceNow.
I've been able to automate it using python and selenium and win32com by following method.
https://test.service-now.com/sys_report_template.do?CSV&jvar_report_id=92a....7aa
And using selenium to access serviceNow as well as modify firefox default download option to dump the file to a folder on windows machine.
However, Since all of this may be ported to a linux server , we would like to port it to SOAP or CURL.
I came across serviceNow libraries for python here.
I tried it out and following code is working if I set login , password and instance-name as listed at the site using following from ServiceNow.py
class Change(Base):
__table__ = 'change_request.do'
and following within clientside script as listed on site.
# Fetch changes updated on the last 5 minutes
changes = chg.last_updated(minutes=5)
#print changes client side script.
for eachline in changes:
print eachline
However, When I replace URL with sys_report_template.do, I am getting error
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\SOAPpy\Parser.py", line 1080, in _parseSOAP
parser.parse(inpsrc)
File "C:\Python27\Lib\xml\sax\expatreader.py", line 107, in parse
xmlreader.IncrementalParser.parse(self, source)
File "C:\Python27\Lib\xml\sax\xmlreader.py", line 125, in parse
self.close()
File "C:\Python27\Lib\xml\sax\expatreader.py", line 220, in close
self.feed("", isFinal = 1)
File "C:\Python27\Lib\xml\sax\expatreader.py", line 214, in feed
self._err_handler.fatalError(exc)
File "C:\Python27\Lib\xml\sax\handler.py", line 38, in fatalError
raise exception
SAXParseException: <unknown>:1:0: no element found
Here is relevent code
from servicenow import ServiceNow
from servicenow import Connection
from servicenow.drivers import SOAP
# For SOAP connection
conn = SOAP.Auth(username='abc', password='def', instance='test')
rpt = ServiceNow.Base(conn)
rpt.__table__ = "sys_report_template.do?CSV"
#jvar_report_id replaced with .... to protect confidentiality
report = rpt.fetch_one({'jvar_report_id': '92a6760a......aas'})
for eachline in report:
print eachline
So, my question is , what can be done to make this work?
I looked on web for resources and help, but didn't find any.
Any help is appreciated.
After much research I was able to use following method to get report in csv format from servicenow. I thought I will post over here in case anyone else runs into similar issue.
import requests
import json
# Set the request parameters
url= 'https://myinstance.service-now.com/sys_report_template.do?CSV&jvar_report_id=929xxxxxxxxxxxxxxxxxxxx0c755'
user = 'my_username'
pwd = 'my_password'
# Set proper headers
headers = {"Accept":"application/json"}
# Do the HTTP request
response = requests.get(url, auth=(user, pwd), headers=headers )
response.raise_for_status()
print response.text
response.text now has report in csv format.
I need to next figure out, how to parse the response object to extract csv data in correct format.
Once done, I will post over here. But for now this answers my question.
I tried this and its working as expected.
`import requests
import json
url= 'https://myinstance.service-now.com/sys_report_template.do?CSV&jvar_report_id=929xxxxxxxxxxxxxxxxxxxx0c755'
user = 'my_username'
pwd = 'my_password'
response = requests.get(url, auth=(user, pwd), headers=headers )
file_name = "abc.csv"
with open(file_name, 'wb') as out_file:
out_file.write(response.content)
del response`
I have been following this tutorial to download some page info from fb.
I am using Python 3.5 and the tutorial is using python2.
I was having some issues at first with a HTTP error code 400, basically saying I had to use the https protocol instead of http. So I have test in idle now that the data in coming and it looks to a novice like me as JSON. But when I try passing it to the json.loads it is giving this error
Traceback (most recent call last):
File "C:\Users\Levo\Desktop\facebookscrape.py", line 38, in <module>
testFacebookPageData(page_id, access_token)
File "C:\Users\Levo\Desktop\facebookscrape.py", line 34, in testFacebookPageData
data = json.loads(requests_until_succeed(url))
File "C:\Users\Levo\AppData\Local\Programs\Python\Python35\lib\json\__init__.py", line 312, in loads
s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'
Apart from using the urllib library instead of the urllib2 library and https instead of http I don't what I am doing wrong? is the encryption of https the problem??
def requests_until_succeed(url):
req = urllib.request.Request(url)
success = False
while success is False:
try:
response = urllib.request.urlopen(req)
if response.getcode() == 200:
success = True
except Exception as e:
print(e)
time.sleep(5)
print ("Error for URL %s: %s" % (url, datetime.datetime.now()))
return response.read()
def testFacebookPageData(page_id, access_token):
base = "https://graph.facebook.com/v2.6"
node = "/" + page_id + "/feed"
parameters = "/?access_token=%s" % access_token
url = base + node + parameters
data = json.loads(requests_until_succeed(url))
print(json.dumps(data, indent = 4, sort_keys=True))
testFacebookPageData(page_id, access_token)
json.loads accepts python3 string, which is unicode, and responce.read() returns binary string.
Use data = json.loads(requests_until_succeed(url).decode('utf-8')) because responce is most likely utf-8.
I have a REST PUT request to upload a file using the Django REST framework. Whenever I am uploading a file using the Postman REST client it works fine:
But when I try to do this with my code:
import requests
API_URL = "http://123.316.118.92:8888/api/"
API_TOKEN = "1682b28041de357d81ea81db6a228c823ad52967"
URL = API_URL + 'configuration/configlet/31'
#files = {
files = {'file': open('configlet.txt','rb')}
print URL
print "Update Url ==-------------------"
headers = {'Content-Type' : 'text/plain','Authorization':API_TOKEN}
resp = requests.put(URL,files=files,headers = headers)
print resp.text
print resp.status_code
I am getting an error on the server side:
MultiValueDictKeyError at /api/configuration/31/
"'file'"
I am passing file as key but still getting above error please do let me know what might I am doing wrong here.
This is how my Django server view looks
def put(self, request,id,format=None):
configlet = self.get_object(id)
configlet.config_path.delete(save=False)
file_obj = request.FILES['file']
configlet.config_path = file_obj
file_content = file_obj.read()
params = parse_file(file_content)
configlet.parameters = json.dumps(params)
logger.debug("File content: "+str(file_content))
configlet.save()
For this to work you need to send a multipart/form-data body. You should not be setting the content-type of the whole request to text/plain here; set only the mime-type of the one part:
files = {'file': ('configlet.txt', open('configlet.txt','rb'), 'text/plain')}
headers = {'Authorization': API_TOKEN}
resp = requests.put(URL, files=files, headers=headers)
This leaves setting the Content-Type header for the request as a whole to the library, and using files sets that to multipart/form-data for you.