We Keep Coding

Download file from API server

I need to upload a file to the system from the API server. post method, the server accepts an authorization token and a file id. I tried to do this by writing to a file(.write(responce.content)), but the text of the saved file is: "Unable to get a response from the server."
code:
import requests
def test_download():
headers = {"Authorization": "token"}
data = {"fileId": "43232-31621-51923-31515-59031"}
responce = requests.post("url/api/integration/file/download",
headers=headers, json=data)
with open("download_file.txt", "wb") as code: code.write(responce.content)
print(responce.text)
assert responce.status_code == 200
but I need the text of the file from the server. is it possible to download the file in another way? thank you in advance)

How to send requests to a downloaded html file

I have a .html file downloaded and want to send a request to this file to grab it's content.
However, if I do the following:
import requests
html_file = "/user/some_html.html"
r = requests.get(html_file)
Gives the following error:
Invalid URL 'some_html.html': No schema supplied.
If I add a schema I get the following error:
HTTPConnectionPool(host='some_html.html', port=80): Max retries exceeded with url:
I want to know how to specifically send a request to a html file when it's downloaded.

You are accessing html file from local directory. get() method uses HTTPConnection and port 80 to access data from website not a local directory. To access file from local directory using get() method use Xampp or Wampp.
for accessing file from local directory you can use open() while requests.get() is for accessing file from Port 80 using http Connection in simple word from internet not local directory
import requests
html_file = "/user/some_html.html"
t=open(html_file, "r")
for v in t.readlines():
print(v)
Output:

You don't "send a request to a html file". Instead, you can send a request to a HTTP server on the internet which will return a response with the contents of a html file.
The file itself knows nothing about "requests". If you have the file stored locally and want to do something with it, then you can open it just like any other file.
If you are interested in learning more about the request and response model, I suggest you try a something like
response = requests.get("http://stackoverflow.com")
You should also read about HTTP and requests and responses to better understand how this works.

You can do it by setting up a local server to your html file.
If you use Visual Studio Code, you can install Live Server by Ritwick Dey.
Then you do as follows:
1 - Make the first request and save the html content into a .html file:
my_req.py
import requests
file_path = './'
file_name = 'my_file'
url = "https://www.qwant.com/"
response = requests.request("GET", url)
w = open(file_path + file_name + '.html', 'w')
w.write(response.text)
2 - With Live Server installed on Visual Studio Code, click on my_file.html and then click on Go Live.
and
3 - Now you can make a request to your local http schema:
second request
import requests
url = "http://127.0.0.1:5500/my_file.html"
response = requests.request("GET", url)
print(response.text)
And, tcharan!! do what you need to do.
On a crawler work, I had one situation where there was a difference between the content displayed on the website and the content retrieved with the response.text so the xpaths did not were the same as on the website, so I needed to download the content, making a local html file, and get the new ones xpaths to get the info that I needed.

You can try this:
from requests_html import HTML
with open("htmlfile.html") as htmlfile:
sourcecode = htmlfile.read()
parsedHtml = HTML(html=sourcecode)
print(parsedHtml)

Python Download Zip file from URL with Authentication

First post here so go easy on me. I'm new to Python.
I'm trying to download a .zip file from a website that requires a login to access the link.
In the Zip file is a .csv.
I've tried a few different methods but can't get the file to download.
This doesn't throw any errors but the file doesn't appear.
Sorry i can't give out the password and username.
Am i missing something? Do i need a different way to do it?
Thanks for your help.
UPDATE: still no joy. I managed to do it with VBA in excel but not python. Any ideas?
import requests
from requests.auth import HTTPBasicAuth
theurl= 'https://www.pencarrie.com/export/products.zip'
username = 'Secret'
password = 'Secret'
filename = os.path.basename(urlparse(theurl).path)
r=requests.get(theurl, auth=HTTPBasicAuth(username, password))
I've now switched to a different approach and am still getting the same result. The file that is downloaded is now complete. or is just the zip generated by the write command. the download complete command says its complete.......HELP PLEASE.
import requests
userid = '???????'
password = '??????'
site_url = 'https://www.pencarrie.com/marketing/website-options/data/enhanced-data'
file_url = 'https://www.pencarrie.com/export/products.zip'
o_file = 'products.zip'
# create session
s = requests.Session()
# GET request. This will generate cookie for you
s.get(site_url)
# login to site.
s.post(site_url, data={'_username': userid, '_password': password})
# Next thing will be to visit URL for file you would like to download.
r = s.get(file_url)
# Download file
with open(o_file, 'wb') as output:
output.write(r.content)
print(f"requests:: File {o_file} downloaded successfully!")
# Close session once all work done
s.close()
I have it running with VBA so I assume it's basic authentication but it just downloads html. But I'm not sure.

requests.get does not save whatever it downloads to file, instead it saves it to content field of the Response object.
What you need is to write it to file manually:
f = open(filename, "wb")
f.write(r.content)
f.close()

How to upload a binary/video file using Python http.client PUT method?

I am communicating with an API using HTTP.client in Python 3.6.2.
In order to upload a file it requires a three stage process.
I have managed to talk successfully using POST methods and the server returns data as I expect.
However, the stage that requires the actual file to be uploaded is a PUT method - and I cannot figure out how to syntax the code to include a pointer to the actual file on my storage - the file is an mp4 video file.
Here is a snippet of the code with my noob annotations :)
#define connection as HTTPS and define URL
uploadstep2 = http.client.HTTPSConnection("grabyo-prod.s3-accelerate.amazonaws.com")
#define headers
headers = {
'accept': "application/json",
'content-type': "application/x-www-form-urlencoded"
}
#define the structure of the request and send it.
#Here it is a PUT request to the unique URL as defined above with the correct file and headers.
uploadstep2.request("PUT", myUniqueUploadUrl, body="C:\Test.mp4", headers=headers)
#get the response from the server
uploadstep2response = uploadstep2.getresponse()
#read the data from the response and put to a usable variable
step2responsedata = uploadstep2response.read()
The response I am getting back at this stage is an
"Error 400 Bad Request - Could not obtain the file information."
I am certain this relates to the body="C:\Test.mp4" section of the code.
Can you please advise how I can correctly reference a file within the PUT method?
Thanks in advance

uploadstep2.request("PUT", myUniqueUploadUrl, body="C:\Test.mp4", headers=headers)
will put the actual string "C:\Test.mp4" in the body of your request, not the content of the file named "C:\Test.mp4" as you expect.
You need to open the file, read it's content then pass it as body. Or to stream it, but AFAIK http.client does not support that, and since your file seems to be a video, it is potentially huge and will use plenty of RAM for no good reason.
My suggestion would be to use requests, which is a way better lib to do this kind of things:
import requests
with open(r'C:\Test.mp4'), 'rb') as finput:
response = requests.put('https://grabyo-prod.s3-accelerate.amazonaws.com/youruploadpath', data=finput)
print(response.json())

I do not know if it is useful for you, but you can try to send a POST request with requests module :
import requests
url = ""
data = {'title':'metadata','timeDuration':120}
mp3_f = open('/path/your_file.mp3', 'rb')
files = {'messageFile': mp3_f}
req = requests.post(url, files=files, json=data)
print (req.status_code)
print (req.content)
Hope it helps .

Python 3 script to upload a file to a REST URL (multipart request)

I am fairly new to Python and using Python 3.2. I am trying to write a python script that will pick a file from user machine (such as an image file) and submit it to a server using REST based invocation. The Python script should invoke a REST URL and submit the file when the script is called.
This is similar to multipart POST that is done by browser when uploading a file; but here I want to do it through Python script.
If possible do not want to add any external libraries to Python and would like to keep it fairly simple python script using the core Python install.
Can some one guide me? or share some script example that achieve what I want?

Requests library is what you need. You can install with pip install requests.
http://docs.python-requests.org/en/latest/user/quickstart/#post-a-multipart-encoded-file
>>> url = 'http://httpbin.org/post'
>>> files = {'file': open('report.xls', 'rb')}
>>> r = requests.post(url, files=files)

A RESTful way to upload an image would be to use PUT request if you know what the image url is:
#!/usr/bin/env python3
import http.client
h = http.client.HTTPConnection('example.com')
h.request('PUT', '/file/pic.jpg', open('pic.jpg', 'rb'))
print(h.getresponse().read())
upload_docs.py contains an example how to upload a file as multipart/form-data with basic http authentication. It supports both Python 2.x and Python 3.
You could use also requests to post files as a multipart/form-data:
#!/usr/bin/env python3
import requests
response = requests.post('http://httpbin.org/post',
files={'file': open('filename','rb')})
print(response.content)

You can also use unirest . Sample code
import unirest
# consume async post request
def consumePOSTRequestSync():
params = {'test1':'param1','test2':'param2'}
# we need to pass a dummy variable which is open method
# actually unirest does not provide variable to shift between
# application-x-www-form-urlencoded and
# multipart/form-data
params['dummy'] = open('dummy.txt', 'r')
url = 'http://httpbin.org/post'
headers = {"Accept": "application/json"}
# call get service with headers and params
response = unirest.post(url, headers = headers,params = params)
print "code:"+ str(response.code)
print "******************"
print "headers:"+ str(response.headers)
print "******************"
print "body:"+ str(response.body)
print "******************"
print "raw_body:"+ str(response.raw_body)
# post sync request multipart/form-data
consumePOSTRequestSync()
You can check out this post http://stackandqueue.com/?p=57 for more details