I need a file downloader and I tried to write it but I have problems to download files over a https connection. It's easy to download files over http but for the https connection I have a username and password. I usually connect to the website with this line of code in firefox:
https://username:password#site.com/path
I want to download every single file + (sub)folder in there. How can I do this? This is how it looks like when I'm connected in firefox:
http://img543.imageshack.us/img543/6355/52961177.png
http://img713.imageshack.us/img713/624/55225462.png
Do you need Python ? Using a command line tool as curl should to the job.
http://docs.python-requests.org/en/latest/user/quickstart/ is a good library for implementing HTTP clients.
Related
I am creating a python application that will download a Minecraft server JAR file and use it to automatically create and configure a Minecraft server. My goal is to have the software download a Minecraft server JAR to use, but I do not know how to download all versions of Minecraft server software (how to format a download URL to do so) and how to do that with Python. TLDR: I need to know how to specify in the URL which version of Minecraft software to download.
This will download and store the Minecraft 1.9 server in a Jar file though this link is subject to change, which means you may want to look into hosting the server binaries yourself to get a permanent URL that wont change as Mojang changes the Minecraft website.
import requests
URL = "https://piston-data.mojang.com/v1/objects/f69c284232d7c7580bd89a5a4931c3581eae1378/server.jar"
response = requests.get(URL)
with open("server.jar", "wb") as file:
file.write(response.content)
I want to make a python script that would run a server which I can then send a request to from another pc using curl to send a file. So I would send a file from a pc to the server using curl and then server would save that file.
I tried googling but couldn't seem to find anything like that, but I found that pycurl might be useful. If anyone could show me the direction towards my goal I would appreciate it.
Sending files is typically just a POST with data. You can make a webserver using something such as flask, and make an endpoint that receives post data, then saves it as a file.
On the server, you don't need curl at all, just an HTTP server that accepts requests.
You can use http.server in the standard library...
https://gist.github.com/mdonkers/63e115cc0c79b4f6b8b3a6b797e485c7
The server is mine, I will run the python script on my aws machine.
And the pc is windows 10, I would want to just use curl to send the
file which is pre installed on windows so that I don't have to install
anything or write any scripts. Purely send the file using curl from cmd
Sending the file via curl is likely to be problematic. When I Google curl upload file, I usually find people that have questions that were not answered or advice that does not work. Without writing a script you easy option is to use FTP. Curl would not be required on the Server. Either you write a simple script, or use FTP. The more difficult is on the PC side. Not all that difficult because you control both sides of the transfer.
Have you considered using a command line SFTP client? A secure FTP transfer is very easy using a CMD prompt. One single and simple CMD statement.
I did a web app for a laboratory. Every day PDFs with patient lab results are uploaded to a doctor portal. The docs then go to the portal to view the PDFs. I first wrote this web app 10 years ago. It has been very trouble free.
I was just beginning my answer, gathering some options when I got you message.
Let me know if FTP is a viable option. ASAP. I will get you the command line to FTP a file to the server.
pscp is a free SSH FTP file transfer utility. I have been using this for over 10 years. Very reliable. The date on my pscp.exe, a 316K byte file, is September 2013.
Putty also has another CMD utility psftp. I used it a few times. psftp has a scripting protocol.
Both Putty utilities are very popular and well documented.
This is a sample pcsp CMD. The RUN is the command from my Window app programming language as well.
RUN('CMD /c ECHO ON & pscp -v -p -pw $password $filename user#domain.com:server path & EXIT')
I went to Fillzilla Pro CLI (command line interface) Even though Filezilla CLI is poorly documented, Filezilla Pro supports transfers to and from These storage services:
Amazon S3
Backblaze B2
Box
Dropbox
Google Cloud Storage
Google Drive
Microsoft Azure File Storage Service
Microsoft Azure Blob Storage Service
Microsoft OneDrive
Microsoft OneDrive for Business
Microsoft SharePoint
OpenStack Swift
Rackspace Cloud
WebDAV
I would recommend the free psftp utility from Putty
The sftp.exe utility is a stand alone 734 KB file. No installation required just copy sftp.exe to any PC where you need it.
If FTP is an option.
Here is a link to the docs for psftp
These are the FTP commands it supports:
6.2.4 The quit command: end your session
6.2.5 The close command: close your connection
6.2.6 The help command: get quick online help
6.2.7 The cd and pwd commands: changing the remote working directory
6.2.8 The lcd and lpwd commands: changing the local working directory
6.2.9 The get command: fetch a file from the server
6.2.10 The put command: send a file to the server
6.2.11 The mget and mput commands: fetch or send multiple files
6.2.12 The reget and reput commands: resuming file transfers
6.2.13 The dir command: list remote files
6.2.14 The chmod command: change permissions on remote files
6.2.15 The del command: delete remote files
6.2.16 The mkdir command: create remote directories
6.2.17 The rmdir command: remove remote directories
6.2.18 The mv command: move and rename remote files
Content-Type: application/octet-stream
Content-Disposition: attachment
Content-Disposition: form-data; name="fieldName"; filename="filename.jpg"
Content-Disposition: form-data; name="fieldName"
<input type="file">
I created an application in Flask Python with a frontend in HTML and Twilio SDK JavaScript version 2.3.0.
A video call is working on local host, but if I try the same app over HTTPS with a valid certificate, it is fetching the token correctly for the video call, but it can't establish a connection to Twilio.
I have tried forcing the browser permissions for the camera and audio to allow and it did not make a difference.
So to load the Twilio script, you need to enable https in both the website and the script. Otherwise, it will mix up HTTP and HTTPS and fail. So there are two simple things to do:
put your website over HTTPS
load the https://media.twiliocdn.com/sdk/js/video/releases/2.3.0/twilio-video.min.js file over HTTPS in file index.html.
This will solve your problem.
I need to implement a project to upload/download file from/to localhost, by Python, from the command line. But the uploaded files need to be viewable from the browser.
Basically i know i need to have a client, a server, and an endpoint(http://localhost).
(1)upload:
From the client side(command line), i send the file through python request package by http requests. The server side will receive this file and parse this file to get the information in the file. I need to see the uploaded file from the browser.
(2)download:
from the command line of the client side, i ask for the file through http request. Then the request will be parsed by server. Then the file will be saved locally to my host machine.
(3)i know how to use the Python request package.
Question: what do i need to work on the server side and client side?
I read through the similar posts for this question, and they are not helpful for my question.
So I have looked around for a while to try to find a answer for this but has had little luck. My issue is I am trying to build a python script to connect to a web server that is running on HTTPS, however it uses only PEM files to authenticate. A lot of the examples I have found show using PEM files with a username and password.
The website I am referring to is a Verizon SSL certificate manager website. It's how my company (cant name) manages there external SSL certs. My idea is to build a script to connect to the website and then take a list of SSL certs that are expiring and parse through it and email the SSL tech contact and let them know its expiring.
The part I am getting stuck on is the connecting to the website using only the PEM file. I was wondering what would be the best way of doing that ?