I am trying to convert the following cURL request to a python requests request.
I know the format for a GET request but not sure how to format the --output flag.
curl -u usr:pass -X GET "http://url-to-file.com/filefolder/filepath.txt" --output new_file_name.txt
This is what I have so far but it was returning a 405 Method not allowed error.
r = session.get('http://url-to-file.com/filefolder/filepath.txt')
Related
I have the following curl that successfully logs me into a website. It returns a 302 and a cookie:
curl \
--verbose \
--request POST \
--data '__EVENTTARGET=&body%3Ax%3AtxtLogin=kingkong%40mailinator.com&body%3Ax%3AbtnLogin=&crypted_pass=736015615f9e251692a6a4aa8a7baa14' \
"https://emma.maryland.gov/page.aspx/en/usr/login"
Unfortunately, I have to insert the real username & encrypted password of the account. otherwise this curl it won't work. However, this is a completely dummy account with NO private data in it. So please don't get mad at me.
I want to convert it to a Python3 code using requests.post(). So I made this code:
>>> requests.post(
... url='https://emma.maryland.gov/page.aspx/en/usr/login',
... data={
... '__EVENTTARGET': '',
... 'body:x:txtLogin': 'kingkong#mailinator.com',
... 'body:x:btnLogin': '',
... 'crypted_pass': '736015615f9e251692a6a4aa8a7baa14'
... }
... )
<Response [200]>
But the response I get from the Python3 code (200) doesn't match the response I get from the Curl (302). This means that the target server senses a difference between the two requests.
How can I convert the curl to Python3 that sends the exact same underlying HTTP request?
Your requests code is actually smarter than the cURL command.
HTTP 302 - is a redirect, cURL didn't follow it and gave you the first response it got. You can make cURL follow the redirect with -L: Is there a way to follow redirects with command line cURL?
The requests code followed the redirect and gave you the final response, which happened to be a HTTP 200.
Try your curl command with -L and see if you get HTTP 200 or not.
Alternatively, you can ask requests to not follow redirects with the allow_redirects=False option: Is there an easy way to request a URL in python and NOT follow redirects?
I have simple Django DRF application setup which I have implemented JWT authentication.
I used the Django REST framework JWT documentation
I am using curl to test the implementation.
I can successfully get a token using the following notation used in the documentation:
$ curl -X POST -d "username=admin&password=password123" http://localhost:8000/api-token-auth/
The token is returned in following format:
{"token":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VybmFtZSI6InNpdHJ1Y3AiLCJleHAiOjE1MTE2NTEyMTQsInVzZXJfaWQiOjEsImVtYWlsIjoiY3VydGlzLnBva3JhbnRAZ21haWwuY29tIn0.F1TSkxe5tQVpddetUdOJDdAPP1XB9Bimb5U3c75oWd0"}
However, when I try using this other variation, I get an error:
$ curl -X POST -H "Content-Type: application/json" -d '{"username":"admin","password":"password123"}' http://localhost:8000/api-token-auth/
The error I get is:
{"detail":"JSON parse error - Expecting value: line 1 column 1 (char 0)"}
I also had the same error when trying to refresh or verify the token:
Refresh:
$ curl -X POST -H "Content-Type: application/json" -d '{"token":"<EXISTING_TOKEN>"}' http://localhost:8000/api-token-refresh/
Verify:
$ curl -X POST -H "Content-Type: application/json" -d '{"token":"<EXISTING_TOKEN>"}' http://localhost:8000/api-token-verify/
I was adding the token as follows:
curl -X POST -H "Content-Type: application/json" -d '{"token":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VybmFtZSI6InNpdHJ1Y3AiLCJleHAiOjE1MTE2NDg5MjIsInVzZXJfaWQiOjEsImVtYWlsIjoiY3VydGlzLnBva3JhbnRAZ21haWwuY29tIn0.T5h_PSvzvKOZCPTS60x5IUm3DgAsRCRmbMJeGWZk3Tw"}' http://localhost:8800/api-token-refresh/
Am I perhaps adding the token incorrectly? Does it need some other formatting with quotes?
Those requests are sending data in two different ways. The first request sends it as form data (x-www-form-urlencoded) which is what your endpoint is expecting and the second request sends it as application/json.
I'm not sure that the library you're using will handle a json request out of the box so one option would be to create a custom endpoint and use something like the following:
import json
def ParseFormData(self, request):
payload = json.loads(request.body.decode('utf-8'))
// use django auth to authorize request and return token
You can read more about it in this answer here: https://stackoverflow.com/a/29514222/5443056
There's instructions for manually creating auth tokens in your library's documentation. Here's the code:
from rest_framework_jwt.settings import api_settings
jwt_payload_handler = api_settings.JWT_PAYLOAD_HANDLER
jwt_encode_handler = api_settings.JWT_ENCODE_HANDLER
payload = jwt_payload_handler(user)
token = jwt_encode_handler(payload)
i suggest use json.dumps({key:value})
I'm trying to convert the following curl post to a Python request:
curl -k -i -H "Content-Type: multipart/mixed" -X POST --form
'session.id=e7a29776-5783-49d7-afa0-b0e688096b5e' --form 'ajax=upload'
--form 'file=#myproject.zip;type=application/zip' --form 'project=MyProject;type/plain' https://localhost:8443/manager
I used curl.trillworks.com to make an auto conversion but it didn't work. I also tried the following:
sessionID = e7a29776-5783-49d7-afa0-b0e688096b5e
project = 'file.zip'
metadata = json.dumps({"documentType":"multipart/mixed"})
files = {
'meta' : ('', metadata , 'application/zip'),
'data':(project, 'multipart/mixed', 'application/octet-stream')}
data = {'ajax':'upload','project':'test','session.id':sessionId}
cookie = {'azkaban.browser.session.id':sessionId}
response=requests.post('https://'+azkabanURL+'/manager',
data=data,verify=False,files=files)
print response.text
I got the following error:
<p>Problem accessing /manager. Reason:
<pre> INTERNAL_SERVER_ERROR</pre></p><h3>Caused by:</h3><pre>java.lang.NullPointerException
at azkaban.webapp.servlet.ProjectManagerServlet.ajaxHandleUpload(ProjectManagerServlet.java:1664)
at azkaban.webapp.servlet.ProjectManagerServlet.handleMultiformPost(ProjectManagerServlet.java:183)
at azkaban.webapp.servlet.LoginAbstractAzkabanServlet.doPost(LoginAbstractAzkabanServlet.java:276)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at org.mortbay.jetty.security.SslSocketConnector$SslConnection.run(SslSocketConnector.java:713)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Can't figure out what I'm missing here??
I found the answer after trying out some of the examples from requests site and finally it worked.
data = {'ajax':'upload','project':'test','session.id':sessionId}
files = {'file':('projects.zip',open('projects.zip','rb'),'application/zip')}
response=requests.post('https://'+azkabanURL+'/manager',data=data,verify=False,files=files)
print response.text
print response.status_code
For the future reference I recommend you to check this online tool:
http://curl.trillworks.com/
it converts curl to python requests, node and php.
It has helped my multiple times.
I hope I can explain myself. with out making an arse of myself.
I am trying to use python 3.4 to send a url to a sparkcore api.
I have managed to use curl direcly from the windows command line:-
curl https://api.spark.io/v1/devices/xxxxxxxxxxxxxxx/led -d access_token=yyyyyyyyyyyyyyyy -d params=l1,HIGH
All works fine. there is a space between the led and -d, but that is not a problem.
I have read that reting to do this within python using libcurl is a big pain and I saw lots of messaged about using Requests, so I though I would give it a go.
So I wrote a small routine:
import requests
r = requests.get('https://api.spark.io/v1/devices/xxxxxxxxxxxxxxxxxx/led -d access_token=yyyyyyyyyyyyyyyyy -d params=l1,HIGH')
print(r.url)
print(r)
I get as return:
<Response [400]>
When I examine the URL which actually got sent out the spaces in the URL are replaced with %20. This seems to be my actual problem, because the %20 being added by requests are confusing the server which fails
"code": 400,
"error": "invalid_request",
"error_description": "The access token was not found"
I have tried reading up on how to inpractice have the spaces with out having a %20 being added by the encoding, but I really could do with a pointer in the right direction.
Thanks
Liam
URLs cannot have spaces. The curl command you are using is actually making a request to the url https://api.spark.io/v1/devices/xxxxxxxxxxxxxxx/led with some command line arguments (using -d)
The curl man (manual) page says this about the -d command line argument
-d, --data
(HTTP) Sends the specified data in a POST request to the HTTP server, in the same way that a browser does when a user has filled in an HTML form and presses the submit button. This will cause curl to pass the data to the server using the content-type application/x-www-form-urlencoded. Compare to -F, --form.
-d, --data is the same as --data-ascii. To post data purely binary, you should instead use the --data-binary option. To URL-encode the value of a form field you may use --data-urlencode.
If any of these options is used more than once on the same command line, the data pieces specified will be merged together with a separating &-symbol. Thus, using '-d name=daniel -d skill=lousy' would generate a post chunk that looks like 'name=daniel&skill=lousy'.
If you start the data with the letter #, the rest should be a file name to read the data from, or - if you want curl to read the data from stdin. Multiple files can also be specified. Posting data from a file named 'foobar' would thus be done with --data #foobar. When --data is told to read from a file like that, carriage returns and newlines will be stripped out.
So that says -d is for sending data to the URL with the POST request using the content-type application/x-www-form-urlencoded
The requests documentation has a good example of how to do that using the requests library: http://docs.python-requests.org/en/latest/user/quickstart/#more-complicated-post-requests
So for your curl command, I think this should work
import requests
payload = {'access_token': 'yyyyyyyyyyyyyyyy', 'params': 'l1,HIGH'}
r = requests.post("https://api.spark.io/v1/devices/xxxxxxxxxxxxxxx/led", data=payload)
print(r.text)
I am making a python build script for a phonegap project.
I need to open the ios key before i build
I am trying to do this with a http put request through the requests module for python.
If i do it with cURL from command line, it works fine
curl -vvv -d 'data={"password":"myPassWord"}' -X PUT https://build.phonegap.com/api/v1/keys/ios/193686?auth_token=passwordlesstokenphg
But from python like this.
password_for_key = {'password': 'myPassword'}
authentication_token = {'auth_token': 'passwordlesstokenphg'}
requests.put('https://build.phonegap.com/api/v1/keys/ios/193686', data=password_for_key, params=authentication_token)
It just returns the json you would recieve if you did a cURL without the data.
For me it seems like the data is not being sent to phonegap correctly.
API reference from build.phonegap.com
docs.build.phonegap.com/en_US/2.9.0/developer_api_write.md.html
Please help :)
So when you do
curl -d "..." -X PUT https://example.com
curl sends exactly what's in that string. requests does not translate so directly to curl. To do something similar in requests you need to do the following:
import json
password_for_key = {'password': 'myPassword'}
authentication_token = {'auth_token': 'passwordlesstokenphg'}
requests.put('https://build.phonegap.com/api/v1/keys/ios/193686',
data={'data': json.dumps(password_for_key)},
params=authentication_token)
What requests will do is build data={"password":"myPassword"} for you if you use the above. First you have to JSON encode the data in password_for_key then pass it in the dictionary to data.