We Keep Coding

Call API in python to run Jenkins Job - problem with authentication

I have got a jenkins job which i can run with making a post request:
curl -u albert405:{mytoken} http://172.31.32.33:8080/job/URL_Job_Trigger/build?token=ozSVoEQfLg
Could you tell me how to place this authentication (albert405:{mytoken}) into my python script:
import requests
url = 'http://172.31.32.33:8080/job/URL_Job_Trigger/build?token=ozSVoEQfLg'
x = requests.post(url)
print(x.text)

I have managed to solve it by this :
http://YOUR_JENKINS_USER_ID:YOUR_API_TOKEN#YOUR_JENKINS_URL/job/YOUR_JENKINS_JOB/build
import requests
build = requests.post("http://YOUR_JENKINS_USER_ID:YOUR_API_TOKEN#YOUR_JENKINS_URL/job/YOUR_JENKINS_JOB/build?token=TokenName")

Where is your authentication code? I see none of it.
Jenkins uses Basic Auth, which is indicated here https://wiki.jenkins.io/display/JENKINS/Remote+access+API.
In order to send auth parameters with requests it's a simple:
res =requests.post(url, auth=("albert405", "password"))
Which is the first documentation you get when googling basic auth requests: https://2.python-requests.org/en/master/user/authentication/

Django service on gunicorn POST request is recieved as GET?

I have a Django rest service running on virutal environment on gunicorn server with the following .wsgi file:
import os, sys import site
site.addsitedir('/opt/valuation/env/lib/python2.7/site-packages')
sys.stdout = sys.stderr
os.environ['DJANGO_SETTINGS_MODULE'] = 'valuation.valuationcont.valuation.settings'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
When I do curl POST call the service works perfectly:
curl -H "Content-Type: application/json" -X POST -d '{...}' -u username:password http://localhost:8000/valuation/predict/
But when I do the same request on API gateway using axios, Django service responds my custom GET response ("GET not supported, try POST").
axios({
method: 'post',
url:'http://localhost:8000/valuation/predict',
headers:{
"Content-Type":"application/json",
"Authorization":"Basic [BASE64 ENCODING]"
},
data:{
...
}
}).then(response=>{
console.log(response.data)
}).catch(err=>{
console.log(err.toString())
})
The request is transformed from GET to POST.
This only happens with the django/gunicorn service.
Since I am new to django/gunicorn I think there is something wrong with the .wsgi file. But how come the curl call then works?
Any help appreciated, been struggling with this for a week now.
Edit:
Managed to recreate the same problem in my local machine. axios POST requests using its API are translated into GET.
Using the axios.post(...) method I managed to get 403 and 201. All while POSTMAN works fine.
I have a suspicion that since the POST fails axios API has a default fallback to GET which then doesn't fail and service responds normally ("GET not supported" as is should).
New step to debug this would be to ask, how do I recreate POSTMAN POST call as close as possible in javascript since POSTMAN is working and it is obviously axios that is causing the problems.

You're not using the same URL. In the curl snippet you request http://localhost:8000/valuation/predict/ but in the second you request http://localhost:8000/valuation/predict - without the final slash.
Django by default redirects URLs that don't end in a slash to one that does, and a redirect is always a GET.

Run a curl tlsv1.2 http get request in python?

I have the following command that I run using curl in linux.
curl --tlsv1.2 --cert ~/aws-iot/certs/certificate.pem.crt --key ~/aws-iot/certs/private.pem.key --cacert ~/aws-iot/certs/root-CA.crt -X GET https://data.iot.us-east-1.amazonaws.com:8443/things/pi_3/shadow
This command returns JSON text that I want. However I want to be able to run the above command in Python3. I do not know what library to use in order to get the same JSON response.
P.S. I replace "data" with my account number in AWS to get JSON

After playing around with it on my own I was able to successfully do it in python using the requests library.
import requests
s = requests.Session()
r = s.get('https://data.iot.us-east-1.amazonaws.com:8443/things/pi_3/shadow',
cert=('/home/pi/aws-iot/certs/certificate.pem.crt', '/home/pi/aws-iot/certs/private.pem.key', '/home/pi/aws-iot/certs/root-CA.crt'))
print(r.text)

Simple not-authorized request to a Github API using Tornado httpclient returns Forbidden

I am trying to make a HTTP Request to a JSON API like https://api.github.com using tornado.httpclient and I found that it always responses with FORBIDDEN 403.
Simplifying, I make a request using the CLI with:
$ python -m tornado.httpclient https://api.github.com
getting a tornado.httpclient.HTTPError: HTTP 403: Forbidden.
In other hand, if I try to request this URL via browser or a simple $ curl https://api.github.com, the response is 200 OK and the proper JSON file.
What is causing this? Should I set some specific Headers on the tornado.httpclient request? What's the difference with a curl request?

You have to put user agent in the request, see Github API for more details:
All API requests MUST include a valid User-Agent header. Requests with
no User-Agent header will be rejected. We request that you use your
GitHub username, or the name of your application, for the User-Agent
header value. This allows us to contact you if there are problems

It might be an issue with their robots.txt. Maybe tornado.httpclient modifies the User-Agent in a way that makes it appear to be a web crawler? I'm not that familiar with it.

I faced similar issue & problem was with configurable-http-proxy so I killed its process & restarted jupyterhub
ps aux | grep configurable-http-proxy
if there are any pid's from above command, kill them with
kill -9 <PID>
and restart ``

Sending data in a POST message to a RESTful web service

I need to send some JSON data in a POST message to a RESTful webservice.
Which python module should I be using for this? Is there some sample code I can refer to?

Which bit are you having trouble with? The JSON, or the POST?
For JSON, the json module has been included in Python since version 2.5. Just do json.dumps(my_data) to convert a data variable to JSON.
For the POST, there are various modules in the standard library, but the best bet is to install the third-party requests library.

Requests is probably the best library for the job. It certainly beats urllib and urllib2. You can get it and see an example at http://pypi.python.org/pypi/requests
or you can just install it with "pip install requests"
There's a few more examples using the Github API with both the requests library and others at https://github.com/issackelly/Consuming-Web-APIs-with-Python-Talk

here is what I've used for post and get requests
import httplib
connection = httplib.HTTPConnection('192.168.38.38:6543')
body_content = 'abcd123456xyz'
connection.request('POST', '/foo/bar/baa.html', body_content)
postResult = connection.getresponse()
connection.request('GET', '/foo/bar/baa.html')
response = connection.getresponse()
getResult = response.read()
It does same as this sequence of CLI commands:
curl -X POST -d "abcd123456xyz" 192.168.38.38:6543/foo/bar/baa.html
curl 192.168.38.38:6543/foo/bar/baa.html