We Keep Coding

Deploying/Updating Marathon Docker image using API

I am writing code to update deployed image on marathon automatically. I use the REST patch method as listed in marathon API reference http://mesosphere.github.io/marathon/api-console/index.html
url = 'https://<my-hostname>:<my-port>/v2/apps'
h = {'Content-type': 'application/json', 'Accept': 'application/json'}
data = {'app': { "id": app,
'container': {
'docker': {
'image': image}}}}
print ('requests.patch(%s, %s)' % (url + app, json.dumps(data)))
r = requests.patch(url + app, headers = h, auth = auth, data = json.dumps(data))
if r.status_code == 200:
print('Deployed %s' % app)
The code ran successfully, I got back a Deployment ID, but nothing was changed from the UI. No new deployment is happening. If I change the patch request into get request without the data, I get back the image I previous updated in using code above.
According to this similar API reference https://docs.mesosphere.com/1.11/deploying-services/marathon-api/#/apps/V2AppsByAppId1
It says " This operation will create a deployment" but nothing happened. From the Marathon GUI, I don't see the configuration getting changed at all. If I restart, it is the same old deployment getting restarted. Am I interpreting the API reference incorrectly?

If I read the API reference guide correctly, the body should be:
{ "id": app,
"container": {
"docker": {
"image": image
}
}
}
Tested this with marathon 1.4.11 and that worked.
Not sure why you would get a deploymentid, if I do it the way you did (with the extra {"app":} layer), I get a 500 error. BTW, I am not sure how sensitive this is for single versus double quotes.

There is open bug in Marathon which cause that request body is not merged with current definition. Therefore, PUT/PATCH request with only .container.docker.image will remove settings like port mappings, volumes, parameters...
Solution is to get container object from current application and replace image key with new Docker image.
Example:
export MARATHON_IMAGE="registry.foo.bar/app:v10"
export BODY=$(curl -s -H "Content-type: application/json" leader.mesos:8080/v2/apps/foo/bar | jq -cr '.app.container | .docker.image = env.MARATHON_IMAGE | {"container": .}')
curl -s -H "Content-type: application/json" -X PATCH -d "${BODY}" leader.mesos:8080/v2/apps/foo/bar

python google geolocation api using wifi mac

I'm trying to use Google's API for geolocation giving wifi data to determine location. This is their intro. And this is my code
#author: Keith
"""
import requests
payload = {
"considerIp": "false",
"wifiAccessPoints": [
{
"macAddress": "00:25:9c:cf:1c:ac",
"signalStrength": -43,
"signalToNoiseRatio": 0
},
{
"macAddress": "00:25:9c:cf:1c:ad",
"signalStrength": -55,
"signalToNoiseRatio": 0
}
],
'key':'<MyAPIKey>'
}
r = requests.post('https://www.googleapis.com/geolocation/v1/geolocate',
params=payload)
print(r.text)
This is the output
{
"location": {
"lat": 32.3643098,
"lng": -88.703656
},
"accuracy": 6061.0
}
The request ignored all of the payload except the key portion and just found the geolocation using my IP address. So I'm sending the json payload incorrectly. I know this is probably really simple, but I'm stuck and couldn't find an example of python being used with requests to do this type of API query. Thanks
Edit:
Picked up the cURL library and executed this command with success:
curl -d #your_filename.json -H "Content-Type: application/json" -i "https://www.googleapis.com/geolocation/v1/geolocate?key=<myapikey>"
and got the output I expected. I just want to be able to do the same thing in requests, but the data I'm trying to send is in "your_filename.json".

Please try the following:
r = requests.post('https://www.googleapis.com/geolocation/v1/geolocate?key='+ '<MyAPIKey>', json=payload)
Note the key was moved to query params (URL) and json argument was used in place of params.

Okay I figured out the error of my ways. Taking a better look at requests.post function, I see that I'm not using the parameters argument correctly. After this, it worked perfectly,
r = requests.post('https://www.googleapis.com/geolocation/v1/geolocate', params=parameters, json=mydata, headers=headers)

Translate this request from Bash to Python?

I was given a request in Bash and I have to translate it to Python 2.7. I did this kind of translations several times, but now I am not able to make it work and I do not understand why.
First of all, I was given this Bash request:
curl -X POST -v -u user#domain:password --data "#file.json" -H "Content-Type:application/json" http://destination_url_a
With the file file.json, whose content is the following one:
{
"username":"user#domain",
"password":"password",
"shortName":"a-short-name",
"visibility":"PRIVATE",
"sitePreset":"site-dashboard",
"title":"A Title",
"description":"A description."
}
If I execute the Bash line in my computer, the result is succesful.
As always, I tried to use requests library in Python to make it work. What I did is:
import requests
from requests.auth import HTTPBasicAuth
import json
data = {
"username": "user#domain",
"password": "password",
"shortName": "a-short-name",
"visibility": "PRIVATE",
"sitePreset": "site-dashboard",
"title": "A Title",
"description": "A description.",
}
headers = {'Content-Type': 'application/json'}
data_json = json.dumps(data)
r = requests.post(
url='http://destination_url_a',
data=data_json,
headers=headers,
auth=HTTPBasicAuth('user#domain', 'password'),
verify=False,
)
Unfortunately, the response, stored in r variable, is an error, despite the status code is 200.
What could be happening? Does anyone find a problem in my code or has any idea?
EDIT
However, this is another example very similar which worked perfectly:
Bash:
curl -v -H "Content-Type:application/json" -X POST --data "#file.json" -u user#domain:password http://destination_url_b
My Python code
import requests
from requests.auth import HTTPBasicAuth
import json
data = {
"userName": "user#domain",
"password": "password",
"firstName": "Firstname",
"lastName": "Lastname",
"email": "email#domain.com",
"disableAccount": "False",
"quota": -1,
"groups": ["a_group",],
}
headers = {'Content-Type': 'application/json'}
data_json = json.dumps(data)
r = requests.post(
url='http://destination_url_b',
data=data_json,
headers=headers,
auth=HTTPBasicAuth('user#domain', 'password'),
verify=False,
)
It seems to be almost the same to the other request, but this works. Different data is sent, and to a different subdomain (both are sent to the same domain). Will these modifications be important if we are talking about the User-Agent you mentioned?

Sometimes, webservices filter on user-agent. User agent of curl and python are different. That may explain.
Try to forge a "legitimate" user-agent by modifying the request header.

Finally, there was no error in the code neither in the User-Agent.
The problem was that the destination application did not accept sites with the same short-name value.
What I was doing is creating the site from Bash, which worked, then removing it from the interface of the app and trying to create it from Python with the same data. I was getting error when doing that because in spite of having removed the site, I had to remove some residual data of it from the trashcan of the app too. If not, app returned an error since it considered that the site I was trying to create already existed.
So if I had introduced different short-name in each attempt, I would not have had any error.

How to fetch CloudFlare API dictionary in Paython

I was browsing around for 1 day already, but I was not able to find how to fetch CloudFlare API responses in Python.
Usually the communication between the host and the CloudFlare's servers is established using cURL:
curl -k -s "https://api.cloudflare.com/host-gw.html" -d 'act=user_lookup' -d 'host_key=8ayf9yaoh2h8faosh2o8dhas' -d 'cloudflare_email=admin#tech-world.info' | python -mjson.tool
So this is the cURL request, which returns the following dictionary:
{
"msg": null,
"request": {
"act": "user_lookup",
"cloudflare_email": "admin#tech-world.info"
},
"response": {
"cloudflare_email": "admin#tech-world.info",
"hosted_zones": [
"tech-world.info"
],
"unique_id": "78yastaig238w9ayfhao8h28hdas",
"user_api_key": "ga7srgfr2i7dfagsd2r89hasd",
"user_authed": true,
"user_exists": true,
"user_key": "d8ha92hiuhda7892houkljhadskju"
},
"result": "success"
}
So I am trying to execute the cURL command from python in order to get the dictionary output (this is just placeholder cURL execution exapmle):
import subprocess
my_var = subprocess.call(["curl", "-X", "GET", "-I", "http://www.tech-world.com"])
But the call() method prints the data it receives, but I want to assign the dict to variable for later use.
I have checked the urllib2, but I did not find a way to use it to accomplish my goals.

Why do you use cURL instead of normal Python HTTP library (such as Requests)?
r = requests.post('https://api.cloudflare.com/host-gw.html', data={'host_key': '8ayf9yaoh2h8faosh2o8dhas', 'cloudflare_email': 'admin#tech-world.info', 'act': 'user_lookup'})
Also there is a library named pyflare for working with CloudFlare using Python.
(it's the first text that I post for urgent warning)
I can't comment (too low reputation), but I can't to not alert that's you have opened your credentials from production server.

Curl POST request into pycurl code

I'm trying to convert following curl request into pycurl:
curl -v
-H Accept:application/json \
-H Content-Type:application/json \
-d "{
name: 'abc',
path: 'def',
target: [ 'ghi' ]
}" \
-X POST http://some-url
I have following python code:
import pycurl, json
c = pycurl.Curl()
c.setopt(pycurl.URL, 'http://some-url')
c.setopt(pycurl.HTTPHEADER, ['Accept: application/json'])
data = json.dumps({"name": "abc", "path": "def", "target": "ghi"})
c.setopt(pycurl.POST, 1)
c.setopt(pycurl.POSTFIELDS, data)
c.setopt(pycurl.VERBOSE, 1)
c.perform()
print curl_agent.getinfo(pycurl.RESPONSE_CODE)
c.close()
Executing this I had an error 415: Unsupported media type, so I have changed:
c.setopt(pycurl.HTTPHEADER, ['Accept: application/json'])
into:
c.setopt(pycurl.HTTPHEADER, [ 'Content-Type: application/json' , 'Accept: application/json'])
This time I have 400: Bad request. But bash code with curl works. Do you have any idea what should I fix in python code?

PycURL is a wrapper on the libcurl library written in C language so its Python API can be bit puzzling. As some people are advocating use of python requests instead I just want to point out that it isn't a perfect replacement. For me, its lack of DNS resolution timeout was a deal breaker. I also find it much slower on my Raspberry Pi. This comparison may be relevant:
Python Requests vs PyCurl Performance
So here's something that doesn't evade OP's question:
import pycurl
import json
from cStringIO import StringIO
curl = pycurl.Curl()
curl.setopt(pycurl.URL, 'http://some-url')
curl.setopt(pycurl.HTTPHEADER, ['Accept: application/json',
'Content-Type: application/json'])
curl.setopt(pycurl.POST, 1)
# If you want to set a total timeout, say, 3 seconds
curl.setopt(pycurl.TIMEOUT_MS, 3000)
## depending on whether you want to print details on stdout, uncomment either
# curl.setopt(pycurl.VERBOSE, 1) # to print entire request flow
## or
# curl.setopt(pycurl.WRITEFUNCTION, lambda x: None) # to keep stdout clean
# preparing body the way pycurl.READDATA wants it
# NOTE: you may reuse curl object setup at this point
# if sending POST repeatedly to the url. It will reuse
# the connection.
body_as_dict = {"name": "abc", "path": "def", "target": "ghi"}
body_as_json_string = json.dumps(body_as_dict) # dict to json
body_as_file_object = StringIO(body_as_json_string)
# prepare and send. See also: pycurl.READFUNCTION to pass function instead
curl.setopt(pycurl.READDATA, body_as_file_object)
curl.setopt(pycurl.POSTFIELDSIZE, len(body_as_json_string))
curl.perform()
# you may want to check HTTP response code, e.g.
status_code = curl.getinfo(pycurl.RESPONSE_CODE)
if status_code != 200:
print "Aww Snap :( Server returned HTTP status code {}".format(status_code)
# don't forget to release connection when finished
curl.close()
There are some more interesting features worth checking out in the libcurl curleasy setopts documentation

In your bash example, the property target is an array, in your Python example it is a string.
Try this:
data = json.dumps({"name": "abc", "path": "def", "target": ["ghi"]})
I also strongly advise you to check out the requests library which has a much nicer API:
import requests
data = {"name": "abc", "path": "def", "target": ["ghi"]}
response = requests.post('http://some-url', json=data)
print response.status_code

I know this is over a year old now, but please try removing the whitespace in your header value.
c.setopt(pycurl.HTTPHEADER, ['Accept:application/json'])
I also prefer using the requests module as well because the APIs/methods are clean and easy to use.

I had similar problem, and I used your code example but updated the httpheader section as follows:
c.setopt(pycurl.HTTPHEADER, ['Content-Type:application/json'])

It's better simple to use requests library. (http://docs.python-requests.org/en/latest)
I append python code for your original curl custom headers.
import json
import requests
url = 'http://some-url'
headers = {'Content-Type': "application/json; charset=xxxe", 'Accept': "application/json"}
data = {"name": "abc", "path": "def", "target": ["ghi"]}
res = requests.post(url, json=data, headers=headers)
print (res.status_code)
print (res.raise_for_status())