We Keep Coding

Import cucumber test result with XRay API using Python

I'm trying to perform an import of a cucumber test with the Xray API on Python, to be more specific I'm trying to translate this curl on Python side (it's a multipart form) :
curl -u usr:pass -F info=#$xrayResultFilePath -F result=#$pathToCucumberJson $jiraUrl/rest/raven/1.0/import/execution/cucumber/multipart
I tried in many different ways the python code I'm stucked on looks something like this:
response = requests.post(
atc_xray_url,
auth=(creds.username, creds.password),
files={"info": open("cucumber.result.json", "rb"),
"result": open("xray_result.json", "rb")},
)
response.raise_for_status()
I also tried to change the tags, to add them in a tuple like I found on the internet, solutions found here, but no result everytime I get this error:
<status><status-code>404</status-code><message>null for uri:
The curl is working, but the Python code is not. I could use the subprocess library but this shoud be a multiplatform solution so if this could be done with a thing in Python, it would be nice.

This repository that I made available some time ago provides several code snippets, including one precisely for that use case.
Your code is similar to the following one though; you may use basic auth or personal auth tokens, if you have a Jira DC version >= 8.14.
Given the result code you obtain, the problem may be on the URL that you use, which is not clear whether it's the same or not that you have on your curl. Note that you can also use v2 of the endpoint, as I show ahead.
import requests
import json
jira_base_url = "http://192.168.56.102"
jira_username = "admin"
jira_password = "admin"
personal_access_token = "OTE0ODc2NDE2NTgxOnrhigwOreFoyNIA9lXTZaOcgbNY"
...
files = {
'result': ('cucumber.json', open(r'cucumber.json', 'rb')),
'info': ('info.json', json.dumps(info_json) )
}
# importing results using HTTP basic authentication
# response = requests.post(f'{jira_base_url}/rest/raven/2.0/import/execution/cucumber/multipart', params=params, files=files, auth=(jira_username, jira_password))
# importing results using Personal Access Tokens
headers = {'Authorization': 'Bearer ' + personal_access_token}
response = requests.post(f'{jira_base_url}/rest/raven/2.0/import/execution/cucumber/multipart', files=files, headers=headers)

Python sending POST requests/ multipart/form-data

I'm just working on API conection at my work. I already made some GET and PUT request, but now i have problem with POST. API documantation is here. And here is my code I test but get 400 bad request:
import requests
files = {'files': ('fv.pdf', open(r"C:\python\API\fv.pdf", 'rb'))}
data = {"order_documents":[{'file_name':"fv.pdf", 'type_code':'CUSTOMER_INVOICE' }]}
header = {
'Authorization': '###########################',
}
response = requests.post("https://######.com/api/orders/40100476277994-A/documents", headers=header, files = files, data = data)
print(response.status_code)
print(response.url)
Someone have any idea how i can handle with this?

Looks like you are missing the order_documents parameter, it needs to be an array and also needs to be called order_documents.
Try changing your data variable into:
data = {"order_documents": [ {'file_name':"fv.pdf", 'type_code':'CUSTOMER_INVOICE' } ] }
The API expects files as the parameter name and your dictionary sends file to the server. The parameter name files that you give to session.post is just for requests library and not the actual parameter sent to the server.
The API also expects multiple files in an array, so you need to change your files object.
files = [
('files', ('fv.pdf', open(r"C:\python\API\fv.pdf", 'rb')),
]
Also, I don't think you need to use requests.Session(), just use requests.post(), unless you're planning on using the session object multiple times for subsequent requests.

Creating a CKAN package/dataset with resources using ckanapi and Python

CKAN provides the ckanapi package for accessing the CKAN API via Python or the command line.
I can use it to download metadata, create resources, etc. But I can't create a package and upload resources to it in a single API call. (A package is also referred to as a dataset.)
Internally, ckanapi scans all keys moving any file-like parameters into a separate dict, which it passes to the requests.session.post(files=..) parameter.
This is the closest I can get but CKAN returns an HTTP 500 error (copied from this guide to requests):
with ckanapi.RemoteCKAN('http://myckan.example.com', apikey='real-key', user_agent=ua, username='joe', password='pwd') as ckan:
ckan.action.package_create(name='joe_data',
resources=('report.xls',
open('/path/to/file.xlsx', 'rb'),
'application/vnd.ms-excel',
{'Expires': '0'}))
I've also tried resources=open('path/file'), files=open('file'), shorter or longer tuples, but get the same 500 error.
The requests documentation says:
:param files: (optional) Dictionary of ``'filename': file-like-objects``
for multipart encoding upload.
I can't pass ckanapi resources={'filename': open('file')} as ckanapi doesn't detect the file, attempts to pass it to requests as a normal parameter, and fails ("BufferedReader is not JSON serializable" as it attempts to make the file a POST parameter). I get the same if I try to pass a list of files. But the API is able to create a package and add a number of resources in a single call.
So how do I create a package and multiple resources with a single ckanapi call?

I was curious about this and thought I'd put something together to test it. Unfortunately I haven't played with the CLI you mentioned. But I hope this will help you and others stumbling across this.
I am not positive but I'm guessing your resource dict isn't formatted properly. The resources needs to be a list of dictionaries.
Here's a ruby script to do the single api call insert (my preferred language at this time):
# Ruby script to create a package and resource in one api call.
# You can run this in https://repl.it/languages/ruby
# Don't forget to update URLs and API key.
require 'csv'
require 'json'
require 'net/http'
hash_to_json = {
"title" => 'test1',
"name" => 'test1',
"owner_org" => 'bbb9682e-b58c-4826-bf4b-b161581056be',
"resources" => [
{
"url" => 'http://www.resource_domain.com/doc.kml'
}
]
}.to_json
uri = URI('http://ckan_app_domain.com:5000/api/3/action/package_create')
Net::HTTP.start(uri.host, uri.port) do |http|
request = Net::HTTP::Post.new uri
request['Authorization'] = 'user-api-key'
request.body = hash_to_json
response = http.request request
puts response.body
end
And here's a plain python script to do the same thing (thank you CKAN docs for this template I modified)
#!/usr/bin/env python
import urllib2
import urllib
import json
import pprint
# Put the details of the dataset we're going to create into a dict.
dataset_dict = {
'name': 'my_dataset_name',
'notes': 'A long description of my dataset',
'owner_org': 'bbb9682e-b58c-4826-bf4b-b161581056be',
'resources': [
{
'url': 'example.com'
}
]
}
# Use the json module to dump the dictionary to a string for posting.
data_string = urllib.quote(json.dumps(dataset_dict))
# We'll use the package_create function to create a new dataset.
request = urllib2.Request(
'http://ckan_app_domain.com:5000/api/3/action/package_create')
# Creating a dataset requires an authorization header.
# Replace *** with your API key, from your user account on the CKAN site
# that you're creating the dataset on.
request.add_header('Authorization', 'user-api-key')
# Make the HTTP request.
response = urllib2.urlopen(request, data_string)
assert response.code == 200
# Use the json module to load CKAN's response into a dictionary.
response_dict = json.loads(response.read())
assert response_dict['success'] is True
# package_create returns the created package as its result.
created_package = response_dict['result']
pprint.pprint(created_package)

How to upload files to slack using file.upload and requests

I've been searching a lot and I haven't found an answer to what I'm looking for.
I'm trying to upload a file from /tmp to slack using python requests but I keep getting {"ok":false,"error":"no_file_data"} returned.
file={'file':('/tmp/myfile.pdf', open('/tmp/myfile.pdf', 'rb'), 'pdf')}
payload={
"filename":"myfile.pdf",
"token":token,
"channels":['#random'],
"media":file
}
r=requests.post("https://slack.com/api/files.upload", params=payload)
Mostly trying to follow the advice posted here

Sending files through http requires a bit more extra work than sending other data. You have to set content type and fetch the file and all that, so you can't just include it in the payload parameter in requests.
You have to give your file information to the files parameter of the .post method so that it can add all the file transfer information to the request.
my_file = {
'file' : ('/tmp/myfile.pdf', open('/tmp/myfile.pdf', 'rb'), 'pdf')
}
payload={
"filename":"myfile.pdf",
"token":token,
"channels":['#random'],
}
r = requests.post("https://slack.com/api/files.upload", params=payload, files=my_file)

Writing this post, to potentially save you all the time I've wasted. I did try to create a new file and upload it to Slack, without actually creating a file (just having it's content). Because of various and not on point errors from the Slack API I wasted few hours to find out that in the end, I had good code from the beginning and simply missed a bot in the channel.
This code can be used also to open an existing file, get it's content, modify and upload it to Slack.
Code:
from io import StringIO # this library will allow us to
# get a csv content, without actually creating a file.
sio = StringIO()
df.to_csv(sio) # save dataframe to CSV
csv_content = sio.getvalue()
filename = 'some_data.csv'
token=os.environ.get("SLACK_BOT_TOKEN")
url = "https://slack.com/api/files.upload"
request_data = {
'channels': 'C123456', # somehow required if you want to share the file
# it will still be uploaded to the Slack servers and you will get the link back
'content': csv_content, # required
'filename': filename, # required
'filetype': 'csv', # helpful :)
'initial_comment': comment, # optional
'text': 'File uploaded', # optional
'title': filename, # optional
#'token': token, # Don't bother - it won't work. Send a header instead (example below).
}
headers = {
'Authorization': f"Bearer {token}",
}
response = requests.post(
url, data=request_data, headers=headers
)
OFFTOPIC - about the docs
I just had a worst experience (probably of this year) with Slack's file.upload documentation. I think that might be useful for you in the future.
Things that were not working in the docs:
token - it cannot be a param of the post request, it must be a header. This was said in one of github bug reports by actual Slack employee.
channel_not_found - I did provide an existing, correct channel ID and got this message. This is somehow OK, because of security reasons (obfuscation), but why there is this error message then: not_in_channel - Authenticated user is not in the channel. After adding bot to the channel everything worked.
Lack of examples for using content param (that's why I am sharing my code with you.
Different codding resulted with different errors regarding form data and no info in the docs helped to understand what might be wrong, what encoding is required in which upload types.
The main issue is they do not version their API, change it and do not update docs, so many statements in the docs are false/outdated.

Base on the Slack API file.upload documentation
What you need to have are:
Token : Authentication token bearing required scopes.
Channel ID : Channel to upload the file
File : File to upload
Here is the sample code. I am using WebClient method in #slack/web-api package to upload it in slack channel.
import { createReadStream } from 'fs';
import { WebClient } from '#slack/web-api';
const token = 'token'
const channelId = 'channelID'
const web = new WebClient(token);
const uploadFileToSlack = async () => {
await web.files.upload({
filename: 'fileName',
file: createReadStream('path/file'),
channels: channelId,
});
}

Implementing challenge/response scheme with python-requests

I'm starting to learn how to use the python requests module. For practicing I tried to manage a challenge/response problem: I want to access the data on http://lema.rae.es/drae/srv/search?val=hacer
With the "Tamper Data" plugin for Firefox I inspected the necessary HTTP requests:
GET http://lema.rae.es/drae/srv/search?val=hacer
POST http://lema.rae.es/drae/srv/search?val=hacer
I copied the exact headers that are sent by Firefox in the two HTTP requests and implemented the JavaScript "challenge" function in Python. Then I'm doing the following:
url = "http://lema.rae.es/drae/srv/search?val=hacer"
headers = { ... }
r1 = requests.get(url=url, headers=headers)
html = r1.content.decode("utf-8")
formdata = challenge(html)
headers = { ... }
r2 = requests.post(url=url, data=formdata, headers=headers)
Unfortunately, the server will not answer in the expected way. I checked all the headers I'm sending via "r.request.headers" and they agree perfectly with the headers that firefox sends (according to Tamper Data)
What am I doing wrong?
You can inspect my full code here: http://pastebin.com/7JAZ9B4s
This is the response header I should be getting:
Date[Tue, 10 Feb 2015 17:13:53 GMT]
Vary[Accept-Encoding]
Content-Encoding[gzip]
Cache-Control[max-age=0, no-cache]
Keep-Alive[timeout=5, max=100]
Connection[Keep-Alive]
Content-Type[text/html; charset=UTF-8]
Set-Cookie[TS014dfc77=017ccc203c29467c4d9b347fb56ea0e89a7182e52b9d7b4a1174efbf134768569a005c7c85; Path=/]
Transfer-Encoding[chunked]
And this is the response header I really get:
Content-Length[5798]
Content-Type[text/html]
Pragma[no-cache]
Cache-Control[no-cache]

I found the reason why my code doesn't work:
The server expects the POSTDATA in exactly the same order in which the entries appear as input-elements of the form. In my code the values of the input-elements were stored in a python dict. But this data type does not preserve the order in which values have been declared!
The ruby script (referred to in the comments) however does work because the ruby dict data type seems to preserve the order of declaration!
Furthermore, reimplementing the javascript challenge() function in python was not necessary at all, because the server will be happy to accept any response string (that worked in the past) over and over again!