I have a Confluence page which has a Calendar inside it (please check photo below).
Calendar
I am trying to pull information from this calendar, like how many events in each day. nothing more.
i used a code from stackoverflow that reads Confluence page using API. but the json response does not contain any data about the calendar inside the page.
`import requests
import json
from requests.auth import HTTPDigestAuth
confluence_host = "https://confluence.tools.mycompany.com"
url = confluence_host + '/rest/api/content/'
page_id = "36013799"
page = requests.get(url=url + page_id,
params={'expand': 'body.storage'},
auth=('my_user', 'my_password') ).json()`
Even if i write, html = page['body']['storage']['value'] and check its output, it only gives this:
name="calendar" ac:schema-version="1" ac:macro-id="99a26d73-abaa-45a1-92cc-0edafec567f5">72da4ae5-4888-46dd-9078-0299b51ab815,743a55b4-7b3b-4e00-b102-90d95916de8d
Is there any way to get the calendar info ?
Thanks
You are using Team Calendar in your page and Team Calendar is a plugin in your page. Technically, /rest/api/content only gives you the content of the page not the Content of the Plugins. As far as I know, Team Calendar doesn't have Public Rest API as you can see on CONFSERVER-51323 but you can get the data that you want from the database instead of REST API since Team Calendar has already creates couple of AO Tables in your database.
I found it easiest to get the subscribe link to the calendar then use an iCalendar library to parse the data. Make sure the subscribe button gives you a link with a {guid}.ics and not undefined.ics - To solve that I had to go to the calendars link in the main confluence space and then select it from the dropdown. You may have to create an empty calendar so you can select a cal.
I was able by looking at the GET and PUT, there is a rest API used by the javascript plugin (rest/calendar-services/1.0/calendar/events.json):
you need to find out your: subCalendarId='yourID'
urlC = 'https://yourconfluence.com/rest/calendar-services/1.0/calendar/events.json?subCalendarId=40xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx&userTimeZoneId=America%2FMexico_City&start=2018-11-28T00%3A00%3A00Z&end=2018-11-28T00%3A00%3A00Z'
r = requests.get(urlC, auth=("myuser", "mypass"), timeout=15)
that will return all the records on that period:
a = r.json()
a.keys()
[u'events', u'success']
a['success']
True
type(a['events'])
list
len(a['events'])
61
Use the following data in a PUT to add new events:
data = {
"subCalendarId": "xxx-xxx-xxx",
"eventType": "custom",
"customEventTypeId": "xxx-xxx-xxx",
"what": "My Test",
"person": "xxxxxxxxxxxxxxxxx",
"startDate": "28-Nov-2018",
"startTime": "15:00",
"endDate": "28-Nov-2018",
"endTime": "16:00",
"allDayEvent": "false",
"editAllInRecurrenceSeries": "true",
"where": "Some Place",
"description": "My testing Case",
"userTimeZoneId": "America/Mexico_City",}
urlC = 'https://yourconfluence.com/rest/calendar-services/1.0/calendar/events.json'
r = requests.put(urlC, auth=('username', 'pass'), data=data, timeout=15)
that will return a 'success': true with the new entry:
u'{"success":true,"subCalendar":{"reminderMe":false,........}}
After going through the developer console in chrome, After analyzing the format of the payload and required authentication details I found solution for this,
My problem statement was little different I have to add the event to confluence calendar, Both adding event and extracting event will follow the same process.
There are few cookies which are required for authentication like JSESSIONID and seraphConfluence Which will be stored in the application -> cookies in chrome developer tool.
and also we require the subCalenderid and Id Type which can be taken from the application-> local storage in in chrome developer tool.
and also , The confluence will send request using 'application/x-www-form-urlencoded' as Content-Type, So in the data we should have it in encoded format,
For this we can use below code to convert to that format
import urllib
urllib.parse.quote_plus('May 4, 2022')
output:
'May+4%2C+2022'
And also date and type should be in the MMM D, YYYY and h:MM A format you can use arrow python package to do the work
arrow.utcnow.format(MMM D, YYYY)
output
May 4, 2022
Below there is a string from the request payload in chrome when it send put request when we click add event , if we analyse the string we can see that we have
confirmRemoveInvalidUsers=false&childSubCalendarId=&customEventTypeId=asdfghjk-asdf-asdf-asfg-sdfghjssdfgh&eventType=custom&isSingleJiraDate=false&originalSubCalendarId=&originalStartDate=&originalEventType=&originalCustomEventTypeId=&recurrenceId=&subCalendarId=asdfghjk-asdf-asdf-asdg-asdfghjkl&uid=&what=test&startDate=May+4%2C+2022&startTime=&endDate=May+4%2C+2022&endTime=&allDayEvent=true&rruleStr=&until=&editAllInRecurrenceSeries=true&where=&url=&description=&userTimeZoneId=America%2FNew_York
After analysing it we can come to conclution we have to replace start date,enddate , start and end time, what , where , subcalendar id and type and other fields with our code and send the request.
Below is the code which will do that
def addEventtoCalender():
reqUrl = 'https://confluence.yourdomain.com/rest/calendar-services/1.0/calendar/events.json'
authDetails = getConfluenceAuthenticationDetails()
what=urllib.parse.quote_plus('WHAT field data')
startDate = urllib.parse.quote_plus(arrow.utcnow().format('MMM D, YYYY'))
startTime=urllib.parse.quote_plus(arrow.utcnow().format('h:MM A'))
endDate=urllib.parse.quote_plus(arrow.utcnow().format('MMM D, YYYY'))
endTime=urllib.parse.quote_plus(arrow.utcnow().shift(hours=+1).format('h:MM A'))
where=urllib.parse.quote_plus('WHERE field data')
url=urllib.parse.quote_plus('https://yoururl.com')
description=urllib.parse.quote_plus('test test test')
customEventTypeId = authDetails['CONFLUENCE_CUSTOM_EVENT_TYPE_ID'] #subcalender type
subCalendarId = authDetails['CONFLUENCE_SUBCALENDAR_ID']
seraphConfluence = authDetails['CONFLUENCE_SERAPH_CONFLUENCE']
JSESSIONID = authDetails['CONFLUENCE_JSESSION_ID']
data = f'confirmRemoveInvalidUsers=false&childSubCalendarId=&customEventTypeId={customEventTypeId}&eventType=custom&isSingleJiraDate=false&originalSubCalendarId=&originalStartDate=&originalEventType=&originalCustomEventTypeId=&recurrenceId=&subCalendarId={subCalendarId}&uid=&what={what}&startDate={startDate}&startTime={startTime}&endDate={endDate}&endTime={endTime}&allDayEvent=false&rruleStr=&until=&editAllInRecurrenceSeries=true&where={where}&url={url}&description={description}&userTimeZoneId=America%2FNew_York'
headers = {
'Content-Type': 'application/x-www-form-urlencoded',
'Cookie': f'seraph.confluence={seraphConfluence}; JSESSIONID={JSESSIONID}'
}
res = requests.put(url=reqUrl,data=data,headers=headers,verify=False)
Above code will replicate the whole process of adding event to calendar. You can use the same approach to replicate getting all event between particular dates.
Related
I have been recently exploring the CloudKit and related frameworks. I got the communication with my app working, as well as with my website using CloudKitJS. Where I am struggling is the Server-to-Server communication (which I would need for exporting data from public database in csv.
I have tried Python package requests-cloudkit, which others were suggesting. I have created a Server-to-Server token, and have copied only the key between START and END line once creating the eckey.pem file. I then got this code:
from requests_cloudkit import CloudKitAuth
from restmapper import restmapper
import json
KEY_ID = '[my key ID from CK Dashboard]'
SECRET_FILE_KEY = 'eckey.pem'
AUTH = CloudKitAuth(KEY_ID, SECRET_FILE_KEY)
PARAMS = {
'query':{
'recordType': '[my record type]'
},
}
CloudKit = restmapper.RestMapper("https://api.apple-cloudkit.com/database/1/[my container]/development/")
cloudkit = CloudKit(auth=AUTH)
response = cloudkit.POST.public.records.query(json.dumps(PARAMS))
I am then getting the 401 Authentication failed response. I am stuck on this for days, so I would be grateful for any help or advice. 😊
Creating the server-to-server key is an important first step, but in order to make HTTP requests after that, you have to sign each request.
Look for the Authenticate Web Service Requests section near the bottom of this documentation page.
It's a little bit convoluted, but you have to carefully construct signed headers to include with each request you make. I'm not familiar with how to do it in Python, but here's how I do it in NodeJS which may help:
//Get the timestamp in a very specific format
let date = moment().utc().format('YYYY-MM-DD[T]HH:mm:ss[Z]')
//Construct the subpath
let endpoint = '/records/lookup'
let path = '/database/1/iCloud.*****/development/public'
let subpath = path+endpoint
//Get the key file
let privateKeyFile = fs.readFileSync('../../'+SECRET_FILE_KEY, 'utf8')
//Make a string out of your JSON query
let query = {
recordType: '[my record type]'
}
let requestBody = JSON.stringify(query)
//Hash the query
let bodyHash = crypto.createHash('sha256').update(requestBody, 'utf8').digest('base64')
//Assemble the components you just generated in a special format
//[Current date]:[Request body]:[Web service URL subpath]
let message = date+':'+bodyHash+':'+subpath
//Sign it
let signature = crypto.createSign('RSA-SHA256').update(message).sign(privateKeyFile, 'base64')
//Assemble your headers and include them in your HTTP request
let headers = {
'X-Apple-CloudKit-Request-KeyID': KEY_ID,
'X-Apple-CloudKit-Request-ISO8601Date': date,
'X-Apple-CloudKit-Request-SignatureV1': signature
}
This is a bit hairy at first, but I just put all this stuff in a function that I reuse whenever I need to make a request.
Apple's documentation has pretty much been abandoned and it's hard to find good help with CloudKit Web Services these days.
I have a list of manually selected Unsplash photo IDs that I want to display on a website. Now I want to make sure to print out proper attribution via the Unsplash API.
The .get_attribution function in PyUnsplash is perfect for my use case. However, it requires a username or search query to return the attribution.
I was hoping to simply query the API with the photo ID, but sadly, that does not work.
So if there's any suggestions how to get from photo-ID to username via the API, I would love to hear them.
_
Alternative question titles:
Get Unsplash user’s profile by photo id
github.com/salvoventura/ released a new version of PyUnsplash. Now this is possible.
PyUnsplash v1.0.0rc1 (release candidate, v1)
This is the first release candidate for production ready 1.0.0 of
PyUnsplash. Thanks to all the early adopters, there have been a number
of improvements and bugfixes and now should be a good time to start
the rc process.
This release brings one useful addition:
Introduction of SinglePhoto which allows instantiating a Photo object
using a photo_id Deprecated format value 'str' in favor of 'txt' in
get_attribution()
I just did some troubleshooting for unsplash queries with Paw to a photos/:id endpoint. Using the requests library I was able to pull the username out of the returned json object:
import requests
def send_request():
# Request
# GET https://api.unsplash.com/photos/4Fjjyhg1YFc
try:
response = requests.get(
url="https://api.unsplash.com/photos/4Fjjyhg1YFc",
params={
"client_id": "YOUR_API_KEY",
},
headers={
"Cookie": "",
},
)
data = response.json()
print('Response HTTP Status Code: {status_code}'.format(
status_code=response.status_code))
print('Unsplash API Username: {uname}'.format(
uname=data['user']['username']))
print('Unsplash API Full Name: {fname}'.format(
fname=data['user']['name']))
print('Unsplash API Location: {location}'.format(
location=data['user']['location']))
print('Unsplash API Portfolio: {portfolio}'.format(
portfolio=data['user']['portfolio_url']))
print('Unsplash API Instagram: {ig}'.format(
ig=data['user']['instagram_username']))
except requests.exceptions.RequestException:
print('HTTP Request failed')
send_request()
For more info, checkout the 'Response' section under Get A Photo in the docs. It shows all the fields returned, you may find more you'd like to use.
I'm sending a GET request to /users/{id}/calendar/calendarView?startDateTime={start_datetime}&endDateTime={end_datetime} in order to get events of a user and after that I'm filtering the event subject and content.
What happens is that in the majority of times the subject in the JSON response is just the name of the event organizer and the content is just a blank HTML instead of the values that can be seen in the calendar normally.
I tried finding another fields in the JSON that could provide the correct information but there aren't any.
It looks like some people also had the same problem (here and here), but no solution was found or the one given is not what I need.
The following code is what I'm using:
graph_client = OAuth2Session(token=token)
headers = {
'Prefer' : 'outlook.timezone="America/Manaus"'
}
response = graph_client.get(f'{self.graph_url}/users/{room}/calendar/calendarView?startDateTime={start_datetime}&endDateTime={end_datetime}',
headers=headers)
print(response.json()['value'])
I am very newbie with the Google DialogFlow, As I am exploring with the Api.ai. I got to know about the userEntity(V1) or SessionEntity(V2).
I have created the session entity using fulfilment with the below code:
import requests
REQUIRED_SCOPES = 'https://www.googleapis.com/auth/dialogflow'
access_token = generate_access_token()
url = 'https://dialogflow.googleapis.com/v2/'+ session_id +'/entityTypes'
headers = {'Authorization': 'Bearer '+ access_token,'Content-Type': 'application/json'}
entity_data = json.dumps({"entityOverrideMode": "ENTITY_OVERRIDE_MODE_OVERRIDE", "entities": [{"synonyms":["sugarcane", "sugar"],"value": "sweet"}], "value": str(session_id)+'fruit'})
response = requests.post(url,headers=headers,data=entity_data)
It creates the sessionEntity with success and also able to list the entity from the session.
It's working while I tried from the same page of console with TryMe-Intents.
But while I tried using simulator or mobile to pass some phrases for my Intent I am not getting the newly created entity to map with the matching phrases.
For Example, I want Sugarcane, at this point, I suppose to get sugarcane as mapped with the entity with the current session for followup intents.
But it was not happening, don't know if I am missing anything. Let me know if anything is wrong or missing.
Suggestions are always welcome.
Reference Links which I have referred to achieve:
https://cloud.google.com/dialogflow-enterprise/docs/reference/rest/v2/projects.agent.sessions.entityTypes/create
https://cloud.google.com/dialogflow-enterprise/docs/entities-session
I need to figure out the simplest method of grabbing the length of a youtube video programmatically given the url of said video.
Is the youtube API the best method? It looks somewhat complicated and I've never used it before so it's likely to take me a bit to get accommodated, but I really just want the quickest solution. I took a glance through the source of a video page in the hopes it might list it there, but apparently it does not (though it lists recommended video times in a very nice list that would be easy to parse). If it is the best method, does anyone have a snippit?
Ideally I could get this done in Python, and I need it to ultimately be in the format of
00:00:00.000
but I'm completely open to any solutions anyone may have.
I'd appreciate any insight.
All you have to do is read the seconds attribute in the yt:duration element from the XML returned by Youtube API 2.0. You only end up with seconds resolution (no milliseconds yet). Here's an example:
from datetime import timedelta
from urllib2 import urlopen
from xml.dom.minidom import parseString
for vid in ('wJ4hPaNyHnY', 'dJ38nHlVE78', 'huXaL8qj2Vs'):
url = 'https://gdata.youtube.com/feeds/api/videos/{0}?v=2'.format(vid)
s = urlopen(url).read()
d = parseString(s)
e = d.getElementsByTagName('yt:duration')[0]
a = e.attributes['seconds']
v = int(a.value)
t = timedelta(seconds=v)
print(t)
And the output is:
0:00:59
0:02:24
0:04:49
(I'm not sure what "pre-download" refers to.)
The simplest way to get the length of VIDEO_ID is to make an HTTP request for
http://gdata.youtube.com/feeds/api/videos/VIDEO_ID?v=2&alt=jsonc
and then look at the value of the data->duration element that's returned. It will be set to the video's duration in seconds.
With python and V3 youtube api this is the way for every videos.
You need the API key, you can get it here: https://console.developers.google.com/
# -*- coding: utf-8 -*-
import json
import urllib
video_id="6_zn4WCeX0o"
api_key="Your API KEY replace it!"
searchUrl="https://www.googleapis.com/youtube/v3/videos?id="+video_id+"&key="+api_key+"&part=contentDetails"
response = urllib.urlopen(searchUrl).read()
data = json.loads(response)
all_data=data['items']
contentDetails=all_data[0]['contentDetails']
duration=contentDetails['duration']
print duration
Console response:
>>>PT6M22S
Corresponds to 6 minutes and 22 seconds.
You can always utilize Data API v3 for this.
Just do a videos->list call.
GET https://www.googleapis.com/youtube/v3/videos?part=contentDetails%2C+fileDetails&id={VIDEO_ID}&key={YOUR_API_KEY}
In response get the contentDetails.duration in ISO 8601 format.
Or you can get duration in ms from fileDetails.durationMs.
If you're using Python 3 or newer you can perform a GET request against the YouTube v3 API URL. For this you will need the enable the YouTube v3 API in your Google Console and you'll need to create an API credential after you enable the YouTube v3 API.
Code examples below:
import json
import requests
YOUTUBE_ID = 'video_id_here'
API_KEY = 'your_youtube_v3_api_key'
url = f"https://www.googleapis.com/youtube/v3/videos?part=contentDetails&id={YOUTUBE_ID}&key={API_KEY}"
response = requests.get(url) # Perform the GET request
data = response.json() # Read the json response and convert it to a Python dictionary
length = data['items'][0]['contentDetails']['duration']
print(length)
Or as a reusable function:
import json
import requests
API_KEY = 'your_youtube_v3_api_key'
def get_youtube_video_duration(video_id):
url = f"https://www.googleapis.com/youtube/v3/videos?part=contentDetails&id={video_id}&key={API_KEY}"
response = requests.get(url) # Perform the GET request
data = response.json() # Read the json response and convert it to a Python dictionary
return data['items'][0]['contentDetails']['duration']
duration = get_youtube_video_duration('your_video_id')
Note: You can only get fileDetails from the API if you own the video, so you'll need to use the same Google account for your YouTube v3 API key as your YouTube account.
The response from Google will look something like this:
{
"kind": "youtube#videoListResponse",
"etag": "\"SJajsdhlkashdkahdkjahdskashd4/meCiVqMhpMVdDhIB-dj93JbqLBE\"",
"pageInfo": {
"totalResults": 1,
"resultsPerPage": 1
},
"items": [
{
"kind": "youtube#video",
"etag": "\"SJZWTasdasd12389ausdkhaF94/aklshdaksdASDASddjsa12-18FQ\"",
"id": "your_video_id",
"contentDetails": {
"duration": "PT4M54S",
"dimension": "2d",
"definition": "hd",
"caption": "false",
"licensedContent": false,
"projection": "rectangular"
}
}
]
}
Where your video duration is: PT4M54S which means 4 Minutes 54 Seconds
Edit: To convert the YouTube duration to seconds, see this answer: https://stackoverflow.com/a/49976787/2074077
Once you convert to time to seconds, you can convert seconds into your format with a timedelta.
from datetime import timedelta
time = timedelta(seconds=duration_in_seconds)
print(time)