I've found many examples of using the Office365-REST-Python-Client however, none of them are correctly obtaining the access token. I've registered an app under the Azure Portal, granted it API permissions using 'Application permissions', created a secret and used the client_secret and client_id in my settings dictionary to use in the below code.
def read_folder_and_files(context, list_title):
"""Read a folder example"""
list_obj = context.web.lists.get_by_title(list_title)
folder = list_obj.root_folder
context.load(folder)
context.execute_query()
print("List url: {0}".format(folder.properties["ServerRelativeUrl"]))
files = folder.files
context.load(files)
context.execute_query()
for cur_file in files:
print("File name: {0}".format(cur_file.properties["Name"]))
folders = context.web.folders
context.load(folders)
context.execute_query()
for folder in folders:
print("Folder name: {0}".format(folder.properties["Name"]))
if __name__ == '__main__':
ctx_auth = AuthenticationContext(url=settings['url'])
if ctx_auth.acquire_token_for_app(client_id=settings['client_credentials']['client_id'],
client_secret=settings['client_credentials']['client_secret']):
ctx = ClientContext(settings['url'], ctx_auth)
read_folder_and_files(ctx, "Documents")
# read_folder_and_files_alt(ctx, "Documents")
# upload_file_into_library(target_library, name, content)
# download_file(ctx)
else:
print(ctx_auth.get_last_error())
When I run the above code I get the following error:
File "/usr/local/lib/python3.7/site-packages/office365/runtime/auth/acs_token_provider.py", line 76, in get_authorization_header
return 'Bearer {0}'.format(self.access_token["access_token"])
KeyError: 'access_token'
My end goal is to upload files to a Sharepoint Document Libary with metadata from a python data pipeline. Sharepoint is not hosted locally and is included in our 365 licences.
Kind Regards
So it looks like this error can happen when you're not getting an access token.
I fixed this by ditching the client and secret in my Azure Portal and instead generated them in the SharePoint site under the following URL:
URL: https://[tenant].sharepoint.com/_layouts/15/appregnew.aspx
To find out what you should use in the space of [tenant] look at your SharePoint URL and pick out the text between 'https://' and '.sharepoint.com'. This is assuming your SharePoint is hosted by Microsoft.
Click the generate buttons, use a relevant Title and unless you know better just enter localhost for the App Domain and Redirect URL. (My project is just a simple upload script). Take a copy of the Client ID and Secret.
If you want your App to have full access then navigate to:
https://[tenant]-admin.sharepoint.com/_layouts/15/appinv.aspx
There is another link 'https://[tenant].sharepoint.com/_layouts/15/appinv.aspx' but this won't let you apply for full control permissions.
Paste in the client id into the App id, where would be the fun in using the same field name, or linking the form together? Click lookup and use the below XML to grant full control.
<AppPermissionRequests AllowAppOnlyPolicy="true">
<AppPermissionRequest Scope="http://sharepoint/content/sitecollection" Right="FullControl" />
</AppPermissionRequests>
Click create and on the next page click trust. You will need to be logged in as a site owner with full admin permissions to grant this control.
I found all this info here which goes into more detail if you need it:
https://www.anupams.net/app-only-policy-with-tenant-level-permissions-in-sharepoint-online/
Bonus Info:
Our Sharepoint has two 'sites' so passing the base URL of 'https://[tenant].sharepoint.com' took me to the wrong site by default and meant that the document libraries I was looking didn't exist. To fix this using Office365-REST-Python-Client in your settings dictionary make sure the URL setting has the fill path to your site like this:
https://[tenant]-admin.sharepoint.com/sites/[site]
Hope this helps, this info cost me far to much time!
Related
I made a script that works perfectly when it runs locally but isn't able to authenticate in my Azure Functions app. At first, I thought it was because it couldn't read the .cache file.
After looking at the logs it's because it can't open a browser window to create the authentication token. I'm fairly new to Azure functions so I'm not sure how I can enable opening a browser if that's even possible.
I looked into the oauth2.py file in spotipy and found it's optional to open the browser but instead, the console asks for the redirected URL. Is there a way for me to get the redirected URL and enter it into the console instead?
Here's my code:
def create_playlist(cred):
"""Creates the playlist for Discover weekly to be copied to"""
# Gain authorization to create playlist
logging.info('Authenticating spotify secrets to create new playlist...')
# spotipy.CacheFileHandler(cache_path='')
sp = spotipy.Spotify(auth_manager=SpotifyOAuth(client_id=cred[0],
client_secret=cred[1],
redirect_uri=cred[2],
scope='playlist-modify-private',
open_browser=False,
))
# Get returned list from get_playlist_info function
logging.info('Determining playlist descriptors...')
info = get_playlist_info()
# New playlist for Discover Weekly
logging.info('Creating new playlist...')
new_playlist = sp.user_playlist_create(user=cred[4],
name=info[0],
public=False,
collaborative=False,
description=info[1])
logging.info('Returning new playlist.')
return new_playlist
In Redirect URIs you enters one or more addresses which you want to whitelist with Spotify. And this URI enables the Spotify authentication service to automatically re-launch your app every time the user logs in.
You can set your Redirect URL by navigating to your Spotify developer dashboard and open the project you are working on. Then click "edit settings" and look for the redirect URIs field and put your redirect URI in the field and lastly save it.
The Authorization Code Flow needs you to add a redirect URI to your application at My Dashboard. The redirect_uri argument or SPOTIPY_REDIRECT_URI environment variable must match the redirect URI added to your application in your Dashboard.
Following are some examples that you can use as redirect URIs.
http://example.com, http://localhost or http://127.0.0.1:9090
The correct way in which you should authenticate yourself is shown in the code snippet below.
token = util.prompt_for_user_token(
username=USERNAME,
scope=SCOPE,
client_id=CLIENT_ID,
client_secret=CLIENT_SECRET,
redirect_uri=REDIRECT_URI)
You should also export your redirect_uri, client ID, and secret before running your script and trying to authenticate. If you don't pass the values directly to the util.prompt_for_user_token function, it will read them from the environment.
Try running these in your terminal before running your script.
export SPOTIPY_CLIENT_ID="YOUR CLIENT ID"
export SPOTIPY_CLIENT_SECRET="YOUR CLIENT SECRET"
export SPOTIPY_REDIRECT_URI='http://localhost/'
For more information you can read this Authorization Code Flow document for Spotipy.
I am trying to get access to a Sharepoint site from Python and it seems like AADSTS has a conditional access policy that’s preventing me from getting a token.
Has anyone used the Office365 Python Rest Client to successful connect to a Sharepoint Rest Endpoint?
This is the actual error I am seeing
"C:\Python\envs\dev\python.exe c:\Users\xxxxxx.vscode\extensions\ms-python.python-2020.8.105369\pythonFiles\lib\python\debugpy\launcher 52801 -- c:\xxxx\Code\PythonSharepointUpdate\readsharepoint.py " An error occurred while retrieving token from XML response: AADSTS53003: Access has been blocked by Conditional Access policies. The access policy does not allow token issuance. An error occurred while retrieving auth cookies from https://microsoft.sharepoint.com/_vti_bin/idcrl.svc/
I am trying to execute this code from the Sample
from office365.runtime.auth.user_credential import UserCredential
from office365.sharepoint.client_context import ClientContext
from office365.runtime.auth.authentication_context import AuthenticationContext
site_url = 'https://microsoft.sharepoint.com/teams/'
ctx = ClientContext(site_url).with_credentials(UserCredential("user#domain.com", "My Complex Password”))
web = ctx.web
ctx.load(web)
ctx.execute_query()
print("Web title: {0}".format(web.properties['Title']))
Also – is there a better way to authenticate to Sharepoint – I hate to have to type my Password in clear text in code ☹
Has anyone use ClientCredential? Is there a seperate setup on the Sharepoint Site to enable ClientCredential? Does our Sharepoint let us do that?
The trick is to use ClientCredentials and not UserCredentials.
One will have to create a new app on the Sharepoint site with a new CliendID and Client Secret. After creating a new app, the newly created app has to be given permissions on the site. Once the permissions are set, you will have to Trust the app.
The full explanation and the steps are documented here https://github.com/vgrem/Office365-REST-Python-Client/wiki/How-to-connect-to-SharePoint-Online-and-and-SharePoint-2013-2016-2019-on-premises--with-app-principal
Set-up
I have an empty Google Sheet invoice template in a folder.
Using Python, Gspread and PyDrive I duplicate the template and populate the copy with data.
Question
Customers need to receive a pdf of the invoice, not a google sheet.
I came across this question where is stated how to export a sheet locally into a pdf, namely,
file_obj = drive.CreateFile({'id': 'sfkldjlfsdkjfgdklj'})
file_obj.GetContentFile('temp.pdf', mimetype='application/pdf')
How can I create a PDF of the google sheet, but instead of downloading the PDF locally store it in the same Google Drive folder as the Google Sheet?
You want to convert Google Spreadsheet to a PDF format, and create it as a file on the same folder of the Google Spreadsheet.
In this case, the PDF file is created on Google Drive.
You want to achieve this using pydrive with Python.
Issue and workaround:
Unfortunately, the Google Spreadsheet cannot be directly converted and export to Google Drive using Google API. In this case, it is required to use the workarounds. In this answer, I would like to propose the following 2 workarounds.
Pattern 1:
In this pattern, your goal is achieved using pydrive. In this case, the Spreadsheet is downloaded to the local PC as a PDF file, and upload the PDF file to the same folder of the Spreadsheet on Google Drive.
Sample script:
spreadsheet_id = '###' # Please set the Spreadsheet ID.
pdf_filename = 'sample.pdf'
drive = GoogleDrive(gauth)
# Download Spreadsheet as PDF file.
dl_file = drive.CreateFile({'id': spreadsheet_id})
dl_file.GetContentFile(pdf_filename, mimetype='application/pdf')
# Get parent folder ID of the Spreadsheet.
url = 'https://www.googleapis.com/drive/v3/files/' + spreadsheet_id + '?fields=parents'
headers = {'Authorization': 'Bearer ' + drive.auth.credentials.token_response["access_token"]}
res = requests.get(url, headers=headers)
# Upload the PDF file to the same folder of Spreadsheet.
up_file = drive.CreateFile({'parents': [{'id': res.json()['parents'][0]}]})
up_file.SetContentFile(pdf_filename)
up_file.Upload()
print('title: %s, mimeType: %s' % (up_file['title'], up_file['mimeType']))
In order to retrieve the parent folder ID of Spreadsheet, I used the method of files.get in Drive API using requests.
Pattern 2:
In this pattern, your goal is achieved using Web Apps created by Google Apps Script. In this case, using Web Apps, the Spreadsheet is converted to a PDF format and create a file to the same folder of Spreadsheet. So the process can be run at only Google side. The python script sends only the spreadsheet ID by the request. For this, please do the following flow.
1. Create new project of Google Apps Script.
Sample script of Web Apps is a Google Apps Script. So please create a project of Google Apps Script.
If you want to directly create it, please access to https://script.new/. In this case, if you are not logged in Google, the log in screen is opened. So please log in to Google. By this, the script editor of Google Apps Script is opened.
2. Prepare script.
Please copy and paste the following script (Google Apps Script) to the script editor. This script is for the Web Apps.
function doGet(e) {
var spreadsheetId = e.parameter.id;
var file = DriveApp.getFileById(spreadsheetId);
var blob = file.getBlob();
var folder = file.getParents().next();
var pdfFile = folder.createFile(blob.setName(file.getName() + ".pdf"));
return ContentService.createTextOutput(pdfFile.getId());
}
In this case, the GET method is used.
3. Deploy Web Apps.
On the script editor, Open a dialog box by "Publish" -> "Deploy as web app".
Select "Me" for "Execute the app as:".
By this, the script is run as the owner.
Select "Anyone, even anonymous" for "Who has access to the app:".
In this case, no access token is required to be request. I think that as the test case, I recommend this setting.
Of course, you can also use the access token. At that time, please set this to "Anyone".
Click "Deploy" button as new "Project version".
Automatically open a dialog box of "Authorization required".
Click "Review Permissions".
Select own account.
Click "Advanced" at "This app isn't verified".
Click "Go to ### project name ###(unsafe)"
Click "Allow" button.
Click "OK".
Copy the URL of Web Apps. It's like https://script.google.com/macros/s/###/exec.
When you modified the Google Apps Script, please redeploy as new version. By this, the modified script is reflected to Web Apps. Please be careful with this.
3. Run the function using Web Apps.
This is a sample python script for requesting Web Apps. Please set your Web Apps URL and spreadsheetId.
import requests
spreadsheet_id = '###' # Please set the Spreadsheet ID.
url = 'https://script.google.com/macros/s/###/exec?id=' + spreadsheet_id
res = requests.get(url)
print(res.text)
References:
Download file content
Upload and update file content
Web Apps
Taking advantage of Web Apps with Google Apps Script
You could use the Drive API's Files.export method, to convert the sheet into a 'application/pdf' mime-type.
Try it out in the Files API explorer.
I'm trying to use python to download an excel file that is hosted in a sharepoint which is part of the Microsoft Azure platform. The sharepoint is password protected, and I have an account and a password which I can use to login in via my browser,
In order to authenticate with a python script I followed the method suggested in: Sharepoint authentication with python.
Which uses the O365 rest python client library and goes as follows:
from office365.runtime.auth.authentication_context import AuthenticationContext
from office365.sharepoint.client_context import ClientContext
url = 'https://organization.sharepoint.com/sites/something/somepage.aspx'
username = 'userx#organization.com'
password = 'fakepass'
ctx_auth = AuthenticationContext(url)
if ctx_auth.acquire_token_for_user(username, password):
ctx = ClientContext(url, ctx_auth)
else:
print(ctx_auth.get_last_error())
But I'm getting an error message back:
An error occurred while retrieving token: AADSTS50076: Due to a configuration
change made by your administrator, or because you moved to a new location, you
must use multi-factor authentication to access ''.
I do connect to this account from multiple devices (browser), and just once I was required to use MFA to log in (SMS message). Is there a way to get around this? Note that I'm not the admin of the system.
The error message is pretty intuitive, user credentials auth is not supported when Multi-Factor Authentication (MFA) enabled.
To circumvent this error, SharePoint App-Only flow could be utilized instead (supported by Office365-REST-Python-Client library).
Setting up an app-only principal with tenant permissions section describes how to configure it, to summarize it consist of two steps:
register App principal (think of it as a "service account")
grant a permissions
Once app principal is created and consented, it could be utilized to access SharePoint resource as demonstrated below:
from office365.sharepoint.client_context import ClientContext
from office365.runtime.auth.client_credential import ClientCredential
site_url = 'https://contoso.sharepoint.com/'
app_principal = {
'client_id': '--client-id-goes-here--',
'client_secret': '--client-secret-goes-here--',
}
credentials = ClientCredential(app_principal['client_id'], app_principal['client_secret'])
ctx = ClientContext(url).with_credentials(credentials)
web = ctx.web
ctx.load(web)
ctx.execute_query()
print("Web site title: {0}".format(web.properties['Title']))
Here is an instruction on how to configure SharePoint App-Only flow:
Note: app principal registration operation(steps 1 through 5)
needs to be performed once per tenant. Although the operation for
granting permissions ( steps 6-9) could be applied either per tenant
or site collection:
permissions granted per site collection and requires a site collection administrator (in the provided instruction the permissions
are granter per site collection)
If you prefer to grant permissions on tenant level, visit tenant administration site instead, the URL must include -admin to access
the tenant administration site, for example,
https://{tenant}-admin.sharepoint.com/_layouts/15/appinv.aspx. That
operation requires a tenant administrator permissions
Steps:
Go to the appregnew.aspx page in your SharePoint Online site. For example, https://{tenant}.sharepoint.com/_layouts/15/appregnew.aspx.
On this page, click the Generate buttons next to the Client ID and Client Secret fields to generate their values.
Store the client ID and client secret securely as these credentials can be used to read or update all data in your SharePoint Online environment. You will also use them to configure the SharePoint Online connection in application.
Under Title, specify a title. For example, Python console. Under App Domain, specify localhost. Under Redirect URI, specify https://localhost.
Note: Sometimes, if you specify a actual domain, e.g. sharepoint.com domain in the App Domain and Redirect URI fields, instead of localhost, the error message An unexpected error has occurred might encounter. Check the appregnew.aspx page and make sure both fields include the proper localhost URI.
Click Create.
Go to the appinv.aspx page on the site collection. For example, https://example.sharepoint.com/_layouts/15/appinv.aspx to grant site-scoped permissions.
Specify your client ID in the App Id field and click Lookup to find your app.
To grant permissions to the app, copy the XML below to the App’s permission request XML field:
<AppPermissionRequests AllowAppOnlyPolicy="true">
<AppPermissionRequest Scope="http://sharepoint/content/sitecollection" Right="FullControl" />
</AppPermissionRequests>
Note: For tenant level scope, permission request XML looks as follows:
<AppPermissionRequests AllowAppOnlyPolicy="true">
<AppPermissionRequest Scope="http://sharepoint/content/tenant" Right="FullControl" />
</AppPermissionRequests>
Click Create.
On the confirmation dialog, click Trust It to grant the permissions.
Anyone know if this is possible?
I just want to automate dropping some documents into my onedrive for business account.
I tried
import onedrivesdk
from onedrivesdk.helpers import GetAuthCodeServer
from onedrivesdk.helpers.resource_discovery import ResourceDiscoveryRequest
redirect_uri = 'http://localhost:8080'
client_id = 'appid'
client_secret = 'mysecret'
discovery_uri = 'https://api.office.com/discovery/'
auth_server_url='https://login.live.com/oauth20_authorize.srf?scope=wl.skydrive_update'
#auth_server_url='https://login.microsoftonline.com/common/oauth2/authorize',
auth_token_url='https://login.microsoftonline.com/common/oauth2/token'
http = onedrivesdk.HttpProvider()
auth = onedrivesdk.AuthProvider(http,
client_id,
auth_server_url=auth_server_url,
auth_token_url=auth_token_url)
auth_url = auth.get_auth_url(redirect_uri)
code = GetAuthCodeServer.get_auth_code(auth_url, redirect_uri)
auth.authenticate(code, redirect_uri, client_secret, resource=resource)
# If you have access to more than one service, you'll need to decide
# which ServiceInfo to use instead of just using the first one, as below.
service_info = ResourceDiscoveryRequest().get_service_info(auth.access_token)[0]
auth.redeem_refresh_token(service_info.service_resource_id)
client = onedrivesdk.OneDriveClient(service_info.service_resource_id + '/_api/v2.0/', auth, http)
I registered an APP and got a secret and id. But when I ran this I got scope is invalid errors. Plus it tries to launch a webpage which isn't great for a command line kinda environment. I think this SDK might be outdated as well because originally this script had login.microsoftonline, but that wasn't reachable so I changed it to login.live.com.
I wrote this sample code you posted. You replaced the auth_server_URLwith the authentication URL for Microsoft Account authentication, which can only be used to access OneDrive (the consumer product). You need to continue using the login.microsoftonline.com URL to log into your OneDrive for Business account.
You are correct that this pops up a dialog. However, you can write a little supporting code so that only happens the first time you log into a particular app. Follow these steps (assuming you are using the default implementation of AuthProvider:
Use the sample code above up through the line auth.redeem_refresh_token()
The AuthProvider will now have a Session object, which caches the credentials of the current user and session. Use AuthProvider.save_session() to save the credentials for later.
Next time you start your app, use AuthProvider.load_session() and AuthProvider.refresh_token() to retrieve the previous session and refresh the auth token. This will all be headless.
Take note that the default implementation of SessionBase (found here) uses Pickle and is not safe for product use. Make sure to create a new implementation of Session if you intend to deploy this app to other users.
Onerive's website shows "Not Yet" on "OneDrive SDK for Python" to "OneDrive for Business"
https://dev.onedrive.com/SDKs.htm
The github sample codes did not work for me either, it tried to popup a window of authentication, but IE can not find the address:
http://('https//login.microsoftonline.com/common/oauth2/authorize',)?redirect_uri=http%3A%2F%2Flocalhost%3A8080&client_id=034xxxx9-9xx8-4xxf-bexx-1bc5xxxxbd0c&response_type=code
or removed all the "-" in client id
http://('https//login.microsoftonline.com/common/oauth2/authorize',)?redirect_uri=http%3A%2F%2Flocalhost%3A8080&client_id=034xxxx99xx84xxfbexx1bc5xxxxbd0c&response_type=code
Either way, I got the same result, IE did not show the popup with a line "This page can’t be displayed"