We Keep Coding

Sharepy library freezes at entering username in PyCharm

I'm trying to authenticate to SharePoint Online. Using sharepy v 2.0, pyCharm community edition, and python 3.9.
When I run:
'sharepy.connect('siteurl')'
From within PyCharm, Sharepy will freeze after I input my username in the run dialog box.
If I add the 'username' parameter and run it. Nothing happens. I'm never prompted for a password
If I use the console and enter in sharepy.connect('siteurl') then username and password (same goes for passing those parameters) I will get an error:
Traceback (most recent call last):
File "C:\Users\Andrew\AppData\Local\Programs\Python\Python39\lib\site-packages\sharepy\auth\adfs.py", line 75, in _get_token
token = root.find('.//wsse:BinarySecurityToken', ns).text
AttributeError: 'NoneType' object has no attribute 'text'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Andrew\AppData\Local\Programs\Python\Python39\lib\code.py", line 90, in runcode
exec(code, self.locals)
File "<input>", line 1, in <module>
File "C:\Users\Andrew\AppData\Local\Programs\Python\Python39\lib\site-packages\sharepy\session.py", line 15, in connect
return SharePointSession(site, auth=autoauth)
File "C:\Users\Andrew\AppData\Local\Programs\Python\Python39\lib\site-packages\sharepy\session.py", line 61, in __init__
self.auth.login(self.site)
File "C:\Users\Andrew\AppData\Local\Programs\Python\Python39\lib\site-packages\sharepy\auth\adfs.py", line 27, in login
self._get_token()
File "C:\Users\Andrew\AppData\Local\Programs\Python\Python39\lib\site-packages\sharepy\auth\adfs.py", line 77, in _get_token
raise errors.AuthError('Token request failed. Invalid server response')
sharepy.errors.AuthError: Token request failed. Invalid server response
It should be noted I'm getting O365 from godaddy and the login page is federated? I think is the correct term.
According to the new release of Sharepy, this shouldn't matter.
Has anyone else had this freezing problem happen for them?
How would I authenticate with sharepoint using sharepy given my current situation?

The source of this problem ended up being GoDaddy. As we were federated using GoDaddy as the O365 provider. There was no way to authenticate correctly using sharepy.
The ultimate solution was to defederate away from GoDaddy (pretty easy to do thanks to this guy: Defederation Guide)
The reason we were unable to authenticate was because our provider redirects the login to their own login site. And unfortunately the sharepy builtin method of "auth" wouldn't work with GoDaddy.
I tested this theory before migrating away from GoDaddy. By using a fresh tenant. I also found that when you enable MFA the password/username method of authentication doesn't work.
NOTE: When new tenants are created they utilize a blanket security protocol which forces MFA. Even though MFA is shown as disabled in the Azure AD > Users section. To turn this off you must disable "Security Defaults": portal.azure.com > Azure Active Directory > Properties > "Manage security defaults" (at the bottom of the screen, its a small hyperlink).
A note on MFA and authentication with sharepy. There are methods to leave MFA enabled which work with other sharepoint/python things. I haven't tested them using sharepy yet, but will be turning on MFA and using one of the following methods:
App Password
Sharepoint API client secret
Azure App Registration (Azure App Reg)

Azure ML Python SDK Unable to get MSI token using identity secret

I've been using the Azure ML Python SDK to create pipelines for weeks now, but all of the sudden I started getting this error when trying to get the default datastore
ws = Workspace.from_config()
def_blob_store = ws.get_default_datastore()
Traceback (most recent call last): File "lstm_evaluate_pipeline.py",
line 14, in
def_blob_store = ws.get_default_datastore() File "/opt/anaconda3/envs/azure_ml/lib/python3.8/site-packages/azureml/core/workspace.py",
line 1154, in get_default_datastore
return _DatastoreClient.get_default(self) File "/opt/anaconda3/envs/azure_ml/lib/python3.8/site-packages/azureml/data/datastore_client.py",
line 699, in get_default
return _DatastoreClient._get_default(workspace) File "/opt/anaconda3/envs/azure_ml/lib/python3.8/site-packages/azureml/data/_exception_handler.py",
line 19, in decorated
raise UserErrorException(str(e)) azureml.exceptions._azureml_exception.UserErrorException:
UserErrorException: Message: (UserError) Unable to get MSI token
using identity secret. The application associated with this managed
identity InnerException None ErrorResponse {
"error": {
"code": "UserError",
"message": "(UserError) Unable to get MSI token using identity secret. The application associated with this managed identity"
} }
How can I fix this? I'm running this on MacOS Monterey in a conda environment using Python 3.8. The sdk version is 1.42.0

As noticed that the issue is with blob storage, check the following documentation, how to configure the datastore.
Source Document: Link Link2

python sharepoint - Error validating credentials due to invalid username or password

I want to automate downloading a zipfile on another organization's sharepoint site that they have shared with my organization, inviting us as a guest using a generic company email address so we can access the files. To gain access, I provide the company email address as username and its associated AD password which authenticates access through our MS 365 organization account.
This works fine manually, but when attempting to do it through shareplum I get a credentials error.
Here is my code:
from shareplum import Site
from shareplum import Office365
from shareplum.site import Version
sharepointUsername = 'elvis.presley#suspiciousminds.org'
sharepointPassword = 'oogabooga123'
sharepointSite = 'https://abc.sharepoint.com/sites/ClientLogin/Shared Documents/Graceland/'
authcookie = Office365(website, username=sharepointUsername,
password=sharepointPassword).GetCookies()
site = Site(sharepointSite, version=Version.v365, authcookie=authcookie)
And here is the error:
Traceback (most recent call last):
File "<string>", line 11, in <module>
File "C:\arcgispro-py3-clone2\lib\site-packages\shareplum\office365.py", line 88, in get_cookies
sectoken = self.get_security_token(self.username, self.password)
File "C:\arcgispro-py3-clone2\lib\site-packages\shareplum\office365.py", line 80, in get_security_token
raise Exception('Error authenticating against Office 365. Error from Office 365:', message[0].text)
Exception: ('Error authenticating against Office 365. Error from Office 365:', 'AADSTS50126: Error validating credentials due to invalid username or password.')
Any clue as why this is not working? I've checked the credentials several times using them to log in manually and it works every time (obviously the example does not have the real username/password)?

You’re not alone, this seems to be a common issue with shareplum over the past 6 months. There’s an open issue on GitHub. The code owner is aware and is having difficulty debugging the issue.
GitHub issue: https://github.com/jasonrollins/shareplum/issues/84
I recommend working closely with the code owner to debug and resolve the issue on GitHub, or using a different library/solution altogether. Another user commented on the issue and referenced a solution that utilizes the requests library to retrieve a file that you may find useful: https://github.com/jasonrollins/shareplum/issues/84#issuecomment-627986466 .

Accessing Office 365 ProPlus OneDrive folder using the official Python SDK

We are currently trying to access a folder of an Office 365 ProPlus tenant using the official OneDrive SDK for Python (https://github.com/OneDrive/onedrive-sdk-python). One of our clients would like to use a OneDrive folder as a way of storing and sharing programmatically generated files, therefore, we would like to provide basic file operations.
We have a working solution for a personal OneDrive account, however, when we try to apply the same approach for their OneDrive, we face an issue during the authentication process.
We asked them to register the application in the Azure AD following the steps in the official documentation. Next, they sent us the redirect URI, client ID and client secret that we included in our script. We are trying to use the following code:
redirect_uri = 'REDIRECT_URI'
client_secret = 'CLIENT_SECRET'
client_id='CLIENT_ID'
discovery_uri = 'https://api.office.com/discovery/'
auth_server_url='https://login.microsoftonline.com/common/oauth2/authorize'
auth_token_url='https://login.microsoftonline.com/common/oauth2/token'
http_provider = onedrivesdk.HttpProvider()
auth_provider = onedrivesdk.AuthProvider(http_provider,
client_id,
auth_server_url=auth_server_url,
auth_token_url=auth_token_url)
auth_url = auth_provider.get_auth_url(redirect_uri)
code = GetAuthCodeServer.get_auth_code(auth_url, redirect_uri)
However, we get the following error message when executing the last line:
Traceback (most recent call last):
File "onedrive-test.py", line 25, in
code = GetAuthCodeServer.get_auth_code(auth_url, redirect_uri)
File "/home/username/.local/lib/python3.6/site-packages/onedrivesdk/helpers/GetAuthCodeServer.py",
line 60, in get_auth_code
s = GetAuthCodeServer((host_address, port), code_acquired, GetAuthCodeRequestHandler)
File "/home/username/.local/lib/python3.6/site-packages/onedrivesdk/helpers/GetAuthCodeServer.py",
line 76, in init
HTTPServer.init(self, server_address, RequestHandlerClass)
File "/usr/lib/python3.6/socketserver.py", line 453, in init
self.server_bind()
File "/usr/lib/python3.6/http/server.py", line 136, in server_bind
socketserver.TCPServer.server_bind(self)
File "/usr/lib/python3.6/socketserver.py", line 467, in server_bind
self.socket.bind(self.server_address)
socket.gaierror: [Errno -2] Name or service not known
We also tried opening the auth_url manually, which took us one step further, but still could not authenticate the application with the following error:
AADSTS50020: User account 'USER ACCOUNT' from identity provider
'live.com' does not exist in tenant 'TENANT NAME' and cannot access
the application 'CLIENT ID' in that tenant. The account needs to be
added as an external user in the tenant first. Sign out and sign in
again with a different Azure Active Directory user account.
We have two questions:
What might casue the first error? This is the comment (see below) that can be found in the readme of the SDK about using the GetAuthCodeServer class. It seems to us that the server cannot be run. Are there any not explicitly defined dependencies that we should be aware of before trying to run the webserver? (We are running the script on Ubuntu 18.10)
If you want to remove some of that manual work, you can
use the helper class GetAuthCodeServer. That helper class spins up a
webserver, so this method cannot be used on all environments.
With respect to the second issue, can you recommend proper material for configuring OneDrive for Business for our use-case? We went through a lot of documentation, but after long hours of research, we still could not find the correct way to fix that issue, especially since we do not have direct acces to the tenant and we cannot easily experiment with things. We would need to give a step-by-step cookbook to our client to set up everything on their side.
Any help would be much appreciated! :)

GCS with GKE, 403 Insufficient permission for writing into GCS bucket [duplicate]

This question already has an answer here:
Is it necessary to recreate a Google Container Engine cluster to modify API permissions?
(1 answer)
Closed 5 years ago.
Currently I'm trying to write files into Google Cloud Storage bucket. For this, I have used django-storages package.
I have deployed my code and I get into the running container through kubernetes kubectl utility to check the working of GCS bucket.
$ kubectl exec -it foo-pod -c foo-container --namespace=testing python manage.py shell
I can able to read the bucket but if I try to write into the bucket, it shows the below traceback.
>>> from django.core.files.storage import default_storage
>>> f = default_storage.open('storage_test', 'w')
>>> f.write('hi')
2
>>> f.close()
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/google/cloud/storage/blob.py", line 946, in upload_from_file
client, file_obj, content_type, size, num_retries)
File "/usr/local/lib/python3.6/site-packages/google/cloud/storage/blob.py", line 867, in _do_upload
client, stream, content_type, size, num_retries)
File "/usr/local/lib/python3.6/site-packages/google/cloud/storage/blob.py", line 700, in _do_multipart_upload
transport, data, object_metadata, content_type)
File "/usr/local/lib/python3.6/site-packages/google/resumable_media/requests/upload.py", line 98, in transmit
self._process_response(result)
File "/usr/local/lib/python3.6/site-packages/google/resumable_media/_upload.py", line 110, in _process_response
response, (http_client.OK,), self._get_status_code)
File "/usr/local/lib/python3.6/site-packages/google/resumable_media/_helpers.py", line 93, in require_status_code
status_code, u'Expected one of', *status_codes)
google.resumable_media.common.InvalidResponse: ('Request failed with status code', 403, 'Expected one of', <HTTPStatus.OK: 200>)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/usr/local/lib/python3.6/site-packages/storages/backends/gcloud.py", line 75, in close
self.blob.upload_from_file(self.file, content_type=self.mime_type)
File "/usr/local/lib/python3.6/site-packages/google/cloud/storage/blob.py", line 949, in upload_from_file
_raise_from_invalid_response(exc)
File "/usr/local/lib/python3.6/site-packages/google/cloud/storage/blob.py", line 1735, in _raise_from_invalid_response
raise exceptions.from_http_response(error.response)
google.api_core.exceptions.Forbidden: 403 POST https://www.googleapis.com/upload/storage/v1/b/foo.com/o?uploadType=multipart: Insufficient Permission
>>> default_storage.url('new docker')
'https://storage.googleapis.com/foo.appspot.com/new%20docker'
>>>
Seems like it was completely related to the bucket permissions. So I have assigned Storage admin , Storage object creator roles to google cloud build service account (through bucket -> manage permissions) but still it shows the same error.

A possible explanation for this would be if you haven't assigned your cluster with the correct scope. If this is the case, the nodes in the cluster would not have the required authorisation/permission to write to Google Cloud Storage which could explain the 403 error you're seeing.
If no scope is set when the cluster is created, the default scope is assigned and this only provides read permission for Cloud Storage.
In order to check the clusters current scopes using Cloud SDK you could try running a 'describe' command from the Cloud Shell, for example:
gcloud container clusters describe CLUSTER-NAME --zone ZONE
The oauthScopes section of the output contains the current scopes assigned to the cluster/nodes.
The default read only Cloud Storage scope would display:
https://www.googleapis.com/auth/devstorage.read_only
If the Cloud Storage read/write scope is set the output will display:
https://www.googleapis.com/auth/devstorage.read_write
The scope can be set during cluster creation using the --scope switch followed by the desired scope identifier. In your case, this would be “storage-rw”. For example, you could run something like:
gcloud container clusters create CLUSTER-NAME --zone ZONE --scopes storage-rw
The storage-rw scope, combined with your service account should then allow the nodes in your cluster to write to Cloud Storage.
Alternatively you if you don't want to recreate the cluster you can create a new node pool with the new desired scopes, then delete your old node pool. See the accepted answer for Is it necessary to recreate a Google Container Engine cluster to modify API permissions? for information on how to achieve this.