Python, how find out the path of Dropbox and upload a file there
I want to upload daily csv file to dropbox account, but I'm getting ValidationError and others.
my code:
#finding the path
import pathlib
import dropbox
import os
# Automation is the name of my folder at dropbox
pathlib.Path.home() / "Automation"
Out[37]: WindowsPath('C:/Users/pb/Automation')
dbx = dropbox.Dropbox('My-token here')
dbx.users_get_current_account()
Out[38]: FullAccount(account_id='accid', name=Name(given_name='pb', surname='manager', familiar_name='pb', display_name='pb', abbreviated_name='pb'), email='example#example.com', email_verified=True, disabled=False, locale='en', referral_link='https://www.dropbox.com/referrals/codigo', is_paired=False, account_type=AccountType('basic', None), root_info=UserRootInfo(root_namespace_id='1111111111', home_namespace_id='11111111'), profile_photo_url='https://dl-web.dropbox.com/account_photo/get/sssssssssssssssssss', country='US', team=None, team_member_id=None)
# Now trying to see something in the folder, I just want upload file there
response = dbx.files_list_folder(path='user:/pb/automation')
print(response)
for entry in dbx.files_list_folder('https://www.dropbox.com/home/automation').entries:
print(entry.name)
ValidationError: 'user:/pb/automation' did not match pattern '(/(.|[\r\n])*)?|id:.*|(ns:[0-9]+(/.*)?)'
That error happens because the path parameter that the API is expecting needs to start with a '/'. It could be called out better in the docs.
Is the Automation folder in the root of your Dropbox directory? If so, then '/automation' should be sufficient for path. Try tinkering with the /files/list_folder endpoint in the Dropbox API explorer until you find the correct path.
Your for loop is likely to throw an error too, though. Are you just trying to loop over the results of the list_folder call? I'd suggest changing to
for entry in response:
print entry
Related
I am an absolute beginner when it comes to working with REST APIs with python. We have received a share-point URL which has multiple folders and multiples files inside those folders in the 'document' section. I have been provided an 'app_id' and a 'secret_token'.
I am trying to access the .csv file and read them as a dataframe and perform operations.
The code for operation is ready after I downloaded the .csv and did it locally but I need help in terms of how to connect share-point using python so that I don't have to download such heavy files ever again.
I know there had been multiple queries already on this over stack-overflow but none helped to get to where I want.
I did the following and I am unsure of what to do next:
import json
from office365.runtime.auth.user_credential import UserCredential
from office365.sharepoint.client_context import ClientContext
from office365.runtime.http.request_options import RequestOptions
site_url = "https://<company-name>.sharepoint.com"
ctx = ClientContext(site_url).with_credentials(UserCredential("{app_id}", "{secret_token}"))
Above for site_url, should I use the whole URL or is it fine till ####.com?
This is what I have so far, next I want to read files from respective folders and convert them into a dataframe? The files will always be in .csv format
The example hierarchy of the folders are as follows:
Documents --> Folder A, Folder B
Folder A --> a1.csv, a2.csv
Folder B --> b1.csv, b2.csv
I should be able to move to whichever folder I want and read the files based on my requirement.
Thanks for the help.
This works for me, using a Sharepoint App Identity with an associated client Id and client Secret.
First, I demonstrate authenticating and reading a specific file, then getting a list of files from a folder and reading the first one.
import pandas as pd
import json
import io
from office365.sharepoint.client_context import ClientCredential
from office365.sharepoint.client_context import ClientContext
from office365.sharepoint.files.file import File
#Authentication (shown for a 'modern teams site', but I think should work for a company.sharepoint.com site:
site="https://<myteams.companyname.com>/sites/<site name>/<sub-site name>"
#Read credentials from a json configuration file:
spo_conf = json.load(open(r"conf\spo.conf", "r"))
client_credentials = ClientCredential(spo_conf["RMAppID"]["clientId"],spo_conf["RMAppID"]["clientSecret"])
ctx = ClientContext(site).with_credentials(client_credentials)
#Read a specific CSV file into a dataframe:
folder_relative_url = "/sites/<site name>/<sub site>/<Library Name>/<Folder Name>"
filename = "MyFileName.csv"
response = File.open_binary(ctx, "/".join([folder_relative_url, filename]))
df = pd.read_csv(io.BytesIO(response.content))
#Get a list of file objects from a folder and read one into a DataFrame:
def getFolderContents(relativeUrl):
contents = []
library = ctx.web.get_list(relativeUrl)
all_items = library.items.filter("FSObjType eq 0").expand(["File"]).get().execute_query()
for item in all_items: # type: ListItem
cur_file = item.file
contents.append(cur_file)
return contents
fldrContents = getFolderContents('/sites/<site name>/<sub site>/<Library Name>')
response2 = File.open_binary(ctx, fldrContents[0].serverRelativeUrl)
df2 = pd.read_csv(io.BytesIO(response2.content))
Some References:
Related SO thread.
Office365 library github site.
Getting a list of contents in a doc library folder.
Additional notes following up on comments:
The site path doesn't not include the full url for the site home page (ending in .aspx) - it just ends with the name for the site (or sub-site, if relevant to your case).
You don't need to use a configuration file to store your authentication credentials for the Sharepoint application identity - you could just replace spo_conf["RMAppID"]["clientId"] with the value for the Sharepoint-generated client Id and do similarly for the client Secret. But this is a simple example of what the text of a JSON file could look like:
{
"MyAppName":{
"clientId": "my-client-id",
"clientSecret": "my-client-secret",
"title":"name_for_application"
}
}
I am a python developer and somewhat new to using Google's gMail API to import .eml files into a gMail account.
I've gotten all of the groundwork done getting my oAuth credentials working, etc.
However, I am stuck where I load in the data-file. I need help loading the message data in to place in a variable..
How do I create the message_data variable reference - in the appropriate format - from my sample email file (which is stored in rfc822 format) that is on disk?
Assuming I have a file on disk at /path/to/file/sample.eml ... how do I load that to message_data in the proper format for the gMail API import call?
...
# how do I properly load message_data from the rfc822 disk file?
media = MediaIoBaseUpload(message_data, mimetype='message/rfc822')
message_response = service.users().messages().import_(
userId='me',
fields='id',
neverMarkSpam=True,
processForCalendar=False,
internalDateSource='dateHeader',
media_body=media).execute(num_retries=2)
...
You want to import an eml file using Gmail API.
You have already been able to get and put values for Gmail API.
You want to achieve this using google-api-python-client.
service in your script can be used for uploading the eml file.
If my understanding is correct, how about this answer? Please think of this as just one of several possible answers.
Modification point:
In this case, the method of "Users.messages: insert" is used.
Modified script:
Before you run the script, please set the filename with the path of the eml file.
eml_file = "###" # Please set the filename with the path of the eml file.
user_id = "me"
f = open(eml_file, "r", encoding="utf-8")
eml = f.read()
f.close()
message_data = io.BytesIO(eml.encode('utf-8'))
media = MediaIoBaseUpload(message_data, mimetype='message/rfc822', resumable=True)
metadata = {'labelIds': ['INBOX']}
res = service.users().messages().insert(userId=user_id, body=metadata, media_body=media).execute()
print(res)
In above script, the following modules are also required.
import io
from googleapiclient.http import MediaIoBaseUpload
Note:
In above modified script, {'labelIds': ['INBOX']} is used as the metadata. In this case, the imported eml file can be seen at INBOX of Gmail. If you want to change this, please modify this.
Reference:
Users.messages: insert
If I misunderstood your question and this was not the result you want, I apologize.
I am trying to upload a spreadsheet on Sharepoint for which I am using REST API function.
The code that I am using for generating the url as well as uploading the file is-
import sys
import requests, os
from requests_ntlm import HttpNtlmAuth
sharePointUrl = 'https://Sharepoint.asr.ith.itl.com/Skt/patchboard'
folderUrl = '/Documents/Patch_automation_work_area'
fileName='/abc/asc/roj/skx/skx_val/rsingh/Patch/Excel.xlsm'
#Setting up the url for requesting a file upload
requestUrl = sharePointUrl + '/_api/web/getfolderbyserverrelativeurl(\'' + folderUrl + '\')/Files/addas(url=\'' + fileName + '\',overwrite=true)'
print(requestUrl)
When printing the URL generated getting the output as-
https://Sharepoint.asr.ith.itl.com/Skt/patchboard/_api/web/getfolderbyserverrelativeurl('/Documents/Patch_automation_work_area')/Files/addas(url='/abc/asc/roj/skx/skx_val/rsingh/Patch/Excel.xlsm',overwrite=true)
So the complete URL is not generated for uploading the file and it is showing 404 error when accessing the link using requests module in python. Can somebody please help me why I am getting this erroe and how to generate link for uploading the document??
EDIT
my link for upload is something like this
https://sharepoint.asr.ith.itl.com/sites/SK/patchboard/_layouts/Upload.aspx?List={CE897D7B-8DC4-4F9C-AF4D-D41DB89DA6D3}&RootFolder=%2Fsites%2FSKX%2Fpatchboard%2FDocuments%2FPatch%5Fautomation%5Fwork%5Farea
This link brings me to a page where in I need to browse the complete path to the file and then after giving the path I would be able to upload the document.
My file path is-
/abc/asc/roj/skx/skx_val/rsingh/Patch/Excel.xlsm
Now I want to concatenate this file path to my above url so that a path for direct upload can be formed.Direct Concatenation is not working as I think direct concatenation does not knows the meaning of browse option and may be that's while its unable to put the file path at its desired location.
Can somebody tell me how to resolve it.
I have resolved the problem. Instead of giving the url link from the browser,I have given the base url for the sharepoint like-
https://sharepoint.asr.ith.itl.com
and then added path to the desired location in the sharepoint where I wanted to upload the file like-
sites/SK/patchboard/shared_documents/patch_work_area
This formed the complete link as-
https://sharepoint.asr.ith.itl.com/sites/SK/patchboard/shared_documents/patch_work_area
then I have used the command as-
curl --ntlm --user username:password --upload-file <filename> https://sharepoint.amr.ith.intel.com/sites/SK/patchboard/shared_documents/patch_work_area/<file_name to upload>
This had worked for me.
I am trying to download an image from dropbox to my desktop using Python. The script below runs to completion without issues and creates a JPEG file on the desktop (about 200+ KB in size). But when I try to open it, I get a file damaged / Preview cannot read file error message:
import requests
from requests.auth import HTTPBasicAuth
import shutil
url = 'https://www.dropbox.com/rest_of_the_url'
db_username = 'user_name'
db_password = 'password'
downloaded_file = requests.get(url, auth=HTTPBasicAuth(db_username, db_password))
dest_file = open('/Users/aj/Desktop/test.jpg', 'w+')
dest_file.write(downloaded_file.content)
What am I doing wrong here?
EDIT: Found the solution. It had to do with the 'dl' parameter in the dropbox link. This parameter needs to be set to 0.
Original link:
https://www.dropbox.com/s/3xujisscbp92to/2.jpg?dl=0
Need to set the dl parameter to 1:
https://www.dropbox.com/s/3xujisscbpj92to/2.jpg?dl=1
Found the solution. It had to do with the 'dl' parameter in the dropbox link. This parameter needs to be set to 0.
Original link:
https://www.dropbox.com/s/3xujisscbp92to/2.jpg?dl=0
Need to set the dl parameter to 1:
https://www.dropbox.com/s/3xujisscbpj92to/2.jpg?dl=1
I need to get the content of the resources received in command line. The user can write a relative path to a file or an URL. Is it possible to read from this resource regardless if it is a path to a file or an URL?
In Ruby I have something like the next, but I'm having problems finding a Python alternative:
content = open(path_or_url) { |io| io.read }
I don't know of a nice way to do it, however, urllib.request.urlopen() will support opening normal URLs (http, https, ftp, etc) as well as files on the file system. So you could assume a file if the URL is missing a scheme component:
from urllib.parse import urlparse
from urllib.request import urlopen
resource = input('Enter a URL or relative file path: ')
if urlparse(resource).scheme == '':
# assume that it is a file, use "file:" scheme
resource = 'file:{}'.format(resource)
data = urlopen(resource).read()
This works for the following user input:
http://www.blah.com
file:///tmp/x/blah
file:/tmp/x/blah
file:x/blah # assuming cwd is /tmp
/tmp/x/blah
x/blah # assuming cwd is /tmp
Note that file: (without slashes) might not be a valid URI, however, this is the only way to open a file specified by relative path, and urlopen() works with such URIs.