I'm using the simple-smartsheet library for read data from a sheet in Smartsheet and download existing attachments on each row of the sheet.
I can already read the data for each row, however I cannot download existing attachments.
import config
from simple_smartsheet import Smartsheet
sheet = smartsheet.sheets.get(id=config.SHEET_ID)
for row in sheet.rows:
attachments = row.attachments
print(attachments)
when executing the above command I get as a result:
[]
simple-smartsheet
I use the simple-smartsheet library as it is the only one that supports python versions 3.6+
my python version 3.7.5
You can use list_row_attachments to find information of the attachments that belongs to a row.
The code might look like this:
import config
from simple_smartsheet import Smartsheet
sheet = smartsheet.sheets.get(id=config.SHEET_ID)
for row in sheet.rows:
response = smartsheet_client.Attachments.list_row_attachments(
config.SHEET_ID,
row.id,
include_all=True
)
attachments = response.data
print(attachments)
my solution is not very pythonic, but works, it consist of 2 steps
Get the attachment links
Save the file to a local HDD (I'm doing backups too) as a pivot place
1. to get the list of attachments:
import smartsheet
import urllib.request
smart = smartsheet.Smartsheet()
att_list = smart.Attachments.list_all_attachments(<sheet_id>, include_all=True)
2. Downloading the attachments to local disk, you need to create a loop to go through the list of attachments, you can also add your own conditions to discriminate which ones to download:
for attach in att_list:
att_id = attach.id #get the id of the attachment
att_name = attach.name # get the name of the attachment
retrieve_att = smart.Attachments.get_attachment(<sheet id>, att_id) #downloads the atachment
dest_dir = "C:\\path\\to\\folder\\"
dest_file = destd+str(att_name) # parsing the destination path
dwnld_url = retrieve_att.url # this link gives you access to download the file for about 5 to 10 min. before expire
urllib.request.urlretrieve(dwnld_url, dest_file) ## retrieving attachement and saving locally
Now you have the file and you can do whatever you need with it
It looks like that library has not implemented logic for dealing with the attachments yet.
as an alternative to solving this problem I implemented a solution with the code below:
import requests
#token = 'Your smartsheet Token'
#sheetId = 'Your sheet id'
r = requests.get('https://api.smartsheet.com/2.0/sheets/{sheetId}/rows/{rowId}/attachments', headers={'Authorization': f'Bearer {token}'})
response_json = r.json()
print(response_json)
see Get Attachments for more details on handling attachments Smartsheets
Related
I'm trying to use python to get some data that is in an attachment on an outlook email and then use that data in python. I've managed to write the code that will get into the outlook inbox and folder I want and then get the attachments of a specific message, however I'm not sure how to view the content of that attachment. A lot of the other questions and tutorials I've found seem to be more related to saving the attachment in a folder location rather than viewing the attachment in python itself.
For context the data I'm trying to get to is an exported report from adobe analytics, this report is a csv file that is attached to an email as a zip file. The CSV file shows some data for a specific time period and I'm planning on scheduling this report to run weekly so what I want to do is get python to look through all the emails with this report on then stack all this data into one dataframe so that I have all the history plus the latest week's data in one place then export this file out.
Please find the code below that I've written so far. If you need more details or I haven't explained anything very well please let me know. I am fairly new to python especially the win32com library so there might be obvious stuff I'm missing.
#STEP 1---------------------------------------------
#import all methods needed
from pathlib import Path
import win32com.client
import requests
import time
import datetime
import os
import zipfile
from zipfile import ZipFile
import pandas as pd
#STEP 2 --------------------------------------------
#connect to outlook
outlook = win32com.client.Dispatch("Outlook.Application").GetNamespace("MAPI")
#STEP 3 --------------------------------------------
#connect to inbox
inbox = outlook.GetDefaultFolder(6)
#STEP 4 --------------------------------------------
#connect to adobe data reports folder within inbox
adobe_data_reports_folder = inbox.Folders['Cust Insights'].Folders['Adobe data reports']
#STEP 5 --------------------------------------------
#get all messages from adobe reports folder
messages_from_adr_folder = adobe_data_reports_folder.Items
#STEP 6 ---------------------------------------------
#get attachement for a specific message (this is just for testing in real world I'll do this for all messages)
for message in messages_from_adr_folder:
if message.SentOn.strftime("%d-%m-%y") == '07-12-22':
attachment = message.Attachments
else:
pass
#STEP 7 ----------------------------------------------
#get the content of the attachment
##????????????????????????????
With the Outlook Object Model, the best you can do is save the attachment as a file (Attachment.SaveAsFile) - keep in mind that MailItem.Attachments property returns the Attachments collection, not a single Attachment object - loop through all attachments in the collection, figure out which one you want (if there is more than one), and save it as file.
To access file attachment data directly without saving as a file, you will need to use Extended MAPI (C++ or Delphi only) or Redemption (any language, I am its author).
Dmitry mentioned below that there isn't the option to view attachment content with an outlook object model.
So I've come up with a solution for this which basically involves using the save method to save the attachment into a folder location on the current working directory and then once that file is save just load that file back up into python as a dataframe. The only thing to note is that I've added an if statement that only saves files that are csvs, obviously this part can be removed if needed.
If you wanted to do this with multiple files and stack all of these into a single dataframe then I just created a blank dataframe at the start (with the correct column names of the file that will be loaded) and concatenated this blank dataframe with the "importeddata" then added this code into the "attachment" for loop so that each time it's appending the data that is saved and loaded from the attachment
#STEP 1---------------------------------------------
#import all methods needed
from pathlib import Path
import win32com.client
import requests
import time
import datetime
import os
import zipfile
from zipfile import ZipFile
import pandas as pd
#STEP 1b ---------------------------------------------
#create a directory where I can save the files
output_dir = Path.cwd() / "outlook_testing"
#STEP 2 --------------------------------------------
#connect to outlook
outlook = win32com.client.Dispatch("Outlook.Application").GetNamespace("MAPI")
#STEP 3 --------------------------------------------
#connect to inbox
inbox = outlook.GetDefaultFolder(6)
#STEP 4 --------------------------------------------
#connect to adobe data reports folder within inbox
adobe_data_reports_folder = inbox.Folders['Cust Insights'].Folders['Adobe data
reports']
#STEP 5 --------------------------------------------
#get all messages from adobe reports folder
messages_from_adr_folder = adobe_data_reports_folder.Items
#STEP 6 ---------------------------------------------
#get attachement for a specific message (this is just for testing in real world
#I'll do this for all messages)
for message in messages_from_adr_folder:
body = message.Body
if message.SentOn.strftime("%d-%m-%y") == '07-12-22':
attachments = message.Attachments
for attachment in attachments:
stringofattachment = str(attachment)
#STEP 6b - if the attachment is a csv file then save the attachment to a folder
if stringofattachment.find('.csv') != - 1:
attachment.SaveAsFile(output_dir / str(attachment))
print(output_dir / str(attachment))
#STEP 6C - reload the saved file as a dataframe
importeddata = pd.read_csv(output_dir / str(attachment))
else:
print('NOT CSV')
pass
else:
pass
I'm looking for a quick way to get the source code of a smartcontract. I tried using the following python code:
import requests
import json
address = "0xBB9bc244D798123fDe783fCc1C72d3Bb8C189413"
api_key = my_api_key
request_string = f'''https://api.etherscan.io/api?module=contract&action=getsourcecode&address={address}&apikey={api_key}'''
response = requests.get(request_string)
print(response.text)
data = json.loads(response.text)['result'][0]['SourceCode']
file = open("contract.sol", "w")
a = file.write(data)
file.close()
So while this works for the given address, it doesn't work if the source code consists of multiple files (like with this address: 0xED5AF388653567Af2F388E6224dC7C4b3241C544). You can see there's 13 individual files. So is there a quick and easy way to save all of them into one file? Or do I just have to create a separate file for each one of the files?
I am an absolute beginner when it comes to working with REST APIs with python. We have received a share-point URL which has multiple folders and multiples files inside those folders in the 'document' section. I have been provided an 'app_id' and a 'secret_token'.
I am trying to access the .csv file and read them as a dataframe and perform operations.
The code for operation is ready after I downloaded the .csv and did it locally but I need help in terms of how to connect share-point using python so that I don't have to download such heavy files ever again.
I know there had been multiple queries already on this over stack-overflow but none helped to get to where I want.
I did the following and I am unsure of what to do next:
import json
from office365.runtime.auth.user_credential import UserCredential
from office365.sharepoint.client_context import ClientContext
from office365.runtime.http.request_options import RequestOptions
site_url = "https://<company-name>.sharepoint.com"
ctx = ClientContext(site_url).with_credentials(UserCredential("{app_id}", "{secret_token}"))
Above for site_url, should I use the whole URL or is it fine till ####.com?
This is what I have so far, next I want to read files from respective folders and convert them into a dataframe? The files will always be in .csv format
The example hierarchy of the folders are as follows:
Documents --> Folder A, Folder B
Folder A --> a1.csv, a2.csv
Folder B --> b1.csv, b2.csv
I should be able to move to whichever folder I want and read the files based on my requirement.
Thanks for the help.
This works for me, using a Sharepoint App Identity with an associated client Id and client Secret.
First, I demonstrate authenticating and reading a specific file, then getting a list of files from a folder and reading the first one.
import pandas as pd
import json
import io
from office365.sharepoint.client_context import ClientCredential
from office365.sharepoint.client_context import ClientContext
from office365.sharepoint.files.file import File
#Authentication (shown for a 'modern teams site', but I think should work for a company.sharepoint.com site:
site="https://<myteams.companyname.com>/sites/<site name>/<sub-site name>"
#Read credentials from a json configuration file:
spo_conf = json.load(open(r"conf\spo.conf", "r"))
client_credentials = ClientCredential(spo_conf["RMAppID"]["clientId"],spo_conf["RMAppID"]["clientSecret"])
ctx = ClientContext(site).with_credentials(client_credentials)
#Read a specific CSV file into a dataframe:
folder_relative_url = "/sites/<site name>/<sub site>/<Library Name>/<Folder Name>"
filename = "MyFileName.csv"
response = File.open_binary(ctx, "/".join([folder_relative_url, filename]))
df = pd.read_csv(io.BytesIO(response.content))
#Get a list of file objects from a folder and read one into a DataFrame:
def getFolderContents(relativeUrl):
contents = []
library = ctx.web.get_list(relativeUrl)
all_items = library.items.filter("FSObjType eq 0").expand(["File"]).get().execute_query()
for item in all_items: # type: ListItem
cur_file = item.file
contents.append(cur_file)
return contents
fldrContents = getFolderContents('/sites/<site name>/<sub site>/<Library Name>')
response2 = File.open_binary(ctx, fldrContents[0].serverRelativeUrl)
df2 = pd.read_csv(io.BytesIO(response2.content))
Some References:
Related SO thread.
Office365 library github site.
Getting a list of contents in a doc library folder.
Additional notes following up on comments:
The site path doesn't not include the full url for the site home page (ending in .aspx) - it just ends with the name for the site (or sub-site, if relevant to your case).
You don't need to use a configuration file to store your authentication credentials for the Sharepoint application identity - you could just replace spo_conf["RMAppID"]["clientId"] with the value for the Sharepoint-generated client Id and do similarly for the client Secret. But this is a simple example of what the text of a JSON file could look like:
{
"MyAppName":{
"clientId": "my-client-id",
"clientSecret": "my-client-secret",
"title":"name_for_application"
}
}
I was working on the python confluence API for downloading attachment from confluence page, I need to download only files with .mpp extension. Tried with glob and direct parameters but didnt work.
Here is my code:
file_name = glob.glob("*.mpp")
attachments_container = confluence.get_attachments_from_content(page_id=33110, start=0, limit=1,filename=file_name)
print(attachments_container)
attachments = attachments_container['results']
for attachment in attachments:
fname = attachment['title']
download_link = confluence.url + attachment['_links']['download']
r = requests.get(download_link, auth = HTTPBasicAuth(confluence.username,confluence.password))
if r.status_code == 200:
if not os.path.exists('phoenix'):
os.makedirs('phoenix')
fname = ".\\phoenix\\" +fname
glob.glob() operates on your local folder. So you can't use that as a filter for get_attachments_from_content(). Also, don't specify a limit of since that gets you just one/the first attachment. Specify a high limit or whatever default will include all of them. (You may have to paginate results.)
However, you can exclude the files you don't want by checking the title of each attachment before you download it, which you have as fname = attachment['title'].
attachments_container = confluence.get_attachments_from_content(page_id=33110, limit=1000)
attachments = attachments_container['results']
for attachment in attachments:
fname = attachment['title']
if not fname.lower().endswith('.mpp'):
# skip file if it's not got that extension
continue
download_link = ...
# rest of your code here
Also, your code looks like a copy-paste from this answer but you've changed the actual "downloading" part of it. So if your next StackOverflow question is going to be "how to download a file from confluence", use that answer's code.
I am a python developer and somewhat new to using Google's gMail API to import .eml files into a gMail account.
I've gotten all of the groundwork done getting my oAuth credentials working, etc.
However, I am stuck where I load in the data-file. I need help loading the message data in to place in a variable..
How do I create the message_data variable reference - in the appropriate format - from my sample email file (which is stored in rfc822 format) that is on disk?
Assuming I have a file on disk at /path/to/file/sample.eml ... how do I load that to message_data in the proper format for the gMail API import call?
...
# how do I properly load message_data from the rfc822 disk file?
media = MediaIoBaseUpload(message_data, mimetype='message/rfc822')
message_response = service.users().messages().import_(
userId='me',
fields='id',
neverMarkSpam=True,
processForCalendar=False,
internalDateSource='dateHeader',
media_body=media).execute(num_retries=2)
...
You want to import an eml file using Gmail API.
You have already been able to get and put values for Gmail API.
You want to achieve this using google-api-python-client.
service in your script can be used for uploading the eml file.
If my understanding is correct, how about this answer? Please think of this as just one of several possible answers.
Modification point:
In this case, the method of "Users.messages: insert" is used.
Modified script:
Before you run the script, please set the filename with the path of the eml file.
eml_file = "###" # Please set the filename with the path of the eml file.
user_id = "me"
f = open(eml_file, "r", encoding="utf-8")
eml = f.read()
f.close()
message_data = io.BytesIO(eml.encode('utf-8'))
media = MediaIoBaseUpload(message_data, mimetype='message/rfc822', resumable=True)
metadata = {'labelIds': ['INBOX']}
res = service.users().messages().insert(userId=user_id, body=metadata, media_body=media).execute()
print(res)
In above script, the following modules are also required.
import io
from googleapiclient.http import MediaIoBaseUpload
Note:
In above modified script, {'labelIds': ['INBOX']} is used as the metadata. In this case, the imported eml file can be seen at INBOX of Gmail. If you want to change this, please modify this.
Reference:
Users.messages: insert
If I misunderstood your question and this was not the result you want, I apologize.