I am trying to use the gspread Python package to import CSV data into a Google sheet from the command line.
Using this guide, I got everything working, and was able to both read and write to cells.
However updating cells 1-by-1 is too slow, so I am now trying to use the import_csv() method. The docs say:
import_csv(file_id, data)
Imports data into the first page of the spreadsheet.
Parameters: data – A CSV string of data.
file_id is not described here, and I can't work out what it should be. A few other methods also use a file_id and for them it is described as:
file_id – a spreadsheet ID (aka file ID.)
I am not sure where I find spreadsheet ID, and no matter what I try I get a permissions error. Since I am able to use update_cell(), as described above, I think I have permissions working fine but am using the wrong file_id.
Here's simplified code:
import gspread
from oauth2client.service_account import ServiceAccountCredentials
scope = ['https://spreadsheets.google.com/feeds']
creds = ServiceAccountCredentials.from_json_keyfile_name('client_secret.json', scope)
client = gspread.authorize(creds)
sheet = client.open("SheetTitle").sheet1
# This works fine, so I think permissions etc are all set up correctly
sheet.update_cell(1, 1, 'Foo')
# Now try importing CSV data from a string
csv="""2016, 2017
1,2
3,4
"""
# Does not work
client.import_csv(sheet, csv);
# Using the spreadsheet_id in the URL as described here https://developers.google.com/sheets/api/guides/concepts#spreadsheet_id
client.import_csv('11x...', csv);
# Using the "#gid=0" value in the query string in the browser when looking at this sheet
client.import_csv(0, csv);
Here's the error I get, no matter which of the above I try:
Traceback (most recent call last):
File "./simple.py", line 22, in <module>
client.import_csv(sheet, csv);
File "/Library/Python/2.7/site-packages/gspread/client.py", line 297, in import_csv
headers=headers
File "/Library/Python/2.7/site-packages/gspread/httpsession.py", line 82, in put
return self.request('PUT', url, params=params, data=data, **kwargs)
File "/Library/Python/2.7/site-packages/gspread/httpsession.py", line 69, in request
response.status_code, response.content))
gspread.exceptions.RequestError: (403, '403: {\n "error": {\n "errors": [\n {\n "domain": "global",\n "reason": "insufficientPermissions",\n "message": "Insufficient Permission"\n }\n ],\n "code": 403,\n "message": "Insufficient Permission"\n }\n}\n')
Add https://www.googleapis.com/auth/drive to your scope variable and it will work:
scope=[
'https://spreadsheets.google.com/feeds',
'https://www.googleapis.com/auth/drive'
]
The reason you need the Google Drive in the scope is because import_csv actually makes an HTTP request to Google Drive API call and not to Google Sheets API.
Came across this while having issues of my own. Although already accepted by the author, let me share my 2 cents.
First: yes, you need to have the correct path in your scope. But to clarify what's the id:
From gspread API Reference:
http://gspread.readthedocs.io/en/latest/
The model spreadsheet has an id field. And that's what should be used into import_csv(file_id, data) function.
Notice that there are 3 different models:
- spreadsheet
- worksheet
- cell.
From your sample code, you're actually getting the worksheet object.
sheet = client.open("SheetTitle").sheet1
Which also has an id, but I'm fairly sure this won't work.
You need to get the spreadsheet object and use its id.
sheet = client.open("SheetTitle")
client.import_csv(sheet.id, csv);
You can verify if the id is the same as the one you're using to input directly by printing it
print(sheet.id)
Related
I have a list of dictionaries that contain album information which I'm trying to use to search within Spotify and then add to users' saved albums. I tried following the examples given in the spotipy documentation, however, I'm getting an issue when using the search function via spotipy.
SCOPE = 'user-library-modify'
sp = spotipy.Spotify(auth_manager=SpotifyOAuth(CLIENT_ID, CLIENT_SECRET, REDIRECT_URI, scope=SCOPE))
for album in newMusic:
albumName = album["Title"]
artistName = f"{album['Last Name']}+{album[' First Name']}"
getRecord = sp.search(q = f"artist:{artistName}&album:{albumName}", type='album')
print(getRecord)
I provided the redirect url when prompted after running, but this results in a 401 response for each album result as such:
{
"error": {
"status": 401,
"message": "No token provided"
}
}
I have the .cache file with the access and refresh tokens but it still says no token is provided. I thought maybe I was entering the query incorrectly, but I don't see anything wrong. This is how an example end url looks:
https://api.spotify.com/v1/search?query=artist:band+name&album:album+name&type=album&offset=0&limit=10
What am I doing wrong here? How can I get my access code recognized?
Found the issue; the query entry format was wrong. I found this answer to a past question that explains my problem: Using python to get a track from the spotify API by using search Endpoint
The artist, album, etc. do not need to be declared separately (e.g. "artist: _", "album: _"). They should just be combined together like:
https://api.spotify.com/v1/search?query=first+name+last+name+album+title&type=album&offset=0&limit=10
Weirdly, this contradicts what is in the Spotify API documentation where an example query value is given as such:
Example value:
"remaster%20track:Doxy+artist:Miles%20Davis"
https://developer.spotify.com/documentation/web-api/reference/#/operations/search
I'm trying to get the direct download link for a file in Google Drive using the Google Drive API (v3), but I'm also trying to do this without making the file publicly shared.
Here is what I've tried:
https://www.googleapis.com/drive/v3/files/**FILE_ID**?alt=media&supportsAllDrives=True&includeItemsFromAllDrives=True&key=**API_KEY**
Now this works if the file is shared publicly. But when the file isn't shared publicly you get this message:
{'error': {'errors': [{'domain': 'global', 'reason': 'notFound', 'message': 'File not found: 10k0Qogwcz7k0u86m7W2HK-LO7bk8xAF8.', 'locationType': 'parameter', 'location': 'fileId'}], 'code': 404, 'message': 'File not found: 10kfdsfjDHJ38-UHJ34D82.'}}
After doing some googling I found a post on stack overflow saying that I need to add a request header with my access token, but this doesn't work and the application just hangs
Here is the full code:
### SETTING UP GOOGLE API
scopes = 'https://www.googleapis.com/auth/drive'
store = file.Storage('storage.json')
credentials = store.get()
if not credentials or credentials.invalid:
flow = client.flow_from_clientsecrets('client_secret.json', scopes)
credentials = tools.run_flow(flow, store)
accessToken = credentials.access_token
refreshToken = credentials.refresh_token
drive = build('drive', 'v3', credentials=credentials)
### SENDING REQUEST
req_url = "https://www.googleapis.com/drive/v3/files/"+file_id+"?alt=media&supportsAllDrives=True&includeItemsFromAllDrives=True&key="+GOOGLE_API_KEY
headers={'Authorization': 'Bearer %s' % accessToken}
request_content = json.loads((requests.get(req_url)).content)
print(request_content)
------------------ EDIT: ------------------
I've gotten really close to an answer, but I can't seem to figure out why this doesn't work.
So I've figured out previously that alt=media generates a download link for the file, but when the file is private this doesn't work.
I just discovered that you can add &access_token=.... to access private files, so I came up with this API call:
https://www.googleapis.com/drive/v3/files/**FILE_ID**?supportsAllDrives=true&alt=media&access_token=**ACCESS_TOKEN**&key=**API_KEY**
When I go to that url on my browser I get this message:
We're sorry...
... but your computer or network may be sending automated queries. To protect our users, we can't process your request right now.
I find this confusing because if I remove alt=media, I am able to call on that request and I get some metadata about the file.
I believe your goal as follows.
From I'm trying to get the direct download link for a file in Google Drive using the Google Drive API (v3),, I understand that you want to retrieve webContentLink.
The file that you want to retrieve the webContentLink is the files except for Google Docs files.
You have already been able to get the file metadata using Drive API. So your access token can be used for this.
Modification points:
When the file is not shared, the API key cannot be used. By this, https://www.googleapis.com/drive/v3/files/**FILE_ID**?alt=media&supportsAllDrives=True&includeItemsFromAllDrives=True&key=**API_KEY** returns File not found. I think that the reason of this issue is due to this.
When I saw your script in your question, it seems that you want to download the file content.
In your script, headers is not used. So in this case, the access token is not used.
In the method of "Files: get", there is no includeItemsFromAllDrives.
In your script, I think that an error occurs at credentials.access_token. How about this? If my understanding is correct, please try to modify to accessToken = credentials.token.
In Drive API v3, the default response values don't include webContentLink. So in this case, the field value is required to be set like fields=webContentLink.
When your script is modified, it becomes as follows.
Modified script:
file_id = '###' # Please set the file ID.
req_url = "https://www.googleapis.com/drive/v3/files/" + file_id + "?supportsAllDrives=true&fields=webContentLink"
headers = {'Authorization': 'Bearer %s' % accessToken}
res = requests.get(req_url, headers=headers)
obj = res.json()
print(obj.get('webContentLink'))
Or, you can use drive = build('drive', 'v3', credentials=credentials) in your script, you can also use the following script.
file_id = '###' # Please set the file ID.
drive = build('drive', 'v3', credentials=credentials)
request = drive.files().get(fileId=file_id, supportsAllDrives=True, fields='webContentLink').execute()
print(request.get('webContentLink'))
Note:
In this modified script,
When the file is in the shared Drive and you don't have the permissions for retrieving the file metadata, an error occurs.
When your access token cannot be used for retrieving the file metadata, an error occurs.
So please be careful above points.
When * is used for fields, all file metadata can be retrieved.
Reference:
Files: get
Added:
You want to download the binary data from the Google Drive by the URL.
The file size is large like "2-10 gigabytes".
In this case, unfortunately, webContentLink cannot be used. Because in the case of the such large file, webContentLink is redirected. So I think that the method that the file is publicly shared and use the API key is suitable for achieving your goal. But, you cannot publicly shared the file.
From this situation, as a workaround, I would like to propose to use this method. This method is "One Time Download for Google Drive". At Google Drive, when the publicly shared file is downloaded, even when the permission of file is deleted under the download, the download can be run. This method uses this.
Flow
In this sample script, the API key is used.
Request to Web Apps with the API key and the file ID you want to download.
At Web Apps, the following functions are run.
Permissions of file of the received file ID are changed. And the file is started to be publicly shared.
Install a time-driven trigger. In this case, the trigger is run after 1 minute.
When the function is run by the time-driven trigger, the permissions of file are changed. And sharing file is stopped. By this, the shared file of only one minute can be achieved.
Web Apps returns the endpoint for downloading the file of the file ID.
After you got the endpoint, please download the file using the endpoint in 1 minute. Because the file is shared for only one minute.
Usage:
1. Create a standalone script
In this workaround, Google Apps Script is used as the server side. Please create a standalone script.
If you want to directly create it, please access to https://script.new/. In this case, if you are not logged in Google, the log in screen is opened. So please log in to Google. By this, the script editor of Google Apps Script is opened.
2. Set sample script of Server side
Please copy and paste the following script to the script editor. At that time, please set your API key to the variable of key in the function doGet(e).
Here, please set your API key in the function of doGet(e). In this Web Apps, when the inputted API key is the same, the script is run.
function deletePermission() {
const forTrigger = "deletePermission";
const id = CacheService.getScriptCache().get("id");
const triggers = ScriptApp.getProjectTriggers();
triggers.forEach(function(e) {
if (e.getHandlerFunction() == forTrigger) ScriptApp.deleteTrigger(e);
});
const file = DriveApp.getFileById(id);
file.setSharing(DriveApp.Access.PRIVATE, DriveApp.Permission.NONE);
}
function checkTrigger(forTrigger) {
const triggers = ScriptApp.getProjectTriggers();
for (var i = 0; i < triggers.length; i++) {
if (triggers[i].getHandlerFunction() == forTrigger) {
return false;
}
}
return true;
}
function doGet(e) {
const key = "###"; // <--- API key. This is also used for checking the user.
const forTrigger = "deletePermission";
var res = "";
if (checkTrigger(forTrigger)) {
if ("id" in e.parameter && e.parameter.key == key) {
const id = e.parameter.id;
CacheService.getScriptCache().put("id", id, 180);
const file = DriveApp.getFileById(id);
file.setSharing(DriveApp.Access.ANYONE_WITH_LINK, DriveApp.Permission.VIEW);
var d = new Date();
d.setMinutes(d.getMinutes() + 1);
ScriptApp.newTrigger(forTrigger).timeBased().at(d).create();
res = "https://www.googleapis.com/drive/v3/files/" + id + "?alt=media&key=" + e.parameter.key;
} else {
res = "unavailable";
}
} else {
res = "unavailable";
}
return ContentService.createTextOutput(res);
}
3. Deploy Web Apps
On the script editor, Open a dialog box by "Publish" -> "Deploy as web app".
Select "Me" for "Execute the app as:".
Select "Anyone, even anonymous" for "Who has access to the app:". This is a test case.
If Only myself is used, only you can access to Web Apps. At that time, please use your access token.
Click "Deploy" button as new "Project version".
Automatically open a dialog box of "Authorization required".
Click "Review Permissions".
Select own account.
Click "Advanced" at "This app isn't verified".
Click "Go to ### project name ###(unsafe)"
Click "Allow" button.
Click "OK"
4. Test run: Client side
This is a sample script of python. Before you test this, please confirm the above script is deployed as Web Apps. And please set the URL of Web Apps, the file ID and your API key.
import requests
url1 = "https://script.google.com/macros/s/###/exec"
url1 += "?id=###fileId###&key=###your API key###"
res1 = requests.get(url1)
url2 = res1.text
res2 = requests.get(url2)
with open("###sampleFilename###", "wb") as f:
f.write(res2.content)
In this sample script, at first, it requests to the Web Apps using the file ID and API key, and the file is shared publicly in 1 minute. And then, the file can be downloaded. After 1 minute, the file is not publicly shared. But the download of the file can be kept.
Note:
When you modified the script of Web Apps, please redeploy the Web Apps as new version. By this, the latest script is reflected to the Web Apps. Please be careful this.
References:
One Time Download for Google Drive
Web Apps
Taking advantage of Web Apps with Google Apps Script
I'm trying to retrieve sheets with Users.list_org_sheets(include_all=True) from the Python SDK API.
With That I can see all the things I need :
lstorg = ss.Users.list_org_sheets(include_all=True)
for entry in lstorg.data:
print(entry.id)
print(entry.name)
ss is the smartsheet.Smartsheet(token).
So with that i can see all the sheets in my domain, but when i try to download them with get_sheet_as_excel(id, path) I got that error:
print(ss.Sheets.get_sheet('6157394402142084'))
Traceback (most recent call last):
File "<pyshell#7>", line 1, in <module>
print(ss.Sheets.get_sheet('6157394402142084')
File "E:\Python\lib\site-packages\smartsheet\sheets.py", line 525, in get_sheet
response = self._base.request(prepped_request, expected, _op)
File "E:\Python\lib\site-packages\smartsheet\smartsheet.py", line 218, in request
raise the_ex(native, str(native.result.code) + ': ' + native.result.message)
smartsheet.exceptions.ApiError: {"requestResponse": null, "result": {"code": 1006, "message": "Not Found", "name": "ApiError", "recommendation": "Do not retry without fixing the problem. ", "refId": null, "shouldRetry": false, "statusCode": 404}}
I guess that this error mean that he didn't find the file (Error 404) but I don't know why, just before i make the command to get a list of all sheets and then when i pick one of that (that i'm not owner with this token but the token is SysAdmin)
Thanks For the help
The situation you're describing is due to how permissions work in Smartsheet. Using the SysAdmin token, you can successfully get the list of all Sheets owned by members of the account, but the SysAdmin token will not allow you to access the contents of those sheets, unless that SysAdmin user has been explicitly granted access to the sheets. You're getting the "not found" error when you try to retrieve the sheet because the SysAdmin user hasn't explicitly been granted access to that sheet in Smartsheet.
To actually retrieve a sheet that's listed in the "List Org Sheets" response, you'll need to add the Assume-User header to the "Get Sheet" API request to impersonate the sheet owner. You can find documentation about the Assume-User header here: https://smartsheet-platform.github.io/api-docs/#http-headers.
I am trying to update a file that I created with some new data. Essentially, i am trying to append contents to an already created file. I have tried different ways using different attributes but nothing seems to work so far. The migration from v2 to v3 seems to have made things harder to develop in python.
This is my code so far
def updateFile(fileID, contents):
credentials = get_credentials() #this function gets the credentials for oauth
http = credentials.authorize(httplib2.Http())
service = discovery.build('drive', 'v3', http=http)
# First retrieve the file from the API.
#fileCreated = service.files().get(fileId=fileID).execute()
#print(dir(fileCreated.update()))
# File's new content.
file_metadata = { 'name' : 'notes.txt', 'description' : 'this is a test' }
fh = BytesIO(contents.encode())
media = MediaIoBaseUpload(fh,
mimetype='text/plain')
# Send the request to the API.
#print(BytesIO(contents.encode()).read())
print(fileID)
updated_file = service.files().update(
body=file_metadata,
#uploadType = 'media',
fileId=fileID,
#fields = fileID,
media_body=media).execute()
I have tried using MediaFileUpload (it works only for uploading a file) but I am appending a 'string' that is real time generated using a voice interface and is not stored in any file. So I ended up using MediaIoBaseUpload.
The code runs without any error, but the file is not updated. Any help is appreciated.
okay, so i figured out what was wrong with my code. Apparently, nothing is wrong with this chunk of code that I posted. I realized that I have not converted the document I created to a "google document" hence the code didnt update the document. I changed the mime type of the document i create to be a google document, and now the code works fine :)
You can also use service.files().get_media(fileId=fileID).execute() to get the file content and append new content to it.
(For clarity, this post relates to the difference between the Google Documents List API and Google Drive API on Google App Engine with Python)
With the [now deprecated] Documents list API I was able to edit Google Documents by exporting as HTML, modifying the HTML and then re-uploading, either as a new document or as a modification to the original. This was useful for things like generating PDF documents from a template. I have been trying to replicate this functionality with the new Drive API (V2), however seem unable to.
Have come up with this ...
http = # authenticated http client
drive_service = build('drive', 'v2', http=http)
# get the file ...
file = drive_service.files().get(fileId=FILE_ID).execute()
# download the html ...
url = file['exportLinks']['text/html']
response, content = http.request(url)
# edit html
html = content.replace('foo', 'bar')
# create new file with modified html ...
body = {'title':'Modified Doc',
'description':'Some description',
'mimeType':'text/html'}
media_body = MediaIoBaseUpload(StringIO.StringIO(html), 'text/html', resumable=False)
drive_service.files().insert(body=body, media_body=media_body)
The above code uploads an html file as a file into Google Drive, rather then rendering the HTML into a Google Document. Fair enough, this makes sense. But how to I get it render as a Google Doc, as I was able to do with the Documents List API?
One other thing - if I set resumable=True it throws the following error on App Engine - '_StreamSlice' has no len(). Couldn't figure out how to get resumable=True to work?
And one last thing - the sample code in the docs uses a MediaInMemoryUpload object, however if you look at the source it is now deprecated, in favour of MediaIoBaseUpload. Should the sample code be updated?!
i suspect the issue is that the default for conversion has changed from true to false. You must explicitly set convert=true on the upload. See https://developers.google.com/drive/v2/reference/files/insert