BELIEVED SOLVED: Python API only supports v1, while watch later was added in v2. SOURCE
SOLUTION: Use "Experimental" API v3
I am attempting to use the Youtube API to access my Watch Later playlist. Below is the code I am using.
import gdata.youtube
import gdata.youtube.service
yt_service = gdata.youtube.service.YouTubeService()
yt_service.ssl = True
yt_service.developer_key = 'REDACTED'
yt_service.email = 'REDACTED'
yt_service.password = 'REDACTED'
yt_service.ProgrammaticLogin()
playlist_uri = 'https://gdata.youtube.com/feeds/api/users/default/watch_later?v=2'
playlist_video_feed = yt_service.GetYouTubePlaylistVideoFeed(uri=playlist_uri)
for playlist_video_entry in playlist_video_feed.entry:
print playlist_video_entry.title.text
I am receiving the following error.
Traceback (most recent call last):
File "Youtube.py", line 21, in <module>
playlist_video_feed = yt_service.GetYouTubePlaylistVideoFeed(uri=playlist_uri)
File "/Library/Python/2.6/site-packages/gdata/youtube/service.py", line 393, in GetYouTubePlaylistVideoFeed
uri, converter=gdata.youtube.YouTubePlaylistVideoFeedFromString)
File "/Library/Python/2.6/site-packages/gdata/service.py", line 1108, in Get
'reason': server_response.reason, 'body': result_body}
gdata.service.RequestError: {'status': 400, 'body': 'Invalid request URI', 'reason': 'Bad Request'}
It would seem the URI https://gdata.youtube.com/feeds/api/users/default/watch_later?v=2 is invalid. However this is the one stated to be used in the google documents. Am I using it wrong, or is there another issue here?
In addition if I change the URI to http://gdata.youtube.com/feeds/api/playlists/63F0C78739B09958 it works as expected.
You should check your authentication. According to Retrieving and updating a user's 'Watch Later' playlist:
Again, the link will only be present in a profile entry if either of
the following conditions is true:
You submit an authenticated request to retrieve the logged-in user's
own profile.
The watch_later playlist is publicly available for the user whose
profile you are retrieving.
The API server will return a 40x HTTP response code if you try to
retrieve a watch_later playlist and neither of the above conditions is
true.
The second link would work most likely due to the second publicly available condition being met. One thing I do notice missing from your example is the client id/source:
# A complete client login request
yt_service.email = 'jo#gmail.com'
yt_service.password = 'mypassword'
yt_service.source = 'my-example-application'
yt_service.developer_key = 'ABC123...'
yt_service.client_id = 'my-example-application'
yt_service.ProgrammaticLogin()
You should look into that and ensure that your authentication is happening properly.
Related
I am trying to download an Excel file using Drive API. Here is my code:
def downloadXlsx(vars, file, creds):
try:
service = build('drive', 'v3', credentials=creds)
fileId = file['id']
fileName = file['name']
# request = service.files().get_media(fileId=fileId)
request = service.files().get_media(fileId=fileId, acknowledgeAbuse=True)
# request = service.files().get(fileId=fileId, supportsTeamDrives=True, fields='*').execute()
fh = io.BytesIO()
downloader = MediaIoBaseDownload(fh, request)
done = False
while done is False:
status, done = downloader.next_chunk()
print("Download %d%%." % int(status.progress() * 100))
fh.seek(0)
print('%s%s' % (vars.download_directory, fileName))
with open('%s%s' % (vars.download_directory, fileName), 'wb') as f:
shutil.copyfileobj(fh, f, length=131072)
except HttpError as error:
print(f'An error occurred: {error}')
Every time I run it most files return this error
An error occurred: <HttpError 403 when requesting https://www.googleapis.com/drive/v3/files/xxxxxxxxxxxxxxxxxxxx?acknowledgeAbuse=true&alt=media returned "This file has been identified as malware or spam and cannot be downloaded.". Details: "[{'domain': 'global', 'reason': 'cannotDownloadAbusiveFile', 'message': 'This file has been identified as malware or spam and cannot be downloaded.'}]">
I tried adding the acknowledgeAbuse=True flag but it doesn't change anything. Previously it would give me this error:
An error occurred: <HttpError 403 when requesting
https://www.googleapis.com/drive/v3/files/1fr7NwhToKFgvbNgExl0QMgurLJlx8KmV?acknowledgeAbuse=true&alt=media
returned "Only the owner can download abusive files.". Details:
"[{'domain': 'global', 'reason': 'cannotDownloadAbusiveFile',
'message': 'Only the owner can download abusive files.',
'locationType': 'parameter', 'location': 'acknowledgeAbuse'}]">
But I no longer get this error and I'm not sure why as I haven't changed anything.
I tried using this line:
request = service.files().get(fileId=fileId, supportsTeamDrives=True, fields='*').execute()
Which would download the file but it would be corrupted and unable to be opened.
Anyway, does anyone have a clue how I can get around this? Maybe a different method I could try or a way to get the .get() to download the file properly? I don't know why it's saying I'm not the owner - if anyone has knowledge on how Drive API determines 'who' is executing the API that would be helpful.
Edit:
I'm looking at the files.get method documentation here and it reads the following:
By default, this responds with a Files resource in the response body.
If you provide the URL parameter alt=media, then the response includes
the file contents in the response body. Downloading content with
alt=media only works if the file is stored in Drive. To download
Google Docs, Sheets, and Slides use files.export instead. For further
information on downloading files, refer to Download files
Seems like I need to specify alt=media somehow but not sure if that is possible in my situation. Maybe that's referring to get_media?
Fixed! It was a bug with Google Drive API. https://issuetracker.google.com/issues/238551542
There is an optional parm that you can send with your file.get request
acknowledgeAbuse boolean Whether the user is acknowledging the risk of downloading known malware or other abusive files. This is only applicable when alt=media. (Default: false)
try
request = service.files().get(fileId=fileId, supportsTeamDrives=True, fields='*', acknowledgeAbuse='true').execute()
What is the way to pull out all emails from Gmail?
I did full_sync, but that didn't return all of my email - only about 3000 emails, while I know I have more. In the documentation they did not mention about this.
My code snippet:
history = service.users().history().list(
userId='me',
startHistoryId=start_history_id,
maxResults=500,
labelId='INBOX'
).execute()
if "history" in history:
try:
for message in history["history"]:
batch.add(
service.users().messages().get(userId='me', id=message["messages"][0]["id"]),
callback="somecallbak",
request_id=request_id
)
batch.execute()
while 'nextPageToken' in history:
If you are doing a full sync, you should refer to this documentation, that recommends two steps:
listing all the messages with the users.messages.list method
for each of the entry get the required information using the users.messages.get method
So you don't need use the users.history.list as you will have an hard time finding the startHistoryId from which to start.
You can achieve this with a snipped similar to the one below (tested and working on my python 3.x console). As suggested by others I used the python client pagination and batch request functionalities.
from httplib2 import Http
from googleapiclient.discovery import build
from oauth2client import client, tools, file
# callback for the batch request (see below)
def print_gmail_message(request_id, response, exception):
if exception is not None:
print('messages.get failed for message id {}: {}'.format(request_id, exception))
else:
print(response)
# Scopes
SCOPES = ['https://www.googleapis.com/auth/gmail.readonly', ]
# where do we store our credentials?
creds_store = file.Storage('gmail-list.json')
start_creds = creds_store.get()
# standard oauth2 authentication flow
if not start_creds or start_creds.invalid:
# client_id.json is exported from your gcp project
start_flow = client.flow_from_clientsecrets('client_id.json', SCOPES)
start_creds = tools.run_flow(start_flow, creds_store)
# Gmail SDK
http = Http()
gmail_sdk = build('gmail', 'v1', http=start_creds.authorize(http))
# messages.list parameters
msg_list_params = {
'userId': 'me'
}
# messages.list API
message_list_api = gmail_sdk.users().messages()
# first request
message_list_req = message_list_api.list(**msg_list_params)
while message_list_req is not None:
gmail_msg_list = message_list_req.execute()
# we build the batch request
batch = gmail_sdk.new_batch_http_request(callback=print_gmail_message)
for gmail_message in gmail_msg_list['messages']:
msg_get_params = {
'userId': 'me',
'id': gmail_message['id'],
'format': 'full',
}
batch.add(gmail_sdk.users().messages().get(**msg_get_params), request_id=gmail_message['id'])
batch.execute(http=http)
# pagination handling
message_list_req = message_list_api.list_next(message_list_req, gmail_msg_list)
As suggested in this link, you may use batch requests.
Use batch and request 100 messages at a time. You will need to make 1000 requests but the good news is that's quite fine and it'll be easier for everyone (no downloading 1GB response in a single request!).
Also based from this thread, you could save the next page token on every request and use it in your next request. If there is no next page token in the response, you know that you have gotten all messages.
So, I'm a bit confused on how I get past authentication on Youtube using Python and successfully login. I always get error 403 when I try to PragmaticLogin():
yt_service = gdata.youtube.service.YouTubeService()
service.developer_key = 'MY Key'
service.client_id='My ID'
service.email = 'myemail#yahoo.gr'
service.password = 'mypassword'
service.source = 'my_program'
service.ProgrammaticLogin()
What do I have to do?
Update:
I think that it has to do with authentication. Do I need both developer_key and client_id? Where do I get each? I want to have rights to add comments to my videos etc.
Full error:
Traceback (most recent call last):
File "/home/bodhi32/Documents/bot.py", line 9, in <module>
client.ClientLogin(USERNAME, PASSWORD)
File "/usr/lib/pymodules/python2.7/gdata/service.py", line 833, in ClientLogin
self.ProgrammaticLogin(captcha_token, captcha_response)
File "/usr/lib/pymodules/python2.7/gdata/service.py", line 796, in ProgrammaticLogin
raise Error, 'Server responded with a 403 code'
gdata.service.Error: Server responded with a 403 code
ClientLogin is deprecated and has all sorts of errors. Don't use it.
Use OAuth2.
This sample should get you started:
https://github.com/youtube/api-samples/blob/master/python/my_uploads.py
Use your code but make sure you fill the developer_key and client_id fields ( check below how to get them).
yt_service = gdata.youtube.service.YouTubeService()
service.developer_key = 'MY Key'
service.client_id='My ID'
service.email = 'myemail#yahoo.gr'
service.password = 'mypassword'
service.source = 'my_program'
service.ProgrammaticLogin()
To obtain a youtube api go to
https://cloud.google.com/console/project and create a new project, then enable youtube.
Check this video for more info Obtaining a simple API key for use with the YouTube API
I'm trying to get history feed from YouTube of an authenticated user with python.
This is my code :
yt_service = gdata.youtube.service.YouTubeService()
def LogIn():
login_name = raw_input('Email:')
login_pass = getpass.getpass()
try:
yt_service.email = login_name
yt_service.password = login_pass
yt_service.ProgrammaticLogin()
except:
print 'False username or password. Unable to authenticate.'
exit();
def GetHistoryFeed():
uri = 'https://gdata.youtube.com/feeds/api/users/default/watch_history?v=2'
feed = yt_service.GetYouTubeVideoFeed(uri)
#PrintVideoFeed(yt_service.GetYouTubeVideoFeed(uri),'history')
LogIn()
GetHistoryFeed()
and it says gdata.service.RequestError: {'status': 400, 'body': 'Invalid request URI', 'reason': 'Bad Request'} . I know I have to make a authenticated Get request , but i don't know how. What am I doing wrong ?
EDIT
I am facing a major problem. The prog is the same as above , but with yt_service.developer_key = DEVELOPER_KEY added under password line and uri = 'https://gdata.youtube.com/feeds/api/users/default/watch_history?v=2&key=%s'%DEVELOPER_KEY. I tested it in 4 PCs and it runs without errors only in one if them. I get this error :
File "/usr/local/lib/python2.6/dist-packages/gdata/youtube/service.py", line 186, in
return self.Get(uri, converter=gdata.youtube.YouTubeVideoFeedFromString)
File "/usr/local/lib/python2.6/dist-packages/gdata/service.py", line 1108, in Get
'reason': server_response.reason, 'body': result_body}
gdata.service.RequestError: {'status': 400, 'body': 'Invalid request URI', 'reason': 'Bad Request'}
I use python 2.7 and gdata python 2.0 . Why one Pc executes it and the rest of them not? What can i do to fix it ? Please help!
When you attempt to call youtube API, you will first need to register a new application. Reference - https://developers.google.com/youtube/2.0/developers_guide_protocol_authentication
Visit http://code.google.com/apis/youtube/dashboard/ to register your application and retrieve the Developer Key that will be generated for you.
Thereafter, whenever you make a call to youtube APIs, you should include the key query parameter. (reference - https://developers.google.com/youtube/2.0/developers_guide_protocol#Developer_Key)
Your instantiated yt_service will be:-
yt_service.developer_key = DEVELOPER_KEY
where the DEVELOPER_KEY is the one that you get on your newly registered application's dashboard ( http://code.google.com/apis/youtube/dashboard/ ).
Without this DEVELOPER_KEY, google youtube will not know whether your python script is in fact a recognized application, with proper access rights.
I'm trying to use the Yahoo Social Python SDK to get a users contacts through oAuth. This is for a webapp running on App Engine. SO, I have everything set up to run through the oAuth dance, exchanging consumer keys and verifiers and all that jazz. I store the token and can reuse it to retrieve a users contacts until the token the expires an hour later. So, is there anyone out there who has used the Python SDK and can tell me what is wrong with this simple code:
import yahoo.application
CONSUMER_KEY = '####'
CONSUMER_SECRET = '##'
APPLICATION_ID = '##'
CALLBACK_URL = '##'
oauthapp = yahoo.application.OAuthApplication(CONSUMER_KEY, CONSUMER_SECRET, APPLICATION_ID, CALLBACK_URL)
oauthapp.token = yahoo.oauth.AccessToken.from_string(access_token) #access_token is legit string pulled from datastore
oauthapp.token = oauthapp.refresh_access_token(oauthapp.token)
contacts = oauthapp.getContacts()
Running this throws the following error:
'oauth_token'<br>
Traceback (most recent call last):<br>
File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", line 513, in __call__<br>
handler.post(*groups)<br>
File "/base/data/home/apps/testproj/2.345270664321958961/scripteditor.py", line 1249, in post<br>
oauthapp.token = oauthapp.refresh_access_token(oauthapp.token)<br>
File "/base/data/home/apps/testproj/2.345270664321958961/yahoo/application.py", line 90, in refresh_access_token<br>
self.token = self.client.fetch_access_token(request)<br>
File "/base/data/home/apps/testproj/2.345270664321958961/yahoo/oauth.py", line 165, in fetch_access_token<br>
return AccessToken.from_string(self.connection.getresponse().read().strip())<br>
File "/base/data/home/apps/testproj/2.345270664321958961/yahoo/oauth.py", line 130, in from_string<br>
key = params['oauth_token'][0]<br>
KeyError: 'oauth_token'<br>
Basically, if I comment out the line with refresh_access_token, and the token has not expired, this code works and I get the users contacts. But with refresh_acces_token, it fails at that line. Can anyone give a hand?
Looks like something wrong with passing params. Try to debug oauth_token variable.
Solved. For reasons I can't understand, the above code now just works. It might have been a problem on yahoo's end, but I really can't be sure. It's been running fine for two weeks.