We Keep Coding

I am facing an Error while i am scraping YOUTUBE information using Google API

I have list of youtube ids. I wanted to get details about each video such as title, viewcount, likecount, etc. My code was working well until yesterday, suddenly it is throwing an error.
For the experts, initially I have list of video ids. For sample I have used 3 ids. But in actual I have 100+ ids.
video_id=['GGHiHodljug','PY8cEyi2MzM','QTCcJipjgxI']
Later, I have defined a function to retrieve video information and append it to various list.
def get_video_details(youtube, video_id):
all_video_stats = []
for i in video_ids:
request = youtube.videos().list(part='snippet,statistics',id=video_id)
response = request.execute()
for video in response['items']:
video_stats = dict(Title = video['snippet']['title'],
Published_date = video['snippet']['publishedAt'],
Views = video['statistics']['viewCount'],
Likes = video['statistics']['likeCount'],
#Dislikes = video['statistics']['dislikeCount'],
Comments = video['statistics']['commentCount']
#Shares = video['statistics']['']
)
all_video_stats.append(video_stats)
return all_video_stats
Now, I tried to call upon the function. But it throws an error, which i never faced till yesterday.
video_details = get_video_details(youtube,video_id)
The error that was received is shown below.
HttpError: <HttpError 400 when requesting
https://youtube.googleapis.com/youtube/v3/videos?part=snippet%2Cstatistics&id=kwlJUFUJeBU&id=9qjgV9UynUU&id=9BRNhfz9TZU&id=Cm614VNaapI&id=Gk0O1hBZL1g&id=QuIfcSDxEZY&id=lO48EoPn7-A&id=n1S886NBQYA&id=y4DGkrs7KcM&id=vk3ahhtY5qE&id=Zr3cAIU6bZM&id=fLC_LfstdAA&id=s4aneR0tc1s&id=A0FJpd-lieI&id=7fFFfU6Cgmk&id=4VJSE59j1pM&id=ns51Rp7o_Bw&id=LQrMIBpGrbA&id=XOeTeM-1qKc&id=f6Ms318wTj0&id=etfoGZquiSA&id=a1LUlovdL_A&id=nTTBa4-0Z7I&id=6KOYprSy1KA&id=wjPmweQ4peQ&id=Cw0xY42b_Mo&id=-n_mVFPzeuY&id=R8ZNqKerC1o&id=JnZhAPBfPYw&id=8o2yLPDr_d8&id=DOoE4hPiJuw&id=M4HRJXRZZAM&id=cNedzwOOqag&id=Qqj96rMBxTE&id=_0J7VcMtKkQ&id=kZ8ObOkzq4w&id=6P_wnhUb02g&id=kH5UPfi3cFQ&id=qtDl9Yu8or0&id=Xzn-URGHItA&id=xutJ2uQ4tT0&id=bZ71zj8BunU&id=gHtjvLejc6E&id=92pTgb_7QI4&id=KKZkES9gfuE&id=Qj01vTif2yI&id=Qkc1A8cJEmw&id=whBSkar_rCs&id=wOPEC95vVUs&id=pi3Tm9lzvLU&id=Ip-PimAxgjw&id=r1W13UnMwLU&id=z68p5nBZ03k&id=6wMuvqUkgqY&id=b7AYFCw8cQA&id=nMrEiom0S74&id=-bGqNqT8Ckg&id=CSXPZuNVqGw&id=wP09CGKyDdg&id=eY_fPjwZ3hU&id=nT9mgg7BtH4&id=D2SqeQPr38I&id=h-0ELP6rNhQ&id=33OOngFBcnk&id=ojbMrcTlQ_s&id=Kt6xncrWZUQ&id=xUCWiEhCIgI&id=jDEC_sV67B4&id=Dxw9xEpkeYk&id=tFiAsk_sOC0&id=eLLXCtd187I&id=3y499rX5A8Q&id=keTSJlzVu2I&id=RtnXrhDxpgo&id=VmbouDaMoM4&id=NWBNOA_fwcA&id=7mTnG30Y8c8&id=gOM5A0w8V1s&id=rLEDXBhuspk&id=XQ-NNWBySIo&id=_klVNQEzauU&id=ukNGhEZLEhw&id=eI8quor4HmM&id=DLbfMGozcyM&id=GZBAUuUbCFs&id=qwquUC4Wk0s&id=aEsEnJMiRqU&id=EKPQX6LC4Uw&id=HmAn0BgKGhA&id=0jlFNP8nDzs&id=n3iSKVMfbFw&id=Ypaah0zPWIo&id=oyOHOcyp2T8&id=aLmJ5zXsVtk&id=dEsEaJGYAZM&id=KGe7nId4GGs&id=BoRgRqqeiw8&id=GGHiHodljug&id=PY8cEyi2MzM&id=QTCcJipjgxI&key=AIzaSyDpPBFs9LZ-33rVUeSKyHCaz5E0UmWmZXk&alt=json
returned "The request specifies an invalid filter parameter.".
Details: "[{'message': 'The request specifies an invalid filter
parameter.', 'domain': 'youtube.parameter', 'reason':
'invalidFilters', 'location': 'parameters.', 'locationType':
'other'}]">

For pass more than one video_id, the elements are separated by comma.
Instead of:
&id=f6Ms318wTj0&id=etfoGZquiSA&id=a1LUlovdL_A
use:
&id=f6Ms318wTj0,etfoGZquiSA,a1LUlovdL_A

The API endpoint does not accept more than 50 IDs. So if we have more then 50 ids, information about each video can be obtained using a loop.

Print the ID of a YouTube channel using YouTube Data API with Python

I want to print the ID of a YouTube channel using the YouTube Data API, but I don't know how.
If I try this:
request = youtube.channels().list(
part='statistics',
forUsername='TheYTChannel'
)
response = request.execute()
print(response)
it responds with all statistics of the YouTube channel, but I only want the ID of that channel.
How do I print out only the ID of that channel?

According to the official docs of the Channels.list API endpoint, invoking it with the request parameter forUsername, will produce a JSON response text that includes an object representing the channel resource that is associated to the specified user name (if any such channel exists):
{
"kind": "youtube#channelListResponse",
"etag": etag,
"nextPageToken": string,
"prevPageToken": string,
"pageInfo": {
"totalResults": integer,
"resultsPerPage": integer
},
"items": [
channel Resource
]
}
The ID of the channel in question is to be found as the value of the property:
id (string)
The ID that YouTube uses to uniquely identify the channel.
From your perspective (as user of the Google's APIs Client Library for Python), the code would look like:
def find_channel_id_by_username(youtube, user_name):
request = youtube.channels().list(
forUsername = user_name,
fields = 'items/id',
part = 'id'
)
response = request.execute()
resource = response.get('items')
if resource:
return resource[0]['id']
else:
return None
Note that the result of the function find_channel_id_by_username above is either a string representing the ID of the channel associated to the given user_name, if there is such channel. In case there's no channel associated with the given user_name, the function returns None.
Running find_channel_id_by_username with user_name = 'Youtube' returns back the channel ID UCBR8-60-B28hp2BmDPdntcQ. When running the function with user_name = 'TheYTChannel' it returns None.
Also note that above I used the fields parameter; that's a good practice: ask from the API only the info that is of actual use.

Can't get all comments from Youtube Data API V3 [Python]

I have a python function, which allows you to get all comments from a youtube video. Therefore I use the youtube API v3 comments.list method.
key = 'My Key'
textFormat = 'plainText'
part = 'snippet'
maxResult = '100'
order = 'relevance'
nextToken = ''
videoId = 'Ix9NXVIbm2A'
while(True):
response = requests.get("https://www.googleapis.com/youtube/v3/commentThreads?&key="+key+"&part="+part+"&videoId="+idVideo +"&maxResults="+maxResult+"&order="+order+"&pageToken="+nextToken)
data = response.json() #kind - etag - ?nextPageToken
if 'error' in data:
print(data)
break
for item in data['items']:
snippet = item["snippet"]
toplevelcomment = snippet['topLevelComment']
content = toplevelcomment['snippet']
commentid = toplevelcomment['id']
authorname = content['authorDisplayName']
textOriginal = content['textOriginal']
#lists
commentids.append(commentid)
authornames.append(authorname)
textOriginals.append(textOriginal)
if 'nextPageToken' in data:
nextToken = data['nextPageToken']
else:
break
all progress good from pageToken to another. But when it reaches the pageToken number 13, the API always returns
{
'error':
{
'errors':
[
{
'domain': 'youtube.commentThread',
'reason': 'processingFailure',
'message': 'The API server failed to successfully process the request. While this can be a transient error, it usually indicates that the requests input is invalid. Check the structure of the <code>commentThread</code> resource in the request body to ensure that it is valid.',
'locationType': 'other',
'location': 'body'
}
],
'code': 400,
'message': 'The API server failed to successfully process the request. While this can be a transient error, it usually indicates that the requests input is invalid. Check the structure of the <code>commentThread</code> resource in the request body to ensure that it is valid.'
}
}
I'm using a valid key and the pageToken is valid too (returned by the API)
Does anyone have the same problems or am I doing something wrong?

This error come because your api limits is exhausted. Youtube change the limit of api time to time.
And sometimes network problem is also occur. You have to write code for multiple attempt when once request is fail.
You can read full documentation here - [https://developers.google.com/youtube/v3/getting-started#quota][1]
Recently youtube decreased the quota numbers from 1M to 10K per day as of 11-January-2019
currently version 3 allow only 10k units per day.

How to list videos that have been flagged for "Matched Third Party Content" using Python against YouTube API V3,

I'm writing Python code (2.76) to access the YouTube API V3, search a given channel (authenticated with OAuth2.0), and return video details for items in that channel. I'm able to get UniqueID, Position in Playlist, Title, Description, Keywords, Privacy Status, ViewCount, Published Date, # Likes, # Dislikes, # Comments, License, and Upload Status.
However, I'm not able to access whether a given video has been flagged for having "matched third party content" or not. I suspect that it lies in status.rejectionReason.
Valid strings for status.rejectionReason are:
"claim", "copyright", "duplicate", "inappropriate", "length",
"termsOfUse", "trademark", "uploaderAccountClosed", and
"uploaderAccountSuspended".
The description of status.rejectionReason states:
"This value explains why YouTube rejected an uploaded video. This
property is only present if the uploadStatus property indicates that
the upload was rejected."
Valid strings for status.uploadStatus are:
"deleted", "failed", "processed", "rejected", and "uploaded".
However, when looking at status.uploadStatus I see that the status actually is "processed" for videos that I know that have been flagged. I suppose this makes sense as the videos are often uploaded and seem fine until they're flagged later for "Matched Third Party Content".
Occasionally, I can pull back a 'rejected' or 'failed' status but so far, the 'rejected' status only references a 'duplicate' upload and most times, the 'failed' status doesn't include a status.failureReason, which should only be
"codec", "conversion", "emptyFile", "invalidFile", "tooSmall", or
"uploadAborted"
My question is: Is there someplace else that I can look to see where 3rd party copyright flags are set?
This is affecting how we monetize videos. I also don't know where to look to see if a video has been monetized or if its monetization attempt has been rejected. Maybe this would be a better place to collect the information I'm looking for if it's possible.
Relevant sections of my code:
# Call the API's videos.list method to retrieve the video resource.
videos_list_response = youtube.videos().list(
id=playlist_item["snippet"]["resourceId"]["videoId"],
part="snippet,statistics,status"
).execute()
if videos_list_response["items"][0]["status"]["uploadStatus"] == "failed":
print videos_list_response["items"][0]["status"]
try:
WorkSheet.row(i).write(14,videos_list_response["items"][0]["status"]["failureReason"])
except:
WorkSheet.row(i).write(14,"Error on failureReason")
if videos_list_response["items"][0]["status"]["uploadStatus"] == "rejected":
print videos_list_response["items"][0]["status"]
try:
WorkSheet.row(i).write(14,videos_list_response["items"][0]["status"]["rejectionReason"])
except:
WorkSheet.row(i).write(14,"Error on rejectionReason")
Sample responses that I'm getting:
Rejected:
Getting video_id: jt0zTqTBid8 Index #765
{u'license': u'youtube', u'embeddable': True, u'privacyStatus': u'public', u'pub
licStatsViewable': True, u'uploadStatus': u'rejected', u'rejectionReason': u'dup
licate'}
Failed:
Getting video_id: 7ttiQfkl_K0 Index #837
{u'publicStatsViewable': True, u'privacyStatus': u'public', u'uploadStatus': u'f
ailed', u'license': u'youtube', u'embeddable': True}
I should mention that neither of the above videos were flagged for copyright. The last one is just "failed". The Video Manager page in YouTube says "aborted upload". It has the Title, description, and other meta details, just no video. However, the returned object doesn't have status.failedReason associated with it.

Getting length of YouTube video (without downloading the video itself)

I need to figure out the simplest method of grabbing the length of a youtube video programmatically given the url of said video.
Is the youtube API the best method? It looks somewhat complicated and I've never used it before so it's likely to take me a bit to get accommodated, but I really just want the quickest solution. I took a glance through the source of a video page in the hopes it might list it there, but apparently it does not (though it lists recommended video times in a very nice list that would be easy to parse). If it is the best method, does anyone have a snippit?
Ideally I could get this done in Python, and I need it to ultimately be in the format of
00:00:00.000
but I'm completely open to any solutions anyone may have.
I'd appreciate any insight.

All you have to do is read the seconds attribute in the yt:duration element from the XML returned by Youtube API 2.0. You only end up with seconds resolution (no milliseconds yet). Here's an example:
from datetime import timedelta
from urllib2 import urlopen
from xml.dom.minidom import parseString
for vid in ('wJ4hPaNyHnY', 'dJ38nHlVE78', 'huXaL8qj2Vs'):
url = 'https://gdata.youtube.com/feeds/api/videos/{0}?v=2'.format(vid)
s = urlopen(url).read()
d = parseString(s)
e = d.getElementsByTagName('yt:duration')[0]
a = e.attributes['seconds']
v = int(a.value)
t = timedelta(seconds=v)
print(t)
And the output is:
0:00:59
0:02:24
0:04:49

(I'm not sure what "pre-download" refers to.)
The simplest way to get the length of VIDEO_ID is to make an HTTP request for
http://gdata.youtube.com/feeds/api/videos/VIDEO_ID?v=2&alt=jsonc
and then look at the value of the data->duration element that's returned. It will be set to the video's duration in seconds.

With python and V3 youtube api this is the way for every videos.
You need the API key, you can get it here: https://console.developers.google.com/
# -*- coding: utf-8 -*-
import json
import urllib
video_id="6_zn4WCeX0o"
api_key="Your API KEY replace it!"
searchUrl="https://www.googleapis.com/youtube/v3/videos?id="+video_id+"&key="+api_key+"&part=contentDetails"
response = urllib.urlopen(searchUrl).read()
data = json.loads(response)
all_data=data['items']
contentDetails=all_data[0]['contentDetails']
duration=contentDetails['duration']
print duration
Console response:
>>>PT6M22S
Corresponds to 6 minutes and 22 seconds.

You can always utilize Data API v3 for this.
Just do a videos->list call.
GET https://www.googleapis.com/youtube/v3/videos?part=contentDetails%2C+fileDetails&id={VIDEO_ID}&key={YOUR_API_KEY}
In response get the contentDetails.duration in ISO 8601 format.
Or you can get duration in ms from fileDetails.durationMs.

If you're using Python 3 or newer you can perform a GET request against the YouTube v3 API URL. For this you will need the enable the YouTube v3 API in your Google Console and you'll need to create an API credential after you enable the YouTube v3 API.
Code examples below:
import json
import requests
YOUTUBE_ID = 'video_id_here'
API_KEY = 'your_youtube_v3_api_key'
url = f"https://www.googleapis.com/youtube/v3/videos?part=contentDetails&id={YOUTUBE_ID}&key={API_KEY}"
response = requests.get(url) # Perform the GET request
data = response.json() # Read the json response and convert it to a Python dictionary
length = data['items'][0]['contentDetails']['duration']
print(length)
Or as a reusable function:
import json
import requests
API_KEY = 'your_youtube_v3_api_key'
def get_youtube_video_duration(video_id):
url = f"https://www.googleapis.com/youtube/v3/videos?part=contentDetails&id={video_id}&key={API_KEY}"
response = requests.get(url) # Perform the GET request
data = response.json() # Read the json response and convert it to a Python dictionary
return data['items'][0]['contentDetails']['duration']
duration = get_youtube_video_duration('your_video_id')
Note: You can only get fileDetails from the API if you own the video, so you'll need to use the same Google account for your YouTube v3 API key as your YouTube account.
The response from Google will look something like this:
{
"kind": "youtube#videoListResponse",
"etag": "\"SJajsdhlkashdkahdkjahdskashd4/meCiVqMhpMVdDhIB-dj93JbqLBE\"",
"pageInfo": {
"totalResults": 1,
"resultsPerPage": 1
},
"items": [
{
"kind": "youtube#video",
"etag": "\"SJZWTasdasd12389ausdkhaF94/aklshdaksdASDASddjsa12-18FQ\"",
"id": "your_video_id",
"contentDetails": {
"duration": "PT4M54S",
"dimension": "2d",
"definition": "hd",
"caption": "false",
"licensedContent": false,
"projection": "rectangular"
}
}
]
}
Where your video duration is: PT4M54S which means 4 Minutes 54 Seconds
Edit: To convert the YouTube duration to seconds, see this answer: https://stackoverflow.com/a/49976787/2074077
Once you convert to time to seconds, you can convert seconds into your format with a timedelta.
from datetime import timedelta
time = timedelta(seconds=duration_in_seconds)
print(time)