Is there a way to pull traffic information from a particular youtube video (demographics for example, age of users, country, gender, etc.), say using the python gdata module or the youtube API? I have been looking around the module's documentation, but so far nothing.
There used to be a large selection of things in the past, but they are all gone now. So do not think it is possible any more.
Related
I am rn trying to scrape twitter for an nlp research. I already used snscrape to get tweets with the required filters, the issue is that we need tweets from a specific age range. In my head I guess some profiles on twitter have their birthdate public, so maybe we can fetch that. Maybe webscrape that from the profile? Any ideas are welcomed.
Till now I have tried some methods of webscraping but can't find something concrete
Twitter has a pretty well documented API that works very well with Python.
Try to make a simple crawler and see one of the JSONs that you get for a Tweet/User.
You will need to sign up and get some Access Tokens/Keys to use in your script, but other than that you are ready to go:
https://developer.twitter.com/en/docs/twitter-api
The age of Twitter users is not made available via the API (and the website may show birthday but not year). There are also a number of other factors you should read about in relation to analysis of Twitter user data.
I've been looking for a method to extract details from a YouTube clip when only knowing the clip URL within a python script. I specifically need the original channel name.
The YouTube API does not seem to include any method to do so, only to extract details from a known video ID.
For example, this clip: https://www.youtube.com/clip/UgkxnEqNDtOHMOOoS5TyJFr2QOjdKbaTOTlW, with the ID UgkxnEqNDtOHMOOoS5TyJFr2QOjdKbaTOTlW is from this video https://youtu.be/z-e2bDx7tUA, with the ID z-e2bDx7tUA.
z-e2bDx7tUA is searchable with the API, but UgkxnEqNDtOHMOOoS5TyJFr2QOjdKbaTOTlW is not.
Even if there's a more roundabout method, such as getting the clip ID, then getting the video ID and then being able to use that within the API would work. But I've not found an efficient way of doing so, including web scraping.
Any help would be greatly appreciated.
One more time YouTube Data API v3 doesn't provide a basic feature.
I recommend you to use my open-source YouTube operational API. Indeed by using https://yt.lemnoslife.com/videos?part=id&clipId=CLIP_ID you'll get in item['videoId'] the video id associated with the clip.
For example the CLIP_ID UgkxfiAGoXJYA02_JdIzA3k3pvqpLNm90DBx is extracted from the videoId 9bZkp7q19f0.
I am writing a script in Python, that uses tweepy to search for tweets with a given keyword. Here is the snippet:
for tweet in tweepy.Cursor(api.search, q=keyword, lang="en").items(10):
print tweet.id
I have everything authenticated properly and the code works most of the time. However, when I try to search for some keywords (examples below) it doesn't return anything.
The keywords that cause trouble are "digitalkidz" (a tech conference) and "newtrendbg" (a Bulgarian company). If you do a quick search on Twitter for either of those you will see that there are results. However, tweepy doesn't find anything. Again, it does work for pretty much any other keyword I use.
Do you have any ideas what might be the problem and how to fix it?
Thank you
I believe you're forgetting an important aspect of the twitter api, it's not exhaustive.
Taken from the api docs
Please note that Twitter’s search service and, by extension, the Search API is not meant to be an exhaustive source of Tweets. Not all Tweets will be indexed or made available via the search interface.
Regardless of whether you're using the streaming or rest api, you're going to have issues with this if you're looking for specific tweets.
Rest API
When looking for historical tweets, you unfortunately won't be able to obtain anything that is older than a week using api.search(). This is also shown in the docs.
Keep in mind that the search index has a 7-day limit. In other words, no tweets will be found for a date older than one week.
There are other ways of getting older tweets, this post details those options.
Streaming API
While it doesn't sound like you're using twitter's streaming API, it should be noted that this only gives a small sample of twitter's current tweet traffic (~1-2%).
Hopefully this is helpful. Let me know if you have any questions.
I am pretty new to Python. I am currently using version 3.3.2. I have an array of links to sources of streaming videos. What I want to do is sort them between HD(>720p) and non-HD(<720p) videos. I have been searching the internet, but the only closest I got to was a ffmpeg python wrapper https://code.google.com/p/pyffmpeg/.
So I wanted to know if it is even possible? If yes, can you please link me to some resources, or what keywords I should be searching on google?
Regards
The easy way to do this is to use the web service API for each site.
For example, the YouTube API lets you issue a search and get back metadata on all of the matching videos. If you look at the video, properties, you can check definition == 'hd', or you can iterate the videoStreams for the video and check whether heightPixels >= 720 or bitrateBps >= 8*1024*1024 or whatever you think is an appropriate definition for "HD" if you don't like theirs.
You can find the APIs for most sites by just googling "Foo API", but here are links for the ones you asked about:
Daily Motion
Metacafe: I can't find the API docs anymore, but it's just simple RSS feeds.
YouTube
The hard way to do this is to write a universal video downloader—which is very, very hard—and process the file with something like pyffmpeg after you download it (or, if you're lucky, after you've only downloaded part of it).
I've been looking for a method to get a steam users status. This includes things like the game they're playing, if they're online or if they're offline. I was originally going to scrape the page, but I figured there must be an easier way to do it.
I looked around for pages which might be json of the users current status, perhaps even an API. But I haven't found anything yet.
Start with the Steam Web API. I think you are looking to work with GetPlayerSummaries.
Did you see:
http://steamcommunity.com/dev
Steam Web API Documentation
Valve provides these APIs so website developers can use data from
Steam in new and interesting ways. They allow developers to query
Steam for information that they can present on their own sites. At the
moment the only APIs we offer provide item data for Team Fortress 2,
but this list will grow over time.