I am trying to collect tweets and extract the text part for my project. I tried many ways and most of them works just fine for me. Though I stumbled upon this nltk.twitter package and some code snippets to do the same work. The codes are pretty clean and i want to use that. But even the simplest code is giving me 401 error though I have my account at twitter developers' and have all the four keys required.
from nltk.twitter import Twitter
tw = Twitter()
tw.tweets(keywords='love, hate', limit=10)
I took this example from http://www.nltk.org/howto/twitter.html#simple and tried every example that is given. None of them works. And apparently I cannot find why. Thank you for your help in advance.
There is a few things that may have caused this. But I bet it is the time issue as nltk is trying to use the streamer and the time of your computer/server is out of sync.
Also make sure you install nltk completely. Try
import nltk
dl = nltk.downloader.Downloader("http://nltk.github.com/nltk_data/")
dl.download()
Using nltk.twitter requires your credentials.txt file path in TWITTER environment variable and the data inside the text file has to be entered correctly.
For example:
`app_key=YOUR CONSUMER KEY
app_secret=YOUR CONSUMER SECRET
oauth_token=YOUR ACCESS TOKEN
oauth_token_secret=YOUR ACCESS TOKEN SECRET
`
There should be no space before of after the '='. Also, don't put the key in quotes like "YOUR CONSUMER KEY".
This solved my issue with 401.
Related
I am working on already built django project, this is the url;
website_domain/server/rest/services/System/SpatialAnalysisTools/GPServer/JoinFeatures/jobs/{JobId}/results/outputLayer?token={Token}&f=json
I create JobId and Token but when I pass both of them in this url I get {"Value":Null}.
I am new to ArcGIS, I tried every possible way but not getting any solution. Any help would be very appreciated.
I read documentation but I didn't find anything useful regarding token.
I want this above mentioned url to return Value so I would use that value in next step. My whole process is stop due to this problem.
I am attaching a screenshot of my required code.
Any help would be really really appreciated.
I'm new with this whole requests stuff. I tried to get the response from an url and I there were some few times in which I was able to. However, after several tries and adding new lines of code, it stopped working.
This is the part of my code where I'm having trouble:
api_request = requests.get("https://www.airnowapi.org/aq/observation/zipCode/current/?format=application/json&zipCode=12345&distance=5&API_KEY=1234-1234-1234-1234")
api = json.loads(api_request.content)
city = api[0]["Reporting Area"]
it tells me that the variable city is not defined, therefore it follows that the requests.get() part has been unsuccessful.
I read some other people who had the same problem, but theirs had something to do with Headers (I tried copying the solutions but they didn't seem to work). I also tried copying the url on google and it opened the json file correctly, so it's a Python problem.
Thanks!
I don't have enough reputation to comment yet, but wanted to warn you that you accidentally leaked your API key. If you don't want others to exploit it, try to keep it private as much as possible ;)
Try this -
api_request = requests.get("https://www.airnowapi.org/aq/observation/zipCode/current/?format=application/json&zipCode=89129&distance=5&API_KEY=1369737F-5361-4CCC-B7C1-F52625548A41")
api = api_request.json()
city = api[0]["ReportingArea"]
I'm working with a json API using python requests, the company who issued the API doesn't clarify in its documentation how many requests I can make per day/hour/minute.
Sometimes I am getting connection errors with it, is there any way to systematically deal with that? find out what the limits are or some way to test what they are efficiently?
Thank you.
I am not an expert in this and your question was pretty vague so I can't really tell in what way you need to systematically deal with the problems that arise.
If it is something that is already running and only occasionaly fails due connection errors or request limits then I would suggest programming something that helps you track the requests and answers from the server by writing them into a text file so you can check later what the error message was and why it happened. To make it easier to go over this data you can have it as an object in your code and save it as a json or xml file so that you can open it and have it as an object again in python.
For example you could have an object that saves the number of requests you have sent, the number of errors you have gotten, a list of requests you have sent, a list of answers you have gotten from the server and a list with numbers of which requests have failed.
If it is something small scale or something that isn't working yet then I would suggest just going over the errors manually and just try to see if you can fix them. You could send alot of requests manually to check the request limit and you could try to find out what happens when a connection error occurrs and try to recognize that in python and send the request again.
I am trying to get the data correlated with a user's feed tab on Instagram. Every time I google "get user feed" or anything similar, I am presented with how to get the recent photos that a user has posted.
However, I am trying to get the feed tab, or basically the trending posts of other people that I follow. I tried using Lev Pasha's Instagram API module, but the only related function that I could find was getPopularFeed(), which got posts from the explore page.
I know how to connect to the Instagram API and have done it using the code below, but have not managed to collect the feed tab.
from InstagramAPI import InstagramAPI
InstagramAPI = InstagramAPI("myusername", "mypassword")
InstagramAPI.login()
Update (6/2/2021):
I've decided to add a bit of info here regarding the Instagram API. I'd like to say that this answer is slightly "deprecated" in a sense, as the GitHub repo for this project has been taken down, and therefore it will become harder to develop updates and bug fixes for the module. As a result, I'd actually recommend that you don't use this module, as it will get you banned very quickly from the Instagram platform (the last update was in 2018, and therefore any User-Agent fixes and bot-detection workarounds will not be added).
I'd also like to say that as of now, there seem to be no good modules for the task of Instagram automation. As I've just said, the InstagramAPI module written in my post is long gone, and all the other API projects on Github are also archived / have no more moderation or updates.
TLDR: Maintaining an API which is being updated constantly by a huge corporation is not easy, and therefore you will get banned / blocked easily. I'd recommend using Lev Pasha's or ping's Instagram APIs, although they are both deprecated and might get you blocked quickly. The module in the answer below is also pretty good, although I've noticed that it hasn't been updated in almost 2 years. Also, it's built as a wrapper on top of ping's API.
You can use instapi module to solve your problem. Link to repo
Code example:
from instapi import bind, User
bind('myusername', 'mypassword')
user = User.from_username('some username')
for feed in user.iter_feeds():
# do something with feed
To install instapi using pip:
pip install git+https://github.com/uriyyo/instapi
I have been trying to download past broadcasts for a streamer on twitch using python. I found this python code online:
https://gist.github.com/baderj/8340312
However, when I try to call the functions I am getting errors giving me a status 400 message.
Unsure if this is the code I want to download the video (as an mp4) or how to use it properly.
And by video I mean something like this as an example: www(dot)twitch.tv/imaqtpie/v/108909385 //note cant put more than 3 links since I don't have 10 reputation
Any tips on how i should go about doing this?
Here's an example of running it in cmd:
python twitch_past_broadcast_downloader.py 108909385
After running it, it gave me this:
Exception API returned 400
This is where i got the information on running it:
https://www.johannesbader.ch/2014/01/find-video-url-of-twitch-tv-live-streams-or-past-broadcasts/
Huh it's not as easy at it seems ... The code you found on this gist is quite old and Twitch has completely changed its API. Now you will need a Client ID to download videos, in order to limit the amount of video you're downloading.
If you want to correct this gist, here are simple steps you can do :
Register an application : Everything is explained here ! Register you app and keep closely your client id.
Change API route : It is no longer '{base}/api/videos/a{id_}' but {base}/kraken/videos/{id_} (not sure about the last one). You will need to change it inside the python code. The doc is here.
Add the client id to the url : As said in the doc, you need to give a header to the request you make, so add a Client-ID: <client_id> header in the request.
And now I think you will need to start debugging a bit, because it is old code :/
I will try myself to do it and I will edit this answer when I'm finished, but try yourself :)
See ya !
EDIT : Mhhh ... It doesn't seem to be possible anyway to download a video with the API :/ I was thinking only links to API changed, but the chunks section of the response from the video url disappeared and Twitch is no longer giving access to raw videos :/
Really sorry I told you to do that, even with the API I think is no longer possible :/
You can download past broadcasts of Twitch videos with Python library streamlink.
You will need OAuth token which you can generate with the command
streamlink --twitch-oauth-authenticate
Download VODs with:
streamlink --twitch-oauth-token <your-oauth-token> https://www.twitch.tv/videos/<VideoID> best -o <your-output-folder>