I am relatively new to python. I am trying to use the scopus api to create a csv file that includes the text from all of the abstracts of a particular author. Any guidance on where to look for sample code would be much appreciated! I can't find documentation for how to use https://api.elsevier.com/content/search/scopus on python.
There is a a Python module for use with api.elsevier.com, located here:
https://github.com/ElsevierDev/elsapy
Its aim is to make life easier for people who are not primarily programmers, but need to interact with publication and citation data from Elsevier products in a programmatic manner (e.g. academic researchers).
This is not an 'official' SDK and is not guaranteed to always work with Elsevier's APIs, on all platforms, or without eating up all your machine's resources. But we'll do our best to keep it in good shape, are happy to take suggestions for improvements, and are open to collaborations.
License info is here:
There's also the pybliometrics package that we develop - from the Scopus community for the Scopus community. In handles all the difficult parsing and accessing of the website, and also caches the responses for later use.
Here are examples accessing the Scopus Search API using the ScopusSearch() class of pybliometrics: https://pybliometrics.readthedocs.io/en/stable/examples/ScopusSearch.html
For your use case, you can simply do this:
import pandas as pd
from pybliometrics.scopus import ScopusSearch
q = "AU-ID(7004212771)" # any query that works in the Advanced Search on scopus.com
s = ScopusSearch(q)
df = pd.DataFrame(s.results)
df.to_csv(OUTPUT_FILE)
Related
I'm new to Python and am very confused on how to begin this assignment:
Write Python code to connect to Twitter search API at:
https://api.twitter.com/1.1/search/tweets.json
with at least the following parameters:
q for a search topic of your interest;
count to 100 for 100 records
to retrieve twitter data and assign the data to a variable.
I already created the app to access Twitter's API. Thanks.
I had this same issue. According to Twitter, you must register an account to even use basic search features of public tweets and there is no way around it. I followed the guide for the complete setup here and can confirm it works
Edit: the article is for php, but changing it to python shouldn’t be very difficult. It has all the methods outlined in the article
Just as a hint, you might want to use tweepy for this. They have very good documentation and its much easier than trying to reinvent the wheel.
I'm looking for an Google API for python (except the official API), in order to make "eloborate" query like "intitle", "intext","filetype", ...
I made some research on the internet but I don't have find my need. Maybe I misssearched or maybe that doesn't exist.
Can you give me some lead or advice ?
Thank you.
Google Search API has deprecated long ago. You could use something like pygoogle to screen-scrape the results page, but you're liable to be detected by Google's anti-bot captchas if you make a large number of queries.
I was wondering if any OData Python libraries are available to produce and consume OData?
There are implementations for different languages:
http://www.odata.org/libraries/
But I couldn't find Python so far. I don't mean IronPython by the way. The library should be just usable in Python.
i am the author of the library at http://code.google.com/p/odata-py/
it's still in its early stages but it provides the most basic functionalities (create, read, update). Don't hesitate to drop a message if you see a bug or want to contribute ;)
I've recently added some OData modules to a Python package I maintain for an e-Learning project called Pyslet. The project is hosted on Github here: https://github.com/swl10/pyslet
I wrote an introductory blog post demonstrating the OData consumer features here: http://swl10.blogspot.co.uk/2014/02/a-dictionary-like-python-interface-for.html
I started my own OData 4.0 consumer project some time ago. It's based on the requests library and is pure Python. It's rather minimal as I've only implemented things I needed for work. Check it out on my github.
Works kinda like this:
from odata import ODataService
url = 'http://services.odata.org/V4/Northwind/Northwind.svc/'
Service = ODataService(url, reflect_entities=True)
Product = Service.entities['Product']
query = Service.query(Product)
query = query.filter(Product.ProductName.startswith('Queso'))
query = query.order_by(Product.UnitPrice.desc())
for product in query:
print(product.ProductName)
Here is a version that is targeting Google App Engine: http://code.google.com/p/odata-py/
I've been experimenting with the spec and wrote a simple server for Python called MyOhData: https://bitbucket.org/dowski/myohdata/src
please check this link
http://www.odata.org/libraries/
ODataPy (Python)
ODataPy is an open-source Python library that implements the Open Data Protocol (OData). It supports the OData protocol version 4.0. It is built on top of ODataCpp using language binding. It is under development and currently serves only parts of client and client side proxy generation (code gen) aspects of OData.
V4 Client GitHub
ODataStore for CoreData (iOS)
The ODataStore for CoreData is an iOS static library and a Mac OS X Framework to use V3 OData services with the CoreData Framework from Apple. V4 OData services will be supported in the future. The development language is Objective-C.
V3 Both Link
Pyslet Python Package (Python)
Pyslet is a Python package for Standards in Learning Education and Training. It implements a number of standards including OData v2 with both client and server capabilities.
V2 Both Link
OData4ObjC
This library makes it easy for iOS app developers to interact with data in any OData-compliant web service. It supports metadata-aware client-side code generation and full CRUD with query. If someone exposes a data model via OData, OData4ObjC makes it easy to get that model onto your iOS device.
V1-3 Client GitHub
I've looked as well after getting an intro to OData and it looks like there isn't one as of yet unfortunately. I'll be keeping an eye out for one as I'm sure one will surface.
Update 2016
OData Libraries lists two python libraries that support OData. With pyslet looking to be the most active since it has had commits in the last few months and several releases. I haven't tried either of them so I can't really say if they work well or not.
I'm sorry but I'm having some trouble implementing Oauth within my app engine python project.
I've been working from http://github.com/tav/tweetapp, but I don't think I have a strong enough grasp on this platform to understand how to implement this class within my main.py I'm building the rest of my app in.
This maybe a feeble attempt, but here is what I have so far:
twa = twitter_auth
client = twa.OAuthClient('twitter')
I've created a source folder within my project called "twitter_auth" and that contains a file within it called "twitter_auth.py" which contains the above linked library, and a file called __ init__.py (no space) which is completely empty.
I really have no idea what to do from here :/
Let me recommend taking a look at the tweepy library and some example tweepy apps. Specifically here: http://github.com/wasauce/tweepy-examples
This shows how to use oauth to authenticate a user: http://github.com/wasauce/tweepy-examples/tree/master/appengine/oauth_example/
As Hagge said, it sounds like your issue is more with the tweetapp library than with App Engine. However, if you would like to know more about OAuth on App Engine and if I may be allowed to link to myself, my two articles on the topic seem to be reasonably popular.
The tweetapp library was a an early prototype for Twitter OAuth on twitter. Tav did the heavy lifting and I deployed the site http://twitteroauth.appspot.com , using some of the tweetapp library. The actual source of that site is here (I need to update the site to point here): http://github.com/ryanwi/twitteroauth
I am still using it in production, but, it has aged and does not work for all API calls. I'd recommend trying a different, more up to date and maintained library as others have mentioned.
But, take a look at the twitteroauth source if you want to try to get a first attempt working.
These two are on Twitter's list
http://github.com/brosner/python-oauth2
http://code.google.com/p/oauth-python-twitter2/
I'm not familiar with that library, but after a quick look and seeing the warning that it is not maintained I'd search for something better. I implemented a simple Twitter connection based on Tornado's auth: see an example of how to make Twitter API calls here (and an authentication example here). In case you don't want to use tipfy, I recommend implementing the python-twitter library in your framework of choice.
Does anyone have any good information aside from the Google App Engine docs provided by Google that gives a good overview for people with MS SQL background to porting their knowledge and using Google App Engine Data Store API effectively.
For Example, if you have a self created Users Table and a Message Table
Where there is a relationship between Users and Message (connected by the UserID), how would this structure be represented in Google App Engine?
SELECT * FROM Users INNER JOIN Message ON Users.ID = Message.UserID
Here is a good link: One to Many Join using Google App Engine.
http://blog.arbingersys.com/2008/04/google-app-engine-one-to-many-join.html
Here is another good link: Many to Many Join using Google App Engine:
http://blog.arbingersys.com/2008/04/google-app-engine-many-to-many-join.html
Here is a good discussion regarding the above two links:
http://groups.google.com/group/google-appengine/browse_thread/thread/e9464ceb131c726f/6aeae1e390038592?pli=1
Personally I find this comment in the discussion very informative about the Google App Engine Data Store:
http://groups.google.com/group/google-appengine/msg/ee3bd373bd31e2c7
At scale you wind up doing a bunch of
things that seem wrong, but that are
required by the numbers we are
running. Go watch the EBay talks. Or
read the posts about how many database
instances FaceBook is running.
The simple truth is, what we learned
about in uni was great for the
business automation apps of small to
medium enterprise applications, where
the load was predictable, and there
was money enough to buy the server
required to handle the load of 50
people doing data entry into an
accounts or business planning and
control app....
Searched around a bit more and came across this Google Doc Article:
http://code.google.com/appengine/articles/modeling.html
App Engine allows the creation of easy
to use relationships between datastore
entities which can represent
real-world things and ideas. Use
ReferenceProperty when you need to
associate an arbitrary number of
repeated types of information with a
single entity. Use key-lists when you
need to allow lots of different
objects to share other instances
between each other. You will find that
these two approaches will provide you
with most of what you need to create
the model behind great applications.
Can I supplement the excellent answer further above with a link to a video:
http://sites.google.com/site/io/building-scalable-web-applications-with-google-app-engine
It's a great talk by Google's Brett Slatkin who talks for an hour about the special way you need to think about your application before you can expect it to scale well. There are some genuine WTFs (such as no count() in db queries) that will cause you to struggle if you are coming from a relational background.
I think this is the basics : Keys and Entity Groups
look for it in appengine docs. (I'm new here so can't post a link)
I have worked on it but not a expert though Google app engine is very good thing and it is the future as it implements Platform as a Service and Software as a Service. Google app engine provides a non- relational database. So you cantreally write relationships here.
Regards,
Gaurav J
These links are great, but are predominantly python biased, I am using GWT, and therefore have to use the java flavour of GAE, does anyone have any examples of how to achieve these "join" equivalencies in the java version of GAE?
Cheers,
John
The standalone GAE SDK is pretty difficult to use for putting data into and retrieving data from the Google App Engine data store.
"Objectify" is a GAE extension that makes these operations much easier. The Objectify wiki and source code can be found here. I strongly recommend using Objectify in your GAE project.
http://code.google.com/p/objectify-appengine/
Here are a couple of tutorials on using Objectify with the app engine. Follow these tutorials and you will be storing and retrieving data in no time.
http://www.fishbonecloud.com/2010/11/use-objectify-to-store-data-in-google.html