I was hoping to find an answer to my problem with the elasticsearch python framework. Maybe I'm completely blind or doing something absolutely wrong, but I'm very confused right now and can't find an adequate answer.
I'm currently trying to establish a connection to my elastic search API using the elasticsearch python framework, my code looks like this:
from elasticsearch import Elasticsearch
def create_es_connection(host: str, port: int, api_key_id: str, api_key: str, user: str, pw: str) -> Elasticsearch:
return Elasticsearch([f"https://{user}:{pw}#{host}:{port}"])
This is working fine. I then created an API key for my testuser, which I'm giving to the function above as well. I am now trying to do something similar like this: Elasticsearch([f"https://{api_key_id}:{api_key}#{host}:{port}"]) so, I want to leave out the user and password completely, because in regards to the greater project behind this snippet, I'm not feeling very well in saving the user/password credentials in my project (and maybe even pushing it to our git server). Sooner or later, these credentials have to be entered somewhere and I was thinking that only saving the API key and to authenticate with that could be more safe. Unfortunately, this isn't working out like I planned and I couldn't find anything about how to authenticate with the API key only.
What am I doing wrong? In fact, I found so little about this, that I'm questioning my fundamental understanding here. Thank you for your replies!
working configuration for me was :
es = Elasticsearch(['localhost:9200'], api_key=('DuSkVm8BZ5TMcIF99zOC','2rs8yN26QSC_uPr31R1KJg'))
Elasticsearch's documentation shows how to generate and use (at the bottom of the page) an API key: https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html
curl -H "Authorization: ApiKey ..." http://localhost:9200/_cluster/health
-H means "header", so to do the same in Python you will need to set this header. Rummaging through the elasticsearch module source code tells me that you might just be able to do the following:
Elasticsearch([f'http://{host}:{port}'], api_key=api_key)
The reason for this is that **kwargs of the Elasticsearch.__init__ method are passed to Transport.__init__, the **kwargs of that are passed to Connection.__init__, which takes an api_key arg which is then used in
_get_api_key_header_val to construct the appropriate header.
The following code shows that the api key header gets added to the HTTP headers of the request:
import elasticsearch, logging, http.client
http.client.HTTPConnection.debuglevel = 5
logging.basicConfig(level=logging.DEBUG)
c = elasticsearch.Elasticsearch(['localhost'], api_key='TestApiKey')
print(c.cluster.health(wait_for_status='green'))
This is definitely something that should be added to Elasticsearch's docs.
Related
Here is how I want my program to work. Step 2 is what I am unsure of how to implement.
Client makes API call to /email endpoint
/email endpoint has a script run that gather emails from GMAIL API
Put contents into response object
Returns response object back to client
I understand how to make a static api response. But I can't get a python script to run when the api endpoint is hit.
I saw the flask tag in your post.
I only played around with flask for certain interviews, but know enough to say calling a python script outside your running server is somewhat of an antipattern.
I assume your backend is a flask app, so ideally, you'd want to wrap whatever script you have in your python script file in a function and simply call it from your flask method when the endpoint is hit.
Something like:
from flask import Flask
from custom_emails_module import gather_email
#api.route('/email', methods=["GET"])
def method_associated_with_your_endpoint():
# additional
gather_email()
where custom_emails_module should be the module you create for your gather_emails script.
Now, at the end of your gather_emails function, simple remember to return the correct type, usually done with:
return json.dumps("success": True, "data": python_object_with_several_emails)
Use something like PostMan for local debugging and remember to use application/json in header for Content-Type.
Good luck!
The situation is the following: I have a Flask web server, and I need to develop a new API endpoint for it (POST/GET), which would update/read a server status.
My POST API requires an input like this:
{
"name": <name_of_the_server>,
"serverUrl" <url_of_the_server>,
"available": true/false
}
I want to save this information in Redis for a week. I save the whole JSON from the API request as a dictionary value in Redis. I dump and load it with pickle. The key is the "name" paramater.
I also have a GET API endpoint, in which I can specify the name of the server, and get the status information about it. Hence, Python checks in Redis for the key, containing the name of the server (given by the user through the API), and returns the value (the dictionary with all of the information that was saved previously).
This is all good, and works as intended. However, through the GET API, I should also be able to search for server status with the server URL (as well as the server name).
So far my solution is like this:
// Redis is a class, I previously instantiated
redis.set(server_name, server_data, ex=time_to_live)
redis.set(server_url, server_data, ex=time_to_live)
I set two keys, containing the exact same information. Through the GET endpoint, this gives me the chance to check server status by using either the server name or the server URL, as they return the same value. Also, they have the same expiry date, and are always updated together.
This works, however I feel this is more of a "cheat" solution, so I would want to ask for an advice if I should keep it that way, or if there is a better way to achieve the same functionality. Thank you in advance!
I simply want to receive notifications from dropbox that a change has been made. I am currently following this tutorial:
https://www.dropbox.com/developers/reference/webhooks#tutorial
The GET method is done, verification is good.
However, when trying to mimic their implementation of POST, I am struggling because of a few things:
I have no idea what redis_url means in the def_process function of the tutorial.
I can't actually verify if anything is really being sent from dropbox.
Also any advice on how I can debug? I can't print anything from my program since it has to be ran on a site rather than an IDE.
Redis is a key-value store; it's just a way to cache your data throughout your application.
For example, access token that is received after oauth callback is stored:
redis_client.hset('tokens', uid, access_token)
only to be used later in process_user:
token = redis_client.hget('tokens', uid)
(code from https://github.com/dropbox/mdwebhook/blob/master/app.py as suggested by their documentation: https://www.dropbox.com/developers/reference/webhooks#webhooks)
The same goes for per-user delta cursors that are also stored.
However there are plenty of resources how to install Redis, for example:
https://www.digitalocean.com/community/tutorials/how-to-install-and-use-redis
In this case your redis_url would be something like:
"redis://localhost:6379/"
There are also hosted solutions, e.g. http://redistogo.com/
Possible workaround would be to use database for such purpose.
As for debugging, you could use logging facility for Python, it's thread safe and capable of writing output to file stream, it should provide you with plenty information if properly used.
More info here:
https://docs.python.org/2/howto/logging.html
I am trying to update Firebase using the python-firebase library, but cannot get authentication to work, using adapted sample code:
from firebase import firebase as fb
auth = fb.FirebaseAuthentication('<firebase secret>', 'me#gmail.com',
auth_payload={'uid': '<uid>'}) // NB renamed extras -> auth_payload, id -> uid here
firebase = fb.FirebaseApplication('https://<url>.firebaseio.com', authentication=auth)
result = firebase.get('/users', name=None, connection=None,
params={'print': 'pretty'}) // HTTPError: 401 Client Error: Unauthorized
print result
I keep getting (401) Unauthorized, but I notice that the token generated by the library is radically different to one generated by a JavaScript version of FirebaseTokenGenerator - and the latter authenticates fine when I provide the same URL, uid and secret.
I noticed a GitHub issue, questioning why the library did not just use the official Python firebase-token-generator, so I forked and implemented the suggested change just in case it would make a difference, but still get the same result.
Can anyone suggest what might be tripping me up here?
This library is 4 years old which means lots of things have been changed for firebase especially after Google's acquisition. The part of how you access Firebase is completely different.
I will recommend to use the official Firebase Admin Python SDK https://github.com/firebase/firebase-admin-python
A really good alternative but prefer the official is this:
https://github.com/thisbejim/Pyrebase
I have recently taken over support for an app that uses rauth to connect to linkedin. The code that is failing is:
self.linkedin= OAuth1Service(
name='linkedin',
consumer_key=self._consumer_key,
consumer_secret=self._consumer_secret,
request_token_url=self.request_token_url,
access_token_url=self.access_token_url,
authorize_url=self.authorize_url)
self.request_token, self.request_token_secret = \
self.linkedin.get_request_token(method='GET',
oauth_callback=self.callback_url)
The owner of the app says this used to work but now we're getting:
TypeError: request() got an unexpected keyword argument 'oauth_callback'
Can you point me to some doc/examples that would help me re-architect this?
-Jim
It sounds like you're using a later version of rauth than the original author was. You will need to amend the code to conform to the changes in the rauth API. These are mostly small, partly necessitated by the move to Requests v1.0.0 which had many breaking changes in its API.
You should read the upgrade guide. Additionally there's a number of working examples.
Finally this particular error is indicating that an unexpected parameter was passed in, namely oauth_callback. This is because rauth is just a wrapper over Requests. Requests doesn't know what to do with oauth_callback. Instead, you should use the native Requests' API and pass it in, in this case, via the params parameter, e.g.:
linkedin = OAuth1Service(name='linkedin',
consumer_key=consumer_key,
consumer_secret=consumer_secret,
request_token_url=request_token_url,
access_token_url=access_token_url,
authorize_url=authorize_url)
request_token, request_token_secret = \
linkedin.get_request_token(method='GET',
params={'oauth_callback': callback_url})
Hope that helps!