I have the following database entry: "companies: 8":
I have the following database rules, which do not allow a simulated write of "companies: 8" to the database.
{
"rules": {
".read": "auth != null",
".write": "auth != null",
"companies": {
".validate": "(data.exists() && (newData.val() === data.val() + 1)) || (!data.exists() && newData.val() == 0)"
}
}
}
However, when I try to write "companies: 20" to the database with the Firebase Python SDK, which also is not allowed under these rules, it works:
In [1]: import firebase_admin
In [2]: from firebase_admin import credentials, db
In [3]: cred = credentials.Certificate('serviceAccountCredentials_dev_async.json
...: ')
In [4]: firebase_admin.initialize_app(cred, {'databaseURL': 'https://async-testi
...: ng.firebaseio.com/'})
Out[4]: <firebase_admin.App at 0x7fc50c00c080>
In [5]: ref = db.reference()
In [6]: ref.update({'companies': 20})
What am I doing wrong?
You are using the Firebase Admin SDK and it looks like you are initializing with the credentials for a service account. In this case, no security rules are applied, I think not even validation rules.
If there is a reason you must you the Admin SDK and want validation rules to be performed, authenticate with limited priveleges:
As a best practice, a service should have access to only the resources
it needs. To get more fine-grained control over the resources a
Firebase app instance can access, use a unique identifier in your
Security Rules to represent your service. Then set up appropriate
rules which grant your service access to the resources it needs.
Related
I have read the AWS documentation but I couldn't find an example of using Temporary Security Credentials to authenticate to AWS with Python.
I would like an example of using a temporary security credentials provided by the AWS Security Token Service (AWS STS) to sign a request.
There are several ways you can use STS to get temporary credential. The two most common ones would be:
get_session_token - used to get temp credentials for existing IAM user or account
assume_role - used to get credentials when assuming iam role
In both cases the call to these function will give you temp credentials, e.g.:
{
"Credentials": {
"AccessKeyId": "AddsdfsdfsdxxxxxxKJ",
"SecretAccessKey": "TEdsfsdfSfdsfsdfsdfsdclkb/",
"SessionToken": "FwoGZXIvYXdzEFkaDGgIUSvDdfgsdfgsdfgsMaVYgsSxO8OqRfjHc4se90WbaspOwCtdgZNgeasdfasdfasdf5wrtChz2QCTnR643exObm/zOJzXe9TUkcdODajHtxcgR8r+unzMo+7WxgQYyKGN9kfbCqv3kywk0EvOBCapusYo81fpv8S7j4JQxEwOGC9JZQL6umJ8=",
"Expiration": "2021-02-17T11:53:31Z"
}
}
Having these credentials, you create new boto3 session, e.g.:
new_session = boto3.session.Session(<temp credentails>)
The new_session will allow you to make new boto3 client or resource, e.g.:
ec2 = new_session.client('ec2')
s3r = new_session.resource('s3')
And then you can use these new clients/resource as you would normally use them.
I written rules for my firestore documents and using admin sdk credentials, that bypasses rules.
How to make rules work on python?
Python code
import firebase_admin
from firebase_admin import credentials
from firebase_admin import firestore
cred = credentials.Certificate('key.json')
firebase_admin.initialize_app(cred)
db = firestore.client()
try:
doc_ref = db.collection('users').document('test')
doc_ref.set({'password':'testPass','shadow':'testShadow'})
print('success')
except:
print('failed')
Firestore rules
rules_version = '2';
service cloud.firestore {
match /databases/{database}/documents {
match /users/{userId} {
allow read, write: if request.auth.uid == userId;
}
}
}
The Firebase Admin SDKs run with administrative privileges and bypass the security rules for Firestore. There is no way to change that.
If you want your code to follow the security rules, you'll have to use one of the client-side SDKs, but none exists for Python at the moment.
I'm writing a simple Google Cloud Function to learn, with the new Python3.7 option, and I would like to read/write some data to Firebase Realtime db.
I found some information to use Firestore that worked, but now I need to use the realtime Database.
# global
import firebase_admin
from firebase_admin import db
# from firebase_admin import firestore
default_app = firebase_admin.initialize_app()
def check_prices_updates(request):
"""Responds to any HTTP request.
Args:
request (flask.Request): HTTP request object.
Returns:
The response text or any set of values that can be turned into a
Response object using
make_response
"""
collection = 'pricelist'
mybase = db.reference(collection)
....[other code]
return "OK"
This is an excerpt of my code, the documentation says that the initialize_app() must be left empty to collect the project default credentials.
The line where I call db.reference(collection) give me the exception:
ValueError: Invalid databaseURL option: "None". databaseURL must be a non-empty URL string.
as if the configuration parameters did not show the requested key 'databaseURL'. With Firestore, as example I could call firestore.client(), after the import from firebase_admin, adding 'google-cloud-firestore in requirements.txt.
Now, for firebase, what should I fix to read and write on the db?
PS. on my db the ".read" rule is true for all so I haven't trouble on auth!
UPDATE:
After a good sleep I realized that to access the realtime db I need to pass some options to initialize_app, as follow:
firebase_admin.initialize_app(options={'databaseURL': 'https://[PROJECT-NAME].firebaseio.com'})
Cheers
Initiliase the Firebase Admin SDK
This will access the db, even if you put permission restrictions on read, write, as your python will have admin access
If you have multiple database instance (Only available for blaze plan)
Initilise them with different app name inside single python script
#Firebase Admin Imports
import firebase_admin
from firebase_admin import credentials
from firebase_admin import db
#Firebase admin sdk Key - Save it as secret
firebaseval={
"type": "service_account",
"project_id": "<project_id>",
"private_key_id": "<private_key>",
"private_key": "-----BEGIN PRIVATE KEY--xxxxxxxxxxxxxxxxxxxxxxxxxx\n-----END PRIVATE KEY-----\n",
"client_email": "firebase-adminsdk-ipcf5#<project_id>.iam.gserviceaccount.com",
"client_id": "<clinet_id>",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/firebase-adminsdk-ipcf5%40<project_id>.iam.gserviceaccount.com"
}
cred = credentials.Certificate(firebaseval)
print("\n---------------------------")
print("Initialising Firebase...")
#Initilising Database 1- fetching stock and option details
firebase_admin.initialize_app(cred, {
'databaseURL' : 'https://<project_id>-<instance_id1>.firebaseio.com/'
})
# Initilising Database 2- Different instance ID
app2 = firebase_admin.initialize_app(cred, {
'databaseURL': 'https://<project_id>-<instance_id2>..firebaseio.com/'
}, name='app2')
# Initilising Database 3- Different instance ID
app3 = firebase_admin.initialize_app(cred, {
'databaseURL': 'https://<project_id>-<instance_id3>..firebaseio.com/'
}, name='app3')
print("Initialised Firebase")
print("---------------------------\n")
Accessing the different database Instances
# database instance_id2
mainref2=db.reference("PATH",app2)
# database instance_id3
mainref3==db.reference("PATH",app3)
# Reading whole data from
mainref2.get()
# Reading inside child node from db instance_id2
mainref2.child("child_name").get()
# Setting json data to db instance_id3
mainref3.set({"data":"something"})
I have 3 databases in my MongoDB server. I am using pymongo to do some scripting with Python3.
I want to use the latest versions and practices. Once I open the client and pick the database, the API for pymongo.MongoClient.['mydatabase'].authenticate is deprecated.
https://api.mongodb.com/python/current/api/pymongo/database.html
Authentication prior to picking the database (while dialing the client) doesn't seem to flow down toward the database. Not just for pymongo, but also when I use mongo shell. So I have a feeling this is the issue.
script.py
import pymongo
from pymongo import MongoClient
u = getUser() # function which prompts for username
p = getPassword() # getpass.getpass('Password')
uri = formatUri(u, p) # formats 'mongodb://%s:%s#%s'.format(user, password, host)
client = MongoClient(uri)
db = client['mydb']
col = db.mycollection
for doc in col.find():
print(doc)
I get the error that I am not authorized for the database. I know my account works in shell but I have to dial the client first then use the db and then auth.
Here's a mongo shell example:
$ mongo
MongoDB shell version: v3.4.10
Connecting to: mongodb://127.0.0.1:port
MongoDB server version: v3.4.10
> use mydb
switched to mydb
> db.auth("user", "pass")
1
Any idea how I can either auth after picking the database or once I use the db it remembers the context I dialed with?
You seem to be missing some concepts here so I'll basically answer as a "guide" to what you should be doing instead. So "authentication' is not really something you do "after" connection, but rather you need to be "looking in the right place" when you actually attempt to authenticate.
We can start this by essentially following the process outlined in Enable Auth from the core documentation, but specifically altered because you want to be running this "test" under your own user account and local directory.
Revision Steps - Straight from Documentation
So first would would want to pick a local working directory and make a path for the database storage files underneath that. On *nix based systems you can do something like:
mkdir -p scratch/data/db
cd scratch
Then we want to startup a separate MongoDB instance without any other options. Making sure the port does not conflict with any other running instance:
mongod --port 37017 --dbpath data/db
In a new terminal or command line window, you can then connect to the shell:
mongo --port 37017
You always want at least one account with administrative privileges to at least "create accounts" and alter them in case you get in trouble, so create one:
use admin
db.createUser(
{
user: "admin",
pwd: "admin",
roles: [{ role: "userAdminAnyDatabase", db: "admin" }]
}
)
Now exit the shell and close the existing mongod instance running in the other terminal or command prompt and then start it again using --auth:
mongod --auth --port 37017 --dbpath data/db
Specific User - Make sure you follow these
Now you actually want to create a user that will be "used by your application". So these steps are important to ensure you get it right.
Log into a shell using your "adminstrative user":
mongo -u admin -p admin --port 37017 --authenticationDatabase 'admin'
You can alternately do the db.auth() method as shown in the question, but as noted this must be authorised on the "admin" namespace.
The next thing you want to do is create a user with access to "mydb" as a namespace with the readWrite role. For kicks, we are also going to let this user have the readAnyDatabase allowing them to "list" all databases namespaces, if not actually being able to do anything else with them.
IMPORTANT: You create ALL your users in the "admin" namespace. And this will be very important in future releases:
use admin
db.createUser(
{
"user": "myuser",
"pwd": "password",
"roles": [
{ "role": "readWrite", "db": "mydb" },
"readAnyDatabase"
]
}
)
Just for additional output, let's look at the current created users:
db.getUsers()
[
{
"_id" : "admin.admin",
"user" : "admin",
"db" : "admin",
"roles" : [
{
"role" : "userAdminAnyDatabase",
"db" : "admin"
}
]
},
{
"_id" : "admin.myuser",
"user" : "myuser",
"db" : "admin",
"roles" : [
{
"role" : "readWrite",
"db" : "mydb"
},
{
"role" : "readAnyDatabase",
"db" : "admin"
}
]
}
]
See how these have expanded in naming, and particularly the values assigned to the various "db" keys on each user. This should give you a little more insight into how MongoDB looks this up and why.
Python Connection
Finally we just want to connect from python. So presuming you have python and pymongo installed already, then it's just a simple listing to verify:
import pymongo
from pymongo import MongoClient
client = MongoClient('mongodb://myuser:password#localhost:37017');
db = client['mydb']
col = db.test
col.remove()
col.insert_one({ "a": 1 })
for doc in col.find():
print(doc)
Which shows the document created and listed without problem:
{u'a': 1, u'_id': ObjectId('5a08e5e0760108251722a737')}
Note that we don't actually need to make any mention of "admin" here, because this is the default where the driver "expects the accounts to be" and where you really "should" be doing it.
But I did it the wrong way
So let's say you originally got all confused and created the user under "mydb" instead:
use mydb
db.createUser({ "user": "bert", "pwd": "password", "roles": ["readWrite"] })
If you go look in "admin" that user is not there. But if you look on "mydb":
use mydb
db.getUsers()
[
{
"_id" : "mydb.bert",
"user" : "bert",
"db" : "mydb",
"roles" : [
{
"role" : "readWrite",
"db" : "mydb"
}
]
}
]
So you can see where the actual user data is now kept and how it has been recorded.
The simple case here is you "must" tell MongoDB where to obtain the authentication from for this user:
client = MongoClient('mongodb://bert:password#localhost:37017/mydb');
See how we add "mydb" on to the connection string. This is how it's done.
This is actually "in progress" to be made consistent with ALL drivers in how connections are made and where authentication happens as well as where you select the database. But there are basic rules:
If no other database namespace is provided with connection details for authentication credentials, then "admin" is taken to be the default.
Where there is a database namespace provided on the connection string, this will be used for authentication and this is the actual intent of the database namespace on the connection string.
Though other drivers "presently" differ in the role of the database namespace on the connection string, the usage is being changed to be consistent with all drivers that "using" a database namespace is in fact an API call, rather than being assigned from the connection string.
So where you need to authenticate depends on "where you created the user". But you should really be noting that "admin" is the place where you "should" be doing this instead of anywhere else.
Deprecation of Authenticate after connect
Whilst all drivers actually do have a similar method to authenticate(), which is used much like the shell example in the question, this method is now considered DEPRECATED as is mentioned throughout the content of the answer it is "intended" that you actually store your users in the "admin" namespace:
"Changed in version 3.5: Deprecated. Authenticating multiple users conflicts with support for logical sessions in MongoDB 3.6. To authenticate as multiple users, create multiple instances of MongoClient."
This is why the whole answer here is based on NOT using that method as you are meant to creating new connection instances, or using the "sessions" functionality available from MongoDB 3.6 instead.
I have 3 databases in my MongoDB server. I am using pymongo to do some scripting with Python3.
I want to use the latest versions and practices. Once I open the client and pick the database, the API for pymongo.MongoClient.['mydatabase'].authenticate is deprecated.
https://api.mongodb.com/python/current/api/pymongo/database.html
Authentication prior to picking the database (while dialing the client) doesn't seem to flow down toward the database. Not just for pymongo, but also when I use mongo shell. So I have a feeling this is the issue.
script.py
import pymongo
from pymongo import MongoClient
u = getUser() # function which prompts for username
p = getPassword() # getpass.getpass('Password')
uri = formatUri(u, p) # formats 'mongodb://%s:%s#%s'.format(user, password, host)
client = MongoClient(uri)
db = client['mydb']
col = db.mycollection
for doc in col.find():
print(doc)
I get the error that I am not authorized for the database. I know my account works in shell but I have to dial the client first then use the db and then auth.
Here's a mongo shell example:
$ mongo
MongoDB shell version: v3.4.10
Connecting to: mongodb://127.0.0.1:port
MongoDB server version: v3.4.10
> use mydb
switched to mydb
> db.auth("user", "pass")
1
Any idea how I can either auth after picking the database or once I use the db it remembers the context I dialed with?
You seem to be missing some concepts here so I'll basically answer as a "guide" to what you should be doing instead. So "authentication' is not really something you do "after" connection, but rather you need to be "looking in the right place" when you actually attempt to authenticate.
We can start this by essentially following the process outlined in Enable Auth from the core documentation, but specifically altered because you want to be running this "test" under your own user account and local directory.
Revision Steps - Straight from Documentation
So first would would want to pick a local working directory and make a path for the database storage files underneath that. On *nix based systems you can do something like:
mkdir -p scratch/data/db
cd scratch
Then we want to startup a separate MongoDB instance without any other options. Making sure the port does not conflict with any other running instance:
mongod --port 37017 --dbpath data/db
In a new terminal or command line window, you can then connect to the shell:
mongo --port 37017
You always want at least one account with administrative privileges to at least "create accounts" and alter them in case you get in trouble, so create one:
use admin
db.createUser(
{
user: "admin",
pwd: "admin",
roles: [{ role: "userAdminAnyDatabase", db: "admin" }]
}
)
Now exit the shell and close the existing mongod instance running in the other terminal or command prompt and then start it again using --auth:
mongod --auth --port 37017 --dbpath data/db
Specific User - Make sure you follow these
Now you actually want to create a user that will be "used by your application". So these steps are important to ensure you get it right.
Log into a shell using your "adminstrative user":
mongo -u admin -p admin --port 37017 --authenticationDatabase 'admin'
You can alternately do the db.auth() method as shown in the question, but as noted this must be authorised on the "admin" namespace.
The next thing you want to do is create a user with access to "mydb" as a namespace with the readWrite role. For kicks, we are also going to let this user have the readAnyDatabase allowing them to "list" all databases namespaces, if not actually being able to do anything else with them.
IMPORTANT: You create ALL your users in the "admin" namespace. And this will be very important in future releases:
use admin
db.createUser(
{
"user": "myuser",
"pwd": "password",
"roles": [
{ "role": "readWrite", "db": "mydb" },
"readAnyDatabase"
]
}
)
Just for additional output, let's look at the current created users:
db.getUsers()
[
{
"_id" : "admin.admin",
"user" : "admin",
"db" : "admin",
"roles" : [
{
"role" : "userAdminAnyDatabase",
"db" : "admin"
}
]
},
{
"_id" : "admin.myuser",
"user" : "myuser",
"db" : "admin",
"roles" : [
{
"role" : "readWrite",
"db" : "mydb"
},
{
"role" : "readAnyDatabase",
"db" : "admin"
}
]
}
]
See how these have expanded in naming, and particularly the values assigned to the various "db" keys on each user. This should give you a little more insight into how MongoDB looks this up and why.
Python Connection
Finally we just want to connect from python. So presuming you have python and pymongo installed already, then it's just a simple listing to verify:
import pymongo
from pymongo import MongoClient
client = MongoClient('mongodb://myuser:password#localhost:37017');
db = client['mydb']
col = db.test
col.remove()
col.insert_one({ "a": 1 })
for doc in col.find():
print(doc)
Which shows the document created and listed without problem:
{u'a': 1, u'_id': ObjectId('5a08e5e0760108251722a737')}
Note that we don't actually need to make any mention of "admin" here, because this is the default where the driver "expects the accounts to be" and where you really "should" be doing it.
But I did it the wrong way
So let's say you originally got all confused and created the user under "mydb" instead:
use mydb
db.createUser({ "user": "bert", "pwd": "password", "roles": ["readWrite"] })
If you go look in "admin" that user is not there. But if you look on "mydb":
use mydb
db.getUsers()
[
{
"_id" : "mydb.bert",
"user" : "bert",
"db" : "mydb",
"roles" : [
{
"role" : "readWrite",
"db" : "mydb"
}
]
}
]
So you can see where the actual user data is now kept and how it has been recorded.
The simple case here is you "must" tell MongoDB where to obtain the authentication from for this user:
client = MongoClient('mongodb://bert:password#localhost:37017/mydb');
See how we add "mydb" on to the connection string. This is how it's done.
This is actually "in progress" to be made consistent with ALL drivers in how connections are made and where authentication happens as well as where you select the database. But there are basic rules:
If no other database namespace is provided with connection details for authentication credentials, then "admin" is taken to be the default.
Where there is a database namespace provided on the connection string, this will be used for authentication and this is the actual intent of the database namespace on the connection string.
Though other drivers "presently" differ in the role of the database namespace on the connection string, the usage is being changed to be consistent with all drivers that "using" a database namespace is in fact an API call, rather than being assigned from the connection string.
So where you need to authenticate depends on "where you created the user". But you should really be noting that "admin" is the place where you "should" be doing this instead of anywhere else.
Deprecation of Authenticate after connect
Whilst all drivers actually do have a similar method to authenticate(), which is used much like the shell example in the question, this method is now considered DEPRECATED as is mentioned throughout the content of the answer it is "intended" that you actually store your users in the "admin" namespace:
"Changed in version 3.5: Deprecated. Authenticating multiple users conflicts with support for logical sessions in MongoDB 3.6. To authenticate as multiple users, create multiple instances of MongoClient."
This is why the whole answer here is based on NOT using that method as you are meant to creating new connection instances, or using the "sessions" functionality available from MongoDB 3.6 instead.