In a mongo command line I can run
db.my_collection.stats()
I need to get my collections stats from Python so I tried
from pymongo import MongoClient
client = MongoClient()
db = client.test_database
collection = db.test_collection
collection.stats()
But I get
TypeError: 'Collection' object is not callable.
If you meant to call the 'stats' method on a 'Collection' object it is failing because no such method exists.
This is because pymongo does not support this method. How do I send raw mongoDB commands to mongo through Python?
from pymongo import MongoClient
client = MongoClient()
db = client.test_database
print(db.command("collstats", "test_collection"))
Approach 1 with PyMongo:
client = pymongo.MongoClient(host = "127.0.0.1", port = 27017)
db = client.test_database
db.command("dbstats") # prints database stats for "test_db"
db.command("collstats", "test_collection") # prints collection-level stats
This can be done with this approach in Django.
from django.db import connections
database_wrapper = connections['my_db_alias']
eggs_collection = database_wrapper.get_collection('eggs')
eggs_collection.find_and_modify(...)
From django-mongodb-engine docs:
django.db.connections is a dictionary-like object that holds all
database connections – that is, for MongoDB databases,
django_mongodb_engine.base.DatabaseWrapper instances.
These instances can be used to get the PyMongo-level Connection,
Database and Collection objects.
Related
I'm trying to copy my db for testing purposes. I'm using the docs for that, but it still fails:
from pymongo import MongoClient
client = MongoClient(username='root', password='pass')
client.admin.command('copydb', fromdb='src', todb='dst')
OperationFailure: no such command: 'copydb', full error: {'ok': 0.0, 'errmsg': "no such command: 'copydb'", 'code': 59, 'codeName': 'CommandNotFound'}
When trying another command, from another section of the doc, it worked:
from pymongo import MongoClient
client = MongoClient(username='root', password='pass')
db = client.src
db.admin.command('copydb', fromdb='src', todb='dst')
The objects used here are different, but this is what the docs say... I still tried using the db object for the copydb - and failed again:
from pymongo import MongoClient
client = MongoClient(username='root', password='pass')
db = client.src
db.admin.command('buildinfo')
TypeError: 'Collection' object is not callable. If you meant to call the 'command' method on a 'Collection' object it is failing because no such method exists.
which means I use a bad object (makes sense, but I don't get how to make it work)
pymongo.version: '3.11.0'
mongodb version: 4.4.1 (running on docker)
Using the help from #AnuragWagh and this nice answer I managed to have my own script to handle dump & restore:
from bson.json_util import dumps, loads
import os
import pymongo
def backup_db(client, db_name, backup_dir):
if not os.path.isdir(backup_dir):
os.makedirs(backup_dir)
database = client[db_name]
collections = database.list_collection_names()
for i, collection_name in enumerate(collections):
col = getattr(database,collections[i])
collection = col.find()
jsonpath = collection_name + '.json'
jsonpath = os.path.join(backup_dir, jsonpath)
with open(jsonpath, 'wb') as jsonfile:
jsonfile.write(dumps(collection).encode())
backup_dir = './bckp'
client = pymongo.MongoClient(host='localhost', username='root', password='pass')
src_db_name = 'Jenkins'
dst_db_name = 'Test'
backup_db(client, src_db_name, backup_dir)
for backup_collection in filter(lambda path: path.endswith('.json'), os.listdir(backup_dir)):
collection_name = backup_collection.rstrip('.json')
with open(os.path.join(backup_dir, backup_collection), 'rb') as backup_file:
data = loads(backup_file.read())
client[dst_db_name][collection_name].insert_many(data)
This is not the ideal solution. If you can, use mongodump & mongorestore. For my case (pymongo on a host without mongodb installed, this seems like the best solution)
I would like to verify the SSL connection that SQLAlchemy sets up when using create_engine to connect to a PostgreSQL database. For example, if I have the following Python 3 code:
from sqlalchemy import create_engine
conn_string = "postgresql+psycopg2://myuser:******#someserver:5432/somedb"
conn_args = {
"sslmode": "verify-full",
"sslrootcert": "/etc/ssl/certs/ca-certificates.crt",
}
engine = create_engine(conn_string, connect_args=conn_args)
I know that I can print the contents of engine.__dict__, but it doesn't contain any information about the SSL settings (TLS version, cipher suite, etc) that it's using to connect:
{
'_echo': False,
'dialect': <sqlalchemy.dialects.postgresql.psycopg2.PGDialect_psycopg2 object at 0x7f988a217978>,
'dispatch': <sqlalchemy.event.base.ConnectionEventsDispatch object at 0x7f988938e788>,
'engine': Engine(postgresql+psycopg2://myuser:******#someserver:5432/somedb),
'logger': <Logger sqlalchemy.engine.base.Engine (DEBUG)>,
'pool': <sqlalchemy.pool.impl.QueuePool object at 0x7f988a238c50>,
'url': postgresql+psycopg2://myuser:******#someserver:5432/somedb
}
I know I can do something like SELECT * FROM pg_stat_ssl;, but does the SQLAlchemy engine store this kind of information as a class attribute / method?
Thank you!
I don't use postgres so hopefully this holds true for you.
SQLAlchemy takes the info that you provide in the url and passes it down to the underlying dbapi library that is also specified in the url, in your case it's psycopg2.
Your engine instance only connects to the database when needed, and sqlalchemy just passes the connection info along to the driver specified in the url which returns a connection that sqlalchemy uses.
Forgive that this is mysql, but should be fundamentally the same for you:
>>> engine
Engine(mysql+mysqlconnector://test:***#localhost/test)
>>> conn = engine.connect()
>>> conn
<sqlalchemy.engine.base.Connection object at 0x000001614ACBE2B0>
>>> conn.connection
<sqlalchemy.pool._ConnectionFairy object at 0x000001614BF08630>
>>> conn.connection.connection
<mysql.connector.connection_cext.CMySQLConnection object at 0x000001614AB7E1D0>
Calling engine.connect() returns a sqlalchemy.engine.base.Connection instance that has a connection property for which the docstring says:
The underlying DB-API connection managed by this Connection.
However, you can see from above that it actually returns a sqlalchemy.pool._ConnectionFairy object which from it's docstring:
Proxies a DBAPI connection...
Here is the __init__() method of the connection fairy, and as you can see it has a connection attribute that is the actual underlying dbapi connection.
def __init__(self, dbapi_connection, connection_record, echo):
self.connection = dbapi_connection
self._connection_record = connection_record
self._echo = echo
As to what info is available on the dbapi connection object, it depends on the implementation of that particular driver. E.g psycopg2 connection objects have an info attribute:
A ConnectionInfo object exposing information about the native libpq
connection.
That info object has attributes such as ssl_in_use:
True if the connection uses SSL, False if not.
And ssl_attribute:
Returns SSL-related information about the connection.
So you don't have to dig too deep to get at the actual db connection to see what is really going on.
Also, if you want to ensure that all client connections are ssl, you can always force them to.
Here´s a quick and dirty of what SuperShoot spelled out in detail:
>>> from sqlalchemy import create_engine
>>> db_string = "postgresql+psycopg2://myuser:******#someserver:5432/somedb"
>>> db = create_engine(db_string)
>>> conn = db.connect()
>>> conn.connection.connection.info.ssl_in_use
Should return True if using SSL.
In case someone is looking for PostgreSQL and pg8000, see the pg8000 docs.
For SSL defaults, it is:
import sqlalchemy
sqlalchemy.create_engine(url, connect_args={'ssl_context':True})
I instantiate Mongo Client as below. It works fine. However I am trying to read the DB name (primer here) from the configuration. How do I do that?
from pymongo import MongoClient
client = MongoClient()
db = client.primer # want to read "primer" string from a variable
coll = db.dataset
You could do:
db_name = 'primer'
db = getattr(client, db_name)
if you are trying to connect to only one database you can specify the dbname while creating the db object itself
dbname = "primer"
db = MongoClient()[dbname]
This is my code so far:
from pymongo import MongoClient
client = MongoClient()
client = MongoClient('localhost', 27017)
db = client.local
collection = db.orderbook_update
orderbook = collection.find({
"lastUpdated": {"$lt": ts}
}).sort("position",pymongo.DESCENDING).limit(1)
print(orderbook)
When I do that, my print(orderbook) gives me that: <pymongo.cursor.Cursor object at 0x7ff7defef828>
How am I able to print my result in order to use it? My json file on my database has three main components: lastUpdated, asks, bids.
Thank you!
order = list(orderbook)
Note: Once you do do this cursor object will not be available
On the command line, this works:
$ mongo
> show dbs
mydatabase 1.0GB
However, this does not:
$ python
>>> import pymongo
>>> connection = pymongo.MongoClient()
>>> connection.mydatabase.find()
I read through docs here:
http://api.mongodb.org/python/current/tutorial.html
But do not understand how to either...
connect to an existing database (using pymongo)
query what databases exist in the mongodb connection.
Why can't I access my database?
Connect to an existing database
import pymongo
from pymongo import MongoClient
connection = MongoClient()
db = connection.mydatabase
List existing databases
import pymongo
from pymongo import MongoClient
connection = MongoClient()
# connection.database_names() # depreciated
connection.list_database_names()
The question implies user has a local MongoDB. However I found this question trying to connect to a remote MongoDB. I think the tutorial is worth mentioning (no other answer here mentioned how I can specify the host and the port)
The above code will connect on the default host and port. We can also specify the host and port explicitly, as follows:
client = MongoClient('localhost', 27017)
Or use the MongoDB URI format:
client = MongoClient('mongodb://localhost:27017/')
show dbs and find() are totally different commands as such you cannot compare the two.
connection.mydatabase.find()
Will actually do nothing because you cannot find() documents on database level. You are probably looking for:
cursor = connection.mydatabase.mycol.find()
I am no Python programmer but something like that and the foreach the cursor var to get your data.
As an added note you will want to replace mycol with the collection name that contains your documents.
As for querying for a list of databases you can do something like:
databases = connection.mydatabase.command({'listDatabases': 1});
As shown here: http://docs.mongodb.org/manual/reference/command/listDatabases/#listDatabases
However again I am no Python programmer but this should get you started.
On the python command line:
import pymongo
from pymongo import MongoClient
connection = MongoClient() ## connects by default to db at localhost:27017
connection.database_names() ## python binding equivalent to show dbs.
Although there doesn't seem to be a wealth of examples, it appears that the bindings are pretty complete within the Python Driver API Documentation.
database_names() is deprecated. One can use list_database_names() instead.
mongo_db_url will be something like "mongodb://localhost:27017/". 27017 is deafult port number, replace suitably.
from pymongo import MongoClient
client = MongoClient(<mongo_db_url>)
#or client = MongoClient('localhost', 27017)
client.list_database_names()