The Problem
I'm getting started with MongoDB on Python, I have a Ubuntu machine in my local network and MongoDB is installed there. When I try to connect with database using Python from Mac it throughs me an error. I searched about it and found out there is a .service called mongod.service that needs to be started along with mongodb.service. But when I try to start the mongod.service the it says the .service doesn't even exist. I tried both with IP and mongodb url, nothing works.
Ubuntu Terminal
$ sudo service mongod start
$ Failed to start mongod.service: Unit mongod.service not found.
$ sudo systemctl start mongod
$ Failed to start mongod.service: Unit mongod.service not found.
DataBase Link (a)
mongodb://user:password#192.168.0.106/database
Python Script (a)
#!/usr/bin/env python3
from pymongo import MongoClient
client = MongoClient('mongodb://user:password#192.168.0.106/database')
db = client['database']
collection = db['collection']
json = dict(message='hello world', token=0)
collection.insert_one(json)
macOS Terminal (a)
pymongo.errors.ServerSelectionTimeoutError: 192.168.0.106:27017: [Errno 61] Connection refused, Timeout: 30s, Topology Description: <TopologyDescription id: 60e140982a43032aef0dd634, topology_type: Single, servers: [<ServerDescription ('192.168.0.106', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('192.168.0.106:27017: [Errno 61] Connection refused')>]>
DataBase Link (b)
mongodb+srv://user:password#cluster0.h9fmz.mongodb.net/database?retryWrites=true&w=majority
Python Script (b)
#!/usr/bin/env python3
from pymongo import MongoClient
client = MongoClient('mongodb+srv://user:password#cluster0.h9fmz.mongodb.net/database?retryWrites=true&w=majority')
db = client['database']
collection = db['collection']
json = dict(message='hello world', token=0)
collection.insert_one(json)
macOS Terminal (b)
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/pymongo/pool.py", line 1278, in _get_socket
sock_info = self.sockets.popleft()
IndexError: pop from an empty deque
During handling of the above exception, another exception occurred:
.....
.....
.....
pymongo.errors.OperationFailure: bad auth : Authentication failed., full error: {'ok': 0, 'errmsg': 'bad auth : Authentication failed.', 'code': 8000, 'codeName': 'AtlasError'}
Note That
I'm providing the correct username and password for the database.
I'm using a machine on my local network, which is not a live server.
I've also tried the following commands but they did not solve anything.
Ubuntu Terminal
$ mongod --auth --port 27017
$ mongod --port 27017
$ sudo rm /var/lib/mongodb/mongod.lock
$ sudo mongod --repair
For accessing mongodb from another machine in local network. You will need to check the following:
There is no firewall restriction in the server machine or client machine. In case there is a firewall, you will need to add rule exceptions to allow this port to be accessible. Both incoming and outgoing. (Ubuntu firewall)
You will have to add bindIp config to the mongodb config in server machine. Refer to docs here. You will need to add something like this:
net:
bindIp: 0.0.0.0
port: 27017
Make sure you are able to connect using this ip: 192.168.0.106(server in local network) from the server machine itself. This will make sure the server is listening in this ip.
$ Failed to start mongod.service: Unit mongod.service not found.
The solution for this error could be found here
The mongo atlas error might be due to the following reasons:
You will have to create an database user in order to connect to mongodb.
you can find it under the left panel -> Database access -> Add user
This will be because of a mismatch with username and password. In case you have any special characters in your password you will have to url encode them.
Related
I have a django api that streams data from an Active directory source and processes it. My connection looks something like this
from ldap3 import Server, Connection
server = Server(url, get_info=ALL)
conn = Connection(server, username, password, auto_bind=True)
I put this app on a container, the dockerfile is simple and looks like this
FROM python:3.9
EXPOSE 8002
# Install Dependencies
ADD requirements.txt .
RUN pip install -r requirements.txt
ADD . .
CMD ./server.sh
Server.sh is also fairly simple:
#!/usr/bin/env bash
aws s3 cp s3://some_creds .
python manage.py makemigrations && python manage.py migrate && python manage.py runserver 0.0.0.0:80
Now, on local, and on gitpod this connection has no issues. I go on to do searches on the conn without problems. However, when I deploy the same container on ecs via ecr I was running on local, I get this error:
<class 'ldap3.core.exceptions.LDAPSocketOpenError'>, LDAPSocketOpenError('socket connection error while opening: [Errno 110] Connection timed out'), ('xxx.xxx.xxx.xxx', xxx))])
This may be a side effect of accessing the api from ssl, but if that is the case, I simply cannot replicate it on locale.
This error occurs only on POST requests; any other request goes through as expected.
The problem was caused by the Active Directory being only accessible via vpn/proxy. The gitpod instance I was using was company provided so it also had network access to the AD.
This was solved by using a VPC that had default access to this network.
I have tried running the following:
import pymongo
from pymongo import MongoClient
client = MongoClient('mongodb://localhost:27017/')
db = client.test_database
collection = db.test_collection
collection.find_one()
but I get
ServerSelectionTimeoutError: localhost:27017: [Errno 111] Connection refused
Any suggestions on how to fix this? I am running this behind a corporate proxy, but have already set environment variables for that in .bashrc.
EDIT
If I run mongo from the terminal, I get
$ mongo
MongoDB shell version v4.0.13
connecting to: mongodb://127.0.0.1:27017/?gssapiServiceName=mongodb
2019-10-16T10:30:23.269+0100 E QUERY [js] Error: couldn't connect to server 127.0.0.1:27017, connection attempt failed: SocketException: Error connecting to 127.0.0.1:27017 :: caused by :: Connection refused :
connect#src/mongo/shell/mongo.js:344:17
#(connect):2:6
exception: connect failed
Its look like you didn't run any mongo instance on your machine
run the command mongod -f <config_file> and then test it again (this may needs a sudo permissions)
I have code which activates my local Postgres server if it is not currently on, but once this command is sent, then I am unable to re-run anything in my editor. VSCode simply tells me "Code is currently running!" and the Output indicates that it is waiting for the server to disconnect before actually completing the entire script.
I want to be able to connect to postgresql straight-away by using psycopg2 and avoiding having to handle starting / stopping the local server, just as I would be able to with the EnterpriseDB installer version of PostgreSQL. However, if I can start the server, query the database, and then go about my merry way, that would also solve my issue. I want to be able to work on this Python script and others without locking up VSCode.
My issue stems from having to find a work-around for installing PostgreSQL on Windows 10. The installer was leading to a false "COMSPEC" environment variable error, so I unpacked the binaries instead. Unfortunately, I think that there is some issue with the configuration, since I am not able to run a simple query like the one below, which means that Postgres doesn't automatically start when called with psycopg2 in Python :
import psycopg2
conn = psycopg2.connect(
user='postgres',
host='127.0.0.1',
port='5432',
database='postgres'
)
cursor = conn.cursor()
SQL = 'select * from dual'
records = cursor.fetchall()
for record in records:
print('dummy :', record[0],'\n')
cursor.close()
conn.close()
^^^ This will return the following error, which is fixed when I start the server with pg_ctl :
Traceback (most recent call last):
File "c:\Users\UserName\Desktop\Test.py", line 7, in <module>
database='postgres'
File "C:\Users\UserName\AppData\Local\Programs\Python\Python37\lib\site-packages\psycopg2\__init__.py", line 126, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: could not connect to server: Connection refused (0x0000274D/10061)
Is the server running on host "127.0.0.1" and accepting
TCP/IP connections on port 5432?
I have manually gone into my command prompt and run these :
pg_ctl -D "C:\Program Files\pgsql\data" stop
pg_ctl -D "C:\Program Files\pgsql\data" start
Ideally, I would be able to have this handled automatically, i.e. I can run a script and not need to shut off the server in order to re-run. Ideally, the server could get started in a background process which is separate from the script's process.
import os
import psycopg2
import subprocess
pg_ctl = r'C:\Program Files\pgsql\bin\pg_ctl.exe'
data_dir = r'C:\Program Files\pgsql\data'
def server_status(exe,data):
exe=exe
data=data
if (os.path.isfile(exe)) and (os.path.isdir(data)) :
proc = subprocess.Popen([exe,'-D',data,'status'],stdout = subprocess.PIPE)
server_status = proc.communicate()[0].rstrip().decode("utf-8")
elif (os.path.isfile(exe)) and not (os.path.isdir(data)) :
server_status = f'PostgreSQL data does not exist here : \n {data}'
elif not (os.path.isfile(exe)) and (os.path.isdir(data)) :
server_status = f'PostgreSQL Executable "pg_ctl.exe" does not exist here : \n {os.path.dirname(exe)}'
else :
server_status = 'Input parameters cannot be executed.\nPlease check where "pg_ctl.exe" and the database reside'
return server_status
def server_on(exe,data):
exe=exe
data=data
if server_status(exe,data) == 'pg_ctl: no server running':
try:
subprocess.check_call([exe,'-D',data,'start'])
return 'server started'
except (subprocess.CalledProcessError) as ex:
return f'Failed to invoke psql: {ex}'
elif server_status(exe,data) == 'server started':
return 'server started already'
print(server_status(pg_ctl,data_dir))
server_on(pg_ctl,data_dir)
print(server_status(pg_ctl,data_dir))
If the server is off, I get : 'server started' returned as the server_status. Then I cannot run anything until I manually shutdown the server. "Code is currently running!" is what is returned (by VSCode) once I try to edit the code and re-run immediately afterwards.
Install PostgreSQL with Binaries :
Download PostgrSQL Binaries
Unzip the downloaded file in the location that you want to have as your base directory for PostgreSQL
Open your CMD prompt, navigate to your "bin" e.g. "C:\Program Files\pgsql\bin"
Initialize the database : initdb [option...] [ --pgdata | -D ] directory
E.g. : initdb.exe -D ../data --username=postgres --auth=trust
^^^ This will create the directory "data" in the same directory as "bin" then create a username "postgres". Note, no password specified here. Only the directory is a required argument
Start the server : pg_ctl [option...] [ --pgdata | -D ] directory
E.g. pg_ctl.exe start -D ../data
^^^ This will start the server with what was initialized in the "\data" directory
Connect to "postgres" now that the server is up : psql --username=postgres
Execute : ALTER USER postgres WITH PASSWORD "my_password"
Execute : CREATE EXTENSION adminpack;
Connect to a database : psql DBNAME USERNAME
Switch databases : \c DBNAME
Exit : \q
Show all active connections in the CMD prompt : netstat -nat
edit "postgresql.conf" file as needed (within your "\data" directory) --> E.g. "listen_addresses = 'localhost'" and "port = 5432"
Register PostgreSQL as a service : pg_ctl register [-D datadir] [-N servicename] [-U username] [-P password] [-S a[uto] | d[emand] ] [-e source] [-W] [-t seconds] [-s] [-o options]
Links :
PostgreSQL Documentation
PostgreSQL 11 initdb.exe
PostgreSQL 11 pg_ctl.exe
PostgreSQL 11 start the server
Install PostgreSQL Binaries (Windows 10)
Install PostgreSQL Binaries and Register It as a Service
Enable Remote PostgreSQL Connection
Configure PostgreSQL to Allow Remote Connection
Allow Remote Connections
Accept TCPIP Connections
Configure PostgreSQL to Accept Local Connections Only
PostgreSQL Management on Windows
Starting PostgreSQL in Windows w/o Install
StackOverflow :
unix_socket_directories
PostgreSQL Database Service
How to use PostgreSQL in multi thread python program
How to run PostgreSQL as a service in windows?
Register and run PostgreSQL as Windows Service
PostgreSQL pg_ctl Register Service Error under Windows
How can I configure PostgreSQL to start automatically in Windows?
PostgreSQL isn't Listening on Port 5432 in Windows
PostgreSQL initialization on Linux
Update :
I have tried to register PostgreSQL as a service, but I do not have admin privileges. I believe this is the root of my problem, since I only get the error "pg_ctl: could not open service manager" when I try to execute :
pg_ctl.exe register -N postgres -D "C:\Program Files\pgsql\data"
I would either need to disable to firewall or have a batch file kick-off a command to start the PostgreSQL server on a separate thread to my Python scripts. Or I could just switch to Linux and literally none of this would be an issue :D
I am trying to run a simple connection to pymongo but it keeps returning that the connection was refused
Here is what I tried:
>>>from pymongo import Connection
>>>connection = Connection('localhost',27017)
here is what I get
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/pymongo-2.0.1_-py2.7-linux i686.egg/pymongo/connection.py", line 348, in __init__
self.__find_node()
File "/usr/local/lib/python2.7/dist-packages/pymongo-2.0.1_-py2.7-linux- i686.egg/pymongo/connection.py", line 627, in __find_node
raise AutoReconnect(', '.join(errors))
pymongo.errors.AutoReconnect: could not connect to localhost:27017: [Errno 111] Connection refused
How do I fix this?
Removing mongod.lock inside /var/lib/mongodb
sudo rm /var/lib/mongodb/mongod.lock
And then restarting the service should do it. For example, in my Ubuntu installation, restarting the server is something like this:
sudo service mongodb start
If you found this page because you use Docker and you face the connection problem,
try to use in your client initialization the docker container name of the mongodb instead of the localhost:27017 or 0.0.0.0:27017
Steps to fix:
write docker ps in console
find the name of container (it's in the last column of the command output called NAMES
MongoClient('mongodb://CONTAINER_NAME')
PROFIT.
Just try following commands in given order :
sudo rm /var/lib/mongodb/mongod.lock
sudo mongod --repair
sudo service mongodb start
sudo service mongodb status
That's it now you could see following as output of last command:
mongodb start/running, process 2796
For anyone who's having this problem on a remote server rather than the localhost, try enabling external interfaces:
Go to the configuration file (ex. /etc/mongodb.conf)
Find bind_ip=127.0.0.1
Comment out that line with a # at the front
Restart mongod
It looks like you might not be running the MongoDB server. One thing that frequently trips me up is that if the server was shut down uncleanly, it will refuse to start up again until you remove the mongod.lock file from the data directory.
Try following commands :
sudo service mongod start
sudo service mongod status
db.py
import pymongo
from pymongo import MongoClient
#mongo client is connected
client = MongoClient()
db = client['db']
Rather than deleting mongod.lock, I'd recommend running 'mongod --repair'. (I figure it's better to go in through the front door whenever possible. And there may be other things that this catches as well, AFAIK.)
None of the above answers worked for me, as I am using docker-compose so this worked for me:
docker run --rm --volumes-from my-mongo-server mongo unlink "/data/db/mongod.lock"
docker run --rm --volumes-from my-mongo-server mongo --repair
Replace my-mongo-server with your container name/id.
If you're trying to connect from a server (other than your localhost), try checking if your mongo installation is complete :
you should have:
* /var/lib/mongodb file
* /var/log/mongodb file
* mongodb.service as service ( check by starting the service sudo service mongodb start )
If any of those fails, try reinstalling mongo ( Failed to start mongod.service: Unit mongod.service not found )
This solved my problem.
Cheers
For the newer versions(4.x) of MongoDb, you can try:
sudo rm /var/lib/mongodb/mongod.lock
sudo systemctl daemon-reload
sudo systemctl start mongod
First make sure you have installed mongodb using sudo apt install mongodb
Thanks, but for me i just had to stop mongod and then restart it and it worked fine without having to remove anything. PS : pymongo 2.7.2
When I run the command:
fab -H localhost host_type
I receive the following error:
[localhost] Executing task 'host_type'
[localhost] run: uname -s
Fatal error: Low level socket error connecting to host localhost: Connection refused
Aborting.
Any thoughts as to why? Thanks.
Fabfile.py
from fabric.api import run
def host_type():
run('uname -s')
Configuration
Fabric 1.0a0 (installed from the most recent Github commit---b8e1b6a)
Paramiko 1.7.4
PyCrypto 2.0.1
Virtualenv ver 1.3.3
Python 2.6.2+ (release26-maint:74924, Sep 18 2009, 16:03:18)
Mac OS X 10.6.1
The important part isn't the "low level error" part of the message - the important part is the "Connection refused" part. You'll get a "connection refused" message when trying to connect to a closed port.
The most likely scenario is that you are not running an ssh server on your machine at the time that Fabric is running. If you do
ssh localhost
you'll probably get a message similar to
ssh: connect to host localhost: Connection refused
So you'll have to go out and set up an SSH server on your computer before you can proceed with Fabric from there.
I had the same problem, but the reason was different: While I could easily log in to the server via SSH (default port 22), fabric tried to connect on a closed port 9090.
Finally I recognized that I had defined "env.port=9090" in my old fabfile for some WSGI server setup; while that was never a problem, I updated my Python installation some weeks before, and fabric now uses env.port for its SSH connection.
I just renamed that config, and all is well again.
This can also happen in OS X 10.11.4 and Fabric 1.10.1, in the case where you are ssh'ing to a VM using Vagrant, which does port forwarding from localhost. In this case, localhost was resolving to the IPv6 ::1 address (not due to /etc/hosts file), and giving this error.
The fix was to force use of IPv4 by using the 127.0.0.1 address in the fabfile instead of the hostname. Using a hostname in /etc/hosts with this address didn't work.
You might also want to try these useful tips for debugging connection issues in Fabric.
env.roledefs = {
'role1': env.hosts[0:5],
'role2':[env.hosts[5],]
}
I encountered the same error if "role" value IS NOT A LIST. for example, the above code works but the following doesn't.
env.roledefs = {
'role1': env.hosts[0:5],
'role2':env.hosts[5],
}