We Keep Coding

Mongod.service not found => Pop from an empty deque => Authentication failed

The Problem
I'm getting started with MongoDB on Python, I have a Ubuntu machine in my local network and MongoDB is installed there. When I try to connect with database using Python from Mac it throughs me an error. I searched about it and found out there is a .service called mongod.service that needs to be started along with mongodb.service. But when I try to start the mongod.service the it says the .service doesn't even exist. I tried both with IP and mongodb url, nothing works.
Ubuntu Terminal
$ sudo service mongod start
$ Failed to start mongod.service: Unit mongod.service not found.
$ sudo systemctl start mongod
$ Failed to start mongod.service: Unit mongod.service not found.
DataBase Link (a)
mongodb://user:password#192.168.0.106/database
Python Script (a)
#!/usr/bin/env python3
from pymongo import MongoClient
client = MongoClient('mongodb://user:password#192.168.0.106/database')
db = client['database']
collection = db['collection']
json = dict(message='hello world', token=0)
collection.insert_one(json)
macOS Terminal (a)
pymongo.errors.ServerSelectionTimeoutError: 192.168.0.106:27017: [Errno 61] Connection refused, Timeout: 30s, Topology Description: <TopologyDescription id: 60e140982a43032aef0dd634, topology_type: Single, servers: [<ServerDescription ('192.168.0.106', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('192.168.0.106:27017: [Errno 61] Connection refused')>]>
DataBase Link (b)
mongodb+srv://user:password#cluster0.h9fmz.mongodb.net/database?retryWrites=true&w=majority
Python Script (b)
#!/usr/bin/env python3
from pymongo import MongoClient
client = MongoClient('mongodb+srv://user:password#cluster0.h9fmz.mongodb.net/database?retryWrites=true&w=majority')
db = client['database']
collection = db['collection']
json = dict(message='hello world', token=0)
collection.insert_one(json)
macOS Terminal (b)
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/pymongo/pool.py", line 1278, in _get_socket
sock_info = self.sockets.popleft()
IndexError: pop from an empty deque
During handling of the above exception, another exception occurred:
.....
.....
.....
pymongo.errors.OperationFailure: bad auth : Authentication failed., full error: {'ok': 0, 'errmsg': 'bad auth : Authentication failed.', 'code': 8000, 'codeName': 'AtlasError'}
Note That
I'm providing the correct username and password for the database.
I'm using a machine on my local network, which is not a live server.
I've also tried the following commands but they did not solve anything.
Ubuntu Terminal
$ mongod --auth --port 27017
$ mongod --port 27017
$ sudo rm /var/lib/mongodb/mongod.lock
$ sudo mongod --repair

For accessing mongodb from another machine in local network. You will need to check the following:
There is no firewall restriction in the server machine or client machine. In case there is a firewall, you will need to add rule exceptions to allow this port to be accessible. Both incoming and outgoing. (Ubuntu firewall)
You will have to add bindIp config to the mongodb config in server machine. Refer to docs here. You will need to add something like this:
net:
bindIp: 0.0.0.0
port: 27017
Make sure you are able to connect using this ip: 192.168.0.106(server in local network) from the server machine itself. This will make sure the server is listening in this ip.
$ Failed to start mongod.service: Unit mongod.service not found.
The solution for this error could be found here
The mongo atlas error might be due to the following reasons:
You will have to create an database user in order to connect to mongodb.
you can find it under the left panel -> Database access -> Add user
This will be because of a mismatch with username and password. In case you have any special characters in your password you will have to url encode them.

Cannot re-execute code until manually shutdown local PostgreSQL server

I have code which activates my local Postgres server if it is not currently on, but once this command is sent, then I am unable to re-run anything in my editor. VSCode simply tells me "Code is currently running!" and the Output indicates that it is waiting for the server to disconnect before actually completing the entire script.
I want to be able to connect to postgresql straight-away by using psycopg2 and avoiding having to handle starting / stopping the local server, just as I would be able to with the EnterpriseDB installer version of PostgreSQL. However, if I can start the server, query the database, and then go about my merry way, that would also solve my issue. I want to be able to work on this Python script and others without locking up VSCode.
My issue stems from having to find a work-around for installing PostgreSQL on Windows 10. The installer was leading to a false "COMSPEC" environment variable error, so I unpacked the binaries instead. Unfortunately, I think that there is some issue with the configuration, since I am not able to run a simple query like the one below, which means that Postgres doesn't automatically start when called with psycopg2 in Python :
import psycopg2
conn = psycopg2.connect(
user='postgres',
host='127.0.0.1',
port='5432',
database='postgres'
)
cursor = conn.cursor()
SQL = 'select * from dual'
records = cursor.fetchall()
for record in records:
print('dummy :', record[0],'\n')
cursor.close()
conn.close()
^^^ This will return the following error, which is fixed when I start the server with pg_ctl :
Traceback (most recent call last):
File "c:\Users\UserName\Desktop\Test.py", line 7, in <module>
database='postgres'
File "C:\Users\UserName\AppData\Local\Programs\Python\Python37\lib\site-packages\psycopg2\__init__.py", line 126, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: could not connect to server: Connection refused (0x0000274D/10061)
Is the server running on host "127.0.0.1" and accepting
TCP/IP connections on port 5432?
I have manually gone into my command prompt and run these :
pg_ctl -D "C:\Program Files\pgsql\data" stop
pg_ctl -D "C:\Program Files\pgsql\data" start
Ideally, I would be able to have this handled automatically, i.e. I can run a script and not need to shut off the server in order to re-run. Ideally, the server could get started in a background process which is separate from the script's process.
import os
import psycopg2
import subprocess
pg_ctl = r'C:\Program Files\pgsql\bin\pg_ctl.exe'
data_dir = r'C:\Program Files\pgsql\data'
def server_status(exe,data):
exe=exe
data=data
if (os.path.isfile(exe)) and (os.path.isdir(data)) :
proc = subprocess.Popen([exe,'-D',data,'status'],stdout = subprocess.PIPE)
server_status = proc.communicate()[0].rstrip().decode("utf-8")
elif (os.path.isfile(exe)) and not (os.path.isdir(data)) :
server_status = f'PostgreSQL data does not exist here : \n {data}'
elif not (os.path.isfile(exe)) and (os.path.isdir(data)) :
server_status = f'PostgreSQL Executable "pg_ctl.exe" does not exist here : \n {os.path.dirname(exe)}'
else :
server_status = 'Input parameters cannot be executed.\nPlease check where "pg_ctl.exe" and the database reside'
return server_status
def server_on(exe,data):
exe=exe
data=data
if server_status(exe,data) == 'pg_ctl: no server running':
try:
subprocess.check_call([exe,'-D',data,'start'])
return 'server started'
except (subprocess.CalledProcessError) as ex:
return f'Failed to invoke psql: {ex}'
elif server_status(exe,data) == 'server started':
return 'server started already'
print(server_status(pg_ctl,data_dir))
server_on(pg_ctl,data_dir)
print(server_status(pg_ctl,data_dir))
If the server is off, I get : 'server started' returned as the server_status. Then I cannot run anything until I manually shutdown the server. "Code is currently running!" is what is returned (by VSCode) once I try to edit the code and re-run immediately afterwards.
Install PostgreSQL with Binaries :
Download PostgrSQL Binaries
Unzip the downloaded file in the location that you want to have as your base directory for PostgreSQL
Open your CMD prompt, navigate to your "bin" e.g. "C:\Program Files\pgsql\bin"
Initialize the database : initdb [option...] [ --pgdata | -D ] directory
E.g. : initdb.exe -D ../data --username=postgres --auth=trust
^^^ This will create the directory "data" in the same directory as "bin" then create a username "postgres". Note, no password specified here. Only the directory is a required argument
Start the server : pg_ctl [option...] [ --pgdata | -D ] directory
E.g. pg_ctl.exe start -D ../data
^^^ This will start the server with what was initialized in the "\data" directory
Connect to "postgres" now that the server is up : psql --username=postgres
Execute : ALTER USER postgres WITH PASSWORD "my_password"
Execute : CREATE EXTENSION adminpack;
Connect to a database : psql DBNAME USERNAME
Switch databases : \c DBNAME
Exit : \q
Show all active connections in the CMD prompt : netstat -nat
edit "postgresql.conf" file as needed (within your "\data" directory) --> E.g. "listen_addresses = 'localhost'" and "port = 5432"
Register PostgreSQL as a service : pg_ctl register [-D datadir] [-N servicename] [-U username] [-P password] [-S a[uto] | d[emand] ] [-e source] [-W] [-t seconds] [-s] [-o options]
Links :
PostgreSQL Documentation
PostgreSQL 11 initdb.exe
PostgreSQL 11 pg_ctl.exe
PostgreSQL 11 start the server
Install PostgreSQL Binaries (Windows 10)
Install PostgreSQL Binaries and Register It as a Service
Enable Remote PostgreSQL Connection
Configure PostgreSQL to Allow Remote Connection
Allow Remote Connections
Accept TCPIP Connections
Configure PostgreSQL to Accept Local Connections Only
PostgreSQL Management on Windows
Starting PostgreSQL in Windows w/o Install
StackOverflow :
unix_socket_directories
PostgreSQL Database Service
How to use PostgreSQL in multi thread python program
How to run PostgreSQL as a service in windows?
Register and run PostgreSQL as Windows Service
PostgreSQL pg_ctl Register Service Error under Windows
How can I configure PostgreSQL to start automatically in Windows?
PostgreSQL isn't Listening on Port 5432 in Windows
PostgreSQL initialization on Linux
Update :
I have tried to register PostgreSQL as a service, but I do not have admin privileges. I believe this is the root of my problem, since I only get the error "pg_ctl: could not open service manager" when I try to execute :
pg_ctl.exe register -N postgres -D "C:\Program Files\pgsql\data"
I would either need to disable to firewall or have a batch file kick-off a command to start the PostgreSQL server on a separate thread to my Python scripts. Or I could just switch to Linux and literally none of this would be an issue :D

salt-ssh : Error appear when trying to cpmile a sls command

this message error appear when i trying to run test.ping command :
_error:
Failed to return clean data
retcode:
10
stderr:
ERROR: Unable to locate appropriate python command
stdout:
ERROR: salt requires python 2.6 or newer on target hosts, must have same major version as origin host
knowing that the version of python installed in both hosts (master and minion ) is the same

Jenkins error: svn: E170013: Unable to connect to a repository at URL............ svn: E215004: No more credentials or we tried too many times

I am tying to run python code given below:
import re
import subprocess
import os
repoBaseURL = 'https://www.collabnet.xxx.com/xxxx/xxx'
svnPath = 'C:\\Program Files\\TortoiseSVN\\bin\\svn.exe'
REPO_SRC = ('trunk')
REL_NUM = ('9999')
if not REL_NUM:
raise Exception('REL_NUM should be filled in.' )
p = subprocess.Popen(['C:\\Program Files\\TortoiseSVN\\bin\\svn.exe', 'list', repoBaseURL + 'branches'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate()
if not out:
print err
exit(-1)
The script is working fine standalone but while running it using
jenkins I am getting error:
svn: E170013: Unable to connect to a repository at URL..............................................................
svn: E215004: No more credentials or we tried too many times.
Authenticaion Failed
Any suggestions ?
Thanks For your help

If your script is running in standalone, then you have to check your Jenkins configuration, generally Jenkins will use Jenkins stored credentials to connect to SVN repository, if you are not providing any credentials explicitly.
Hope you have already installed SVN plugin in Jenkins, open the Jenkins job configuration, and there you have to provide your credentials to connect SVN, you can test from there it self. If connection is success the same you can try to implement with your scripting.

python, subprocess (call or popen) works in terminal but not in Gnome Schedule User Guide V2.2

I am trying to call wget from my python (2.75) script:
cmd = ['wget', 'https://192.168.0.1/003/', '--tries=5', '--http-user=foo', '--http-passwd=bar', '--auth-no-challenge', '--no-check-certificate', '--output-document=temp/page.html']
proc = subprocess.Popen(cmd)
# same with subprocess.call(cmd)
it works fine in the IDE (eclipse), also in a terminal, but when I try to run it from Gnome Schedule User Guide V2.2, I get the following:
--2013-08-16 11:27:35-- https://192.168.0.1/003/
Connecting to 192.168.0.1:443... connected.
ERROR: cannot verify 192.168.0.1's certificate, issued by `blabla': Self-signed certificate encountered.
ERROR: no certificate subject alternative name matches requested host name `192.168.0.1'.
To connect to 192.168.0.1 insecurely, use `--no-check-certificate'.
--2013-08-16 11:27:37-- http://--tries=5/
Resolving --tries=5 (--tries=5)... failed: Name or service not known.
wget: unable to resolve host address `--tries=5'
--2013-08-16 11:27:37-- http://--http-user=foo/
Resolving --http-user=foo (--http-user=foo)... failed: Name or service not known.
wget: unable to resolve host address `--http-user=foo'
--2013-08-16 11:27:37-- http://--http-passwd=bar/
Resolving --http-passwd=bar (--http-passwd=bar)... failed: Name or service not known.
wget: unable to resolve host address `--http-passwd=bar'
--2013-08-16 11:27:37-- http://--auth-no-challenge/
Resolving --auth-no-challenge (--auth-no-challenge)... failed: Name or service not known.
wget: unable to resolve host address `--auth-no-challenge'
--2013-08-16 11:27:37-- http://--no-check-certificate/
Resolving --no-check-certificate (--no-check-certificate)... failed: Name or service not known.
wget: unable to resolve host address `--no-check-certificate'
--2013-08-16 11:27:37-- http://--output-document=temp/page.html
Resolving --output-document=temp (--output-document=temp)... failed: Name or service not known.
wget: unable to resolve host address `--output-document=temp'
...so instead of taking the command "wget arg1 arg2 arg3...", it seems to try to run:
"wget arg1"
"wget arg2"
"wget arg3"
...
any clue of what could make a different result when run from the scheduler?
the command I use in the scheduler is:
"python /home/python/Download/Download.py"
Thanks

I solved it myself, by moving the second argument in the last position:
cmd = ['wget', '--tries=5', '--http-user=foo', '--http-passwd=bar', '--auth-no-challenge', '--no-check-certificate', '--output-document=temp/page.html', 'https://192.168.0.1/003/']
It works now, but I still wonder why in terminal or in the IDE it worked fine and not when run from the scheduler...