We Keep Coding

"stdin is not a tty" when populating Postgres database

Please help, I have this stdin is not a tty message when i run the command below in my terminal.
psql -U postgres kdc < kdc.psql
kdc is the database and kdc.psql is the psql file with commands to populate the database. I am in the directory that holds the psql file.

I am not sure what causes that message (it does not happen here), but you should be able to avoid it using the -f option:
psql -U postgres -d kdc -f kdc.psql

Postgresql psql: error: FATAL: Peer authentication failed for user "userrole"

I have installed PostgreSQL and created a user 'userrole' with superuser privileges. Also able to connect through python code.
import psycopg2
conn = psycopg2.connect(
database="postgres", user="userrole", password="userroot", host='127.0.0.1', port= '5432'
)
cursor = conn.cursor()
cursor.execute("select version()")
data = cursor.fetchone()
print("Connection established to: ",data)
conn.close()
This is my output:
Connection established to: ('PostgreSQL 12.6 (Ubuntu 12.6-0ubuntu0.20.04.1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0, 64-bit',)
Process finished with exit code 0
The issue I'm facing is through CLI is cannot connect to this user:
(venv) resh#project:~/PycharmProjects/utilities$ sudo su -l userrole
[sudo] password for resh:
su: user userrole does not exist
Sorry, I am new to ubuntu, now I have changed the command and still getting an issue :
(venv) resh#project:~/PycharmProjects/utilities$ sudo psql -U userrole
[sudo] password for resh:
psql: error: FATAL: Peer authentication failed for user "userrole"

Your latest psql attempt is trying to connect over the Unix socket, which is (apparently, based on error message) configured to user peer authentication.
To use the password, you need to connect over TCP, like your python is doing:
psql -U userrole -h 127.0.0.1
Also, the sudo is useless for password-based authentication, so I removed it

userrole is an SQL user. It is not a Linux user. The two user lists are completely separate.

I had the same problem and fixed it by using following command in Ubuntu:
psql -d database -U username -h 127.0.0.1

Open the pg_hba.conf file using any text editor. It is located at /etc/postgresql/[major.minor]/main/pg_hba.conf eg /etc/postgresql/12/main/pg_hba.conf location.
Change the following line
local all postgres peer
to
local all postgres md5

Python script does not recognize Mysql query

I am trying to receive the results of a query by using sshpass, an example below:
sshpass -p password ssh user#ip "mysql -u user -pdbpassword -h ip -P port database -e \"SELECT * FROM database.ViewName;\""
When I launch that command from my local machine, it works.
But when I do on my Python script, it doesnt:
import subprocess
import sys
from subprocess import check_output
command = 'sshpass -p password ssh user#ip "mysql -u user -pdbpassword -h ip -P port database -e \"SELECT * FROM database.ViewName;\""'
output = check_output(command, shell=True)
And it returns this error:
bash: -c: línea 0: EOF inesperado mientras se buscaba un `"' coincidente
bash: -c: línea 1: error sintáctico: no se esperaba el final del fichero
/bin/sh: 1: : Permission denied
Traceback (most recent call last):
File "ipcompare.py", line 8, in <module>
output2 = check_output(command2, shell=True)
File "/usr/lib/python2.7/subprocess.py", line 223, in check_output
raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command 'sshpass -p password ssh user#ip "mysql -u user -pdbpassword -h ip -P port database -e \"SELECT * FROM database.ViewName;\""' returned non-zero exit status 127
I have tried:
chmod 777 to my script, the error is still the same.
Changing doble quotes for single quotes.
Escaping all special characters.

Basing on this line:
/bin/sh: 1: : Permission denied
It appears that you don't have privileges to execute that shell command. You sure that when running it locally, you are not having some elevated privileges (run some command with 'sudo' previously etc.)?
It'd isolate it a bit on your place, eg. start with something simple:
command = 'sshpass -p password ssh user#ip "pwd"'
If it works, you could further try to delegate mysql query.
Also, you are running the command and script on same machine? Sometimes databases have limited set of IP addresses which can access it. Check if that's not the case for your db.

I did this to solve the problem:
import subprocess
import sys
from subprocess import check_output
command = 'sshpass -p password ssh user#ip "mysql -u user -pdbpassword -h ip -P port database -e \\\"SELECT * FROM database.ViewName;\\\""'
output = check_output(command, shell=True)
Escaping the escape character was the answer for this issue.

Python script not running : -bash: ./btltest.py: /usr/bin/python3: bad interpreter: No such file or directory

I have a python script that establishes connection to postgres db and writes results in csv.
When I run it from terminal on Mac, using command
administrators-MacBook-Pro:ml bob$ ./btltest.py -d dev -u temp_viewer -p "xxxxxxx" -l debug -s "2019-08-10" -e "2019-08-20"
it shows error
-bash: ./btltest.py: /usr/bin/python3: bad interpreter: No such file or directory
Using same credentials i.s. pass, user, db name as in the code to connect through pgadmin4 has no problem.
How can I remove this error ?

Cannot re-execute code until manually shutdown local PostgreSQL server

I have code which activates my local Postgres server if it is not currently on, but once this command is sent, then I am unable to re-run anything in my editor. VSCode simply tells me "Code is currently running!" and the Output indicates that it is waiting for the server to disconnect before actually completing the entire script.
I want to be able to connect to postgresql straight-away by using psycopg2 and avoiding having to handle starting / stopping the local server, just as I would be able to with the EnterpriseDB installer version of PostgreSQL. However, if I can start the server, query the database, and then go about my merry way, that would also solve my issue. I want to be able to work on this Python script and others without locking up VSCode.
My issue stems from having to find a work-around for installing PostgreSQL on Windows 10. The installer was leading to a false "COMSPEC" environment variable error, so I unpacked the binaries instead. Unfortunately, I think that there is some issue with the configuration, since I am not able to run a simple query like the one below, which means that Postgres doesn't automatically start when called with psycopg2 in Python :
import psycopg2
conn = psycopg2.connect(
user='postgres',
host='127.0.0.1',
port='5432',
database='postgres'
)
cursor = conn.cursor()
SQL = 'select * from dual'
records = cursor.fetchall()
for record in records:
print('dummy :', record[0],'\n')
cursor.close()
conn.close()
^^^ This will return the following error, which is fixed when I start the server with pg_ctl :
Traceback (most recent call last):
File "c:\Users\UserName\Desktop\Test.py", line 7, in <module>
database='postgres'
File "C:\Users\UserName\AppData\Local\Programs\Python\Python37\lib\site-packages\psycopg2\__init__.py", line 126, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: could not connect to server: Connection refused (0x0000274D/10061)
Is the server running on host "127.0.0.1" and accepting
TCP/IP connections on port 5432?
I have manually gone into my command prompt and run these :
pg_ctl -D "C:\Program Files\pgsql\data" stop
pg_ctl -D "C:\Program Files\pgsql\data" start
Ideally, I would be able to have this handled automatically, i.e. I can run a script and not need to shut off the server in order to re-run. Ideally, the server could get started in a background process which is separate from the script's process.
import os
import psycopg2
import subprocess
pg_ctl = r'C:\Program Files\pgsql\bin\pg_ctl.exe'
data_dir = r'C:\Program Files\pgsql\data'
def server_status(exe,data):
exe=exe
data=data
if (os.path.isfile(exe)) and (os.path.isdir(data)) :
proc = subprocess.Popen([exe,'-D',data,'status'],stdout = subprocess.PIPE)
server_status = proc.communicate()[0].rstrip().decode("utf-8")
elif (os.path.isfile(exe)) and not (os.path.isdir(data)) :
server_status = f'PostgreSQL data does not exist here : \n {data}'
elif not (os.path.isfile(exe)) and (os.path.isdir(data)) :
server_status = f'PostgreSQL Executable "pg_ctl.exe" does not exist here : \n {os.path.dirname(exe)}'
else :
server_status = 'Input parameters cannot be executed.\nPlease check where "pg_ctl.exe" and the database reside'
return server_status
def server_on(exe,data):
exe=exe
data=data
if server_status(exe,data) == 'pg_ctl: no server running':
try:
subprocess.check_call([exe,'-D',data,'start'])
return 'server started'
except (subprocess.CalledProcessError) as ex:
return f'Failed to invoke psql: {ex}'
elif server_status(exe,data) == 'server started':
return 'server started already'
print(server_status(pg_ctl,data_dir))
server_on(pg_ctl,data_dir)
print(server_status(pg_ctl,data_dir))
If the server is off, I get : 'server started' returned as the server_status. Then I cannot run anything until I manually shutdown the server. "Code is currently running!" is what is returned (by VSCode) once I try to edit the code and re-run immediately afterwards.
Install PostgreSQL with Binaries :
Download PostgrSQL Binaries
Unzip the downloaded file in the location that you want to have as your base directory for PostgreSQL
Open your CMD prompt, navigate to your "bin" e.g. "C:\Program Files\pgsql\bin"
Initialize the database : initdb [option...] [ --pgdata | -D ] directory
E.g. : initdb.exe -D ../data --username=postgres --auth=trust
^^^ This will create the directory "data" in the same directory as "bin" then create a username "postgres". Note, no password specified here. Only the directory is a required argument
Start the server : pg_ctl [option...] [ --pgdata | -D ] directory
E.g. pg_ctl.exe start -D ../data
^^^ This will start the server with what was initialized in the "\data" directory
Connect to "postgres" now that the server is up : psql --username=postgres
Execute : ALTER USER postgres WITH PASSWORD "my_password"
Execute : CREATE EXTENSION adminpack;
Connect to a database : psql DBNAME USERNAME
Switch databases : \c DBNAME
Exit : \q
Show all active connections in the CMD prompt : netstat -nat
edit "postgresql.conf" file as needed (within your "\data" directory) --> E.g. "listen_addresses = 'localhost'" and "port = 5432"
Register PostgreSQL as a service : pg_ctl register [-D datadir] [-N servicename] [-U username] [-P password] [-S a[uto] | d[emand] ] [-e source] [-W] [-t seconds] [-s] [-o options]
Links :
PostgreSQL Documentation
PostgreSQL 11 initdb.exe
PostgreSQL 11 pg_ctl.exe
PostgreSQL 11 start the server
Install PostgreSQL Binaries (Windows 10)
Install PostgreSQL Binaries and Register It as a Service
Enable Remote PostgreSQL Connection
Configure PostgreSQL to Allow Remote Connection
Allow Remote Connections
Accept TCPIP Connections
Configure PostgreSQL to Accept Local Connections Only
PostgreSQL Management on Windows
Starting PostgreSQL in Windows w/o Install
StackOverflow :
unix_socket_directories
PostgreSQL Database Service
How to use PostgreSQL in multi thread python program
How to run PostgreSQL as a service in windows?
Register and run PostgreSQL as Windows Service
PostgreSQL pg_ctl Register Service Error under Windows
How can I configure PostgreSQL to start automatically in Windows?
PostgreSQL isn't Listening on Port 5432 in Windows
PostgreSQL initialization on Linux
Update :
I have tried to register PostgreSQL as a service, but I do not have admin privileges. I believe this is the root of my problem, since I only get the error "pg_ctl: could not open service manager" when I try to execute :
pg_ctl.exe register -N postgres -D "C:\Program Files\pgsql\data"
I would either need to disable to firewall or have a batch file kick-off a command to start the PostgreSQL server on a separate thread to my Python scripts. Or I could just switch to Linux and literally none of this would be an issue :D