Cannot debug a PsyNet experiment because a failure to connect to Postgres - python

I cannot debug a PsyNet experiment locally because I see the following error in the terminal (I'm using PsyNet v5.0.2 with Dallinger 7.7):
Exception: Failed to connect to Postgres at postgresql://dallinger:dallinger#localhost/dallinger. Is Postgres running on port 5432?
I tried restarging postgres but did not fix it.
Unfortunately I am unable to share the full experiment code here due to privacy constraints, but I'm hoping someone has familiarity with the error message nonetheless.

I found a solution to the problem above. Sometimes, PostgresSQL crashes and you may get an error like the one above or also like this:
❯❯ There was a problem connecting to the Postgres database!
This problem can be due to several reasons but I found the following procedure to fix this issue robustly:
You can check if PostgreSQL is running normally with postgres -D /usr/local/var/postgres; if the above didn't work, this will return an error.
If there is an error, your db has been corrupted due to the crash. Try the following:
rm /usr/local/var/postgres/postmaster.pid
If this does not work or you cannot find the file (postmaster.pid). Then try the following steps in the same order:
Stop postgress services: brew services stop postgresql
Uninstall postgress in your computer completely: and delete all postgress databases and postgress folders in your computer:
brew uninstall postgresql
rm -rf /usr/local/var/postgres
rm -rf .psql_history .psqlrc .psql.local .pgpass .psqlrc.local
Install postgress again: brew install postgresql
Start postgress: brew services start postgresql
Install db as indicated in the dallinger documentation for creating a new user database (as your computer crashing can cause it to be deleted):
createuser -P dallinger --createdb
(Password: dallinger)
createdb -O dallinger dallinger
createdb -O dallinger dallinger-import
This should solve the problem.

Related

how to get out of "caching_sha2_password" error? [duplicate]

I am connecting MySQL - 8.0 with MySQL Workbench and getting the below error:
Authentication plugin 'caching_sha2_password' cannot be loaded:
dlopen(/usr/local/mysql/lib/plugin/caching_sha2_password.so, 2): image
not found
I have tried with other client tool as well.
Any solution for this?
you can change the encryption of the password like this.
ALTER USER 'yourusername'#'localhost' IDENTIFIED WITH mysql_native_password BY 'youpassword';
Note: For MAC OS
Open MySQL from System Preferences > Initialize Database >
Type your new password.
Choose 'Use legacy password'
Start the Server again.
Now connect the MySQL Workbench
For Windows 10:
Open the command prompt:
cd "C:\Program Files\MySQL\MySQL Server 8.0\bin"
C:\Program Files\MySQL\MySQL Server 8.0\bin> mysql -u root -p
Enter password: *********
mysql> ALTER USER 'root'#'localhost' IDENTIFIED WITH mysql_native_password BY 'newrootpassword';
Query OK, 0 rows affected (0.10 sec)
mysql> exit
Alternatively, you can change the my.ini configuration as the following:
[mysqld]
default_authentication_plugin=mysql_native_password
Restart the MySQL Server and open the Workbench again.
I had the same problem, but the answer by Aman Aggarwal didn't work for me with a Docker container running mysql 8.X.
I loged in the container
docker exec -it CONTAINER_ID bash
then log into mysql as root
mysql --user=root --password
Enter the password for root (Default is 'root')
Finally Run:
ALTER USER 'username' IDENTIFIED WITH mysql_native_password BY 'password';
You're all set.
You can change the encryption of the user's password by altering the user with below Alter command :
ALTER USER 'username'#'ip_address' IDENTIFIED WITH mysql_native_password BY
'password';
OR
We can avoid this error by make it work with old password plugin:
First change the authentication plugin in my.cnf file for Linux / my.ini file in Windows:
[mysqld]
default_authentication_plugin=mysql_native_password
Restart the mysql server to take the changes in affect and try connecting via MySQL with any mysql client.
If still unable to connect and getting the below error:
Unable to load plugin 'caching_sha2_password'
It means your user needs the above plugin. So try creating new user with create user or grant command after changing default plugin. then new user need the native plugin and you will able to connect MySQL.
Thanks
Currently (on 2018/04/23), you need to download a development release. The GA ones do not work.
I was not able to connect with the latest GA version (6.3.10).
It worked with mysql-workbench-community-8.0.11-rc-winx64.msi (from https://dev.mysql.com/downloads/workbench/, tab Development Releases).
Ok, wasted a lot of time on this so here is a summary as of 19 March 2019
If you are specifically trying to use a Docker image with MySql 8+, and then use SequelPro to access your database(s) running on that docker container, you are out of luck.
See the sequelpro issue 2699
My setup is sequelpro 1.1.2 using docker desktop 2.0.3.0 (mac - mojave), and tried using mysql:latest (v8.0.15).
As others have reported, using mysql 5.7 works with nothing required:
docker run -p 3306:3306 --name mysql1 -e MYSQL_ROOT_PASSWORD=secret -d mysql:5.7
Of course, it is possible to use MySql 8+ on docker, and in that situation (if needed), other answers provided here for caching_sha2_password type issues do work. But sequelpro is a NO GO with MySql 8+
Finally, I abandoned sequelpro (a trusted friend from back in 2013-2014) and instead installed DBeaver. Everything worked out of the box. For docker, I used:
docker run -p 3306:3306 --name mysql1 -e MYSQL_ROOT_PASSWORD=secret -d mysql:latest --default-authentication-plugin=mysql_native_password
You can quickly peek at the mysql databases using:
docker exec -it mysql1 bash
mysql -u root -p
show databases;
I was installing MySQL on my Windows 10 PC using "MySQL Web Installer" and was facing the same issue while trying to connect using MySQL workbench. I fixed the issue by reconfiguring the server form the Installer window.
Clicking on the "Reconfigure" option it will allow to reconfigure the server. Click on "Next" until you reach "Authentication Method".
Once on this tab, use the second option "Use Legacy Authentication Method (Retain MySQL 5.x Compatibility)".
Keep everything else as is and that is how I solved my issue.
Note: For Linux (Debian, Ubuntu, Mint)
I got this error:
MySQL Error Message: Plugin caching_sha2_password could not be loaded: /usr/lib/x86_64-linux-gnu/mariadb19/plugin/caching_sha2_password.so: cannot open shared object file: No such file or directory
I solved it with these steps:
Enter on mysql console: $ mysql -u root -p, if you don't have a password for root user, then:
Use mysql db: mysql> use mysql;
Alter your user for solve the problem: mysql> ALTER USER 'root'#'localhost' IDENTIFIED WITH mysql_native_password BY 'password';
Exit... mysql> quit;
Done!
like this?
docker run -p 3306:3306 -e MYSQL_ALLOW_EMPTY_PASSWORD=yes -d mysql --default-authentication-plugin=mysql_native_password
mysql -uroot --protocol tcp
Try in PWD
https://github.com/GitHub30/docs/blob/change-default_authentication_plugin/mysql/stack.yml
or You shoud use MySQL Workbench 8.0.11.
Open MySQL Command Line Client
Create a new user with a new pass
Considering an example of a path to a bin folder on top, here's the code you need to run in the command prompt, line by line:
cd C:\Program Files\MySQL\MySQL Server 5.7\bin
MySQL -u root -p
current password...***
CREATE USER 'nativeuser'#'localhost'
IDENTIFIED WITH mysql_native_password BY 'new_password';
Then, you can access Workbench again (you should be able to do that after creating a new localhost connection and using the new credentials to start using the program).
Set up a new local host connection with the user name mentioned above (native user), login using the password (new_password)
Courtesy: UDEMY FAQs answered by Career365 Team
For Windows 10,
Modify my.ini file in C:\ProgramData\MySQL\MySQL Server 8.0\
[mysqld]
default_authentication_plugin=mysql_native_password
Restart the MySQL Service.
Login to MySQL on the command line, and execute the following commands in MySQL:
Create a new user.
CREATE USER 'user'#'localhost' IDENTIFIED BY 'password';
Grant all privileges.
GRANT ALL PRIVILEGES ON * .* TO 'user'#'localhost';
Open MySQL workbench, and open a new connection using the new user credentials.
I was facing the same issue and this worked.
Although this shouldn't be a real
solution, it does work locally if you are stuck
ALTER USER 'root'#'localhost' IDENTIFIED WITH mysql_native_password BY '';
This is my databdase definition in my docker-compose:
dataBase:
image: mysql:8.0
volumes:
- db_data:/var/lib/mysql
networks:
z-net:
ipv4_address: 172.26.0.2
restart: always
entrypoint: ['docker-entrypoint.sh', '--default-authentication-plugin=mysql_native_password']
environment:
MYSQL_ROOT_PASSWORD: supersecret
MYSQL_DATABASE: zdb
MYSQL_USER: zuser
MYSQL_PASSWORD: zpass
ports:
- "3333:3306"
The relevant line there is entrypoint.
After build and up it, you can test it with:
$ mysql -u zuser -pzpass --host=172.26.0.2 zdb -e "select 1;"
Warning: Using a password on the command line interface can be insecure.
+---+
| 1 |
+---+
| 1 |
+---+
For those using Docker or Docker Compose, I experienced this error because I didn't set my MySQL image version. Docker will automatically attempt to get the latest version which is 8.
I set MySQL to 5.7 and rebuilt the image and it worked as normal:
version: '2'
services:
db:
image: mysql:5.7
I found that
ALTER USER 'username'#'ip_address' IDENTIFIED WITH mysql_native_password BY 'password';
didn't work by itself. I also needed to set
[mysqld]
default_authentication_plugin=mysql_native_password
in /etc/mysql/mysql.conf.d/mysqld.cnf
on Ubuntu 18.04 running PHP 7.0
Here is the solution which worked for me after MySQL 8.0 Installation on Windows 10.
Suppose MySQL username is root and password is admin
Open command prompt and enter the following commands:
cd C:\Program Files\MySQL\MySQL Server 8.0\bin
mysql_upgrade -uroot -padmin
mysql -uroot -padmin
ALTER USER 'root'#'localhost' IDENTIFIED WITH mysql_native_password BY
'admin'
If you are getting this error on GitLab CI like me:
Just change from latest to 5.7 version ;)
# .gitlab-ci.yml
rspec:
services:
# - mysql:latest (I'm using latest version and it causes error)
- mysql:5.7 #(then I've changed to this specific version and fix!)
Open my sql command promt:
then enter mysql password
finally use:
ALTER USER 'username'#'ip_address' IDENTIFIED WITH mysql_native_password BY 'password';
refer:https://stackoverflow.com/a/49228443/6097074
Thanks.
For me this started happening because on a project, I was using Docker image mysql:latest (which was version 5, and which was working fine), and during a later build, the latest version was switched to version 8, and stopped working. I changed my image to mysql:5 and I was no longer getting this error.
This error comes up when the tool being used is not compatible with MySQL8, try updating to the latest version of MySQL Workbench for MySQL8
If you still want to use the new authentication method, the proper solution is to install the mariadb-connector-c package. For Alpine, run:
apk add mariadb-connector-c
This will add the missing caching_sha2_password.so library into /usr/lib/mariadb/plugin/caching_sha2_password.so.
Almost like answers above but may be in simple queries, I was getting this error in my spring boot application along with hibernate after MySQL upgrade. We created a new user by running the queries below against our DB. I believe this is a temp work around to use sha256_password instead of latest and good authentication caching_sha2_password.
CREATE USER 'username'#'localhost' IDENTIFIED WITH mysql_native_password BY 'pa$$word';
GRANT ALL PRIVILEGES ON * .* TO 'username'#'localhost';
MySQLWorkbench 8.0.11 for macOS addresses this.
I can establish connection with root password protected mysql instance running in docker.
If you are trying to connect to a MySQL server from a text-based MySQL client from another computer (be it Docker or not)
Most answers here involve connecting from a desktop client, or ask you to switch to an older authentication method. If you're connecting it with the MySQL client (text-based), I made it work with a Debian Buster in a Docker container.
Say you have the apt system and wget set up, do the following:
sudo apt-get update
sudo apt-get install lsb-release -y
Download a Debian package which update apt sources for you from the MySQL web site.
sudo dpkg -i mysql-apt-config_0.8.13-1_all.deb and select the options you want. In my case I only need MySQL Tools & Connectors to be enabled.
sudo apt-get update
sudo apt-get install mysql-client -y
Done. You can now run the new MySQL client and connect with the new authentication method.
The below solution worked for me
Go to Mysql Workbench -> Server-> Users and Privileges
1.Click Add Account
2.Under Login Tab provide new details and make sure to choose the Authentication Type as standard and choose respective administrative roles and Schema Privileges
Actually MySql allows two type of authentication at the time of installation.
Password Encryption
Legacy Encryption
Read Here
So by checking legacy authentication the issue was resolved.
Try using legacy password while downloading and installing MySql, that helped me.
Or follow the method posted by Santhosh Shivan for Mac OS.
Just downloaded the latest mysqlworkbench which is compatible with the latest encryption:
https://downloads.mysql.com/archives/workbench/
Note: On Mac big Sur, the latest two versions: 8.0.22 and 8.0.23 are buggy and do not work.
Use 8.0.21 until these are fixed
I run docker in M1 (arm64), the direct way of changing in the docker bash does not work for me. Instead, I change the mysql image to be
mysql:8.0.26
and the platform is set as
linux/x86_64
and add default_authentication_plugin=mysql_native_password to my.cnf
Then, you rebuild your container.

Server is freezing while trying to get a backup of MongoDB in docker composer

I have a back-end API server created with Python & Flask. I used MongoDB as my database. I build and run docker-composer every time while I update my source code. Because of this, I always take a backup of my database before stopping and restarting docker container.
From the beginning I am using this command to get a backup in my default folder:
sudo docker-compose exec -T db mongodump --archive --gzip --db SuperAdminDB> backup.gz
This line worked well previously. Then I restore the database again after restarting the the docker-composer to enable my back-end with updated code. I used this command to restore the database.
sudo docker-compose exec -T db mongorestore --archive --gzip < backup.gz
But from today, while I am trying to take a backup from server while the docker is still running (as usual), the server freezes like the image below.
I am using Amazon EC2 server and Ubuntu 20.04 version
First, stop redirecting output of the command. If you don't know whether it is working you should be looking at all available information which includes the output.
Then verify you can connect to your deployment using mongo shell and run commands.
If that succeeds look at server log and verify there is a record of connection from mongodump.
If that works try dumping other collections.
After digging 3 days for right reason I have found that the main reason is the apache.
I have recently installed apache to host my frontend also. While apache is running the server won't allow me to dump mongodb backup. Somehow apache was conflicting with docker.
My solution:
1. Stop apache service
sudo service apache2 stop
2. Then take MongoDB backup
sudo docker-compose exec -T db mongodump --archive --gzip --db SuperAdminDB> backup.gz

Heroku download postgres DB - capture works, curl is unknown

I am doing this task the first time. I have found good documentation on how to download an existing postgres DB to local but none of the informations seem to work for me.
I need to download an existing postgres DB to local. I do this to check the entries and to have a backup because I need to work on the DB.
I have posgresql installed localy and I want to copy the heroku DB into my local DB which is named postgres. The local server has a username postgres and a password.
I have only one DB and one APP on heroku.
I tried 2 solutions.
PG:PULL
heroku pg:pull DATABASE_URL postgres
This gave me:
DL is deprecated, please use Fiddle
! The command "createdb" is written either false or
! could not be found
!
! Unable to create new local database. Ensure your local Postgres is working and try again.
So I tried with password ans username:
PGUSER=postgres PGPASSWORD=mypw heroku pg:pull DATABASE_URL postgres
This gave me:
The command "PGUSER" is written either false or could not be found
PG:BACKUPS
heroku pg:backups capture
This worked well:
Use Ctrl-C at any time to stop monitoring progress; the backup
will continue running. Use heroku pg:backups info to check progress.
Stop a running backup with heroku pg:backups cancel.
DATABASE ---backup---> b003
Backup completed
Then I used curl:
curl -o latest.dump `heroku pg:packups public-url`
Which gave me:
The command "curl" is written either false or could not be found
It seems nobody had such a problem, I dont know what I am missing. I red the whole heroku documentation and there is no such case.
EDIT
I connected to my DB via PgAdmin4 but also here I am getting the same type of error when trying to restore a DB dump file.
The PgAdmin4 log gives me:
The command "pg_restore.exe" is written either false or could not be found
I also tried to add postgresql to global path variable, but it does not seem to help:
;C:\Programme\PostgreSQL\9.6\bin\pg_restore.exe
and I tried:
;C:\Programme\PostgreSQL\9.6\bin

Django not connecting to RDS Mysql

I am trying to deploy my django app on EC2 and am using RDS(Mysql) as the backend.
However when I try to run gunicorn with this, the server does not respond, and the if I run python manage.py dbshell I get the error:
CommandError: You appear not to have the 'mysql' program installed or on your path.
It seems to be trying to connect to a local server of mysql when I have clearly changed the host setting the Database dict to my RDS deployment
Things I have done for solving this and trouble shooting:
1. I have changed the database settings so that they point to the RDS database.
Both the EC2 and the RDS instance are in the same zone of AWS(read this might have been the issue)
Permission could have been a problem, but I checked by trying to login from my local machine, and I was able to access the mysql commandline on my server.
Checked the database settings which django is picking up at runtime,they are correct.
Install Mysql-python, it would not install without a local installation of Mysql, so googled that , people suggested install libmysqlclient-dev first and then you should be able to install Mysql-python.
Has someone face a similar problem? What am I doing wrong here?

Remote query in MySQL using python

How can I query a remote MySQL database, write a select query and insert into my local mysql database using python?
Some tips to keep in mind:
MySQL by default does not listen on a public IP address. This means, even if the server is running; you may not be able to access it remotely.
Even if the server has been reconfigured to listen on the public IP address, your user account needs to be granted permission to connect from remote clients.
Once you have those two taken care of, make sure you are able to connect to server. Use the mysql client:
mysql -H remote.box.com -U yourusername -P
Next, you need to install the MySQL drivers for Python.
On Ubuntu/Kubuntu/Debian: sudo apt-get install python-mysqldb
On RedHat/Fedora/CentOS: sudo yum install MySQL-python
On Windows: http://www.lfd.uci.edu/~gohlke/pythonlibs/ (search for MySQLdb)
On Mac: sudo pip install mysql-python
Finally - read this tutorial which will get you started.

Categories