Remote query in MySQL using python - python

How can I query a remote MySQL database, write a select query and insert into my local mysql database using python?

Some tips to keep in mind:
MySQL by default does not listen on a public IP address. This means, even if the server is running; you may not be able to access it remotely.
Even if the server has been reconfigured to listen on the public IP address, your user account needs to be granted permission to connect from remote clients.
Once you have those two taken care of, make sure you are able to connect to server. Use the mysql client:
mysql -H remote.box.com -U yourusername -P
Next, you need to install the MySQL drivers for Python.
On Ubuntu/Kubuntu/Debian: sudo apt-get install python-mysqldb
On RedHat/Fedora/CentOS: sudo yum install MySQL-python
On Windows: http://www.lfd.uci.edu/~gohlke/pythonlibs/ (search for MySQLdb)
On Mac: sudo pip install mysql-python
Finally - read this tutorial which will get you started.

Related

how to get out of "caching_sha2_password" error? [duplicate]

I am connecting MySQL - 8.0 with MySQL Workbench and getting the below error:
Authentication plugin 'caching_sha2_password' cannot be loaded:
dlopen(/usr/local/mysql/lib/plugin/caching_sha2_password.so, 2): image
not found
I have tried with other client tool as well.
Any solution for this?
you can change the encryption of the password like this.
ALTER USER 'yourusername'#'localhost' IDENTIFIED WITH mysql_native_password BY 'youpassword';
Note: For MAC OS
Open MySQL from System Preferences > Initialize Database >
Type your new password.
Choose 'Use legacy password'
Start the Server again.
Now connect the MySQL Workbench
For Windows 10:
Open the command prompt:
cd "C:\Program Files\MySQL\MySQL Server 8.0\bin"
C:\Program Files\MySQL\MySQL Server 8.0\bin> mysql -u root -p
Enter password: *********
mysql> ALTER USER 'root'#'localhost' IDENTIFIED WITH mysql_native_password BY 'newrootpassword';
Query OK, 0 rows affected (0.10 sec)
mysql> exit
Alternatively, you can change the my.ini configuration as the following:
[mysqld]
default_authentication_plugin=mysql_native_password
Restart the MySQL Server and open the Workbench again.
I had the same problem, but the answer by Aman Aggarwal didn't work for me with a Docker container running mysql 8.X.
I loged in the container
docker exec -it CONTAINER_ID bash
then log into mysql as root
mysql --user=root --password
Enter the password for root (Default is 'root')
Finally Run:
ALTER USER 'username' IDENTIFIED WITH mysql_native_password BY 'password';
You're all set.
You can change the encryption of the user's password by altering the user with below Alter command :
ALTER USER 'username'#'ip_address' IDENTIFIED WITH mysql_native_password BY
'password';
OR
We can avoid this error by make it work with old password plugin:
First change the authentication plugin in my.cnf file for Linux / my.ini file in Windows:
[mysqld]
default_authentication_plugin=mysql_native_password
Restart the mysql server to take the changes in affect and try connecting via MySQL with any mysql client.
If still unable to connect and getting the below error:
Unable to load plugin 'caching_sha2_password'
It means your user needs the above plugin. So try creating new user with create user or grant command after changing default plugin. then new user need the native plugin and you will able to connect MySQL.
Thanks
Currently (on 2018/04/23), you need to download a development release. The GA ones do not work.
I was not able to connect with the latest GA version (6.3.10).
It worked with mysql-workbench-community-8.0.11-rc-winx64.msi (from https://dev.mysql.com/downloads/workbench/, tab Development Releases).
Ok, wasted a lot of time on this so here is a summary as of 19 March 2019
If you are specifically trying to use a Docker image with MySql 8+, and then use SequelPro to access your database(s) running on that docker container, you are out of luck.
See the sequelpro issue 2699
My setup is sequelpro 1.1.2 using docker desktop 2.0.3.0 (mac - mojave), and tried using mysql:latest (v8.0.15).
As others have reported, using mysql 5.7 works with nothing required:
docker run -p 3306:3306 --name mysql1 -e MYSQL_ROOT_PASSWORD=secret -d mysql:5.7
Of course, it is possible to use MySql 8+ on docker, and in that situation (if needed), other answers provided here for caching_sha2_password type issues do work. But sequelpro is a NO GO with MySql 8+
Finally, I abandoned sequelpro (a trusted friend from back in 2013-2014) and instead installed DBeaver. Everything worked out of the box. For docker, I used:
docker run -p 3306:3306 --name mysql1 -e MYSQL_ROOT_PASSWORD=secret -d mysql:latest --default-authentication-plugin=mysql_native_password
You can quickly peek at the mysql databases using:
docker exec -it mysql1 bash
mysql -u root -p
show databases;
I was installing MySQL on my Windows 10 PC using "MySQL Web Installer" and was facing the same issue while trying to connect using MySQL workbench. I fixed the issue by reconfiguring the server form the Installer window.
Clicking on the "Reconfigure" option it will allow to reconfigure the server. Click on "Next" until you reach "Authentication Method".
Once on this tab, use the second option "Use Legacy Authentication Method (Retain MySQL 5.x Compatibility)".
Keep everything else as is and that is how I solved my issue.
Note: For Linux (Debian, Ubuntu, Mint)
I got this error:
MySQL Error Message: Plugin caching_sha2_password could not be loaded: /usr/lib/x86_64-linux-gnu/mariadb19/plugin/caching_sha2_password.so: cannot open shared object file: No such file or directory
I solved it with these steps:
Enter on mysql console: $ mysql -u root -p, if you don't have a password for root user, then:
Use mysql db: mysql> use mysql;
Alter your user for solve the problem: mysql> ALTER USER 'root'#'localhost' IDENTIFIED WITH mysql_native_password BY 'password';
Exit... mysql> quit;
Done!
like this?
docker run -p 3306:3306 -e MYSQL_ALLOW_EMPTY_PASSWORD=yes -d mysql --default-authentication-plugin=mysql_native_password
mysql -uroot --protocol tcp
Try in PWD
https://github.com/GitHub30/docs/blob/change-default_authentication_plugin/mysql/stack.yml
or You shoud use MySQL Workbench 8.0.11.
Open MySQL Command Line Client
Create a new user with a new pass
Considering an example of a path to a bin folder on top, here's the code you need to run in the command prompt, line by line:
cd C:\Program Files\MySQL\MySQL Server 5.7\bin
MySQL -u root -p
current password...***
CREATE USER 'nativeuser'#'localhost'
IDENTIFIED WITH mysql_native_password BY 'new_password';
Then, you can access Workbench again (you should be able to do that after creating a new localhost connection and using the new credentials to start using the program).
Set up a new local host connection with the user name mentioned above (native user), login using the password (new_password)
Courtesy: UDEMY FAQs answered by Career365 Team
For Windows 10,
Modify my.ini file in C:\ProgramData\MySQL\MySQL Server 8.0\
[mysqld]
default_authentication_plugin=mysql_native_password
Restart the MySQL Service.
Login to MySQL on the command line, and execute the following commands in MySQL:
Create a new user.
CREATE USER 'user'#'localhost' IDENTIFIED BY 'password';
Grant all privileges.
GRANT ALL PRIVILEGES ON * .* TO 'user'#'localhost';
Open MySQL workbench, and open a new connection using the new user credentials.
I was facing the same issue and this worked.
Although this shouldn't be a real
solution, it does work locally if you are stuck
ALTER USER 'root'#'localhost' IDENTIFIED WITH mysql_native_password BY '';
This is my databdase definition in my docker-compose:
dataBase:
image: mysql:8.0
volumes:
- db_data:/var/lib/mysql
networks:
z-net:
ipv4_address: 172.26.0.2
restart: always
entrypoint: ['docker-entrypoint.sh', '--default-authentication-plugin=mysql_native_password']
environment:
MYSQL_ROOT_PASSWORD: supersecret
MYSQL_DATABASE: zdb
MYSQL_USER: zuser
MYSQL_PASSWORD: zpass
ports:
- "3333:3306"
The relevant line there is entrypoint.
After build and up it, you can test it with:
$ mysql -u zuser -pzpass --host=172.26.0.2 zdb -e "select 1;"
Warning: Using a password on the command line interface can be insecure.
+---+
| 1 |
+---+
| 1 |
+---+
For those using Docker or Docker Compose, I experienced this error because I didn't set my MySQL image version. Docker will automatically attempt to get the latest version which is 8.
I set MySQL to 5.7 and rebuilt the image and it worked as normal:
version: '2'
services:
db:
image: mysql:5.7
I found that
ALTER USER 'username'#'ip_address' IDENTIFIED WITH mysql_native_password BY 'password';
didn't work by itself. I also needed to set
[mysqld]
default_authentication_plugin=mysql_native_password
in /etc/mysql/mysql.conf.d/mysqld.cnf
on Ubuntu 18.04 running PHP 7.0
Here is the solution which worked for me after MySQL 8.0 Installation on Windows 10.
Suppose MySQL username is root and password is admin
Open command prompt and enter the following commands:
cd C:\Program Files\MySQL\MySQL Server 8.0\bin
mysql_upgrade -uroot -padmin
mysql -uroot -padmin
ALTER USER 'root'#'localhost' IDENTIFIED WITH mysql_native_password BY
'admin'
If you are getting this error on GitLab CI like me:
Just change from latest to 5.7 version ;)
# .gitlab-ci.yml
rspec:
services:
# - mysql:latest (I'm using latest version and it causes error)
- mysql:5.7 #(then I've changed to this specific version and fix!)
Open my sql command promt:
then enter mysql password
finally use:
ALTER USER 'username'#'ip_address' IDENTIFIED WITH mysql_native_password BY 'password';
refer:https://stackoverflow.com/a/49228443/6097074
Thanks.
For me this started happening because on a project, I was using Docker image mysql:latest (which was version 5, and which was working fine), and during a later build, the latest version was switched to version 8, and stopped working. I changed my image to mysql:5 and I was no longer getting this error.
This error comes up when the tool being used is not compatible with MySQL8, try updating to the latest version of MySQL Workbench for MySQL8
If you still want to use the new authentication method, the proper solution is to install the mariadb-connector-c package. For Alpine, run:
apk add mariadb-connector-c
This will add the missing caching_sha2_password.so library into /usr/lib/mariadb/plugin/caching_sha2_password.so.
Almost like answers above but may be in simple queries, I was getting this error in my spring boot application along with hibernate after MySQL upgrade. We created a new user by running the queries below against our DB. I believe this is a temp work around to use sha256_password instead of latest and good authentication caching_sha2_password.
CREATE USER 'username'#'localhost' IDENTIFIED WITH mysql_native_password BY 'pa$$word';
GRANT ALL PRIVILEGES ON * .* TO 'username'#'localhost';
MySQLWorkbench 8.0.11 for macOS addresses this.
I can establish connection with root password protected mysql instance running in docker.
If you are trying to connect to a MySQL server from a text-based MySQL client from another computer (be it Docker or not)
Most answers here involve connecting from a desktop client, or ask you to switch to an older authentication method. If you're connecting it with the MySQL client (text-based), I made it work with a Debian Buster in a Docker container.
Say you have the apt system and wget set up, do the following:
sudo apt-get update
sudo apt-get install lsb-release -y
Download a Debian package which update apt sources for you from the MySQL web site.
sudo dpkg -i mysql-apt-config_0.8.13-1_all.deb and select the options you want. In my case I only need MySQL Tools & Connectors to be enabled.
sudo apt-get update
sudo apt-get install mysql-client -y
Done. You can now run the new MySQL client and connect with the new authentication method.
The below solution worked for me
Go to Mysql Workbench -> Server-> Users and Privileges
1.Click Add Account
2.Under Login Tab provide new details and make sure to choose the Authentication Type as standard and choose respective administrative roles and Schema Privileges
Actually MySql allows two type of authentication at the time of installation.
Password Encryption
Legacy Encryption
Read Here
So by checking legacy authentication the issue was resolved.
Try using legacy password while downloading and installing MySql, that helped me.
Or follow the method posted by Santhosh Shivan for Mac OS.
Just downloaded the latest mysqlworkbench which is compatible with the latest encryption:
https://downloads.mysql.com/archives/workbench/
Note: On Mac big Sur, the latest two versions: 8.0.22 and 8.0.23 are buggy and do not work.
Use 8.0.21 until these are fixed
I run docker in M1 (arm64), the direct way of changing in the docker bash does not work for me. Instead, I change the mysql image to be
mysql:8.0.26
and the platform is set as
linux/x86_64
and add default_authentication_plugin=mysql_native_password to my.cnf
Then, you rebuild your container.

default password for mysql connection in python

I install mysql with pip install mysql-connector-python
but for connect in python i don't know what password
mysqldb=mysql.connector.connect(host="localhost",user="",password="",database="")
Don't use password, just use host.
https://dev.mysql.com/doc/connector-python/en/connector-python-connectargs.html
(Btw. this is just connecting package, you need MySql server too - https://pypi.org/project/MySQL-python/)

psycopg2.connect failing with error 'connection refused' (Django app+db hosted on separate VMs)

I have a Django app where the app and database reside in two separate VMs. I use Azure. In trying to connect to a postgresql backend hosted in a separate Azure VM, I'm writing: conn = psycopg2.connect("dbname='dbname' host='dnsname'"), where value of host was gotten off the Azure portal from here:
I have also tried the Virtual IP address above as the host. Both approaches fail and I get a connection error: could not connect to server: Connection refused. Is the server running on host "myapp.cloudapp.net" (23.132.341.192) and accepting TCP/IP connections on port 5432? Please note that I changed the IP address when pasting the error here.
The Azure VM hosting the postgresql database has port 5432 added to its Endpoints. Moreover, my app to which this postgresql backend belongs is a Django/Python app. My VMs have Ubuntu 14.04. In setting up postgresql, I ran the following installation commands on both app and database VMs separately:
sudo apt-get update
sudo apt-get install PostgreSQL postgresql-contrib
sudo apt-get install postgresql-server-dev-all
sudo apt-get install libpq-dev
And even though both app and database VMs have all of the above installed in them, the database itself was only created in the latter VM. I didn't create a separate user; I instead did:
sudo su postgres
psql -d postgres -U postgres
And then
CREATE DATABASE dbname;
How do I fix the psycopg2.connect error? Ask for more information if needed.
As to separate PostgreSQL into another Azure VM, we need to configure several settings both on Azure VM side and your PostgreSQL service.
1, modify the listen_addresses="*" at /etc/postgresql/9.4/main/postgresql.conf ,to allow other client beside local to request your PostgreSQL service.
2, Add the following line as the first line of /etc/postgresql/9.4/main/pg_hba.conf. It allows access to all databases for all users with an encrypted password:
# TYPE DATABASE USER CIDR-ADDRESS METHOD
host all all 0.0.0.0/0 md5
3, run command sudo service postgresql restart to restart the service
4, login Azure manage portal, add a request rule in Endpoints page of your VM, configure public port 5432 to private port 5432:

Postgres in Azure Flask Web App

I've got Flask up and running as per this tutorial
https://azure.microsoft.com/en-us/documentation/articles/web-sites-python-create-deploy-flask-app/
I want Postgres to be my database system, but I'm in a Web App, so I can't just log into the VM and install it. What can I do here?
Thanks
It seems that we don’t have permission to install PostgreSQL database on Azure Web APP server. You need install PostgreSQL on Azure VM.
For examples:
A. If you created a Windows Server VM, refer to the link https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-windows-tutorial/ and connect your VM.
At the link page http://www.enterprisedb.com/products-services-training/pgdownload#windows, you can download a PostgreSQL Windows Installer file and run it on your VM to install it using default configuration step by step.
The PostgreSQL default port is 2345. Make sure the Windows Server Firewall allow the port accessing and try to test the connection by using VM DNS NAME from your local host, and then you can continue to develop.
B. If you create a Linux VM such as Ubuntu,refer to the link https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux-tutorial/ and connect your VM.
To install PostgreSQL, you can use Linux Package Management Tool.
Refer to the link:https://wiki.postgresql.org/wiki/Detailed_installation_guides
Ubuntu/Debian:
$ sudo apt-get update
$ sudo apt-get install postgresql
RedHat/CentOS:
Refer to the link:http://wiki.postgresql.org/wiki/YUM_Installation
You can use SQLAlchemy ORM Framework in Flask for PostgreSQL, please refer to http://flask.pocoo.org/docs/0.10/patterns/sqlalchemy/ and http://docs.sqlalchemy.org/en/rel_1_0/dialects/postgresql.html.

Setting up psycopg2 for the first time, how do you configure it for some test data bases?

This is my first time using PostgreSQL, I'm used to using MySQL where once you install MySQL server on Ubuntu for the first time, it'll ask you for a MySQL username and password. Once that's finished you can connect to it simply by providing:
MySQL Host: 127.0.0.1
Username: root
Password: 123pie
And from there you're ready to open up your favorite DB software like sequel pro and connect to it via SSH using the localhost IP of MySQL like shown above.
However right now my client is using PostgreSQL and so I need to learn how to set it up fast.
So far I've done this:
pip install psycopg2==2.4.5
Looks like this is just a library required for Django and Python to open up a PostgreSQL database. It doesn't look like something identical to MySQL Server.
Is installing PostgreSQL Server the next thing that I have to do? After I install that via the command line, is it exactly the same as setting up MySQL server?
I've never used PostgreSQL, I expect that it's very identical to using MySQL?
There's a nice, detailed guide on the Ubuntu-specific setup for PostgreSQL on the ubuntu community wiki. That should be your starting point.
For most applications it is sufficient to create a PostgreSQL user for the web server's username:
sudo -u postgres createuser www-data
then create a database owned by that user for the app to use:
sudo -u postgres createdb -O www-data test_django
and configure the app to connect with username www-data to the test_django database with no password. PostgreSQL on most distros, including Ubuntu, defaults to peer authentication where it requires you to have the same unix username as the postgres user you're connecting as, and doesn't require a password.
If the app forces you to supply a password and won't accept a blank one, or if you want the app to use a username different to the user that the web server runs as (i.e. to isolate multiple web apps from each other a bit), you need to add an entry to pg_hba.conf specifying md5 password authentication for that database/username combo - or just all users for all dbs.

Categories