We Keep Coding

How to access remote hive using pyhive

Used this link to try to connect to a remote hive. Below is the code used. The error msg received is also given below
How to Access Hive via Python?
Code
from pyhive import hive
conn = hive.Connection(host="10.111.22.11", port=10000, username="user1" ,database="default")
Error msg
Could not connect to any of [('10.111.22.11', 10000)]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/anaconda3/lib/python3.6/site-packages/pyhive/hive.py", line 131, in __init__
self._transport.open()
File "/opt/anaconda3/lib/python3.6/site-packages/thrift_sasl/__init__.py", line 61, in open
self._trans.open()
File "/opt/anaconda3/lib/python3.6/site-packages/thrift/transport/TSocket.py",line 113, in open
raise TTransportException(TTransportException.NOT_OPEN, msg)
thrift.transport.TTransport.TTransportException: Could not connect to any of [('10.111.22.11', 10000)]
What are the other requirements for successful connection? I am able to connect to the server directly (using putty) and run hive. But when tried from another server X i get this error. Also i can ping the hive server from server X.
Could the port number be the problem? How do i check the correct port number?
As discussed in the below answer i tried to start hiveserver2 . But the command doesnt seem to work. Any help is really appreciated.
Also the port i see in the log when i execute a query from hive shell is 8088. wonder if this should be the port instead of 10000(both did not work anyway)

Could not make it work using pyhive. Had to use paramiko insted
below is the sample code
import os
import paramiko
import time
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.load_host_keys(os.path.expanduser(os.path.join("~", ".ssh", "known_hosts")))
ssh.connect('1.1.1.1', username='uid', password='pwd')
sshin, sshout, ssherr= ssh.exec_command('hive -e "create table test(key varchar(10),keyval varchar(200))"')

HiveServer2 process must be started in your remote Hive host. 10000 is the default port number.
Use this command to start HiveServer2.
$HIVE_HOME/bin/hiveserver2

For PyHive to work, Hive Server 2.0 should be running using transport mode as “binary”. If it's running with transport as Http change it to "binary".

Please try below code to access remote hive table using pyhive:
from pyhive import hive
import pandas as pd
#Create Hive connection
conn = hive.Connection(host="10.111.22.11", port=10000, username="user1")
# Read Hive table and Create pandas dataframe
df = pd.read_sql("SELECT * FROM db_Name.table_Name limit 10", conn)
print(df.head())

OperationalError: (2003, "Can't connect to MySQL server on 'mysql2.000webhost.com' (10060)")

I have been learning some basic MySQL and decided to put it into Python which i have some more experience in. I have got PHP scripts working on this MySQL server so i know the server is working. But every time i connect i get this error:
Traceback (most recent call last):
File "E:/Geekster_Bot/Geekster_Bot Code/Geekster_Bot_alpha_API_MySQL", line 6, in <module>
db="MY DATABASE")# name of the data base
File "C:\Python27\lib\site-packages\MySQLdb\__init__.py", line 81, in Connect
return Connection(*args, **kwargs)
File "C:\Python27\lib\site-packages\MySQLdb\connections.py", line 187, in __init__
super(Connection, self).__init__(*args, **kwargs2)
OperationalError: (2003, "Can't connect to MySQL server on 'mysql2.000webhost.com' (10060)")
Im really not sure why. I have got the MySQL port (3306) open and like i said, the server/database is working. Any ideas?
This is the code of the entire MySQL connection;
import MySQLdb
db = MySQLdb.connect(host="mysql2.000webhost.com", # your host, usually localhost
user="MY USER", # your username
passwd="MY PASSWORD", # your password
db="MY DATABASE")# name of the data base
#you must create a Cursor object. It will let
#you execute all the queries you need
cur = db.cursor()
#Use all the SQL you like
cur.execute("SELECT * twitch_follows_ign")
#print all the first cell of all the rows
for row in cur.fetchall() :
print row[0]
All the blocked out data is entered correctly.

Your database seems to be on an external server "mysql2.000webhost.com", before you connect to the MySQL server from your Python application you'll have to create a database user and grant that user access to the server from your IP address.
If you have a free account on "000webhost" you probably won't be able to create a database user that has access to the database from the outside.
Try this site http://www.freemysqlhosting.net/ they will grant you access to the newly created database from any IP address.

Python Not Executing MySQLdb connect

I am having some issues connecting to my hosted mysql database using python. My code is as follows:
import MySQLdb
db = MySQLdb.connect(host="xxxxxx.db.1and1.com", user="xxxxxxx",passwd="xxxxxxx", db="xxxxxxx")
I get the following error:
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
db = MySQLdb.connect(host="db518597727.db.1and1.com", user="dbo518597727", passwd="OilyOily123", db="db518597727")
File "C:\Python27\lib\site-packages\MySQLdb\__init__.py", line 81, in Connect
return Connection(*args, **kwargs)
File "C:\Python27\lib\site-packages\MySQLdb\connections.py", line 193, in __init__
super(Connection, self).__init__(*args, **kwargs2)
OperationalError: (2003, "Can't connect to MySQL server on 'xxxxxxx.db.1and1.com' (10060)")
My sql database is hosted by 1and1. I am sure I have the right credentials, as when I read from this database on a php website, using the same credentials (I have triple checked) it works fine.
I have tried this both on a Raspberry Pi (where I intend to use it) and on my Windows 8.1 PC, so I am pretty sure that it is not the computer that is the problem.
Any help would be much appreciated.Many thanks.

In MySQL credentials are per client, so we distinguish a couple of client types:
- localhost means the same machine as MySQL server connecting via Unix Socket
- 127.0.0.1 means the same machine as MySQL server connecting via TCP/IP
- A.B.C.D where letters are replaced by numbers in range 0-255 which means IP address of client machine connecting via TCP/IP
- * is a wildcard which means that any client can connect with given credentials via TCP/IP
Every entry in MySQL users table consists of client specification (described above), username, password and some other columns (not relevant here).
So the problem you are facing is that PHP script on 1and1 server can connect to the database hosted on 1and1 since the hosting company sets up their database server to accept connections from their own servers with valid credentials.
But the same credentials are considered invalid for connections coming from client machines unknown to 1and1.
What you can do is to ask 1and1 to give your specific IP access rights to your database.
What you can't to is to overcome this problem on your own.

connecting to SQL database on my web server from Raspberry pi

I am trying to access an SQL database that reside on my web server from a Raspberry Pi - using python.
I tried using the following code:
#!/usr/bin/env python
import MySQLdb as mdb
conn = mdb.connect(host="abc.co.uk.mysql"
user="USER NAME"
passwd="PASSWORD"
db="NAME OF DATABASE")
cur = conn.cursor()
cur.execute("SELECT * FROM TABLE_NAME")
for row in cur.fetchall():
print row[0]
I receive the following error when executing!
Traceback (most recent call last):
File "getDatafromSQL", line 7, in <module>
db='alisoliman_co_u')
File "/usr/lib/python2.7/dist-packages/MySQLdb/__init__.py", line 81, in Connect
return Connection(*args, **kwargs)
File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 187, in __init__
super(Connection, self).__init__(*args, **kwargs2)
_mysql_exceptions.OperationalError: (2003, "Can't connect to MySQL server on 'alisoliman.co.uk.mysql' (111)")
Any idea why could that be? Is there any ways that I can connect to a MySQL database on a server?

I'm sure that you made the right indentation for the print statement under the for loop (the last line of your code).
I don't know which distro are you running on your Raspberry PI, but I've tried your code on my RPI (Arch distro), doing a query on a simple table on the mysql server running in the RPI, and it works well. I've did some changes but I don't believe that these changes are relevant for the right flow of the query.
#!/usr/bin/env python
import MySQLdb as mdb
conn_1=mdb.Connection(db='NAME OF DATABASE', host='localhost', user='USER NAME', passwd='PASSWORD')
cur = conn_1.cursor()
cur.execute("SELECT * FROM rkn_env")
for row in cur.fetchall():
print row[0]
FYI, you can capture the traffic between RPI and the host where the MySQL server is running using tcpdump. This tool has saved me many times...
Lastly, did you check if your MySQL daemon is running and listening (e.g. using netstat -anp) ?

MySQL error: 2013, "Lost connection to MySQL server at 'reading initial communication packet', system error: 0"

I'm having an issue connecting to my local MySQL database using Python's MySQLdb library. The script has been working well previously, but I will occasionally get the MySQL error in the title. There seems to be no explanation for when the error occurs, and the script is always run from the same machine with the same arguments.
The MySQL server is running as a service on Windows XP SP3 using port 3306 (locally hosted phpMyAdmin works), and the script is run from an Ubuntu 10.04 guest operating system in Oracle VM VirtualBox.
I am currently working around this issue by opening a command prompt and executing 'net stop MySQL' then 'net start MySQL'. This allows me to run the script a few times again before resulting in the error, which I've been fixing by restarting the MySQL service.
As I am still making changes to the script, there are occasions when the script raises an exception and doesn't exit gracefully, though I do catch the exception and close the cursor and connection.
The code to connect to the database:
def __init__(self):
try:
print "Connecting to the MySQL database..."
self.conn = MySQLdb.connect( host = "192.168.56.1",
user = "guestos",
passwd = "guestpw",
db = "testdb")
self.cursor = self.conn.cursor(MySQLdb.cursors.DictCursor)
print "MySQL Connection OK"
except MySQLdb.Error, e:
print "MySQLdb error %d: %s" % (e.args[0],e.args[1])
raise
The full error generated when this happens is as follows:
MySQLdb error 2013: Lost connection to MySQL server at 'reading initial communication packet', system error: 0
Traceback (most recent call last):
File "search.py", line 45, in <module>
dataHandler = DataHandler()
File "/home/guestos_user/workspace/Search/src/data_handler.py", line 25, in __init__
db = "testdb")
File "/usr/lib/pymodules/python2.6/MySQLdb/__init__.py", line 81, in Connect
return Connection(*args, **kwargs)
File "/usr/lib/pymodules/python2.6/MySQLdb/connections.py", line 170, in __init__
super(Connection, self).__init__(*args, **kwargs2)
_mysql_exceptions.OperationalError: (2013, "Lost connection to MySQL server at 'reading initial communication packet', system error: 0")

sudo vi /etc/mysql/my.cnf
delete
bind-address = 127.0.0.1
then
sudo reboot now
That's it. Be aware that this will make your mysql server less secure as you are exposing it.

I have seen this happen when child processes try to share the same mysql connection id (solution = create new connections for each child process). I'm not sure if this is also possible when sharing connection objects with multiple threads.
However, that's only one of the many possible causes. See VVS's answer in MySQL Error 2013 for a list of troubleshooting resources.

Do you have in your MySQL server an acount called guestos#YOURIPADDRESS?
You must have an account to access to your MySQL server from YOURIPADDRESS!
For example:
Your IP address is 192.168.56.2; then you must create and account if not exist to access.
mysql> create user guestos#192.168.56.2 identified by 'guestpw';

The problem fixed for me just by restarting my mac. Though there might be a more specific fix for it.

I received a similar error when attempting to connect to my MySQL server remotely through a user with the sufficient permissions.
After editing the /etc/mysql/my.cnf file to include
[mysqld]
bind-address=xx.xx.xxx.xxx
where xx.xx.xxx.xxx is my local IP address, I began experiencing the exact same error as you. From there, I found an answer regarding this issue (answered by Coffee Converter) which worked for me, and can be found here: Lost connection to MySQL server at 'reading initial communication packet', system error: 0 on a windows machine
All I did to fix the issue for myself was edit the /etc/hosts.allow to include
mysqld: ALL: allow
Works great now! I hope this helped :)

Could you change the bind-address=localhost and restart MySQL server? Seems like this issue is related to yours: http://forums.mysql.com/read.php?152,355740,355742#msg-355742
Also this-
If MySQL port is wrong result is MySQL client error 2013 "Lost
connection ...". Note that this error also occurs if port forwarding
is disabled in SSH configuration (the configuration parameter
'AllowTcpForwarding' is set to 'no' in the 'sshd_config' file). It
(here) simply tells that there is no connection from SSH to MySQL for
some reason. But the mySQL client API 'thinks' there was one
connection and that is why is says 'Lost connection ...' and not
'Can’t connect...'. There was one successful connection - but not to
the MySQL server - to the SSH daemon only! But the MySQL client API is
not designed to 'see' the difference!
Refer this.

I run a windows server and from time to time the php-win.exe will load and stay in the processes list on the windows task manager.
If you know the host file is correct, then kill the php-win.exe process and restart iis iisreset
If you are running windows then your problem should be solved.

I've had the exact same mysql error (ERROR 2013 (HY000): Lost connection to MySQL server at 'reading initial communication packet', system error: 0=) and have resolved it by adding a newline to /etc/hosts.deny.

Possibility: your database is corrupted.
I encountered this situation when I was running an UPDATE statement on a specific row of a specific table. (Specifically, I was editing an item in a Django Admin site.) Most of the time the database worked just fine.
I finally resolved the problem by running:
OPTIMIZE TABLE `your_table`
After that everything was OK, no connection lost.
Conclusion:
The problem "Lost connection to MySQL server at 'reading initial communication packet'", sometimes "Can't connect to MySQL server on '127.0.0.1'", could possibly be resolved by running a full database optimization if the database is corrupted. For more info, read this.

Just to further extend the list of possible causes: it could also be as banal as wrong connection data/credentials. I encountered this error in conjunction with sqlalchemy:
sqlalchemy.exc.OperationalError: (mysql.connector.errors.OperationalError) 2013 (HY000): Lost connection to MySQL server at 'reading initial communication packet', system error: 0
In my code I connect to several different databases and once in a while it happens that I don't get the mapping between the db connections and their credentials (e.g. ip address of server, db-name, password etc.) right, which then also results in the 2013-error (in this case wrapped into an sqlalchemy operational error).

setting.py file set like:
'ENGINE': 'django.db.backends.mysql',
'NAME': 'test2',
'USER': 'root',
'PASSWORD': '',
'HOST': 'localhost',
'PORT': '3308',

This bug report might be of interest to you. Don't know if this will help you, but some were able to solve it by using the name of the server rather than the ip address in the connection properties.