I am new to superset.
Going to Sources > Databases for a new connection to my athena.
I have downloaded JDBC driver and writing following connection line:
awsathena+jdbc://AKIAJ2PKWTZYAPBYKRMQ:xxxxxxxxxxxxxxx#athena.us-east-1.amazonaws.com:443/default?s3_staging_dir='s3://aws-athena-query-results-831083831535-us-east-1/' as SQLAlchemy URI. First parameter being access key and 2nd being secret key(Modified a bit for privacy)
I am getting the error:
ERROR: {"error": "Connection failed!\n\nThe error message returned was:\nCan't load plugin: sqlalchemy.dialects:awsathena.jdbc"}
I really wish to explore the open source visualisation using superset on my databases.
As per Superset documentation, you need to escape/encode at least the s3_staging_dir, i.e.,
s3://... -> s3%3A//...
Have you followed that step?
If you are sure you have done pip install "PyAthenaJDBC>1.0.9" in the same python environment as you start your superset. Try restarting Superset in the same environment.
In my case problem was with the special characters in the aws_secret_key and s3_staging_dir. I solved it by putting the output of quote_plus methods into the URI. No quotes were required.
from urllib.parse import quote_plus
secretkey = quote_plus(aws_secret_access_key)
loc = quote_plus(s3_staging_dir)
Further, make sure the schema_name (i.e. database name) already exists in the s3 path. Hope it helps!
Related
I have a django & docker server running on my computer and I have created a database with code from outside this server. I am trying to access this database ('test.sqlite3') within the server.
I made sure the path was the correct one and that the file name was correct as well. When I open the database with DB Browser, I can see the tables and all my data. But I still get the following error text:
OperationalError no such table: NAMEOFTABLE
When I use the exact same code from another python IDE (spyder) it works fine. I'm guessing there's something weird going on with django?
Here is some of the code:
conn = sqlite3.connect("../test.sqlite3")
c = conn.cursor()
c.execute("SELECT firstName, lastName FROM RESOURCES")
conn.close()
(Yes, I have also tried using the absolute path and I get the same error.)
Also to be noted: I get this same error when I try to create the database file & table from within the django code (the path should then be the same but it still get the error in this case).
Update: it seems I have a problem with my path because I can't even open a text file with python and it's absolute path. So if anyone has any idea why that'd be great.
try:
f = open("/Users/XXXXX/OneDrive/XXXXX/XXXX/Autres/argon-dashboard-django/toto.txt")
# Do something with the file
except IOError:
q="File not accessible"
finally:
f.close()
always return the following error 'f referenced before assignment' and q = "File not accesible" so that means I can't even find the text file.
To answer this problem, I used two things:
I moved the sqlite3 file within the app folder and used '/app/db.sqlite3' as the path
Added ; at the ends of my SQL requests
c.execute("SELECT firstName, lastName FROM RESOURCES;")
Not sure which one solved the problem but everything works for me now.
Had a similar issue, possibly something about leaving Django Model metadata files outside of the image. I needed to synchronize the model with the DB using a run-syncdb
RUN ["python", "manage.py", "migrate"]
RUN ["python", "manage.py", "migrate", "--run-syncdb"]
CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]
I am trying to connect to a DB2 JDBC database thru Python with providing the SSLClientKeystoredb.
This is how I have been trying to connect to the DB:
import ibm_db
arg1 = "DRIVER={IBM DB2 ODBC DRIVER};" + "DATABASE=databasename;HOSTNAME=" + "server" + ";PORT=" + "111111" + ";PROTOCOL=TCPIP;UID=" + "userId" + ";PWD=" + "password" + ";SECURITY=ssl" + ";SSLClientKeystoredb=" + "C:/Users/path/db2_ssl_keydb.kdb" + ";SSLClientKeystash=" + "C:/Users/path/db2_ssl_keydb.sth"
conn=ibm_db.connect(arg1, "", "")
I keep getting this error:
SQLCODE=-1109M][CLI Driver] SQL1109N The command was not processed because the database manager failed to load the following DLL: "GSKit Error: 202". SQLSTATE=42724
I installed both GSKit8 Crypt and GSKit SSL 64-bit. Any help would be appreciated !
On Db2-client workstations, you can avoid installing/configuring the GSK8 as a separate component, and still have encrypted SSL connections to Db2-LUW servers.
Note that you might need GSK8 on client-workstations for other reasons (other non-Db2 applications), but that is a separate matter.
On MS-Windows, there are two ways to avoid having to install GSK8 for Db2 SSL connections, but in this answer I mention one way.
Technically this feature became available at V10.5 fixpack 5 Db2-clients, but there were some bugs so I suggest to avoid that fixpack and start with fixpack 8 or higher. This functionality also works in V11.1 Db2-clients.
If you have the server-certificate in ARM format, then you can use the SSLSERVERCERTIFICATE and SECURITY keywords in the connection-string to connect with SSL from Python (or from any tool that uses the Db2 CLI libraries).
With this approach you don't need a keystore and stash to be manually created, and you don't need SSLClientKeystoredb etc in the connection string.
You still need to add appropriate security for the ARM file both at rest and during distribution.
This approach may be easier to manage, and example connection is below:
try:
arg1="DATABASE=whatever;HOSTNAME=whatever;PORT=50443;UID=whavever;PWD=whatever;SSLServerCertificate=/path_to/db2server_instance.arm;SECURITY=ssl;"
conn = ibm_db.connect(arg1,"","")
except:
logging.error('Error: Failed to connect to database: %s', ibm_db.conn_errormsg())
sys.exit(1)
Don't know if it helps, but when I launched the gitBash / command prompt via "run as administrator" it worked for me. After I used the
conn=ibm_db.connect("Database=****DB; Hostname=***.***.***.COM; PORT=****; Security=ssl; SSLClientKeystoredb=c:/keystore/ibmca.kdb; SSLClientKeystash=c:/keystore/ibmca.sth;UID= ; PWD= ;",'','')
"202 - GSK_KEYRING_OPEN_ERROR
Unable to open the key file or the Microsoft Certificate Store. Either the path was specified incorrectly or the file permissions did not allow the file to be opened, or the file format is incorrect."
The arg1 passed to ibm.db is badly formatted and you are missing a semi-colon after you assign your SSLClientKeystash. Try following this: IBM Support
The issue was solved by switching to python 2.7.9, and changing the path of the SSL to "C:\SSL" for the certificates. Not sure if changing the path helped but just wanted to mention that for future reference.
I'm trying to access registry keys under HKEY_LOCAL_MAHINE\SOFTWARE... on a 64 bits system. I have following code but judging by the results it gets redirected to Wow6432Node even though I have _winreg.DisableReflectionKey(_winreg.OpenKey(HKEY_LOCAL_MACHINE, "SOFTWARE")) in my code.
import _winreg
import wmi
c = wmi.WMI(computer="localhost", namespace="root/default").StdRegProv
_winreg.DisableReflectionKey(_winreg.OpenKey(HKEY_LOCAL_MACHINE, "SOFTWARE"))
result, names = c.EnumKey(hDefKey=_winreg.HKEY_LOCAL_MACHINE, sSubKeyName="SOFTWARE")
print names
_winreg.EnableReflectionKey(_winreg.OpenKey(HKEY_LOCAL_MACHINE, "SOFTWARE"))
I have read the following case but it seems that the solution described there doesn't work for either the author or for me:
How can I turn off registry redirection on Python?
Also checked _winreg documentation, but there are no specific examples and I have no idea what I'm doing wrong. Any ideas? Sorry, I didn't have permission to comment in the existing case and had to open a new one.
It looks like "_winreg.KEY_READ | _winreg.KEY_WOW64_64KEY" does the job. To be more precise you need to open the key this way:
_winreg.OpenKey(_winreg.HKEY_LOCAL_MACHINE, "SOFTWARE", 0, _winreg.KEY_READ | _winreg.KEY_WOW64_64KEY)
I got this working when accessing the localhost registry. However, I still haven't figured out how to connect to remote registry on the domain. Trying to use _winreg.ConnectRegistry, but keep getting access denied error.
Hey guys this is my first question on here. I'm trying to make a local copy of the UniprotKB in SQL.
The UniprotKB is 2.1GB, and it comes in XML and a special text format used by SwissProt
Here are my options:
1) Use a SAX parser (XML) - I chose Ruby, and Nokogiri. I started writing the parser, but my initial reaction: how would I map the XML schema to the SAX parser?
2) BioPython - I already have BioSQL/Biopython installed, which literally created my SQL schema for me, and I was able to successfully insert one SwissProt/Uniprot txt file into the database.
I'm running it right now (crosses fingers) on the entire 2.1gb. Here is the code I'm running:
from Bio import SeqIO
from BioSQL import BioSeqDatabase
from Bio import SwissProt
server = BioSeqDatabase.open_database(driver = "MySQLdb", user = "root", passwd = "", host="localhost", db = "bioseqdb")
db = server["uniprot"]
iterator = SeqIO.parse(open("/path/to/uniprot_sprot.dat", "r"), "swiss")
db.load(iterator)
server.commit()
Edit: it's now crashing because the transactions are getting locked (since the tables are Innodb)
Error Number: 1205
Lock wait timeout exceeded; try restarting transaction. I'm using
MySQL version: 5.1.43
Should I switch my database to Postgrelsql ?
Switched to PostgrelSQL for convenience purposes. Some of the issues were resolved by downloading the NCBI taxonomy information (which I did not know was necessary, should have been more clear in the documentation), so I ended up using the Swiss parser from BioPython because it fits so nicely with BioSQL.
The code
#!/usr/bin/env python
import MySQLdb
print "Content-Type: text/html"
print
print "<html><head><title>Books</title></head>"
print "<body>" print "<h1>Books</h1>"
print "<ul>"
connection = MySQLdb.connect(user='me', passwd='letmein', db='my_db') cursor = connection.cursor() cursor.execute(“SELECT name FROM books ORDER BY pub_date DESC LIMIT 10”)
for row in cursor.fetchall():
print "<li>%s</li>" % row[0]
print "</ul>"
print "</body></html>"
connection.close()
I saved it as test.cgi to my web server. I run it by www.mysite.com/test.cgi unsuccessfully
Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
How can you solve the problem?
[edit] after the first answer
test.cgi is executable (I run $ chmod +x test.cgi)
I use Apache.
I have this in .bashrc export PATH=${PATH}:~/bin
Python module MySQLdb is installed.
The code does not have smart quotes.
[edit] after the second answer
you're getting that error because you
haven't installed the MySQLdb module
that Python needs to talk to a MySQL
database
I installed MySQLdb to my system. The module works, since I can import them.
However, I still get the same error whet I go to the www.[mysite].com/test.cgi.
[edit]
I am not sure about the questions
Are the connect() parameters correct? Is MySQL running on localhost
at the default port?
I run MySQL on my server. Is the question about the connect() parameters relevant here?
Is the SELECT statement correct?
You mean whether I have my SQL statements such as SELECT statement correct?
I have not used any SQL queries yet.
Do I need them here?
Any number of issues can cause the error you are seeing:
Is test.cgi executable (chmod 755) on the server?
Is the directory in which you placed test.cgi designated as a ScriptAlias location or have the ExecCGI option enabled (or equivalent if you're not using Apache)?
Is python in the system PATH or in the PATH in the Web server's startup environment?
Is the MySQLdb Python library installed?
Are the connect() parameters correct? Is MySQL running on localhost at the default port?
Is the SELECT statement correct?
If you're sure that python is found (test using the simplest possible script or by logging into the Web server if you can and typing which python) then you can get much better debug output by adding the following to the top of your script just below the shebang:
import cgitb
cgitb.enable()
More details: http://docs.python.org/library/cgitb.html
Additionally, if you have shell access to the Web server, try running python and just typing:
>>> import MySQLdb
If the command returns with no error, you have your answer for #4 above. If an error is printed, you will need to get MySQLdb installed into the Web server's Python installation.
EDIT: Looking more closely at the top of your question, I see that the code was scraped from an illustrative example at the very beginning of the Django Book. As such, I might expand #5 above to include the caveat that, of course, the requisite database, tables, user, and permissions need to be set up on the MySQL installation available to the Web server.
I've tidied up the code a bit by inserting linebreaks where necessary and replacing smart quotes with " and '. Do you have any more luck with the following? Can you run it from a terminal just by typing python test.cgi?
#!/usr/bin/env python
import MySQLdb
print "Content-Type: text/html"
print
print "<html><head><title>Books</title></head>"
print "<body>"
print "<h1>Books</h1>"
print "<ul>"
connection = MySQLdb.connect(user='me', passwd='letmein', db='my_db')
cursor = connection.cursor()
cursor.execute("SELECT name FROM books ORDER BY pub_date DESC LIMIT 10")
for row in cursor.fetchall():
print "<li>%s</li>" % row[0]
print "</ul>"
print "</body></html>"
connection.close()
The error in http://dpaste.com/8866/ is occurring because you are using "curly quotes" instead of standard ASCII quotation marks.
You'll want to replace the “ and ” with ". Just use find and replace in your text editor.
Make sure you're saving the file with correct line endings. i.e. LF only on unix, or CR/LF for Windows. I just recently had the exact same problem...