We Keep Coding

Problems connecting to MariaDB using the Python mysql.connector

I am trying to use Python 3.7 to connect to various MySQL and MariaDB databases using ver 8.0.18 of the mysql.connector (installed via pip as the mysql-connector-python package).
In this particular instance, I am trying to connect to a MariaDB 5.5.52 instance, but seem to be having the same problem on other systems.
If I attempt to connect thus:
cnx = mysql.connector.connect(user=c['user'], password=c['password'], host=c['host'], database=c['database'])
I get
mysql.connector.errors.ProgrammingError: 1045 (28000): Access denied for user '<user name>'#'<ip address>' (using password: YES)
The mysterious thing is that I can use a client application (JetBrains DataGrip) to connect from the same PC to the databases in question without any problems, so I am confident that the credentials are valid and there aren't any network or similar problems preventing the connection (i.e. port 3306 is open).
The only common factor I can find seems to be the mysql.connector. I've checked the manual and it looks like the syntax is correct.
UPDATE Following #makozaki's advice to use a different connector (pymysql) the code works. So it would definitely appear to be the mysql.connector that's the problem. I might try rolling it back to a previous version to see if that fixes it (unless anyone out there knows of a workaround).

I was able to fix this same error message for mysql.connector by adding in the additional parameter 'tls_versions':
cnx = mysql.connector.connect(user=c['user'], password=c['password'], host=c['host'], database=c['database'], 'tls_versions'=['TLSv1.1', 'TLSv1.2'])
This works because, as of the update 8.0.18, you can now specify the TLS version if it doesn't match the version of your database.

So far as I can tell, this turned out to be something to do with character encoding. The password I was using had some strange characters in it, including a British pound sign and an accented foreign (European) character.
For reasons I don't yet understand, the DataGrip client passed these without any problem, yet the mysql.connector somehow nobbled them.I suspect that this is something to do with encoding, although everything is set (or defaults to) utf-8.
I've changed the password(s) to ones encoded in base64 and the problem appears to have been solved, although I am frustrated that I haven't got to the bottom of why it occurred in the first place.

How do I access Production Datastore from my local development server?

I have a existing Website deployed in Google App Engine for Python. Now I have setup the local development server in my System. But I don't know how to get the updated DataBase from live server. There is no Export option in Google's developer console.
And, I don't want to read the data for each request from Production Datastore, I want to set it up locally for once. The google manual says that it stores the local datastore in sqlite file.
Any hint would be appreciated.

First, make sure your app.yaml enables the "remote" built-in, with a stanza such as:
builtins:
- remote_api: on
This app.yaml of course must be the one deployed to your appspot.com (or whatever) "production" GAE app.
Then, it's a job for /usr/local/google_appengine/bulkloader.py or wherever you may have installed the bulkloader component. Run it with -h to get a list of the many, many options you can pass.
You may need to generate an application-specific password for this use on your google accounts page. Then, the general use will be something like:
/usr/local/google_appengine/bulkloader.py --dump --url=http://your_app.appspot.com/_ah/remote_api --filename=allkinds.sq3
You may not (yet) be able to use this "all kinds" query -- the server only generates the needed statistics for the all-kinds query "periodically", so you may get an error message including info such as:
[ERROR ] Unable to download kind stats for all-kinds download.
[ERROR ] Kind stats are generated periodically by the appserver
[ERROR ] Kind stats are not available on dev_appserver.
If that's the case, then you can still get things "one kind at a time" by adding the option --kind=EntityKind and running the bulkloader repeatedly (with separate sqlite3 result files) for each kind of entity.
Once you've dumped (kind by kind if you have to, all at once if you can) the production datastore, you can use the bulkloader again, this time with --restore and addressing your localhost dev_appserver instance, to rebuild the latter's datastore.
It should be possible to explicitly list kinds in the --kind flag (by separating them with commas and putting them all in parentheses) but unfortunately I think I've found a bug stopping that from working -- I'll try to get it fixed but don't hold your breath. In any case, this feature is not documented (I just found it by studying the open-source release of bulkloader.py) so it may be best not to rely on it!-)
More info about the then-new bulkloader can be found in a blog post by Nick Johnson at http://blog.notdot.net/2010/04/Using-the-new-bulkloader (though it doesn't cover newer functionalities such as the sqlite3 format of results in the "zero configuration" approach I outlined above). There's also a demo, with plenty of links, at http://bulkloadersample.appspot.com/ (also a bit outdated, alas).

Check out the remote API. This will tunnel your database calls over HTTP to the production database.

Python 2.7->2.6 problems

I'm a beginner with Python and Django.
I'm setting up a program i've written locally. After almost finishing getting the app to work on the server, i've learned that the server is running python 2.6, while my local system runs 2.7. This is seemingly giving me problems when retrieving paramters from urls.
I'm using a server from Openshift. I don't know much about servers, but my current setup is that I have a local clone of the file, and I work on everything locally, and the push them via git to the server. The server was set up using a predefined quick setup from inside the Openshift interface.
I'm using the following urlpattern, which works just fine locally on my computer.
url(r'^website/(?P<url>[:\w/\-.]+)$', 'page'),
However, on my server version i'm running into some problems. The following url, returns two different urls to the view, depending on whether i'm on the server or running local.
#when using this url
website/http://example.com
#local view called page, retrives this argument
http://example.com
#server version retrieves almost the same, but with one / in the beginner less.
http:/example.com
It seems to me that a backslash is being chopped off somewhere. How can I change it to parse the argument with both backslashes?
# the receiving view
def page(request, url):
p = Page.objects.get(url=url)
domain = p.website.url
return render_to_response('seo/page.html', {'domain': domain, 'page': p}, context_instance=RequestContext(request))
The local version is returning the desired page just fine. The server version returns this:
DoesNotExist at /website/http:/coverme.dk/collections/iphone-sleeves-covers
I noticed that one of the backslashes in http:// was missing here, and assumed the error was based on it being sent to the view incorrectly.
I've just tested with an url that does not exist in the database on the local version, and it displays the error message correctly.
I've also double checked that the object for url='http://coverme.dk/collections/iphone-sleeves-covers' actually exists. I've also checked with several others.
I've experimented with messing around with the input url, and it seems to working just fine, except when I use double, triple of more backslashes. All backslashes succesively after the first are ignored in the url.
/website/http://////coverme.dk////collections/iphone-sleeves-covers
#gives the same as
/website/http:/coverme.dk/collections/iphone-sleeves-covers.
Any kind of help is much appreciated. A link to some documentation that could help me out would be greatly appreciated as well.
EDIT: Updating django solved this issue.

From a comment by the author of the question:
Using /website/http%3A%2F%2Fcoverme.dk%2Fcollections%2Fiphone-sleeves-covers as the url returns: The requested URL /website/coverme.dk/collections/iphone-sleeves-covers was not found on this server. Django version on the server is 1.4 and the one being used locally is 1.5.1. I've still to understand why i'm seeing different results locally and on the server, but i'm starting to think i should just switch to an url pattern that doesn't use //?
Updating Django solved the issue for me

Python ejabberd Auth Script not responding to changes in Database

I have an authentication script in ejabberd (XMPP server) that based off of THIS LINK
I have slightly modified the script so that instead of setting the variable out, it just returns true or false.
I'm using Ubuntu, MySQL, ejabberd, and Python.
I can authenticate all the records that are already on the database. But, when I add or remove records (I do this through phpMyAdmin), the script doesn't seem to know that the database has changed (I remove a user in phpMyAdmin and it still authenticates the user). The only time when the script recognizes the new records is when I restart or force-reload the ejabberd server. I've already been told its not a mySQL caching problem. I made sure I turned off external authentication caching for ejabberd.
That's all I can think of right now. I'll add more information if I can think of it. Any help is appreciated. I have no idea what is going on.
Addition: I turned on the MySQL logs, and all the queries there so there is not skipping queries.

I managed to fix this problem by changing the database engine back to MYISAM rather than INNODB. But I would like to know if this can be fixed for INNODB.
Edit: to fix it in innodb, set autocommit to true

Server Upgrade Script

Does anyone have or know of a good template / plan for doing automated server upgrades? In this case I am upgrading a python/django server, but am going to have to apply this update to many machines, and want to be sure that the operation is fully testable and recoverable should anything go wrong.
Am picturing something along the lines of:
remotely fetch new code
verify code download (e.g. hash of files)
take down server, display "you are upgrading dialog"
backup database(s)
backup code directory
apply new code updates
verify code update (e.g. hash of files)
apply database update (if necessary)
run tests
if success
startup server
verify server update
else
restore old database
restore old code
report error
startup server
verify server restore
I'm sure that this isn't exhaustive, and there are many other error conditions to consider, but am wondering if something like this already exists as a formalized process/best practices checklist to follow? Ideally this whole thing should of course be done by a single script call.

Once you have a plan (and yours looks pretty good), the Fabric site should be your next stop.

I think you're pretty much covering everything. Identify what's important to you and you're business practices: that's what counts.