mysqldbcompare and mysqldiff fails to compare - python

When I compare two databases, mysqluc stucks in halfway and it shows error as follows:
**Traceback (most recent call last):
File "G:\ade\build\sb_0-14553893-1424966082.93\Python-2.7.6-windows-x86-32bit\
lib\site-packages\cx_Freeze\initscripts\Console.py", line 27, in <module>
File "scripts\mysqldiff.py", line 245, in <module>
File ".\mysql\utilities\command\diff.py", line 149, in database_diff
File ".\mysql\utilities\command\diff.py", line 92, in object_diff
File ".\mysql\utilities\common\dbcompare.py", line 646, in diff_objects
File ".\mysql\utilities\common\dbcompare.py", line 466, in _check_tables_struc
ture
File ".\mysql\utilities\common\database.py", line 1206, in get_object_definiti
on
File ".\mysql\utilities\common\server.py", line 1263, in exec_query
File ".\mysql\connector\cursor.py", line 339, in close
mysql.connector.errors.InternalError: Unread result found.**
I have tried with both mysqldiff and mysqldbcompare command.
mysqldiff --server1=username:password#hostname:3307 --server2=username:pasword#localhost:3306 DB1:DB1 --force --difftype=sql --changes-for=server2
mysqldbcompare --server1=username:password#hostname:3307 --server2=username:pasword#localhost:3306 DB1:DB1 --run-all-test --difftype=sql changes-for=server2
It gets stuck on both commands.
I have tried to compared on both windows 7, windows 8.1 .

I found the problem. Database is Huge and some columns in tables are in "utf8" character set with "utf8_general_ci" collation and some in "latin1" with "latin1_swedish_ci" collation. So sometimes its getting stuck i don't know why. I changed the character set of every column to "latin1" now its not getting stuck.
Execute this query to check the character set of columns.
SELECT table_name,column_name,character_set_name,data_type FROM information_schema.`COLUMNS`
WHERE table_schema = "DbName"
For character set conversion you can take help from this link
http://dev.mysql.com/doc/refman/5.7/en/charset-conversion.html

Related

How to use dates in the yahoo_fin Python Package

I recently installed yahoo_fin and I tired the following example:
get_calls('NFLX' ')
It worked. I then tired the following:
get_calls('NFLX', '11/8/2019')
It failed. Here is what I got:
get_calls('NFLX', '11/8/2019')
Traceback (most recent call last):
File "", line 1, in
get_calls('NFLX', '11/8/2019')
File "C:\Users\rsher\Anaconda3\lib\site-packages\yahoo_fin\options.py", line 48, in get_calls
options_chain = get_options_chain(ticker, date)
File "C:\Users\rsher\Anaconda3\lib\site-packages\yahoo_fin\options.py", line 32, in get_options_chain
tables = pd.read_html(site)
File "C:\Users\rsher\Anaconda3\lib\site-packages\pandas\io\html.py", line 906, in read_html
keep_default_na=keep_default_na)
File "C:\Users\rsher\Anaconda3\lib\site-packages\pandas\io\html.py", line 743, in _parse
raise_with_traceback(retained)
File "C:\Users\rsher\Anaconda3\lib\site-packages\pandas\compat__init__.py", line 344, in raise_with_traceback
raise exc.with_traceback(traceback)
ValueError: No tables found
I am using version 3.6.3 of Python and I am also using Spyder.
Am I doing something wrong? Do you think I have found a bug?
I updated my version of yahoo_fin. Not really sure it was out of date. I now get the following error messages when I run the command: get_calls("nflx", "1/31/20")
Traceback (most recent call last):
File "", line 1, in
get_calls("nflx", "1/31/20")
File "C:\Users\rsher\Anaconda3\lib\site-packages\yahoo_fin\options.py", line 48, in get_calls
options_chain = get_options_chain(ticker, date)
File "C:\Users\rsher\Anaconda3\lib\site-packages\yahoo_fin\options.py", line 32, in get_options_chain
tables = pd.read_html(site)
File "C:\Users\rsher\Anaconda3\lib\site-packages\pandas\io\html.py", line 906, in read_html
keep_default_na=keep_default_na)
File "C:\Users\rsher\Anaconda3\lib\site-packages\pandas\io\html.py", line 743, in _parse
raise_with_traceback(retained)
File "C:\Users\rsher\Anaconda3\lib\site-packages\pandas\compat__init__.py", line 344, in raise_with_traceback
raise exc.with_traceback(traceback)
ValueError: No tables found
It should work the way you have it.
from yahoo_fin.options import get_calls
get_calls("nflx", "1/31/20")
Are you using the most recent version of yahoo_fin? It should be (as of this writing) version 0.8.4. Another possible issue is that there could have been a problem with Yahoo Finance's page for that option chain at that particular time.

How do I track down the string that is causing this mysql exception?

I have been brought on to fix some issues with a peice of software, I have never used mysql before, and when running one of the python scripts that interacts with mysql I get a error.
Here is the Traceback:
Assign scans to examsets
Traceback (most recent call last):
File "analyse.py", line 16, in <module>
scans.AssignScansToExamSets()
File "/home/connor/billing/billing/scans.py", line 1190, in AssignScansToExamSets
scanTable = scanTable, examSetsTable = examSetsTable)
File "/home/connor/billing/billing/funcautoassign.py", line 234, in AutoAssignScansToExamSets
debug = examSetsTable.Flush(log, startTime)
File "/home/connor/billing/billing/scandb.py", line 1175, in Flush
examSetIds = self.db.multiple_insert("examset", vals)
File "/usr/local/lib/python2.7/dist-packages/web.py-0.37-py2.7.egg/web/db.py", line 840, in multiple_insert
self._db_execute(db_cursor, q1)
File "/usr/local/lib/python2.7/dist-packages/web.py-0.37-py2.7.egg/web/db.py", line 587, in _db_execute
out = cur.execute(query, params)
File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 176, in execute
if not self._defer_warnings: self._warning_check()
File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 92, in _warning_check
warn(w[-1], self.Warning, 3)
_mysql_exceptions.Warning: Invalid utf8 character string: 'A1421C'
How do I find where that string is? And any advice on fixing this issue?
The mysql data base is currently using latin1 as the charset, which as far as I can tell should be fine with utf8 stuff. How can you set a mysql database to import into a particular charset?

Unable to ping managed nodes using ansible-2.0

I downloaded the ansible-2.0.0-0.2.alpha2.tar.gz and installed it on my control machine. However now I'm not able to ping any of my machines. Previously using v1.9.2 i could communicate with them. Now it gives the following error:
Unexpected Exception: lstat() argument 1 must be encoded string without NULL bytes, not str
the full traceback was:
Traceback (most recent call last):
File "/usr/bin/ansible", line 79, in
sys.exit(cli.run())
File "/usr/lib/python2.6/site-packages/ansible/cli/adhoc.py", line 111, in run
inventory = Inventory(loader=loader, variable_manager=variable_manager, host_list=self.options.inventory)
File "/usr/lib/python2.6/site-packages/ansible/inventory/init.py", line 77, in init
self.parse_inventory(host_list)
File "/usr/lib/python2.6/site-packages/ansible/inventory/init.py", line 133, in parse_inventory
host.vars = combine_vars(host.vars, self.get_host_variables(host.name))
File "/usr/lib/python2.6/site-packages/ansible/inventory/init.py", line 499, in get_host_variables
self.vars_per_host[hostname] = self.get_host_variables(hostname, vault_password=vault_password)
File "/usr/lib/python2.6/site-packages/ansible/inventory/__init.py", line 529, in get_host_variables
vars = combine_vars(vars, self.get_host_vars(host))
File "/usr/lib/python2.6/site-packages/ansible/inventory/__init_.py", line 653, in get_host_vars
return self.get_hostgroup_vars(host=host, group=None, new_pb_basedir=new_pb_basedir)
File "/usr/lib/python2.6/site-packages/ansible/inventory/__init_.py", line 702, in _get_hostgroup_vars
base_path = os.path.realpath(os.path.join(basedir, "host_vars/%s" % host.name))
File "/usr/lib64/python2.6/posixpath.py", line 365, in realpath
if islink(component):
File "/usr/lib64/python2.6/posixpath.py", line 132, in islink
st = os.lstat(path)
TypeError: lstat() argument 1 must be encoded string without NULL bytes, not str
Any help would be appreciated.
This is a known bug due to some Unicode changes made to the playbook parser in 2.0. Several versions of Python shipped with a version of shlex.split() that fails horribly on Unicode input- you likely have one of them installed. The bug has been worked around and will be included in the next drop. See https://github.com/ansible/ansible/issues/12257

SyntaxError using gdata-python to get worksheets feed

I'm getting an occasional error when trying to fetch a list of worksheets from gdata. This does not happen for all spreadsheets, but will consistently happen to the same spreadsheet for a period of several days to weeks. I suspected permissions, but was unable to find any special permissions for the spreadsheets that cause the error. I'm using OAuth2, gdata 2.0.18, and Python 2.6.8.
Traceback (most recent call last):
File "/mnt/shared_from_host/snake/base/fetchers/google_spreadsheet/common.py", line 176, in get_worksheet_list
feed = client.get_worksheets(spreadsheet_id)
File "/home/ubuntu/.virtualenvs/snakeenv/lib/python2.6/site-packages/gdata/spreadsheets/client.py", line 108, in get_worksheets
**kwargs)
File "/home/ubuntu/.virtualenvs/snakeenv/lib/python2.6/site-packages/gdata/client.py", line 640, in get_feed
**kwargs)
File "/home/ubuntu/.virtualenvs/snakeenv/lib/python2.6/site-packages/gdata/client.py", line 278, in request
version=get_xml_version(self.api_version))
File "/home/ubuntu/.virtualenvs/snakeenv/lib/python2.6/site-packages/atom/core.py", line 520, in parse
tree = ElementTree.fromstring(xml_string)
File "<string>", line 86, in XML
SyntaxError: no element found: line 1, column 0
This seems to be from the request getting an empty string as the response.
Does anybody have any idea on why this might not work, or troubleshooting ideas? Thanks.

Server Error (Google Apps Engine) when executing custom bulkloader

I had not succeeded in using the bulkloader option that is officially provided, so i wrote my own bulkloader script (actually post handler that would incrementally load entities from csv into the datastore).
The solution worked as follows:
1. I would copy data from a csv file
2. Paste it into a text area in a form on the app
3. Post the form
4. The handler parses the incoming text for headers (column names)
5. Stores rest of lines in list
6. incrementally fetches 100 lines from the list and for each line creates and stores an entity (the type of entity is resolved from the type specified from a select field from the form)
Now, this technique would work on my dev server for small inputs of up to 1000 lines, beyond that it would show the following error:
Traceback (most recent call last):
File "/host/_Hive/Lab/ACTIVE WORKS/UG VOTES/google_appengine/google/appengine/ext/webapp/init.py", line 513, in call
File "/host/_Hive/Lab/ACTIVE WORKS/UG VOTES/google_appengine/myApps/ugvotes/ugvotes.py", line 241, in post
File "/host/_Hive/Lab/ACTIVE WORKS/UG VOTES/google_appengine/google/appengine/ext/db/init.py", line 893, in put
File "/host/_Hive/Lab/ACTIVE WORKS/UG VOTES/google_appengine/google/appengine/api/datastore.py", line 291, in Put
File "/host/_Hive/Lab/ACTIVE WORKS/UG VOTES/google_appengine/google/appengine/api/datastore.py", line 195, in _MakeSyncCall
File "/host/_Hive/Lab/ACTIVE WORKS/UG VOTES/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 499, in check_success
File "/host/_Hive/Lab/ACTIVE WORKS/UG VOTES/google_appengine/google/appengine/api/apiproxy_rpc.py", line 149, in _WaitImpl
File "/host/_Hive/Lab/ACTIVE WORKS/UG VOTES/google_appengine/google/appengine/api/datastore_file_stub.py", line 863, in MakeSyncCall
File "/host/_Hive/Lab/ACTIVE WORKS/UG VOTES/google_appengine/google/appengine/api/apiproxy_stub.py", line 80, in MakeSyncCall
File "/host/_Hive/Lab/ACTIVE WORKS/UG VOTES/google_appengine/google/appengine/api/datastore_file_stub.py", line 933, in _Dynamic_Put
File "/host/_Hive/Lab/ACTIVE WORKS/UG VOTES/google_appengine/google/appengine/api/datastore_file_stub.py", line 806, in __WriteDatastore
File "/host/_Hive/Lab/ACTIVE WORKS/UG VOTES/google_appengine/google/appengine/api/datastore_file_stub.py", line 836, in __WritePickled
IOError: [Errno 24] Too many open files: '/tmp/tmpOfgvm3'
At first I thought this was an error due to the limitations of the development server, but when i tried to do the same task from the
production server, i got the following error:
Error: Server Error
The server encountered an error and could not complete your request.
If the problem persists, please report your problem and mention this error message and the query that caused it.
Who knows what could have gone wrong, and what i can do about it?
thanks.
This is a known issue: Python dev SDK 1.3.8 datastore_file_stub.py appears to leak file handles.
There's an unofficial patch available.

Categories