I would like to copy the contents of a MySQL database from one server to another using a third server. This could be done from the shell prompt using this:
mysqldump --host=hostname1 --user=username --password="mypwd" acme | mysql --host=hostname2 --user=username --password="mypwd" acme
However, how do I do this from within a Python script without using os.system or any of the other subprocess methods? I've read through the MySQLdb docs, but don't see a way to do a bulk export/import. Thank you!
If you dont want to use mysqldump from the command line (using the os.system methods) you are kind of tied to get the data straight from MySQL and then put it to the other server. In that respect your question looks very similar to Get Insert Statement for existing row in MySQL
you can use a query to get the schema creation sql
SHOW CREATE TABLE MyTable;
And then you need to implement a script that just querys data and inserts it to the other server.
You could also look into third party applications that allows you to copy data from one database to another.
Related
I have basic csv report that is produced by other team on a daily basis, each report has 50k rows, those reports are saved on sharedrive everyday. And I have Oracle DB.
I need to create autoscheduled process (or at least less manual) to import those csv reports to Oracle DB. What solution would you recommend for it?
I did not find such solution in SQL Developer, since it is upload from file and not a query. I was thinking about python cron script, that will autoran on a daily basis and transform csv report to txt with needed SQL syntax (insert into...) and then python will connect to Oracle DB and will ran txt file as SQL command and insert data.
But this looks complicated.
Maybe you know other solution that you would recommend yo use?
Create an external table to allow you to access the content of the CSV as if it were a regular table. This assumes the file name does not change day-to-day.
Create a scheduled job to import the data in that external table and do whatever you want with it.
One common blocking issue that prevents using 'external tables' is that external tables require the data to be on the computer hosting the database. Not everyone has access to those servers. Or sometimes the external transfer of data to that machine + the data load to the DB is slower than doing a direct path load from the remote machine.
SQL*Loader with direct path load may be an option: https://docs.oracle.com/en/database/oracle/oracle-database/19/sutil/oracle-sql-loader.html#GUID-8D037494-07FA-4226-B507-E1B2ED10C144 This will be faster than Python.
If you do want to use Python, then read the cx_Oracle manual Batch Statement Execution and Bulk Loading. There is an example of reading from a CSV file.
Good day to All,
I need to communicate to DBs like here is the sample using python & peewee.
First i need to put an entry in the types table of the MainDb. only the Name & type am having & the ID will get auto increment after the entryin the table, then using the auto incremented id(MainID), i need to put a data entry in respective type table of the respective type db.
then i need to do some operation using the select queries with MainID in the python code and finally i need to delete the entries of the MainID in the MainDb as well as in the respective type db, if the operations in the python code gets executed successfully or not (throws any error or exception).
Currently i am using the simple db (MySQLdb) connection & close with cursor to execute the queries & to get the last auto incremented id am using lastrowid of the cursor(cursor.lastrowid).
And also i referred the question in stackoverflow as per my understanding its ok if i have an single db (like main db alone) but what am i need to do for my situation.
OS: Windows7 64bit,
Db: MySQL Db v5.7,
Python : v2.7,
peewee : v2.10
Thanks in advance
I have a python script( script 1) that adds elements in a table in MySQL database using mysql.connector module, and I have another python script( script 2) that reads data from that table, I want to put a trigger on that table and each time a new element is added script 2 notice the new added element and display a message on the console.
I'm looking for something like the nodeJS npm package "mysql-events" that watches a MySQL database and runs callbacks on matched functions
I know this is an old question. But I was searching the same today for my work and came across this link - mysql - Run python script on Database event. Look at the answer which has been submitted by the person asking question. It might help you/someone.
I'm writing an application in Python and I'm using the Adodbapi package to handle my sql queries. For one part of the code, I need to communicate with an Access '97 (no, it's not a typo) database, and I want to get a list of the table names via an SQL statement. I've tried a bunch of statements using MSysObjects but I run into permissions isssues - no I don't have the option of opening the db in access and changing the permissions. I would imagine that Adodbapi has a way of accessing Schema information but I can't manage to find it.
Thanks in advance for your help.
I'm using python and psycopg2 to remotely query some psql databases, and I'm trying to figure out the best way to select the data I need from the remote table, and insert it into a table on a separate DB (local application server).
Most of the stuff I've read has directed me to avoid executemany and look toward COPY operations, but I'm unsure how to implement this on a specific select statement as opposed to the entire table. Should I be headed this way or am I completely off?
but I'm unsure how to implement this on a specific select statement as opposed to the entire table
COPY isn't limited to tables, you can use a query as the source as well, check out the examples in the manual, it shows how to use COPY to create a text file based on a query:
http://www.postgresql.org/docs/current/static/sql-copy.html#AEN59055
(3rd example)
Take a look at http://ryrobes.com/featured-articles/using-a-simple-python-script-for-end-to-end-data-transformation-and-etl-part-1/
Granted, this is pulling from Oracle and inserting into SQL Server, but the concepts should be the same.