Do mysql querys in python with zip file - python

Im trying to do some querys on my data base, but the file is to large and is spending to much time, exists some way to upload one zip file with those querys to my table?
ps: the file have between 350Mb to 500Mb


Store the zip file somewhere else on your server and simply store the name or the file location string in your dB.
Mysql really isn't intended to store large files or better yet zip folders.
Then when you go to retrieve it just unload the file location into an <a> tag it will link to it.

Related

What's the best way to handle large files upload with aiohttp (server)

I'm currently working on kind of cloud files manager. You can basically upload your files through a website. This website is connected to a python backend which will store your files and manage them using a database. It will basically put every of your files inside a folder and rename them with their hash. The database will associate the name of the file and its categories (kindof folders) with the hash so that you can retrieve the file easily.
My problem is that I would like the file upload to be really user friendly: I have a quite bad connection and when I try to download or upload a file on the internet I often get problems like at 90% of file uploading, the upload fails and I need to restart it. I'ld like to avoid that.
I'm using aiohttp to achieve this, how could I allow a file to be uploaded in multiple times? What should I use to upload large files.
In a previous code which managed really small files (less than 10MB), I was using something like that:
data = await request.post()
data['file'].file.read()
Should I continue to do it this way for large files?

What should I do to read dbf file and insert data to Oracle db?

What language (which programming) should I use to develop an .exe file to read dbf file and insert datas from it to oracle db??
The program will have a button.
I want to make a program that reads a dbf file saved in C drive and excute a query to insert datas of the dbf file to oracle db.
I learned web service is not an option since it cannot fetch files from one's computer.
My purpose is to upload datas to server which is made and saved in one's computer. Therefore, datas can be shared to public(web) by acessesing DB.
I found in google python can be an option.
Please give me a hint if you have the same experience or know the answer.
Thank you in advance.

How Do I store downloaded pdf files to Mongo DB

I download the some of pdf and stored in directory. Need to insert them into mongo database with python code so how could i do these. Need to store them by making three columns (pdf_name, pdf_ganerateDate, FlagOfWork)like that.

You can use GridFS. Please check this URL.
It will help you to store any file to mongoDb and get them. In other collection you can save file metadata.

import a set of text file to mysql using mysql or python code

I have a folder which contains a set of file .txt with the same structure.
The folder directory is E:\DataExport
Which contain 1000 .txt file: text1.txt, text2.txt,....
Each txt file contain price data of one. The data is updated on a daily. An example of one .txt file is as below
Ticker,Date/Time,Open,High,Low,Close,Volume
AAA,7/15/2010,19.581,20.347,18.429,18.698,174100
AAA,7/16/2010,19.002,19.002,17.855,17.855,109200
AAA,7/19/2010,19.002,19.002,17.777,17.777,104900
....
My question is, I want to:
Load all the file into mysql data base through a line of code, I can do it one by one in MySql command line but do not know how to import all at once, into one table in MySql (I have already create the table)
Everyday, when I get new data, how could I only update the new data to the table in MySql
I would like to get solution using either Python or MySql. I have tried to google and apply some solution but cannot success, the data do not load into mySQL

You could use python package pandas.
Read data with read_csv into DataFrame and use to_sql method with proper con and schema.
You will have to keep track of what is imported. You could for example keep it in file or database that last imported 54th line on 1032th file. And perform an update that reads the rest and imports that.

How to store and download a zip file in database postgres

I'm trying to store and then download a zip file from Postgres database. I know that this is not the best approach (i should only save the path to file) but i need to do this way, just for learning and practice.
I did a python script to store the content of the file into a bytea field but this was not my final goal. I really want to know how to save the zip file.
Any ideas? I just know python so i'm trying to this in python
Thank you guys!

If you can store the file as a bytea field then storing a zipped file is just the same.
Postgres doesn't have a concept of "file" field - you simply store the content (as you did for the original content) in bytea field.
If you're asking about zipping a file on the fly,
Take a look at zlib it's one of the common modules for such tasks.
Regards
Jony

Categories