How can i use Hive on Cassandra with Python coding - python

I am using Python and my data is stored on Cassandra.But I want to write query with Hive(maybe it will faster than). But I can not found any API or library.Have you any suggestion for make this combination?

Related

How can I perform database CRUD operations with text files with Python?

I've made an address book app using MySQL in PHP but now I want to make it in Python using a text file. Is it possible to perform CRUD operations with a text file instead of a database?
You can use the shelve module that's in the Python standard library. It basically gives you a dictionary that is easy to save to a file. However, you don't get a lot of relational database features like joining tables; it's just a key-value dictionary.
The documentation for it is at https://docs.python.org/3/library/shelve.html
This doesn't really scale as well as using a database though.

Extract webscraped python data to SQLite, excel or xml?

I'm kinda new to Python and webscraping but I'm currently at a point where I need to extract data to a database. Can someone tell me the pros and cons by using sqlite, excel or xml?
I've read that sqlite should be the fastest, so I may go for that database structure, but can someone then tell me what IDE you use to handle sqlite data after I've extracted it from python?
Edit: I hope my post makes sense. I'm currently trying to use a web scraper from here: https://github.com/gingeleski/odds-portal-scraper
Thanks in advance.
For the short term, Excel is a good way to examine your data and prototype analysis and visualizations. It gets old using it for very large datasets, or multiple similar datasets. Basically as soon as you start doing the same thing more than twice or writing VB code you should switch to the pandas/matplotlib solution.
It looks like the scraper you are using already puts the results in an SQLITE database, but if you have your data in a list or dictionary, I'd suggest using pandas to do calculations and matplotlib for visualizations, as that will give you a robust, extensible solution over the long term. It is very easy to read and write data between an SQLITE database and pandas.
A good way of viewing the data in the DB is a must. I'm currently using SQLiteStudio.
When you say IDE, I'm assuming you're looking for a way to view the SQLite data? If so, DBeaver is a free, open source SQL client. You could use this to view the data quite easily.

How to import data from mongodb to MySQL?

More specifically , can i use some bridge for this like first i should copy data to excel fro mongodb and then that excel sheets data could easily be imported into mysql by some scripts like as in Python.
MongoDB does not offer any direct tool to do this, but you have many options to achieve this.
You can:
Write your own tool using your favorite language, that connect to MongoDB & MySQL and copy the data
Use mongoexport to create files and mysqlimport to reimport them into MySQL
Use an ETL (Extract, Transform, Load) that connect to MongoDB, allow you to transform the data and push them into MySQL. You can for example use Talend that has connector for MongoDB, bu you have many other solutions.
Note: Keep in mind that a simple document could contains complex structures such as Array/List, sub-documents, and even an Array of sub-documents. These structures can not be imported directly into a single table record, this is why most of the time you need a small transformation/mapping layer.

Using Data in MongoDB through Pymongo(Python)

I am interested in but am ignorant to the best method of extracting quantitative data fast and efficiently that I have inserted into MongoDB.
I will explain my process. I used MongoDB to hold a variety of quantiative data that I inserted from multiple .log files.
Now that the information is inserted, I would like to extract certain data through queries, format it into an array, and display it in a form of a GUI (matplotlib).
I am confused on how to go about the best method of extracting the data. Thank you.
There are some good tutorials on using the python and mongodb such as this http://api.mongodb.org/python/current/tutorial.html
Also there is some more information on SO on matplotplib with mongodb such as this one Mongodb data statistics visualization using matplotlib
It's probably better to start trying some things and then asking specific questions on SO when you get stuck.
use the standard query API
use Map-Reduce
use the new MongoDB aggregation framework

Use Python to load data into Mysql

is it possible to set up tables for Mysql in Python?
Here's my problem, I have bunch of .txt files which I want to load into Mysql database. Instead of creating tables in phpmyadmin manually, is it possible to do the following things all in Python?
Create table, including data type definition.
Load many files one by one. I only know this LOAD DATA LOCAL INFILE command to load one file.
Many thanks
Yes, it is possible, you'll need to read the data from the CSV files using CSV module.
http://docs.python.org/library/csv.html
And the inject the data using Python MySQL binding. Here is a good starter tutorial:
http://zetcode.com/databases/mysqlpythontutorial/
If you already know python it will be easy
It is. Typically what you want to do is use an Object-Retlational Mapping library.
Probably the most widely used in the python ecosystem is SQLAlchemy, but there is a lot of magic going on in it, so if you want to keep a tighter control on your DB schema, or if you are learning about relational DB's and want to follow along what the code does, you might be better off with something lighter like Canonical's storm.
EDIT: Just thought to add. The reason to use ORM's is that they provide a very handy way to manipulate data / interface to the DB. But if all you will ever want to do is to do a script to convert textual data to MySQL tables, than you might get along with something even easier. Check the tutorial linked from the official MySQL website, for example.
HTH!

Categories