inserting big json file into mongodb - python

I am working on flask and mongodb. I am new to mongodb, I realized that mongodb do not let me insert json file to database, the file size is greater than 16mb
Is there any other database that is similar to mongodb (JSON object based) ?
Thank You

You can make use of PostgreSQL also for json kind of data to store, based on your Application requirements.
Please refer their official documentation for more information.
Hope the below links can be useful.
1.Postgre Data Types
2.Postgre SQL and Json

Related

Export mongodb data to json

After running the mongodb docker image I created a database called MovieFlix and some collections with items inside the database.
What I want is to find a way to store all the data from the MovieFlix db in a json file to have it saved for later user for docker-compose.
Should I do it with python code using pymongo or is there a simpler way ?
Mongodb has a command line tool for exporting to json or csv. The keyword for this is dump. Hope it helps!

import json data from an api to mysql database in python

I need to import JSON data from an API to MySQL database using python. I can get the json data in python script but no idea how to insert this data to a mysql database.
finally, did it. Saved the JSON file locally first, then parsed it using key values in a for loop & lastly ran a query to insert into MySQL table.
Serializing JSON to string and insert string to DB, and make it Deserializing when you want to use it. I use this method but I don't know if it is optimum
Working With JSON Data in Python

Pulling Files from Mongo DB into CSV using python

I am new to python and I was trying to pull the data from mongodb using python and import it into a csv file.Can some one tell me from where should I start.
I am so new to both the technologies,But trying to learn.
If someone could help me it would be really helpful.
Check out pymongo: http://api.mongodb.com/python/current/tutorial.html
The are also some great introductory videos to MongoDB on youtube demonstrating basic CRUD operations (create, read, update, delete) and working in the mongo shell, which is the core interface to MongoDB.

Storing unstructured data with ramses to be searched with Ramses-API?

I would like to give my users the possibility to store unstructured data in JSON-Format, alongside the structured data, via an API generated with Ramses.
Since the data is made available via Elasticsearch, I try to achieve that this data is indexed and searchable, too.
I can't find any mentioning in the docs or searching.
Would this be possible and how would one do it?
Cheers /Carsten
I put an answer here because needed to give a several docs links and this is a new SO account limited to a couple: https://gitter.im/ramses-tech/ramses?at=56bc0c7a4dfe1fa71ffc0b61
This is Chrisses answer, copied from gitter.im:
You can use the dict field type for "unstructured data", as it takes arbitrary json. If the db engine is postgres, it uses jsonfield under the hood, and if the db engine is mongo, it's converted to a bson document as usual. Either way it should index automatically as expected in ES and will be queryable through the Ramses API.
The following ES queries are supported on documents/fields: nefertari-readthedocs-org/en/stable/making_requests.html#query-syntax-for-elasticsearch
See the docs for field types here, start at the high level (ramses) and it should "just work", but you can see what the code is mapped to at each level below down to the db if desired:
ramses: ramses-readthedocs-org/en/stable/fields.html
nefertari (underlying web framework): nefertari-readthedocs-org/en/stable/models.html#wrapper-api
nefertari-sqla (postgres-specific engine): nefertari-sqla-readthedocs-org/en/stable/fields.html
nefertari-mongodb (mongo-specific engine): nefertari-mongodb-readthedocs-org/en/stable/fields.html
Let us know how that works out, sounds like it could be a useful thing. So far we've just used that field type to hold data like user settings that the frontend wants to persist but for which the API isn't concerned.

how will Python DB-API read json format data into an existing database?

If we have a json format data file which stores all of our database data content, such as table name, row, and column, etc content, how can we use DB-API object to insert/update/delete data from json file into database, such as sqlite, mysql, etc. Or please share if you have better idea to handle it. People said it is good to save database data information into json format, which will be much convenient to work with database in python.
Thanks so much! Please give advise!
There's no magic way, you'll have to write a Python program to load your JSON data in a database. SQLAlchemy is a good tool to make it easier.

Categories