For fetching data from databases, I usually see people create a class in models.py and they migrate it, and then they fetch data from databases and show it on the front-end.
my question is, i already have a database which has student_info table and has lots of data.
I don't want to define student_info class in models.py and do migration. Instead, since i already have student_info data in sql database. so I want to fetch it directly. can i do so?
one method i know of doing the same, is directly connecting sql server in views.py as;
mydb=mysql.connector.connect(host="localhost",user="root",password="******",database=database_name)
and fetching the data and passing it as variable in index.html. But the problem is, it fetches data only once, and if i refresh the page where local server is running, all the content will disappear, and server will report Broken pipe from ('127.0.0.1', 59579) and to see the data, again i need to restart the sever.
Kindly looking for a better approach to show data on front-end from the existing databases
What you could do is a simple thing for the solution.
Create a StudentData(models.Model) class which will store your data in them as model objects And define all the attributes/ columns in models according to the data and which is in the student_info table.
Write a simple script ( using pymysql ) to retrive all the data from the student_table , store it in text file . ( This part is easy )
take that text file and now in your django project , run a shell command or create a new script to transfer the text_file student data to model objects and save them .
Now you have your data .
I think this will work fine .
If you also want code . I'll include it.
The code for the solution :
An Update : I have used pymysql for this scripts , i think mysqlconnector will also do the work.
Create a folder inside your project named "Creating Database model" ( you can choose whatever you want but I created the files and all script inside this folder )
first_file : "data retreiving.py"
import pymysql as psql
import os
connection = psql.connect(
user = "root" ,
password = os.environ.get("MYSQL_PASSWORD") ,
database = "college" ,
)
cursor = connection.cursor()
cursor.execute("select * from student") ;
data_text_file = open("Creating Database model/text_data.txt" , "w")
for row in cursor :
temp_string = ""
for data in row :
temp_string+= "{} ".format(str(data))
data_text_file.write(temp_string+"\n")
data_text_file.close()
Enter your password for connection. Here I am using college database which has student table with some student data .
Then I am creating a file text_data.txt which will have the text data of the table .
I used string formatting here for convinence and better retrival of data on the other side.
My student table schema has five fields :
id , first_name, last_name , age , branch
so my models.py according to this database is here . You create your own according to your requirement.
from django.db import models
class StudentData(models.Model) :
first_name = models.CharField(max_length = 100 , required = True)
last_name = models.CharField(max_length = 100 , required = True)
age = models.IntegerField(max_length = 100 , required = True)
branch = models.CharField(max_length = 100 , required = True)
def __str__(self) :
return "{} {}".format(first_name , last_name)
second file : "data entry in models.py"
This file takes data and then saves it to the model . And you get your database in django .
with open("Creating Database model/text_data.txt") as f :
data = f.read()
new_data_list = []
data = data.split("\n")
new_data_list = []
for i in data :
temp_list = i.split(" ")[:-1]
new_data_list.append(temp_list)
data = new_data_list[:-1]
for i in data :
student_info = StudentModel(
first_name = i[1] ,
last_name = i[2] ,
age = i[3] ,
branch = i[4]
)
studnet_info.save()
These all are within the "Creating Database model" folder.
This will do all the work you require. Make sure to debug and understand before committing to database.
Related
I working in a project in which i have different projects with the same database architecture,
so i used peewee Model in which:
dynamic_db = SqliteDatabase(None)
class BaseModel(Model):
class Meta:
database = dynamic_db
class KV (BaseModel):
key = TextField()
value = IntegerField()
And whenever i new project is created i will call a function
dynamic_db.init(r'{}\database.db'.format(ProjectName.upper()))
dynamic_db.connect()
dynamic_db.create_tables([KV])
dynamic_db.close()
The problem is that once this database is created, i can't access with peewee.
When i try to create a record:
KV.create(key = 'Saul', value = 123)
I get this error:
peewee.InterfaceError: Error, database must be initialized before opening a connection.
I would appreciate any help or cookbook for peewee.
I believe something is incorrect, either in your question description, or in the error you are receiving. The call you are making to .init() is what initializes the database. After that, you should have no problems using it.
Full example which works fine:
from peewee import *
db = SqliteDatabase(None)
class Base(Model):
class Meta:
database = db
class KV(Base):
key = TextField()
value = IntegerField()
db.init('foo.db') # database is now initialized
db.connect()
db.create_tables([KV]) # no problems.
db.close()
I was finally able to create a record.
I didn't mention that i was trying to create them in another file, but the procedure is the same as the one coleifer posted on the answer.
The file in which i create the peewee models is databases.py, so in the other file i do the following:
import databases
databases.db.init('foo.db')
databases.KV.create(name = 'Saul', value= 123)
Thanks!
First off, I'm new to Django and Python overall - so I might be asking the wrong question, if that's the case please tell me where to look. If not, continue:
I'm working with a Django app that queries a Wordpress database, I have created a connection and wrote the query as such:
cnxn = MySQLdb.connect(host="localhost",
user="root",
passwd="password",
db="dbname")
query = '''SELECT post_id, label, meta_value
FROM table'''
The issue is that the data is in a flat format, where the label should be the column name and the meta_value should be the value, output:
post_id label meta_value
------- ----- ----------
1 name example name
1 email example#mail.com
2 name example name 2
2 email example2#mail.com
Please keep in mind that there are 24 labels for each post_id, so each form entry adds 24 lines to the table.
How would I got about creating a Django model that appends the data from the table in a pivot style format to be able to display it nicely in the template? The goal is to be able to display 4-5 labels on the main page with a link on each line that shows the full input on a separate page.
Desired output:
post_id name email
------- ---- -----
1 example name example#mail.com
2 example name 2 example2#mail.com
Thank you!
I ended up converting the initial output to a pivot using Pandas, then uploaded the df into a new model that I created with the same exact field names.
def users(request):
df = pd.read_sql(query, cnxn)
pvt = df.pivot(columns='label', index='post_id', values='meta_value')
pvt = pvt.reset_index()
pvt['id'] = pvt.index
pvt.to_sql('table_name', cnxn, if_exists='replace')
The model configured includes all the fields in the queried table, so the outcome matched the model and I can treat it as an object.
Hope this helps.
I am currently working with Access 2013. I have built a database that revolves around applicants submitting for a Job. The database is set up so that a person can apply for many different jobs, when the same person applies for a job through our website (uses JotForms) it automatically updates the database.
I have a Python script pulling the applicants submission information from an email which updates the database. The problem that I am running into is that within the database I have the applicants primary email set to "no duplicates", thus not allowing the same person to apply for many different jobs as the Python script is trying to create a new record within the database causing an error.
Within my Access form (VBA) or in Python what do I need to write to tell my database if the primary emails are the same only create a new record within the position applied for table that is related to the persons primary email?
Tables:
tblPerson_Information tblPosition_Applied_for
Personal_ID (PK) Position_ID
First_Name Position_Personal_ID (FK)
Last_Name Date_of_Submission
Clearance_Type
Primary_Phone
Primary_email
Education_Level
Simply look up the email address in the [tblPerson_Information] table:
primary_email = 'gord#example.com' # test data
crsr = conn.cursor()
sql = """\
SELECT Personal_ID FROM tblPerson_Information WHERE Primary_email=?
"""
crsr.execute(sql, (primary_email))
row = crsr.fetchone()
if row is not None:
personal_id = row[0]
print('Email found: tblPerson_Information.Personal_ID = {0}'.format(personal_id))
else:
print('Email not found in tblPerson_Information')
I have code that works for getting models and fields from a Django database. However, it only works on the default database.
This function wants a database name, and I'd like to get the tables and fields for that database.
def browse_datasource(request, dbname):
table_info = []
# This is what I'd /like/ to be able to do, but it doesn't work:
# tables = connections[dbname].introspection.table_names()
tables = connection.introspection.table_names()
found_models = connection.introspection.installed_models(tables)
for model in found_models:
tablemeta = model._meta
columns = [field.column for field in model._meta.fields]
table_info.append([model.__name__, columns])
How can I perform introspection on the non-default databases? Is there a correct way to get connection.introspection for a database with the name "example", for example?
I found the solution. The trick is getting the database connection from the connections list, then getting a cursor and passing that to introspection.table_names, like so:
table_info = []
conn = connections[dbname]
cursor = conn.cursor()
tables = conn.introspection.table_names(cursor)
found_models = conn.introspection.installed_models(tables)
for model in found_models:
tablemeta = model._meta
I'm fairly new to Django and I'm trying to add some 'host' data to 'record' using django's hook for using SQL to initialise (a SQL file in lowercase in the app folder & sql subfolder)
Here's the models:
class Record(models.Model):
species = models.TextField(max_length = 80)
data=models.TextField(max_length = 700)
hosts = models.ManyToManyField('Host')
class Host(models.Model):
hostname = models.TextField()
I've used a ManyToManyField as each record should be able to have multiple hosts, and hosts should be 'reusable': ie be able to appear in many records.
When I'm trying to insert via SQL I have
INSERT INTO myapp_record VALUES ('Species name', 'data1', XYZ);
I'm not sure what to put for XYZ (the ManytoMany) if I wanted hosts 1, 2 and 3 for example
Separating them by commas doesn't work obviously, and I tried a tuple and neither did that.
Should I be trying to insert into the intermediary table Django makes? Does that have a similar hook to the one I'm using? If not, how can I execute SQL inserts on this table?
The use of initial SQL data files is deprecated. Instead, you should be using a data migration, which might look something like this:
from django.db import models, migrations
def create_records(apps, schema_editor):
# We can't import the Person model directly as it may be a newer
# version than this migration expects. We use the historical version.
Record = apps.get_model("yourappname", "Record")
Host = apps.get_model("yourappname", "Host")
host1 = Host.objects.get(hostname='host1')
record = Record.objects.create(name='Species name', data='Data')
record.hosts.add(host1)
...etc...
class Migration(migrations.Migration):
dependencies = [
('yourappname', '0001_initial'),
]
operations = [
migrations.RunPython(create_records),
]