First question so please excuse any formalities I miss.
I am developing a system - the backend in Python, the frontend in PHP. The system will leverage a MongoDB database for a number of reasons. I plan to use an Object Document Mapper to make database lookups, and relationships, between pieces of data relatively trivial.
Adding this information as requested
The plan I envision is that the PHP interface would allow users to view the data, and navigate through to connected information, leveraging the relationship defined in the database. For example
Organisation > IP Ranges > IPs > TCP ports open on this IPs
The Python side is the system collecting this information, entering it into the database, and defining/establishing the relationship between the data (i.e. this IP range belongs to this organisation)
I can see that there are many options for both PHP, and Python, independently - but there dont appear to be any solutions that support both PHP and Python at the same time. Is there a way I can utilise the same ODM across both languages to ensure that data can be referenced/'looked up' in the same manner - or, is there a better solution for this that I am unaware of?
perhaps the simplified version of the question is - is there a language agnostic way I can/should be querying data, with defined relationships in that data, with mongodb?
Thanks! :-)
Related
I am an 11th grade student and I'm learning how to build a web app, with my teammates. Currently, We're making a website showing the school schedule (also to show students' marks) and helping the users to create their to-do lists, of course this web serves students like me. In the backend of the web, we use Python as the main language, Flask as the framework and MySQL to manipulate our database. Now, everything is ok and we're trying to make something like an admin interface for people who host the web. Specifically, it is where teachers can insert their students' grade, and maybe adjust the school timetable. The problem is, we're learning how to use Flask Admin to code that function, and we've found out this tech is only compatible with SQL Server. However, we have a better understand in MySQL therefore we could create multiple tasks, in contrast, we don't know how to use SQL Server to create those funcs. Now I have 2 main questions:
Could we use 2 different SQL in the backend of our web? It is the quickest way we know, however we have to learn how to use SQL Server.
Could we use 2 different Python backend frameworks in the backend of our web? We haven't searched which framework to use yet because we don't know if it's possible to do this.
We don't know any other ways to solve this problem except getting rid of MySQL and use SQL Server instead. However this is not the way we prefer and we hope those 2 questions answered. If there is anything wrong in our knowledge please just straightly comment to let us know, and we greatly welcome any other solutions. Thanks for answering!!
Directly from the flask-admin docs https://flask-admin.readthedocs.io/en/latest/advanced/#using-different-database-backends is the following:
Using Different Database Backends Other than SQLAlchemy… There are
five different backends for you to choose from, depending on which
database you would like to use for your application. If, however, you
need to implement your own database backend, have a look at Adding A
Model Backend.
If you don’t know where to start, but you’re familiar with relational
databases, then you should probably look at using SQLAlchemy. It
is a full-featured toolkit, with support for SQLite, PostgreSQL,
MySQL, Oracle and MS-SQL amongst others. It really comes into its own once you have lots of data, and a fair amount of relations between
your data models. If you want to track spatial data like
latitude/longitude points, you should look into GeoAlchemy, as well.
Regarding the original question, it is possible to use two different frameworks in the backend of a web app. One way to do so would be to set up a reverse proxy server (see https://www.nginx.com/resources/glossary/reverse-proxy-server/#:~:text=A%20reverse%20proxy%20server%20is,traffic%20between%20clients%20and%20servers.), but I would recommend giving SQLAlchemy before doing so.
Why do you think that flask-admin is tied to SqlServer? Flask (and flask-admin) can handle different connections to various databases:
https://github.com/flask-admin/flask-admin#introduction
https://flask-admin.readthedocs.io/en/v1.0.9/db/
My guess is you are currently using SqlAlchemy. As explained here, you can use different backends:
The string form of the URL is dialect[+driver]://user:password#host/dbname[?key=value..], where dialect is a database name such as mysql, oracle, postgresql, etc., and driver the name of a DBAPI, such as psycopg2, pyodbc, cx_oracle, etc.
Alternatively, the URL can be an instance of URL.
(https://docs.sqlalchemy.org/en/13/core/engines.html#sqlalchemy.create_engine)
More on Engine here
Flask-admin is a admin view on your database tables - you cannot run it on a "different" db then the tables it should modify. It needs to have access to the database tables you want it to modify, so you cannot run "admin" on MS and "your data backend" on another database(-connection).
Some other things to think about:
MS-(T-)Sql and MySql are infrastructural choices, language wise they are very closely related so adapting MySql knowledge to T-SQL Syntax should be possible. Choosing SqlServer may force you to license it - and afaik that comes with fees (eiter on premise or as a azure subscription wich might or might not be free for schools (no idea - but you should check that)).
This sounds like an ambitious project for school - depending on where you live, privat data protection laws come into play especially considering you connect names with schedules with grades which would need you to implement a lot more to comply.
For first question, I checked flask-admin documentation and found that the framework already included serval built-in ORM library, i.e. SQLAlchemy, MongoEngine, pymongo and Peewee. This means you can just directly import the ORM library from the flask-admin package and use it to access your database. For your case, you should look for SQLAlchemy as you are using SQL Database. Both SQL Server and MySQL are supported by SQLAlchemy.
The Adding Model Views section in their official doc also mentioned it as well:
https://flask-admin.readthedocs.io/en/latest/introduction/#getting-started
For the second question, it is technically not possible to apply two different frameworks in one single backend application.
I was hoping someone with more in-depth knowledge of MongoDB could provide some feedback on the implementation for my database requirements.
I am currently implementing a web application using Flask and MongoDB for my client who's business has multiple physical locations. They, as the Managing Director, want to be able to access information for each of the locations individually but they want to restrict the managers of each location to just their information.
I can envisage two different solutions to this but am not familiar enough with the underlying technology/design of MongoDB to fully appreciate the pros and cons of each implementation.
For each solution I have a user model that contains which location(s) a user has access to.
Solution 1 would involve each document having a field to record which location the object belonged to and then including that as a filter in every query. This seems the simplest to implement but would result in a single large database, could there be scaling issues?
Solution 2 involves having a separate user database, and an individual database for each location. When a request is made the application would have to establish from the user database which location they are accessing and connect to that database. This at first appears more complex to implement but would result in smaller isolated databases which may make it easier to scale but possibly harder to manage.
I would greatly appreciate any feedback on my solutions or any other solutions I have not considered.
Many thanks in advance.
At my work, we use Oracle for our database. Which works great. I am not the main db admin, but I do work with it. One thing I like is that the DB has a built in logic layer using PL/SQL which ca handle logic related to saving the data and retrieve it. I really like this because it allows our MVC application (PHP/Zend Framework) to be lighter, and makes it easier to tie in another platform into the data, such as desktop or mobile.
Although, I have a personal project where I want to use couchdb or mongodb, and I want to try and accomplish a similar goal. outside of the mvc/framework, I want to have an API layer that the main applications talk to. they dont actually talk directly to the database. They specify the design document (couchdb) or something similar for mongo, to get the results. And that API layer will validate the incoming data and make sure that data itself is saved and updated properly. Such as saving a new user, in the framework I only need to send a json obejct with the keys/values that need to be saved and the api layer saves the data in the proper places where needed.
This API would probably have a UI, but only for administrative purposes and to make my life easier. In general it will always reply with json strings, or pre-rendered/cached html in some cases. Since each api layer would be specific to the application anyways.
I was wondering if anyone has done anything like this, or had any tips on nethods I could accomplish this. I am currently looking to write my application in python, and the front end will likely be something like Angularjs. Although I am also looking at node.js for a back end.
We do this exact thing at my current job. We have MongoDB on the back end, a RESTful API on top of it and then PHP/Zend on the front end.
Most of our data is read only, so we import that data into MongoDB and then the RESTful API (in Java) just serves it up.
Some things to think about with this approach:
Write generic sorting/paging logic in your API. You'll need this for lists of data. The user can pass in things like http://yourapi.com/entity/1?pageSize=10&page=3.
Make sure to create appropriate indexes in Mongo to match what people will query on. Imagine you are storing users. Make an index in Mongo on the user id field, or just use the _id field that is already indexed in all your calls.
Make sure to include all relevant data in a given document. Mongo doesn't do joins like you're used to in Oracle. Just keep in mind modeling data is very different with a document database.
You seem to want to write a layer (the middle tier API) that is database agnostic. That's a good goal. Just be careful not to let Mongo specific terminology creep into your exposed API. Mongo has specific operators/concepts that you'll need to mask with more generic terms. For example, they have a $set operator. Don't expose that directly.
Finally after having a decent amount of experience with CouchDB and Mongo, I'd definitely go with Mongo.
I am currently in the planning stages of building a simple Django web app (for learning proposes). Basically, teacher’s can login, input student grades, and then the students can login and view their grades. Each class has a group number.
Here is a template of a user:
Username: Johny098
Password: Y98uj*?877!(
Name: John Doe
Gender: Male
Group: 32
Secondary: 5
Taking into consideration that I am starting with django and web developpement in general, I find confusing the number of database systems that are available to me: MySQL, CouchDB, MongoDB, SQLite, etc. And I am having a hard time deciding which database system to use for my purpose (I have no prior experience with databases).
After some research I found Couchdb (and SQLite) which seems fairly simple to pick up and fun to use, but that's just me, and that's why I need help. I know there are numerous debates on SQL vs NoSQL, but I don't really know if this will have an impact for my use of the databases. Ideally, the database system should integrate well with django and be easy enough to pick up in a couple of days.
So, coming back to the question: What database system should I use for my web app?
Any resources would also be appreciated.
Thanks,
-Raphael
For learning purposes I'd recommend SQLite: no setup, no background daemons running, everything is bundled with Python, it's SQL (so it maps well with Django ORM), and it's a very simple DBMS. In fact, some people use SQLite for prototyping and then switch to MySQL/PostgreSQL in production.
As for NoSQL, I would not recommend to use it unless you know exactly why and for what purpose you need it.
Oh, and one last thing: store password hashes (md5 or sha1), not the raw passwords. It's not necessary in your case, but in real-world apps it's mandatory.
The Django website has a page that lists the database backends supported. However, Django's database access layer is designed to work more or less the same no matter which database backend you're using. There are some differences described on the page I linked, but they shouldn't come up in normal usage if you're writing just a basic web app. So as far as effects on how you write your web app, among the choices listed it really doesn't matter.
Note that all the database backends Django supports are SQL-based, I believe. Accessing the database through Django does eliminate some of the security issues that I believe prompted the NoSQL movement... in any case, NoSQL is something you can pretty much ignore for now.
In your case, I would suggest picking SQLite, simply because it's easier to set up, and you don't want to spend time worrying about how to configure the database when you should be worrying about how to build your web app. The difference between SQLite and most other DBMSs (database management systems) is that SQLite stores each database in a regular file, and the SQLite client works directly with that file. Other DBMSs (like MySQL, PostgreSQL, Oracle, etc.) have a central location for the databases, a server to manage them, and a client that connects to the server and handles all the database access. A server-based DBMS works well for a busy web app, because it has features to handle many simultaneous requests to the database, but since you're just using this as a learning project, you don't need those features.
I would go for MySQL ... why? because among the ones you named, it is the most popular and, better than that, it will give you more credit when looking for a job (for instance, you can go to monster dot com and find how many jabs require you to know MySQL vs how many SQL-Lite).
Also, MySQL is simple and easy to learn, there are many GUI clients (some of which are Open Source) and it has pretty good documentation.
At the company I work for, we are building a cluster of web applications for collaboration. Things like accounting, billing, CRM etc.
We are using a RESTfull technique:
For database we use CouchDB
Different applications communicate with one another and with the database via http.
Besides, we have a single sign on solution, so that when you login in one application, you are automatically logged to the other.
For all apps we use Python (Pylons).
Now we need to add instant messaging to the stack.
We need to support both web and desktop clients. But just being able to chat is not enough.
We need to be able to achieve all of the following (and more similar things).
When somebody gets assigned to a task, they must receive a message. I guess this is possible with some system daemon.
There must be an option to automatically group people in groups by lots of different properties. For example, there must be groups divided both by geographical location, by company division, by job type (all the programers from different cities and different company divisions must form a group), so that one can send mass messages to a group of choice.
Rooms should be automatically created and destroyed. For example when several people visit the same invoice, a room for them must be automatically created (and they must auto-join). And when all leave the invoice, the room must be destroyed.
Authentication and authorization from our applications.
I can implement this using custom solutions like hookbox http://hookbox.org/docs/intro.html
but then I'll have lots of problems in supporting desktop clients.
I have no former experience with instant messaging. I've been reading about this lately. I've been looking mostly at things like ejabberd. But it has been a hard time and I can't find whether what I want is possible at all.
So I'd be happy if people with experience in this field could help me with some advice, articles, tales of what is possible etc.
Like frx suggested above, the StropheJS folks have an excellent book about web+xmpp coding but since you mentioned you have no experience in this type of coding I would suggest talking to some folks who have :) It will save you time in the long run - not that I'm saying don't try to implement what frx outlines, it could be a fun project :)
I know of one group who has implemented something similar and chatting with them would help solidify what you have in mind: http://andyet.net/ (I'm not affiliated with them at all except for the fact that the XMPP dev community is small and we tend to know each other :)
All goals could be achieved with ejabberd, strophe and little server side scripting
When someone gets assigned to task, server side script could easily authenticate to xmpp server and send message stanza to assigned JID. That its trivial task.
To group different people in groups, it is easily can be done from web chat app if those user properties are stored somewhere. Just join them in particular multi user chat room after authentication.
Ejabberd has option to automatically create and destroy rooms.
Ejabberd has various authorization methods including database and script auth
You could take look at StropheJS library, they have great book (paperback) released. Really recommend to read this book http://professionalxmpp.com/