I'm writing a little chatserver and client. There I got the idea to let users connect (nice :D) and when they want to protect their account by a password, they send /password <PASS> and the server will store the account information in a sqlite database file, so only users, who know the passphrase, are able to use the name.
But there's the problem: I totally forgot, that sqlite3 in python is not thread-safe :( And now its not working. Thanks to git I can undo all changes with the storage.
Does anyone have an idea how to store this stuff, so that they are persistent when stopping/starting the server?
Thanks.
OK, I'm using a simple JSON text file with automatic saving every minute
Related
I have a Python registration and login code using Tkinter.
But, how and where can I save this recorded data so that a person who created the account on one computer can log on to another?
On google, I only find people saving the data in a local database (.db file on the computer) or in a .txt
Thanks in advance!
To use .db you can use sqlite3 and working with .txtis builtin.
I would recommend hosting your data then so it is not a actual file but on the cloud for stuff like that I would recommend checking out MongoDB
You can use a sqlite3 database, but that would not solve your problem.
The database or any other form of data source must be stored on a server that you can access. This can be on the internet or on the local network. (You didn't specify your environment)
you could access this data source by https (don't use http as it is clear text and not a good idea for password requests), connect to an online database or write some kind of socket server that you can connect to and get the needed information.
I have a basic background in DS with Python and I try now the first time to build an application and I need a bit advise which infrastructure to choose on AWS and how to structure the application. The code I can develop/google on my own :)
The main question is: Where/On which platform of AWS should happen Step 2. I guess I miss there some basic knowledge of applications and therefore I have problems to google the problem myself.
What should happen by the application:
On a website a user types in values in a form and this values are sended somewhere so be processed. (Already coded)
Now, this values (so far an email with the values) has to be sent somewhere to be processed. Here I do not know in which infrastructure of AWS I can write an application that can receive this values (/email) directly and process it automatically?
3./4. Automated process of values, pdf creation and sending etc.
Goal is that always when a user uses the website and sends the email, that the automated process is triggered.
Thank you for your help! :)
I am assuming that you have access to the mailbox to which user form data will be sent via email, You can then read the email data using imap module of python and extract the required information either by using regex or by some html to dict conversion module, please find below link for html to dict conversion.
How to convert an HTML table into a Python dictionary.
Having said all that I would strongly recommend you to use AWS EC2 instance to host your application, NGNIX as web server, postgress as database and most importantly Django as the webframe work, you should have the user fill the require data in form and send that form directly to the back end server which can then save it directly to your database (there is no need to send the data via email), if you have any queries please let me know.
I would suggest you use a "fanning out" architecture with something like Eventbridge or SNS topic.
When your user submits form, you publish a message to an SNS topic.
That topic can send an email, and also send the data to a backend service like lambda to save to something like DynamoDB or something like RDS MySQL.
I have created django app that I uploaded on "pythonanywhere website".
This app use quite a big database which I have to update every day and I wanted this database to be on my computer only.
Inside on of my django view I have created TCP client and after user type some information to app, this client send query to database on address when my server with database is running (my computer IP).
Everything was good on local network. I could connect to database from one computer to another and receive answer from it. Unfortunatelly I am unable to do it online on "pythonanywhere" website. I have read that it's impossible to do it through this website right now and I am looking for some alternative way to solve this problem.
Long story short. I am looking for a way to deploy my django website and connect it with server on my computer.
If there is some other, easier way to do it, please advise, but I would rather keep my database on computer.
Thank you.
I am trying to learn how to large organisations that use python structure their code so that I can maybe apply some of their theories to my own code.
Currently I am looking through reddit's code and am interested how they have implemented the sending of emails generated as part of the app's operations. See: https://github.com/reddit/reddit/blob/master/r2/r2/lib/emailer.py (their emailing library) and https://github.com/reddit/reddit/blob/master/r2/r2/models/mail_queue.py
I think mail_queue.py contains some form of SqlAlchemy table backed email queue.
Is this the case. Does that mean the table is kept in memory? Could somebody make this a little clearer for me?
Cheers from Down Under.
P.S. May I suggest if anybody is trying to get a good understanding of how to structure python apps they do the same as I am. Reading and understanding other peoples code has allowed me to structure and write noticeably better code.. :) Open source stuff is great!
Traditionally, the mail queue on e-mail servers has been some sort of disk storage. The reason for this is so that the chance of mail getting lost is minimized. For example, the mail server would receive a message and not send back a successful return code to the sending mail client until the entire message was successfully written to disk via synchronous write.
Yes, the reddit code is using a database as a email data store via SqlAlchemy.
As far as the table being stored in memory, I wouldn't imagine that it would be. From reading the SqlAchemy documentation, the Table object is SqlAlchemy is just a proxy to the underlying table in whatever database is backing the system. In general, you wouldn't want the table in memory since you don't know how many messages the system will process, how big the e-mail messages are, and how many messages need to be queued in case of a temporary mail sending failure.
I have a Python script that is connecting to the database. To that, obviously, I need the password. I need to hide it somewhere.
My problem is that this code is stored in a folder that everybody who has access to the server can look. So, if I write this password encrypted in a file, in the code will appear the key to discover it and people can figured it out.
So, please, if anyone has an idea..
You're using a scripting language and accessing a database directly with a password. No matter what you do, at some level that password is going to be easily accessible. Obscuring it doesn't really buy you much.
You have to rely on the machine's security and permissions, and perhaps the database (restricting access from that particular machine and user).
Don't store the database connection credentials in the Python file at all. Instead, store them in a secure place, readable only by the user account that the script will run under.
For example, create a user account for running this job, and create a file in that user account's home directory (readable only by that user) called database.ini and put the database connection string and password there. Then use the Python ConfigParser class in the standard library to read the file in.
Then the job can be always run under that user account. You can also run it under your account by putting a database.ini file in your home directory with the correct credentials, but anyone who doesn't have the credentials cannot run it.
Check out this question. They suggest encoding the password in base64 (outside of the script) then including that string in the script and converting it back before you make the connection
Just to reinforce what Brian said, if a program runs automatically (i.e., without the opportunity to prompt the user for a password), any program that runs under the same user authority has the same access to any password. It's not clear what else you could do. Perhaps if the (trusted) operating system on the client machine could certify to the host that it was being accessed by a program run from a particular path, the host could be told "Only open the database to /var/lib/tomcat/bin/tomcat on appserver.example.com". If you accomplish all that, an attacker would have to compromise the tomcat executable to get to the database.
A more advanced idea is to do the mysql authentication manually. That is, learn the mysql protocol (it's a standard handshake with a challenege and a response) and do the procedure yourself. This way, you never send the password directly.