I have a basic background in DS with Python and I try now the first time to build an application and I need a bit advise which infrastructure to choose on AWS and how to structure the application. The code I can develop/google on my own :)
The main question is: Where/On which platform of AWS should happen Step 2. I guess I miss there some basic knowledge of applications and therefore I have problems to google the problem myself.
What should happen by the application:
On a website a user types in values in a form and this values are sended somewhere so be processed. (Already coded)
Now, this values (so far an email with the values) has to be sent somewhere to be processed. Here I do not know in which infrastructure of AWS I can write an application that can receive this values (/email) directly and process it automatically?
3./4. Automated process of values, pdf creation and sending etc.
Goal is that always when a user uses the website and sends the email, that the automated process is triggered.
Thank you for your help! :)
I am assuming that you have access to the mailbox to which user form data will be sent via email, You can then read the email data using imap module of python and extract the required information either by using regex or by some html to dict conversion module, please find below link for html to dict conversion.
How to convert an HTML table into a Python dictionary.
Having said all that I would strongly recommend you to use AWS EC2 instance to host your application, NGNIX as web server, postgress as database and most importantly Django as the webframe work, you should have the user fill the require data in form and send that form directly to the back end server which can then save it directly to your database (there is no need to send the data via email), if you have any queries please let me know.
I would suggest you use a "fanning out" architecture with something like Eventbridge or SNS topic.
When your user submits form, you publish a message to an SNS topic.
That topic can send an email, and also send the data to a backend service like lambda to save to something like DynamoDB or something like RDS MySQL.
Related
I have a Python registration and login code using Tkinter.
But, how and where can I save this recorded data so that a person who created the account on one computer can log on to another?
On google, I only find people saving the data in a local database (.db file on the computer) or in a .txt
Thanks in advance!
To use .db you can use sqlite3 and working with .txtis builtin.
I would recommend hosting your data then so it is not a actual file but on the cloud for stuff like that I would recommend checking out MongoDB
You can use a sqlite3 database, but that would not solve your problem.
The database or any other form of data source must be stored on a server that you can access. This can be on the internet or on the local network. (You didn't specify your environment)
you could access this data source by https (don't use http as it is clear text and not a good idea for password requests), connect to an online database or write some kind of socket server that you can connect to and get the needed information.
So I have been using firebase with Pyrebase-python and it has been working out well so far for a prototype. For login I get the email and password, post through ajax and then use sign_in_with_email(email, password). Boom, the user is verified and I have a refresh token to access the firebase db with.
This worked out well until I realized that I needed to query the data on different fields. Errror!! So the work around is adding a field like accountUUID_campaignUUID and querying on that. Ok, not so bad. However, now they have released Firestore, which includes multiple queries.
I would like to implement this feature, however, if I use the python sdk, it only validated with Google IAM and not the Firebase Auth. So what do I do now? All my models and backend is written in python. If I move all the models to the client JS, that would solve one problem, however, I still need to perform backend functions on the data. So do I get the data in the JS frontend, then send it to the python backend, perform function, then send to frontend? This seems pointless and redundant. At this point, I could just drop the realtime function as its not 100% necessary and just go with MongoDB and write a custom JWT authorization. Firebase Realtime Database/Firestore has really been pissing me off lately. Why is everything forced in the front end? But I guess that's my fault for trying to force Realtime Database/Firestore with python... Maybe I'll try it again with my next project, maybe not...
Firebase Firestore in Python
That link should be all you need to get started.
As to your comment on IAM vs Firebase Auth: You can set up security rules to deal with your issue even setting up variables and conditionals for access rules based on auth.uid
https://firebase.google.com/docs/firestore/security/get-started?authuser=0
New to programming, using Python 3.
I work in sales and want to make a program using the Podio API which is going to take information about potential clients from an excel sheet and use it to create subpages in Podio with their information. To get an API-key, Podio wants a redirect-URL for the purposes described here and here, a whole bunch of text I don't really understand. Does it mean I have to authenticate myself in my program (using my Podio login info?), which sends me to Podio (where I log in to Podio manually, using the same login info?), which sends me to the redirect URL, which sends me back to Podio? I can't really make sense of this.
I googled and found some similar questions but none of the answers explained exactly what the actual functions of these authentication flows are. When do I need them? Do I need them if I'm just going to be using this program myself? Do I always need them to gain access to my Podio account through my program?
Thanks in advance.
If you are only going to use your program yourself, then username/password flow is what you need. It is simplest to understand and use flow of authenticating with Podio API. Here are all needed details for it: https://developers.podio.com/authentication/username_password
To be short: yes, you can enter localhost as full domain (without protocol) of your return URL
I am currently learning how to use django. I have a standalone python script that I want to communicate with my django app. However, I have no clue how to go about doing this. My django app has a login function and a database with usernames and passwords. I want my python script to talk to my app and verify the persons user name and password and also get some account info like the person's name. How do I go about doing this? I am very new to web apps and I am not really sure where to begin.
Some Clarifications: My standalone python program is so that the user can access some information about their account. I am not trying to use the script for login functionality. My django app already handles this. I am just trying to find a way to verify that they have said account.
For example: If you have a flashcards web app and you want the user to have a program locally on their computer to access their flashcards, they need to login and download the cards from the web app. So wouldn't the standalone program need to communicate with the app to get login information and access to the cards on that account somehow? That's what I am trying to accomplish.
If I understand you correctly, you're looking to have an external program communicate with your server. To do this, the server needs to expose an API (Application Interface) that communicates with the external program. That interface will receive a message and return a response.
The request will need to have two things:
identifying information for the user - usually a secret key - so that other people can't access the user's data.
a query of some sort indicating what kind of information to return.
The server will get the request, validate the user's secret key, process the query, and return the result.
It's pretty easy to do in Django. Set up a url like /api/cards and a view. Have the view process the request and return the response. Often, these days, these back and forth messages are encoded in JSON - an easy way to encapsulate and send data. Google around with the terms django, api, and json and you'll find a lot of what you need.
I am trying to learn how to large organisations that use python structure their code so that I can maybe apply some of their theories to my own code.
Currently I am looking through reddit's code and am interested how they have implemented the sending of emails generated as part of the app's operations. See: https://github.com/reddit/reddit/blob/master/r2/r2/lib/emailer.py (their emailing library) and https://github.com/reddit/reddit/blob/master/r2/r2/models/mail_queue.py
I think mail_queue.py contains some form of SqlAlchemy table backed email queue.
Is this the case. Does that mean the table is kept in memory? Could somebody make this a little clearer for me?
Cheers from Down Under.
P.S. May I suggest if anybody is trying to get a good understanding of how to structure python apps they do the same as I am. Reading and understanding other peoples code has allowed me to structure and write noticeably better code.. :) Open source stuff is great!
Traditionally, the mail queue on e-mail servers has been some sort of disk storage. The reason for this is so that the chance of mail getting lost is minimized. For example, the mail server would receive a message and not send back a successful return code to the sending mail client until the entire message was successfully written to disk via synchronous write.
Yes, the reddit code is using a database as a email data store via SqlAlchemy.
As far as the table being stored in memory, I wouldn't imagine that it would be. From reading the SqlAchemy documentation, the Table object is SqlAlchemy is just a proxy to the underlying table in whatever database is backing the system. In general, you wouldn't want the table in memory since you don't know how many messages the system will process, how big the e-mail messages are, and how many messages need to be queued in case of a temporary mail sending failure.