We Keep Coding

Gathering bug reports?

Premise: I am a beginner in search for an easy way to send bug reports from users over sea.
I've made a script for some friends that are living on the other side of the sea (US - EUROPE)... I will like to gather automatic bug reports whenever they happen. So my first idea was to send myself an email with the smtplib module. It works fine when testing home, but as soon as the sender "sends", my email provider (gmail) blocks the connection because of course, its from an "unknown device". I've already enabled "Allow less secure apps" as someone suggested but with no avail.
What I am searching its a simple way of dealing with this.
Yes I could make the script to ignore the error if the email its not being sent, and then go into my google account and enable those devices so at least it will work from the second run..
But it doesn't seem what a programmer would do in this case. I am learning so a solution withing the language is what I am after.
A different provider that has no restriction its also a good start but I tried Yahoo, Live, Yandex but I couldn't make them work. Are there any?
So my question is: how others do? what is the best solution for some one like me?
I've read about sentry or other error/bug tracking but its obviously way too much for want I need

You should certainly not incorporate e.g. Gmail credentials in the code that is remotely executed on devices you do not control, given I understand correctly the Gmail less secure device issue happens as every "user" is running this code and using your credentials. This holds true for any other provider.
Now this won't exactly be simple but one way to go about it would be to create a server side API endpoint that can accept HTTP(s) or any other protocol requests that then will authenticate in a little more secure way on the server side with Gmail.
The concept for emails is:
Bug > Python Script > API call > Email
This could be implemented using Python on the API side (Flask e.g.) using an AWS Lambda Function with Amazon API Gateway, but again that is something to get through and understand by itself which will take a good chunk of time.
You need to touch a lot of concepts, like auth tokens to make this really secure.
Could you elaborate a little on where the code needs to run and if you are willing to try AWS or any other cloud provider, or would have access to an internet connected server ? This makes it easier to provide you with a full example on the solution in a hackish way while I would highlight the problems you could face on the security side.

I understand that this is not the way to go but as for my needs and my level of experience it works for me!
Yandex allows you to send email from different ip so Yandex is the way to go. What I was doing wrong in the first place was to use the wrong port (587 instead of 465)

Trigger an event on a raspi behind a firewall

I have a raspi which monitors solar panels on my roof. It is installed behind a firewall.
I can send emails from that raspi (e.g. in case of errors).
Now I want to be able to get an email reporting the actual status at the moment whenever I am interested, triggering this by an email from my mobile.
The only solution I can think of is sending an email from my mobile to an email account which is checked regularily by the raspberry, using code such as:
http://mitchtech.net/raspberry-pi-physical-gmail-notifier/
Are there any other methods (also other than email) which I have not thought of? I don't think that there is a way to do that without active polling from the raspi.
If I have to use active polling (e.g. every minute or so), are there other methods than email which generate less permanent traffic?
I am aware that this question may be borderline for stackoverflow, since I am looking for a concept or a hint to functions rather than an actual problem in my code, and apologize for that.

I suggest you to use cloud-based log management and analysis. Some of them are free for very limited features (usually on log retention or amount of notifications).

pulling info from a log file and sending a notification email using python

I am working on a project for work that requires me to pull information from a logfile and send a notification anytime it finds a the specific information. For example the exact issue I am working on is I am needing to create a python script that will look into may /var/log/auth.log (FreeBSD system) and pull any invalid SSH login attempts, then proceed to email me and another co-worker anytime there is an offense.
I've been looking all over for a few days now and have had minimal success any help would be greatly appreciated.

I think what you're really after is a daemon like fail2ban, which is specifically designed to examine log files for intrusion attempts.
From the fail2ban wiki:
Fail2ban scans log files (e.g. /var/log/apache/error_log) and bans IPs
that show the malicious signs -- too many password failures, seeking
for exploits, etc. Generally Fail2Ban then used to update firewall
rules to reject the IP addresses for a specified amount of time,
although any arbitrary other action (e.g. sending an email, or
ejecting CD-ROM tray) could also be configured. Out of the box
Fail2Ban comes with filters for various services (apache, curier, ssh,
etc).
This would probably work better than any solution you baked yourself.
That said, if you did want to roll your own, the naive way to implement periodic checking of a file is simply to read it every five minutes and see if it's changed.
The smarter way is to use the operating system's file monitoring service, which hooks into the filesystem driver and notifies you as soon as the file changes. This has the dual benefits that your code will take less CPU time, and it will respond immediately whenever the file changes.
On Linux the service is called inotify. BSD and Windows have an equivalent feature.

You could run a cron job every few minutes that checks for changes in that file. If there are any changes, it will email you, by using, for example, smtplib. Here is an example of smtplib usage with sendgrid: http://docs.sendgrid.com/documentation/get-started/integrate/examples/python-email-example-using-smtp/
How do you find out if a file was modified?
You keep a copy of the file as it looked in the previous script run, and compare that to the current contents
You check the file's last modification time.
This is just a general idea that can be tweaked, and all the 'ingredients' can be found on google, so you should be able to implement it by googling yourself.
Hope this helps.

As a rough idea for a cron job:
with open('/var/log/auth.log') as auth:
for line in auth:
if 'blahblah' in line:
# send email
You'll want to check out the email module for emailing details. You'll also want a way to keep track of what's already been scanned, so you don't end up sending duplicate emails.

Process dynamic email addresses using python

I need to do the following and I was wondering if anyone has done something similar, and if so what they did.
I need to write a program that will handle incoming emails for different clients, process them, and then depending on the email address, do something (add to database, reply, etc).
The thing that makes this a little more challenging is that the email addresses aren't static they are dynamic. For example. The emails would be something like this. dynamic-email1#dynamic-subdomain1.domain.com . The emails are grouped by client using a dynamic subdomain in this example it would be 'dynamic-subdomain1'. A client would have their own subdomain that is assigned to them. Each client can create their own email address under their subdomain, and assign an event to that email. These email addresses and subdomains can change all of the time, new ones added, old ones removed, etc.
So for example if an email comes in for the email 'dynamic-email1#dynamic-subdomain1.domain.com' then I would need to look up in the database to find out which client is assigned the 'dynamic-subdomain1' subdomain and then look to see which event maps to the email address of 'dynamic-email1' and then execute that event. I have the event processing already, I'm just not sure how to map the email addresses to the event.
Since the email addresses are dynamic, it would be a real pain to handle this with file based configuration files, it would be nice to look up in a database instead. I did some research and I found some projects that do something similar but not exactly. The closest that I found is Zed Shaw's Lamson project: http://lamsonproject.org
More background:
I'm using python, django, linux, mysql, memcached currently.
Questions:
Has anyone used Lamson to do what I'm looking to do, how did you like it?
Is there any other projects that do something similar, maybe in a different language besides python?
How would I setup my DNS MX record to handle something like this?
Thanks for your help.
Update:
I did some more research on the google app engine suggestion and it might work but I would need to change too many things and it would add too many moving parts. I would also need a catch all emailer forwarder, anyone know of any good cheap ones? I prefer to deploy on system that handles all email. It looks like people have used postfix listening on port 25 and forwarding requests to lamson. This seems reasonable, I'm going to try it out and see how it goes. I'll update with my results.
Update 2:
I did some more research and I found a couple of websites that do something like this for me, so I'm going to look at them next.
http://mailgun.net
http://www.emailyak.com

I've done some work on a couple projects using dynamic email addresses, but never with dynamic subdomains at the same time. My thoughts on your questions:
I've never used Lamson, so I can't comment on that.
I usually use App Engine's API to receive and handle incoming messages, and it works quite well. You could easily turn each received message into a basic POST request on your own server with e.g. To, From, Subject, and Message fields and handle those with standard django.
One downside with GAE email is having to use *#yourappname.appspotmail.com, but you could get around that by setting up a catch-all email forwarder for *#yourdomain.com to direct everything to secretaddress#yourappname.appspotmail.com. That would let you receive the messages on the custom domain and handle them with GAE.
The other issue/benefit with GAE is using Google's servers instead of your own (at least for the email bit).
For the subdomain issue, you could try setting up wildcard DNS for the MX records, which (in theory) would direct all mail sent to any subdomain to the same server(s). This would enable you to receive email on all subdomains (for better or worse--look out for spam!)

For lamson, have you tried something as simple as:
#route("(address)#(subdomain).(host)", address=".+", subdomain="[^\.]+")
def START(message, address=None, subdomain=None, host=None):
....

Text-based one-on-one chat with Flash interface: what to power the backend?

I'm building a website where I hook people up so that they can anonymously vent to strangers. You either choose to be a listener, or a talker, and then you get catapulted into a one-on-one chat room.
The reason for the app's construction is because you often can't vent to friends, because your deepest vulnerabilities can often be leveraged against you later on. (Like it or not, this is a part of human nature. Sad.)
I'm looking for some insight into how I should architect everything. I found this neat tutorial, http://giantflyingsaucer.com/blog/?p=875, which suggests using python & stackless + flash. Someone else suggested I should try using p2p sockets, but I don't even know where to begin to look for info on that.
Any other suggestions? I'd like to keep it simple. :^)

Unless you expect super high load, this is simple enough that it doesn't really matter what you use on the backend: just pick something you're comfortable with. PHP, Python, Ruby, Even a bash script using CGI - your skill level with the language is likely to make more difference that the language features themselves.

I would use an XMPP server like ejabberd or OpenFire to power the backend. XMPP contains everything you need for creating chat/real-time applications. You can use a Flex/Flash Actionscript library like Actionscript 3 XIFF to communicate with the XMPP server.

Flash is user-unfriendly for UI (forms, etc) and it is relatively easy to do what you want using HTML and Javascript on the front-end.
One possible approach for reading the messages would be to regularly do an Ajax request from the server for any new messages. Format the new message and insert it into the DOM.
You will probably need to answer at least these questions before you continue, though:
1) Are you recreating IRQ (everyone sees your posts), or is this a random one-to-one chat, like chatroulette?
1a) Is this a way for a specific person to talk to another specific person, or is this more like twitter?
2) What is your plan for scaling up if this idea takes off? Memcached should probably be a method of last-resort ("bandaid over a bullet-hole"). What's your roadmap for eventually handling a large volume of messages?
3) Is there any way to ignore users? Talk to certain users? Hide your rants from users?

Hey Zach I had to create a socket server for a flash game I made. I built my server in C#, but I would use whatever language your familiar with. If you let me know what your most comfortable with I could try to help find a good tutorial.
The one thing I spent many hours on was getting flash to work from a website with a socket server. With the newer versions of Flash you need to send back a policy file. In my case this needed to be the first chunk of data sent back to the client when they connected to the socket server.
Not sure what to tell you about structuring the back end. I need to know a little bit more about your programming experience. I had an array of all user connections, and was placing them in different "Rooms" so they could play each other. So just some simple arrays and understanding how to send messages to the clients would help you here.
If you have any familiarity with C# I would have no problem sending you the source code for my socket server.