We Keep Coding

PYTHON - Itertools.combinations results in memory error

I am creating a brute force script (for my studies) on python which connects to my login.php form and tries a combinaison of every characters of the alphabet to find the password.
The thing is that, after something like 110 000 combinaison, I am getting a memory error.
I already looked up on the net to find solutions which seems to be :
--> gc.collect() & del var
I tried to add the gc.collect() on the blank field of my function bruteforce so everytime i % 100 == 0, I clear the memory space but It dosen't work.
I can't find the way to add it in my script to free memory.
I think the problem is that I can't clear the memory space when my function is running. Maybe I should play with many threads like :
start 1
stop 1
clear space
start 2
stop 2
clear space
etc...
Do you guys have any suggestions ?
Here is my code :
import itertools, mechanize, threading, gc, os, psutil
charset='abcdefghijklmnopqrstuvwxyz'
br=mechanize.Browser()
combi=itertools.combinations(charset, 2)
br.open('http://192.168.10.105/login.php')
def check_memory():
process = psutil.Process(os.getpid())
mem_info = process.memory_info()
return mem_info.rss
def bruteforce():
i = 0
for x in combi:
br.select_form(nr=0)
br.form ['username'] = 'myusername'
tried = br.form ['password'] = ''.join(x)
i = i+1
print "Checking : ", tried, " Number of try : ", i
response=br.submit()
if response.geturl()=="http://192.168.10.105/index.php":
print("The password is : ", ''.join(x))
break
if i % 100 == 0:
mem_before = check_memory():
print "Memory before : ", mem_before
print "Free memory "
mem_after = check_memory()
print "Memory after : ", mem_after
x = threading.Thread(target = bruteforce)
x.start()
Thank you !

Thanks to #chepner, I found that the issue was with mechanize, not with itertools.
A good way to fix it is by clearing the history : "Out of Memory" error with mechanize

Python Code for RPM-measurement with Raspberry Pi

I'm working on a project where I use a photoelectric sensor to detect a reflector attached to a motor axis. Every time the reflector reflects the light from the LED from the sensor it sends a pulse (voltage) to the raspberry pi (with a voltage divider). With some help I've come to the following code and I've noticed that the results that I'm getting are way too high. Anybody got any suggestions on how to improve it? I've seen other examples of RPM code online, but I wanted to learn it myself and the ones that I've found did not use the same method as me. Thanks in advance!
import time
from gpiozero import Button
sensor = Button(17)
i=1
timestampeven=0
timestamponeven=0
def pulsen():
global i
global timestampeven
global timestamponeven
if (i % 2) ==0:
timestampeven = time.time_ns()
i+=1
elif (i % 2) == 1:
timestamponeven = time.time_ns()
i+=1
periode = timestamponeven-timestampeven
frequentie = 1/(periode*10e09)
rpm = frequentie*60
print("rpm=" + str(rpm))
print("teller = " +str(i))
print("periode = " +str(periode))
print("frequentie = " +str(frequentie))
sensor.when_pressed = pulsen
while True:
pass

How to automate control of Wemo light switch based on iPhone GPS

I'm writing a program to toggle the lights at my house based on my iPhone's GPS coordinates. Below is what I have so far. However, I feel like there must be a better way to do this. Is there a way to get GPS data without pinging my phone every five minutes?
So far I've tried the following with no joy:
Using Shortcuts and Scriptable I tried to write some JavaScript that would trigger when I got close to home. However, I could not figure out how to use await require('wemo-client') using scriptablify. I kept getting an error, "ReferenceError: Can't find variable: require".
IFTTT does not have a variable timed trigger so the lights won't turn off after 15 minutes. Also, I plan on adding a motion sensor trigger that is unsupported.
Pythonista is $10. Yes, I am that cheap.
Apple HomeKit does not support the model I'm using, Wemo Smart Light Switch F7C030.
The code below works, but I hate that I have to ping my phone every five minutes. I'd rather save battery life by firing this code once or twice a day, as needed.
Any suggestions would be greatly appreciated.
Code:
import sys
import time
import datetime
import os
from pyicloud import PyiCloudService
import pywemo
APPLE_ID = os.getenv('APPLE_ID') # Apple ID username
APPLE_ID_PASSWORD = os.getenv('APPLE_ID_PASSWORD') # Apple ID password
API = PyiCloudService(APPLE_ID, APPLE_ID_PASSWORD)
IPHONE = API.devices[1]
LOCATION = IPHONE.location()
FIVE = 300 # 5 * 60 seconds
FIFTEEN = 900 # 15 * 60 seconds
ONEMILE = 0.01449275362318840579710144927536 # one mile is 1/69 degrees lat or long
HOMELAT = # my home's latitude
HOMELONG = # my home's longitude
WEMOS = pywemo.discover_devices()
LEN_WEMOS = range(len(WEMOS))
# Two factor authentication to retrieve iPhone data
if API.requires_2fa:
import click
print("Two-step authentication required. Your trusted devices are:")
DEVICES = API.devices
for i, device in enumerate(DEVICES):
print(" %s: %s" % (i, device.get('deviceName', "SMS to %s" % device.get('phoneNumber'))))
DEF_DEVICE = click.prompt('Which device would you like to use?', default=0)
DEVICE = DEVICES[DEF_DEVICE]
if not API.send_verification_code(DEVICE):
print("Failed to send verification code")
sys.exit(1)
CODE = click.prompt('Please enter validation code')
if not API.validate_verification_code(DEVICE, CODE):
print("Failed to verify verification code")
sys.exit(1)
# Turn off the lights when I leave
def leavehome():
timenow = datetime.datetime.now()
print("Left home on {}".format(timenow.strftime("%B %d, %Y at %H:%M:%S")))
for wemo in LEN_WEMOS:
WEMOS[wemo].off()
# Turn on the lights for 15 minutes when I get home
def arrivehome():
timenow = datetime.datetime.now()
print("Arrived home on {}".format(timenow.strftime("%B %d, %Y at %H:%M:%S")))
# Loop through all Wemo devices
for wemo in LEN_WEMOS:
WEMOS[wemo].on()
time.sleep(FIFTEEN)
for wemo in LEN_WEMOS:
WEMOS[wemo].off()
# Automatically turn off the lights after 15 minutes - save electricity
def timeoff():
time.sleep(FIFTEEN)
for wemo in LEN_WEMOS:
WEMOS[wemo].off()
# Ping my phone for GPS data
def pingphone(prev):
mylat = LOCATION["latitude"]
mylong = LOCATION["longitude"]
logic(prev, mylat, mylong)
time.sleep(FIVE)
# Perform logic to determine if I'm home, out, arriving, or leaving
def logic(prev, lat, long):
inrange = (HOMELAT+ONEMILE >= lat >= HOMELAT-ONEMILE and HOMELONG+ONEMILE >= long >= HOMELONG-ONEMILE)
current = bool(inrange)
previous = prev
if current and not previous:
arrivehome()
elif previous and not current:
leavehome()
else:
timeoff()
pingphone(current)
# Run the script
pingphone(False)

Adding a parcel repository using Cloudera Manager Python API

I'm trying to install CDH5 parcels on Hadoop-cluster using Cloudera Manager Python API. I'm doing this using following code:
test_cluster = ... # configuring cluster
# adding hosts ...
for parcel in test_cluster.get_all_parcels():
if parcel.product == 'CDH' and 'cdh5':
parcel.start_download().wait()
parcel.start_distribution().wait()
success = parcel.activate().wait().success
But I catch such error:
cm_api.api_client.ApiException: Parcel for CDH : 5.8.0-1.cdh5.8.0.p0.42 is not available on UBUNTU_TRUSTY. (error 400)
The CDH 5.8.0-1.cdh5.8.0.p0.42 was in AVAILABLE_REMOTELY, as we can see if print a string representation on this parcel:
<ApiParcel>: CDH-5.8.0-1.cdh5.8.0.p0.42 (stage: AVAILABLE_REMOTELY) (state: None) (cluster: TestCluster)
After the execution of code, parcel changes its stage to DOWNLOADED.
It seems, I should add a new parcel repository, compatible with Ubuntu Trusty (14.04). But I don't know of doing this using Cloudera Manager API.
How I can specify the new repository for installing correct CDH?

You may want to be more specific about the parcel you are acting on. I use something like this for the same purpose, the important part for your question is the combined check on parcel.version and parcel.product. After that (yes I am verbose in my output) I print the list of parcels to verify I am trying to only install the 1 parcel I want.
I'm sure you've been here, but if not the cm_api github site has some helpful examples too.
cdh_version = "CDH5"
cdh_version_number = "5.6.0"
# CREATE THE LIST OF PARCELS TO BE INSTALLED (CDH)
parcels_list = []
for parcel in cluster.get_all_parcels():
if parcel.version.startswith(cdh_version_number) and parcel.product == "CDH":
parcels_list.append(parcel)
for parcel in parcels_list:
print "WILL INSTALL " + parcel.product + ' ' + parcel.version
# DISTRIBUTE THE PARCELS
print "DISTRIBUTING PARCELS..."
for p in parcels_list:
cmd = p.start_distribution()
if not cmd.success:
print "PARCEL DISTRIBUTION FAILED"
exit(1)
# MAKE SURE THE DISTRIBUTION FINISHES
for p in parcels_list:
while p.stage != "DISTRIBUTED":
sleep(5)
p = get_parcel(api, p.product, p.version, cluster_name)
print p.product + ' ' + p.version + " DISTRIBUTED"
# ACTIVATE THE PARCELS
for p in parcels_list:
cmd = p.activate()
if not cmd.success:
print "PARCEL ACTIVATION FAILED"
exit(1)
# MAKE SURE THE ACTIVATION FINISHES
for p in parcels_list:
while p.stage != "ACTIVATED":
p = get_parcel(api, p.product, p.version, cluster_name)
print p.product + ' ' + p.version + " ACTIVATED"

How do I get Boto to return EC2 instances - S3 works fine

I'm having some issues with the EC2 bit of Boto (Boto v2.8.0, Python v2.6.7).
The first command returns a list of S3 Buckets - all good! The second command to get a list of EC2 instances blows up with a 403 with "Query-string authentication requires the Signature, Expires and AWSAccessKeyId parameters"
s3_conn = S3Connection(AWSAccessKeyId, AWSSecretKey)
print s3_conn.get_all_buckets()
ec2_conn = EC2Connection(AWSAccessKeyId, AWSSecretKey)
print ec2_conn.get_all_instances()
Also, my credentials are all good (Full admin) - I tested them using the Ruby aws-sdk, both EC2 and S3 work fine.
I also noticed that the host attribute in the ec2_conn object is s3-eu-west-1.amazonaws.com, "s3"...? Surely thats wrong? I've tried retro fixing it to the correct endpoint but no luck.
Any help would be great appreciate
Thanks

Here's some working code I use to list all my instances across potentially multiple regions.
Its doing a lot more than you need, but maybe you can pare it down to what you want.
#!/usr/bin/python
import boto
import boto.ec2
import sys
class ansi_color:
red = '\033[31m'
green = '\033[32m'
reset = '\033[0m'
grey = '\033[1;30m'
def name(i):
if 'Name' in i.tags:
n = i.tags['Name']
else:
n = '???'
n = n.ljust(16)[:16]
if i.state == 'running':
n = ansi_color.green + n + ansi_color.reset
else:
n = ansi_color.red + n + ansi_color.reset
return n
def pub_dns( i ):
return i.public_dns_name.rjust(43)
def pri_dns( i ):
return i.private_dns_name.rjust(43)
def print_instance( i ):
print ' ' + name(i) + '| ' + pub_dns(i) + ' ' + pri_dns(i)
regions = sys.argv[1:]
if len(regions)==0:
regions=['us-east-1']
if len(regions)==1 and regions[0]=="all":
rr = boto.ec2.regions()
else:
rr = [ boto.ec2.get_region(x) for x in regions ]
for reg in rr:
print "========"
print reg.name
print "========"
conn = reg.connect()
reservations = conn.get_all_instances()
for r in reservations:
# print ansi_color.grey + str(r) + ansi_color.reset
for i in r.instances:
print_instance(i)

There is the connect_to_region command:
import boto.ec2
connection = boto.ec2.connect_to_region('eu-west-1', aws_access_key_id=AWSAccessKeyId,
aws_secret_access_key=AWSSecretKey)
The Boto tutorial gives another way. That method would basically work like this:
import boto.ec2
for region in boto.ec2.regions():
if region.name == 'my-favorite-region':
connection = region.connect()
break
This has not been working on older versions of Boto.

Do you have your IAM credentials in order? The given access key should have rights for EC2. If you're not sure, you can add the policy AmazonEC2FullAccess to test, and later tune this down.