I'm trying to animate every step of some code programmatically like in this video(code here). Basically, I want to visualize every step of the algorithm for a given input.
Doing the animations is trivial, but if you look at the code, the author does everything by hand. I'm trying to do this programmatically, so at every step, I need the current line, some variables, call hierarchy etc. so I can make the animations work.
For example, let's say bfs.py contains the following:
def bfs(G, v):
def visit(v):
pass #print(v)
visited = [False] * len(G)
queue = [v]
while len(queue) > 0:
v = queue.pop(0)
if not visited[v]:
visit(v)
visited[v] = True
for w in G[v]:
if not visited[w]:
queue.append(w)
And I want to do something similar to the video without modifying this code: I run this from inside a different file in a way that enables stepping and peeking in the way I want.
I tried Pdb but it forces interactivity, and I couldn't find a way to use it how I want to. I also tried nopdb, but it seems very complicated to use it for my purposes, so before further work, I'd like to explore other options.
Related
I want to utilize all my computer's cores to run the following Pseudocode (the actual code is too long):
def function2(modifiedList):
process modifiedList
return value
mainList = [a, b, c,...,z]
def function1(mainList):
process mainList
create modifiedList
result = function2(modifiedList)
return result
calculator(function1)
Not sure how to do multiporcessing or multi-threads when a function is called from inside another function.
You should look at https://docs.python.org/3.6/library/multiprocessing.html module in standard library.
However, the code you posted would not be easy to parallelize, because of it's sequential nature.
A much better approach would be to find an easily parallelizable part of your problem and work from there.
As the documentation says, if your list is long enough, and you can process each element independently, you could try to substitute
for i in l:
result.append(f(i))
with
result = p.map(f, l)
I am struggling with a program to tell whether a created disk touches the edge of a predefined box. The parser keeps saying things such as
NameError: global name 'disksdescription' is not defined
Warning: variable 'left' is assigned to, but never used
The code so far is:
import numpy.random as nr
def touch_sides(disk, radius, x, y):
for disk in disksdescription:
for left in disk.values()[2]:
if x <= radius:
left = True
else:
left = False
for right in disk.values()[3]:
if x >= 1-radius:
right = True
else:
right = False
def run_code():
radius = 0.1
coordinates = nr.uniform(size=(1,2))
print coordinates
disksdescription = []
for row in range(0, coordinates):
x = coordinates[row, 0]
y = coordinates[row, 1]
disksdescription.append({'x': x, 'y': y, 'left': left, 'right': right})
touch_sides(coordinates, radius, x, y)
print disksdescription
Overview
You have quite a few basic problems in this code. I'm going to back off and try to solve a couple of your process problems.
Most of all, I strongly recommend that you adopt incremental programming: write a few lines of code, just a single step or two of logic. Then test that until you're sure it does what you want. Only then do you write more code.
The advantage of this is that problems are usually easy to debug: only the new 2-5 lines are suspect. Contrast this with your current situation: you have several errors; fixing any one of them won't give you an appreciably better result, so it's hard for you to learn what you did wrong until you get all of them at once.
What is a function?
Think of it like some sort of security set-up, perhaps like the pass-through window to a diner's kitchen. The server sticks an order (slip of paper) through the window. When the cook is done, plates of food come back. All of the communication goes through that window. When we ask for an omelet, for all we know, the cook is slicing meat bits off a full side of beef on a rotisserie ... or nuking some frozen stuff. Can't see it, can't tell; that's a separate room with opaque walls.
Which means what to me?
You have three "rooms" in your program: the (trivial) main program, run_code, and touch_sides. None of them can see inside the others. The variable left in run_code has no relation to the variable left in touch_sides. These are entirely separate entities, no more connected than, say, Kim Jong Il and Kim Kardashian.
In particular, the first line of touch_sides is an ultimate huh? moment for the Python compiler. There is no defined thing called diskdescription. Farther down, you spend work to get values for left and right, but then you don't slide them back through the window. run_code has no idea that you did anything useful in there.
What to fix
Go back and do the simple exercises that you skipped. Their purpose was to keep you from making all these mistakes at once. Write trivial functions and trivial calls. Learn to use the input parameters and return values.
Practice incremental programming. Comment out the bodies of your functions. Now, run the program, a simple call to run_code(). It won't do anything, but it does so without error.
Next, put "stub" code into touch_sides: don't compute anything, but print the parameter values. Ignore the input parameters and return False for left and right. Add code to run_code that calls touch_sides and prints the results.
At this point, you know that you can reliably pass values between the functions. Now you can add the actual code.
What else?
Get comfortable with the concept of a boolean expression. Learn that your original code
if x <= radius:
left = True
else:
left = False
Reduces directly to
left = (x <= radius) # the parentheses aren't necessary
Note that you have two variables, left and right, which are trying to describe properties of a list of disks.
Note that these are the same variables that you're using to iterate through a list of coordinates. They can't do both at once -- make a couple more storage areas, such as left_touches and right_touches.
That should be enough for now. Live Long, And Program.
I'm working on a web crawler. The crawler is built for web page which has many categories. These categories can have subcategories, the same for subcategories etc.
So it can seems like this:
So I made a recursive method which provides deep first search.
def deep_search(url):
if is_leaf(url):
return get_data(url)
for url in get_subcategories(url):
deep_search(url)
This method works fine but it takes a long time to finish so there are situations when connection falls or another error raises.
What would you do to remember state in case that error occures and next time it continues from this state?
I can't just remember last 'url' or category since there are loops and the program would not know what 'urls' and categories has been stored in upper loops.
If the order of search paths is stable (every time your script visits sub-categories in the same order), then you can maintain a branch number list in your DFS, and make it persistent - save it in a file or database:
current_path = [] # save the path currently visited
def deep_search(url, last_saved_path=None):
if is_leaf(url):
if last_saved_path:
# Continue where you left off
if path_reached(last_saved_path):
data = get_data(url)
else: # first run
data = get_data(url)
# save the whole path persistently
save_to_file(current_path)
# add data to result
else:
for index, url in enumerate(get_subcategories(url)):
current_path.append(index)
deep_search(url, last_saved_path)
del current_path[-1]
def path_reached(old_path):
print old_path, current_path
# if the path has been visited in last run
for i,index in enumerate(current_path):
if index < old_path[i]:
return False
elif index > old_path[i]:
return True
return True
When running the crawler for a second time, you can load the saved path and start where you left off:
# first run
deep_search(url)
# subsequent runs
last_path = load_last_saved_path_from_file()
deep_search(url, last_path)
That said, I think in a web crawler there are 2 kind of tasks: traversing the graph and downloading data. And it's better to keep them separate: use the above DFS algorithm (plus logic to skip paths that have been visited) to traverse the links, and save the download urls in a queue; Then start a bunch of workers to take urls from the queue and download. This way, you just need to record the current position in queue if interrupted.
And I recommend scrapy to you, I haven't read scrapy source, but I guess it implements all of the above, and more.
As a simple hint you can use a try-except statement to handle your Errors and save the relative url and as a good choice for such task you can use collections.deque with 1 capacity,and check it in next iterations.
Demo :
from collections import deque
def deep_search(url,deq=deque(maxlen=1)):
if is_leaf(url):
return get_data(url)
try:
for url in get_subcategories(url):
if deq[0]==url:
deep_search(url,deq)
except : #you can put the error title after except
deq.append(url)
But as a more pythonic way for dealing with networks you can use networkx.
NetworkX is a Python language software package for the creation, manipulation, and study of the structure, dynamics, and functions of complex networks.
I need to try all possible paths, branching every time I hit a certain point. There are <128 possible paths for this problem, so no need to worry about exponential scaling.
I have a player that can take steps through a field. The player
takes a step, and on a step there could be an encounter.
There are two options when an encounter is found: i) Input 'B' or ii) Input 'G'.
I would like to try both and continue repeating this until the end of the field is reached. The end goal is to have tried all possibilities.
Here is the template, in Python, for what I am talking about (Step object returns the next step using next()):
from row_maker_inlined import Step
def main():
initial_stats = {'n':1,'step':250,'o':13,'i':113,'dng':0,'inp':'Empty'}
player = Step(initial_stats)
end_of_field = 128
# Walk until reaching an encounter:
while player.step['n'] < end_of_field:
player.next()
if player.step['enc']:
print 'An encounter has been reached.'
# Perform an input on an encounter step:
player.input = 'B'
# Make a branch of player?
# perform this on the branch:
# player.input = 'G'
# Keep doing this, and branching on each encounter, until the end is reached.
As you can see, the problem is rather simple. Just I have no idea, as a beginner programmer, how to solve such a problem.
I believe I may need to use recursion in order to keep branching. But I really just do not understand how one 'makes a branch' using recursion, or anything else.
What kind of solution should I be looking at?
You should be looking at search algorithms like breath first search (BFS) and depth first search (DFS).
Wikipedia has this as the pseudo-code implementation of BFS:
procedure BFS(G, v) is
let Q be a queue
Q.enqueue(v)
label v as discovered
while Q is not empty
v← Q.dequeue()
for all edges from v to w in G.adjacentEdges(v) do
if w is not labeled as discovered
Q.enqueue(w)
label w as discovered
Essentially, when you reach an "encounter" you want to add this point to your queue at the end. Then you pick your FIRST element off of the queue and explore it, putting all its children into the queue, and so on. It's a non-recursive solution that is simple enough to do what you want.
DFS is similar but instead of picking the FIRST element form the queue, you pick the last. This makes it so that you explore a single path all the way to a dead end before coming back to explore another.
Good luck!
I'm trying to build a program that controls an RGB LED through a RaspberryPi.
I was able to build a simple fade program in python using pi-blaster, which works fine but doesn't let me do what I want.
Here's my code:
import time
import sys
import os
blue = 21
green = 23
red = 22
def fade(color, direction, step):
if direction == 'up':
for i in range(0, 100, step):
f=(i/float(100))
os.system('echo "%d=%f" > /dev/pi-blaster' % (color, f))
return fade(color, 'down', step)
else:
step=-step
for i in range (100, 0, step):
f=(i/float(100))
os.system('echo "%d=%f" > /dev/pi-blaster' % (color, f))
return fade(color, 'up', step)
input = raw_input("Choose a color (r, g, b): ")
if input == 'r':
fade(red, 'up', 1)
if input == 'g':
fade(green, 'up', 1)
if input == 'b':
fade(blue, 'up', 1)
The problem is that I want to be able to control the fade through an external script / program.
My idea is to have a script which is always listening for user input, so when I type "red" it stops the ongoing fade and starts a new red one, by calling the function I posted before.
I also want to be able to change the speed of the loop but this is already implemented into the "fade" function.
I don't really understand how I could do this. I've read some things on here and I think I may have to use the Threading functions, but I don't really understand how those could work for my project.
Thanks in advance for your help and sorry for my English which is not very good :)
EDIT:
I solved it using a loop checking continuously for keyboard inputs which then calls the fade function using multiprocessing.
I can now kill the fade process using processName.terminate()
to get much better performance (and as a clarification to the first answer) it is much better to write using python - but you must put a new line after each write with a '\n'
Otherwise the code #daveydave400 gave is spot on
I'm doing something like this:
f = open('/dev/pi-blaster', 'w')
f.write('0=120\n')
#or
f.write('0=%s\n'%amount)
my actual code is in a function - in case you want to use that:
def changer(pin, amount):
f = open('/dev/pi-blaster', 'w')
f.write('%d=%s\n'%(pin, str(amount)))
if doing for loops using the the system command method you might find that python stops listening for keyboard input intermittently. This was certainly happening for me. I found that keyboardinterrupt was very hard to achieve with ctrl-c and had to repeatedly hit those keys to get the program to halt. Now i'm writing with python i'm not getting that issue.
There are a lot of ways to doing this, I'll describe a hard way and an easy way to get the basic functionality you want and hopefully they give you some ideas on how you want to solve this. The hard way involves creating a daemon-like script that sits in the background and reads a socket or file for when to change what it's doing. This is probably a good learning experience so don't be afraid of trying this. This method is also similar to the easy method I describe below. You would create a python script that opens a UDP or TCP socket, then it starts a separate process that calls the fade function. The main process listens to the socket, when it gets a new valid command (red, green, blue, quit, etc) it kills the child process and starts another with the new color. You could use threads, but I'm pretty sure killing threads from the parent is not so pretty, so you'd have to use events/locks and maybe some global variables. There are plenty of different ways to do this, but the general idea of one process/thread listens and one does the fading will be the same.
The easy/fast way is to have a bash script that wraps your color changes (red.sh, green.sh, blue.sh). In these scripts they kill any previously running scripts and then start the new fade command. Knowing what to kill can be done a couple of ways, the easiest is probably to write the PID to a file, then when it needs to be killed read that PID number in and kill it.
I know I didn't provide any code and this is long, but there are a lot of ways to do what you are trying to do. Also, you should be able open/write the "/dev/pi-blaster" directly in python (not sure opening device files this way will work):
blaster_file = open("/dev/pi-blaster", "a")
for i in range(100, 0, step):
blaster_file.write(...)
blaster_file.close()
To change the speed of the loop you could add a delay by using sleep and specify the duration with a keyword:
from time import sleep
def fade(color, direction, step, delay=0):
...
for i in range(100, 0, step):
...
sleep(delay)
return fade(color, direction, step, delay=delay)