We Keep Coding

What could be a better alternative to this For?

I'm a beginner trying to write up a game about trading in Python 3.x.
I have two classes (Depot and Good). Instances of the class Good are stored in a dictionary inside depot.inv (the instance as Key, and the amount of it as Value). When the user is asked to write what does he want to take, he will write the name of the instance ('Iron' instead of ir). So I took a For loop and searched through everything the Depot had in it's inventory. If he finds that the reply of the user is = to the name of any instance (ir.name, for example), then the program gains access to the instance as a whole.
The question is, how can I do this without a For-Loop?
I imagine that searching the whole inventory each time a reply is made is not optimal, even less if it's a linear search.
class Depot:
def __init__ (self, name, inv, bank):
self.name = name
self.inv = inv
self.bank = bank
class Good:
def __init__(self, name, weight, rare):
self.name = name
self.weight = weight
self.rare = rare
ir = Good('Iron', 1, 0.1)
gd = Good('Gold', 0.4, 2)
sl = Good('Silver', 0.7, 6.3)
mars = Depot('Mars', {ir: 10500, gd: 800, sl: 6000}, 1000)
player = {ir: 100}
reply = input('What do you want to take?')
i, q = reply.split()
for k in mars.inv.keys(): #This is the crux of the problem
if i in k.name:
x = k
print('Got it')
if x in mars.inv:
if int(q) > mars.inv[x]:
print('We dont have that much.')
elif int(q) <= mars.inv[x]:
mars.inv[x] -= int(q)
if x in player:
player[x] += int(q)
elif i not in player:
player[x] = int(q)

Using list comprehension and next() allows you to only find the first occurance of the item you're looking for, and then stop. Breaking the for loop after finding the key you're looking for would achieve the same thing. Both are O(n) there isn't really that much room to optimize. A one liner would be
x = next((k for k in mars.inv.keys() if i in k.name), None)

I'd probably duplicate the name as the key of the dictionary and map it to a tuple of the other information:
{"Iron": (ir, 10500), "Gold": (gd, 800), "Silver": (sl, 6000}
Then you can do mars.inv["Iron"] to get the tuple of (ir, 10500), and you can extract whichever data you want from that.
Note though, this requires an exact match on the name. You're currently using in to do the check, which will prevent any optimizations. Since from your description, you're doing lookups based on exact names (ignoring case, since that can be easily accounted for), in is unnecessary.

How can I multithread a function that reads a list of objects in Python? Astrophysics example code

This is my first post to stack overflow. I'll try to include all the necessary information, but please let me know if there's more info I can provide to clarify my question.
I'm trying to multithread a costly function for an astrophysical code in python using pool.map. The function takes as an input a list of objects. The basic code structure is like this:
There's a class of Stars with physical properties:
Class Stars:
def __init__(self,mass,metals,positions,age):
self.mass = mass
self.metals = metals
self.positions = positions
self.age = age
def info(self):
return(self.mass,self.metals,self.positions,self.age)
and there's a list of these objects:
stars_list = []
for i in range(nstars):
stars_list.append(Stars(mass[i],metals[i],positions[i],age[i]))
(where mass, metals, positions and age are known from another script).
There's a costly function that I run with these star objects that returns a spectrum for each one:
def newstars_gen(stars_list):
....
return stellar_nu,stellar_fnu
where stellar_nu and stellar_fnu are numpy arrays
What I would like to do is break the list of star objects (stars_list) up into chunks, and then run newstars_gen on these chunks on multiple threads to gain a speedup. So, to do this, I split the list up into three sublists, and then try to run my function through pool.map:
p = Pool(processes = 3)
nchunks = 3
chunk_start_indices = []
chunk_start_indices.append(0) #the start index is 0
delta_chunk_indices = nstars / nchunks
for n in range(1,nchunks):
chunk_start_indices.append(chunk_start_indices[n-1]+delta_chunk_indices)
for n in range(nchunks):
stars_list_chunk = stars_list[chunk_start_indices[n]:chunk_start_indices[n]+delta_chunk_indices]
#if we're on the last chunk, we might not have the full list included, so need to make sure that we have that here
if n == nchunks-1:
stars_list_chunk = stars_list[chunk_start_indices[n]:-1]
chunk_sol = p.map(newstars_gen,stars_list_chunk)
But when I do this, I get as the error:
File "/Users/[username]/python2.7/multiprocessing/pool.py", line 250, in map
return self.map_async(func, iterable, chunksize).get()
File "/Users/[username]/python2.7/multiprocessing/pool.py", line 554, in get
raise self._value
AttributeError: Stars instance has no attribute '__getitem__'
So, I'm confused as to what sort of attribute I should include with the Stars class. I've tried reading about this online and am not sure how to define an appropriate __getitem__ for this class. I'm quite new to object oriented programming (and python in general).
Any help is much appreciated!

So, it looks like there may be a couple things wrong here and that could be cleaned up or made more pythonic. However, the key problem is that you are using pool.multiprocessing.Pool.map incorrectly for what you have. Your newstars_gen function expects a list, but p.map is going to break up the list you give it into chunks and hand it one Star at a time. You should probably rewrite newstars_gen to operate on one star at a time and then throw away all but the first and last lines of your last code block. If the calculations in newstars_gen aren't independent between Stars (e.g., the mass of one impacts the calculation for another), you will have to do a more dramatic refactoring.
It also looks like it would behoove you to learn about list comprehensions. Be aware that the other built in structures (e.g., set, dict) have equivalents, and also look into generator comprehensions.

I've written a function for distributing the processing of an iterable (like your list of stars objects) among multiple processors, which I'm pretty sure will work well for you.
from multiprocessing import Process, cpu_count, Lock
from sys import stdout
from time import clock
def run_multicore_function(iterable, function, func_args = [], max_processes = 0):
#directly pass in a function that is going to be looped over, and fork those
#loops onto independant processors. Any arguments the function needs must be provided as a list.
if max_processes == 0:
cpus = cpu_count()
if cpus > 7:
max_processes = cpus - 3
elif cpus > 3:
max_processes = cpus - 2
elif cpus > 1:
max_processes = cpus - 1
else:
max_processes = 1
running_processes = 0
child_list = []
start_time = round(clock())
elapsed = 0
counter = 0
print "Running function %s() on %s cores" % (function.__name__,max_processes)
#fire up the multi-core!!
stdout.write("\tJob 0 of %s" % len(iterable),)
stdout.flush()
for next_iter in iterable:
if type(iterable) is dict:
next_iter = iterable[next_iter]
while 1: #Only fork a new process when there is a free processor.
if running_processes < max_processes:
#Start new process
stdout.write("\r\tJob %s of %s (%i sec)" % (counter,len(iterable),elapsed),)
stdout.flush()
if len(func_args) == 0:
p = Process(target=function, args=(next_iter,))
else:
p = Process(target=function, args=(next_iter,func_args))
p.start()
child_list.append(p)
running_processes += 1
counter += 1
break
else:
#processor wait loop
while 1:
for next in range(len(child_list)):
if child_list[next].is_alive():
continue
else:
child_list.pop(next)
running_processes -= 1
break
if (start_time + elapsed) < round(clock()):
elapsed = round(clock()) - start_time
stdout.write("\r\tJob %s of %s (%i sec)" % (counter,len(iterable),elapsed),)
stdout.flush()
if running_processes < max_processes:
break
#wait for remaining processes to complete --> this is the same code as the processor wait loop above
while len(child_list) > 0:
for next in range(len(child_list)):
if child_list[next].is_alive():
continue
else:
child_list.pop(next)
running_processes -= 1
break #need to break out of the for-loop, because the child_list index is changed by pop
if (start_time + elapsed) < round(clock()):
elapsed = round(clock()) - start_time
stdout.write("\r\tRunning job %s of %s (%i sec)" % (counter,len(iterable),elapsed),)
stdout.flush()
print " --> DONE\n"
return
As a usage example, let's use your star_list, and send the result of newstars_gen to a shared file. Start by setting up your iterable, file, and a file lock
star_list = []
for i in range(nstars):
stars_list.append(Stars(mass[i],metals[i],positions[i],age[i]))
outfile = "some/where/output.txt"
file_lock = Lock()
Define your costly function like so:
def newstars_gen(stars_list_item,args): #args = [outfile,file_lock]
outfile,file_lock = args
....
with file_lock:
with open(outfile,"a") as handle:
handle.write(stellar_nu,stellar_fnu)
Now send your list of stars into run_multicore_function()
run_multicore_function(star_list, newstars_gen, [outfile,file_lock])
After all of your items have been calculated, you can go back into the output file to grab the data and carry on. Instead of writing to a file, you can also share the state with multiprocessing.Value or multiprocessing.Array, but I've ran into the occasional issue with data getting lost if my list is large and the function I'm calling is fairly fast. Maybe someone else out there can see why that's happening.
Hopefully this all makes sense!
Good luck,
-Steve

MiniMax Recursive Algorithm (Python)

I am pretty new so I apologize if I make any errors in posting here... I did a search but didn't come up with much that would help me. I am writing a miniMax algorithm for a variation on Tic Tac Toe. This variation allows either player to put an X or an O anywhere on the board. I am having trouble with the recursion and was hoping I could get a bit of guidance.
class TicTacToeBoard:
def __init__(self, initGameBoard):
#initGameBoard is a string
self.board = initGameBoard
def getEmptySpaces(self):
return self.board.count("-")
def markX(self, index):
self.board = self.board[:index] + "x" + self.board[index+1:]
def markO(self, index):
self.board = self.board[:index] + "o" + self.board[index+1:]
def endGame(self):
#determines if someone has won
endGameStates = [[0,1,2],[3,4,5],[6,7,8],[0,3,6],[1,4,7],[2,5,8],[0,4,8],[2,4,6]]
for x in range(len(endGameStates)):
trySlice = self.board[endGameStates[x][0]] + self.board[endGameStates[x][1]] + \
self.board[endGameStates[x][2]]
if trySlice[0] == trySlice[1] == trySlice[2] and "-" not in trySlice:
return True
return False
def draw(self):
#determines if there has been a draw
if "-" not in self.board:
endGameStates = [[0,1,2],[3,4,5],[6,7,8],[0,3,6],[1,4,7],[2,5,8],[0,4,8],[2,4,6]]
for x in range(len(endGameStates)):
trySlice = self.board[endGameStates[x][0]] + self.board[endGameStates[x][1]] + \
self.board[endGameStates[x][2]]
if trySlice[0] == trySlice[1] == trySlice[2] and "-" not in trySlice:
return False
return True
else:
return False
def __str__(self):
boardStr = ""
for char in self.board:
boardStr += char
return boardStr
Above is my board class. I'm just using strings, not doing anything too fancy. I'm also using a very simple Node class that just stores data (though I suppose I might be able to just use strings too I guess...)
from tic_tac_toe_board import TicTacToeBoard
from node import Node
nodeQueue = []
def fitnessFunction(gameBoard):
#only runs if end game or if all full
if gameBoard.draw():
return 0
else:
emptySpaces = gameBoard.getEmptySpaces()
if emptySpaces %2 == 0:
#max won
return (emptySpaces + 1) *1
else:
#max lost
return (emptySpaces + 1) *-1
def miniMax(gameBoard):
if gameBoard.endGame() or if "-" not in gameBoard:
#end game checks for winner, second clause checks for full/draw
return fitnessFunction(gameBoard)
else:
emptyIndexes = [] #keeps track of which indexes are empty
count = 0
for char in gameBoard:
if char == "-":
emptyIndexes.append(count)
count +=1
if len(emptyIndexes) %2 != 0:
#max's turn
for index in emptyIndexes:
childNode = Node(gameBoard.markX(index))
nodeQueue.append(childNode)
childNode = Node(gameBoard.markO(index))
nodeQueue.append(childNode)
return miniMax()
The fitnessFunction returns a score based on the number of empty spaces left. I'm having trouble with my recursive miniMax method. What I need to do is check for the base cases (either player winning, or a draw) and if those base cases are not true, I figure out whose move it is based on the number of empty spaces left. I think I've gotten that far, but I don't know what to do next (the recursive part). I also need to be able to get the min or max of children, depending on whose turn it is. I guess I am lost with the recursion. I'm new to CS and haven't touched much on it. Any hints would be greatly appreciated! :)

If you are looking for a mini-max algorithm, you don't need an example upon which to apply it. Most games require a mini-max algorithm.
So if you want one, decide how it must behave. Then write a test for at least one example of that behavior.
Post your test. (Not a game, but just a test of the data that would result from a game.)
If your algorithm does not work, your game program can't work.
Hint: A mini-max algorithm depends only upon the evaluations of game paths, not upon the game being played.

How do I keep a list of numbers I have already enountered?

I am currently working on a BASIC simulator in Python, as the title suggests. Here is my code for this problem:
def getBASIC():
l = []
x = 1
while x == 1:
i = input()
l.append(i)
if len(i.split()) != 3:
x = 0
return l
def findLine(prog, target):
for l in range(0, len(prog)):
progX = prog[l].split()
if progX[0] == target:
return l
def execute(prog):
location = 0
visited = [False] * len(prog)
while True:
T = prog[location].split()[2]
location = findLine(prog, T)
visited[location] = True
if visited[len(visited)-1] == False:
return "infinite loop"
else:
return "success"
The first function does what it is intended to do -- convert input of BASIC code into a list. The second function, findLine also does what it is intended to do, in that it finds the item which contains the string equal to the input. The last function, however, I cannot get to work. I know what I have to do, and that is to check whether or not a part of it has been visited twice. I cannot figure out how to do this, due to the existence of the while loop. As a result of this, the second half of that function is just placeholder. If you could help me figure out how to solve this, it would be greatly appreciated. Thanks.

You keep a list of places that have been visited (you already do this) and then when you encounter a goto, you check if it does to a line that already have been visited, and if it has been visited, you exit.
One mistake right now is that you make a list that is as long as the program is. That's pretty pointless. Just keep a list of the visited line numbers instead, and check with
if current_line in visited:

Try adding an if statement declaring a line in the visited list to be true when it is encountered in the loop. This is my solution:
def execute(prog):
location = 0
visited=[False]*len(prog)
while True:
if location==len(prog)-1:
return "success"
if visited[location]==True:
return "infinite loop"
if visited[location]==False:
visited[location]=True
line2strings=prog[location].split()
T=line2strings[-1]
location=findLine(prog, T)

Does anybody have gdb pretty-printer code for an fd_set on linux?

Modern versions of gdb allow integration of python code to "pretty print" complex data structures. There are some great pretty printer implementations for C++'s STL classes as well as some of the more common boost.org types.
In network programming, one commonly encounters select/poll calls. While poll() uses an array of data structures, select() uses fd_set.
Has anyone run across a pretty printer implementation for fd_set, preferably portable, but even platform-specific would be okay. Ideally, it'd be linux/x86, but I'd take anything and hope to be able to adapt.

Alrighty, here's something I wrote which seems to work for me under Linux. Let me know if it works for you:
anonprint.py
import gdb
class fd_set_printer:
"""
Prints an fd_set, which is normally an opaque
array of ints, each bit representing one file descriptor
"""
def __init__(self, val, val_array):
self.val = val
self.val_array = val_array
#staticmethod
def find_set_bits(bit_array):
"""
Finds set bits in a long bit list.
Expects a gdb.Value which contains a C array,
such as int[10], and treats it as a bitlist
of int_size * 10 bits long. Returns an array of
bit positions, starting with 0, for which the bits
are on.
e.g. for int foo[] = [1, 6], it will return [ 0, 33, 34 ]
The array should be given as a gdb.Value
"""
set_bits = []
bits_length = bit_array[0].type.sizeof * 8
current_bit = 0
# Can not use 'for current_byte in gdb.Value:' even if
# gdb.Value.type.code == gdb.TYPE_CODE_ARRAY
# So iteration happens this ugly C-style way
for current_byte_pos in xrange(*bit_array.type.range()):
current_byte = bit_array[current_byte_pos]
for bit in xrange(0, bits_length):
bit_mask = 1 << bit
if bit_mask & current_byte == bit_mask:
set_bits.append(current_bit)
current_bit += 1
return set_bits
def to_string(self):
fd_list = self.find_set_bits(self.val_array)
if len(fd_list) == 0:
output = "Empty file descriptor set."
else:
output = "File descriptor set: "
output += ', '.join(map(str, fd_list))
return output
def anon_struct_lookup_function(val):
"""
Checks if the given value looks like an fd_set.
If it does, delegates printing to the printer
"""
lookup_tag = val.type.tag
if lookup_tag == None:
return None
if lookup_tag != "<anonymous struct>":
return None
fields = val.type.fields()
val_array = None
if len(fields) == 1 and fields[0].name == 'fds_bits':
val_array = val['fds_bits']
elif len(fields) == 1 and fields[0].name == '__fds_bits':
val_array = val['__fds_bits']
if not val_array is None:
return fd_set_printer(val, val_array)
return None
def add_fd_set_printer(obj = gdb):
"Adds the fd_set pretty printer to the given object"
obj.pretty_printers.append(anon_struct_lookup_function)
And then make your ~/.gdbinit:
python
import sys
sys.path.insert(0, '/home/user/anonprint_py_directory_here')
from anonprint import add_fd_set_printer
add_fd_set_printer()
end
This is the first time I tried to interact with gdb internals via Python, so commentary and suggestions are welcome.