We Keep Coding

How to transform a OpenAI gym Box Space into a Discrete

To use Q-learning I need to create a bi-dimensional NumPy array with size [nº of actions][nº of states].
My actions are a list of directions that I can easily convert to integers.
My state is a Box: gym.spaces.Box(low=np.int8(0), high=np.int8(2), shape=(5, 5), dtype=np.int8)
I need to find a way to take my state and convert it to an integer to be able to create the Q-table. I'm aware of wrappers but don't know how to use them and which one should be used.
The previous section is a quick summary, here's the problem in more detail:
I'm creating an OpenAi gym environment for the game Neutreeko. The states are the 8 directions and the environment is the Box (or a NumPy array). The creator of the game states:
There are 3,450,516 valid board positions. source
So I need to find a way to map each board to an ID to use as the index for the Q-table.
I asked the creator for a bit of aid on how he got to this number and he answered:
I am not entirely sure how I arrived at 3,450,515 (this would date back around 19 years). I have probably assumed that the Next player (N) does not already have three in a row, which gives them C(25, 3) - 48 = 2252 possible positions for their pieces, where C(n, k) denotes the binomial coefficient (n over k). The Previous player (P) should then have C(25 - 3, 3) = 1540 possible position for each of them, for a total of 2252 x 1540 = 3,468,080 positions. Most likely I have subtracted positions that are impossible to reach because there is no position from which the Previous player could have made a legal move and reached the current position.
Taking a simple example of 5 available positions and 3 pieces, there are 10 possible dispositions for the pieces, but I can't find a way to turn each disposition into an id using the available information (index of pieces allocated). Here's a try:

set variable number of input nodes in python neat

i have this simple game where there is a ball bouncing on the screen and the player can move left and right of the screen and shoot an arrow up to pop the ball, every time the player hits a ball, the ball bursts and splits into two smaller balls until they reach a minimum size and disappear.
I am trying to solve this game with a genetic algorithm based on the python neat library and on this tutorial on flappy bird https://www.youtube.com/watch?v=MMxFDaIOHsE&list=PLzMcBGfZo4-lwGZWXz5Qgta_YNX3_vLS2, so I have a configuration file in which I have to specify how many input nodes must be in the network, I had thought to give as input the player's x coordinate, the distance between the player's x-coordinate and the ball's x-coordinate and the distance between the player's y-coordinate and the ball's y-coordinate.
My problem is that at the beginning of the game I have only one ball but after a few moves I could have more balls in the screen so I should have a greater number of input nodes,the more balls there are on the screen the more input coordinates I have to provide to the network.
So how to set the number of input nodes in a variable way?
config-feedforward.txt file
"""
# network parameters
num_hidden = 0
num_inputs = 3 #this needs to be variable
num_outputs = 3
"""
python file
for index,player in enumerate(game.players):
balls_array_x = []
balls_array_y = []
for ball in game.balls:
balls_array_x.append(ball.x)
balls_array_x.append(ball.y)
output = np.argmax(nets[index].activate(("there may be a number of variable arguments here")))
#other...
final code
for index,player in enumerate(game.players):
balls_array_x = []
balls_array_y = []
for ball in game.balls:
balls_array_x.append(ball.x)
balls_array_y.append(ball.y)
distance_list = []
player_x = player.x
player_y = player.y
i = 0
while i < len(balls_array_x):
dist = math.sqrt((balls_array_x[i] - player_x) ** 2 + (balls_array_y[i] - player_y) ** 2)
distance_list.append(dist)
i+=1
i = 0
if len(distance_list) > 0:
nearest_ball = min(distance_list)
output = np.argmax(nets[index].activate((player.x,player.y,nearest_ball)))

This is a good question and as far as I can tell from a quick Google search hasn't been addressed for simple ML algorithms like NEAT.
Traditionally resizing methods of Deep NN (padding, cropping, RNNs, middle-layers, etc) can obviously not be applied here since NEAT explicitly encodes each single neuron and connection.
I am also not aware of any general method/trick to make the input size mutable for the traditional NEAT algorithm and frankly don't think there is one. Though I can think of a couple of changes to the algorithm that would make this possible, but that's of no help to you I suppose.
In my opinion you therefore have 3 options:
You increase the input size to the maximum number of balls the algorithm should track and set the x-diff/y-diff value of non-existent balls to an otherwise impossible number (e.g. -1). If balls come into existence you actually set the values for those x-diff/y-diff input neurons and set them to -1 again when they are gone. Then you let NEAT figure it out. Also worth thinking about concatenating 2 separate NEAT NNs, with the first NN having 2 inputs, 1 output and the second NN having 1 (player pos) + x (max number of balls) inputs and 2 outputs (left, right). The first NN produces an output for each ball position (and is identical for each ball) and the second NN takes the first NNs output and turns it into an action. Also: The maximum number of balls doesn't have to be the maximum number of displayable balls, but can also be limited to 10 and only considering the 10 closest balls.
You only consider 1 ball for each action side (making your input 1 + 2*2). This could be the consideration of the lowest ball on each side or the closest ball on each side. Such preprocessing can make such simple NN tasks however quite easy to solve. Maybe you can add inertia into your test environment and thereby add a non-linearity that makes it not so straightforward to always teleport/hurry to the lowest ball.
You input the whole observation space into NEAT (or a uniformly downsampled fraction), e.g. the whole game at whatever resolution is lowest but still sensible. I know that this observation space is huge, but NEAT works quite well in handling such spaces.
I know that this is not the variable input size option of NEAT that you might have hoped for, but I don't know about any such general option/trick without changing the underlying NEAT algorithm significantly.
However, I am very happy to be corrected if someone knows a better option!

Fast great circle for multiple points - Python geopy

Is it possible to speed up the great_circle(pos1, pos2).miles from geopy if using it for multiple thousand points?
I want to create something like a distance matrix and at the moment my machine needs 5 seconds for 250,000 calculations.
Actually pos1 is always the same if it helps.
Another "restriction" in my case is that I only want all points pos2 which have a distance less than a constant x.
(The exact distance doesn't matter in my case)
Is there a fast method? Do I need to use a faster function than great_circle which is less accurate or is it possible to speed it up without losing accuracy?
Update
In my case the question is whether a point is inside a circle.
Therefore it is easily possible to first get whether a point is inside a square.
start = geopy.Point(mid_point_lat, mid_point_lon)
d = geopy.distance.VincentyDistance(miles=radius)
p_north_lat = d.destination(point=start, bearing=0).latitude
# check whether the given point lat is > p_north_lat
# and so on for east, south and west

How to generate statistically probably locations for ships in battleship

I made the original battleship and now I'm looking to upgrade my AI from random guessing to guessing statistically probably locations. I'm having trouble finding algorithms online, so my question is what kinds of algorithms already exist for this application? And how would I implement one?
Ships: 5, 4, 3, 3, 2
Field: 10X10
Board:
OCEAN = "O"
FIRE = "X"
HIT = "*"
SIZE = 10
SEA = [] # Blank Board
for x in range(SIZE):
SEA.append([OCEAN] * SIZE)
If you'd like to see the rest of the code, I posted it here: (https://github.com/Dbz/Battleship/blob/master/BattleShip.py); I didn't want to clutter the question with a lot of irrelevant code.

The ultimate naive solution wold be to go through every possible placement of ships (legal given what information is known) and counting the number of times each square is full.
obviously, in a relatively empty board this will not work as there are too many permutations, but a good start might be:
for each square on board: go through all ships and count in how many different ways it fits in that square, i.e. for each square of the ships length check if it fits horizontally and vertically.
an improvement might be to also check for each possible ship placement if the rest of the ships can be placed legally whilst covering all known 'hits' (places known to contain a ship).
to improve performance, if only one ship can be placed in a given spot, you no longer need to test it on other spots. also, when there are many 'hits', it might be quicker to first cover all known 'hits' and for each possible cover go through the rest.
edit: you might want to look into DFS.
Edit: Elaboration on OP's (#Dbz) suggestion in the comments:
hold a set of dismissed placements ('dissmissed') of ships (can be represented as string, say "4V5x3" for the placement of length 4 ship in 5x3, 5x4, 5x5, 5x6), after a guess you add all the placements the guess dismisses, then for each square hold a set of placements that intersect with it ('placements[x,y]') then the probability would be:
34-|intersection(placements[x,y], dissmissed)|/(3400-|dismissed|)
To add to the dismissed list:
if guess at (X,Y) is a miss add placements[x,y]
if guess at (X,Y) is a hit:
add neighboring placements (assuming that ships cannot be placed adjacently), i.e. add:
<(2,3a,3b,4,5)>H<X+1>x<Y>, <(2,3a,3b,4,5)>V<X>x<Y+1>
<(2,3a,3b,4,5)>H<X-(2,3,3,4,5)>x<Y>, <(2,3a,3b,4,5)>V<X>x<Y-(2,3,3,4,5)>
2H<X+-1>x<Y+(-2 to 1)>, 3aH<X+-1>x<Y+(-3 to 1)> ...
2V<X+(-2 to 1)>x<Y+-1>, 3aV<X+(-3 to 1)>x<Y+-1> ...
if |intersection(placements[x,y], dissmissed)|==33, i.e. only one placement possible add ship (see later)
check if any of the previews hits has only one possible placement left, if so, add the ship
check to see if any of the ships have only possible placement, if so, add the ship
adding a ship:
add all other placements of that ship to dismissed
for each (x,y) of the ships placement add placements[x,y] with out the actual placement
for each (x,y) of the ships placement mark as hit guess (if not already known) run stage 2
for each (x,y) neighboring the ships placement mark as miss guess (if not already known) run stage 1
run stage 3 and 4.
i might have over complicated this, there might be some redundant actions, but you get the point.

Nice question, and I like your idea for statistical approach.
I think I would have tried a machine learning approach for this problem as follows:
First model your problem as a classification problem.
The classification problem is: Given a square (x,y) - you want to tell the likelihood of having a ship in this square. Let this likelihood be p.
Next, you need to develop some 'features'. You can take the surrounding of (x,y) [as you might have partial knowledge on it] as your features.
For example, the features of the middle of the following mini-board (+ indicates the square you want to determine if there is a ship or not in):
OO*
O+*
?O?
can be something like:
f1 = (0,0) = false
f2 = (0,1) = false
f3 = (0,2) = true
f4 = (1,0) = false
**note skipping (1,1)
f5 = (1,2) = true
f6 = (2,0) = unknown
f7 = (2,1) = false
f8 = (2,2) = unknown
I'd implement features relative to the point of origin (in this case - (1,1)) and not as absolute location on board (so the square up to (3,3) will also be f2).
Now, create a training set. The training set is a 'labeled' set of features - based on some real boards. You can create it manually (create a lot of boards), automatically by a random generator of placements, or by some other data you can gather.
Feed the training set to a learning algorithm. The algorithm should be able to handle 'unknowns' and be able to give probability of "true" and not only a boolean answer. I think a variation of Naive Bayes can fit well here.
After you have got a classifier - exploit it with your AI.
When it's your turn, choose to fire upon a square which has the maximal value of p. At first, the shots will be kinda random - but with more shots you fire, you will have more information on the board, and the AI will exploit it for better predictions.
Note that I gave features based on a square of size 1. You can of course choose any k and find features on this bigger square - it will give you more features, but each might be less informative. There is no rule of thumb which will be better - and it should be tested.

Main question is, how are you going to find statistically probable locations. Are they already known or you want to figure them out?
Either case, I'd just make the grid weighed. In your case, the initial weight for each slot would be 1.0/(SIZE^2). The sum of weights must be equal to 1.
You can then adjust weights based on the statistics gathered from N last played games.
Now, when your AI makes a choice, it chooses a coordinate to hit based on weighed probabilities. The quick and simple way to do that would be:
Generate a random number R in range [0..1]
Start from slot (0, 0) adding the weights, i.e. S = W(0, 0) + W(0, 1) + .... where W(n, m) is the weight of the corresponding slot. Once S >= R, you've got the coordinate to hit.
This can be optimised by pre-calculating cumulative weights for each row, have fun :)

Find out which ships are still alive:
alive = [2,2,3,4] # length of alive ships
Find out spots where you have not shot, for example with a numpy.where()
Loop over spots where you can shoot.
Check the sides of the given position. Go left and right, how many spaces? Go up and down, how many spaces? If you can fit a boat in that many spaces, you can fit any smaller boat, so this loop I'd do it from the largest ship downwards, and I'd add to the counts in this position as many +1 as ships smaller than the one that fits.
Once you have done all of this, the position with more points should be the most probable to attack and hit something.
Of course, it can get as complicated as you want. You can also ask yourself, instead of which is my next hit, which combinations of hits will give me the victory in less number of hits or any other combination/parametrization of the problem. Good luck!

"Ticket to Ride" board game logic for gray routes

I enjoy playing Ticket to Ride, so I decided to play around with implementing parts of the game logic in Python as a side programming project. The game board is essentially a weighted multigraph, so replicating the basic structure of the game with NetworkX was a cinch.
The one part I'm having trouble with is analyzing whether a particular path through the board is possible given an inventory of train cards the player possesses. I think it's more of a math problem than a programming problem per se, and I can probably piece together a brute force method for figuring things out, but thought there must be a more efficient way.
For those who don't know the game: at any given time, each player has a number of train cards in one of eight colors, plus a special "locomotive" category that serves as a wild card. These colors correspond to the color of train lines on the game board (shown here) except for the gray lines, where you can use any color, as long as all cars in the segment are the same color. (There are edge cases involving tunnels and ferries, but we'll leave those aside for now.)
With the code as it stands now, I can find all paths between two given cities and get back how many train cards of each color are needed to take that particular path, unless the paths involve gray segments. I do the non-gray segments first since they're more straightforward -- either you have enough red/green/blue cards for each red/green/blue segment in the path or you don't. With gray, because you can pick any color to use for each segment, it gets a bit more involved.
For paths with just one gray segment, it's still easy -- either you have enough cards of any one color to fill it in or not. With multiple gray segments, however, one can run into situations where the color chosen for the first segment makes completing the second or third segment impossible.
As an example, suppose a player's card inventory is 4 red, 2 green, 3 blue, and we're trying to figure out if he can get from Paris to Vienna. Looking at the board, it's pretty easy to see that the only possible route for this card combination involves going Paris --(3 gray)--> Zurich --(2 green)--> Venice --(2 gray)--> Zagrad --(2 gray)--> Vienna. My algorithm for figuring this out starts with the green segment, and allocates the two green cards there. Then it needs to decide how to use the remaining 4 red and 3 blue cards to cover the gray segments of lengths 3, 2, and 2.
The answer, of course, is to use the 3 blue cards between Paris and Zurich, and 2 red cards each for Venice to Zagrad and Zagrad to Vienna. But how does one write a generalized algorithm that solves this problem for less obvious cases involving more colors and more segments?
My code for this right now looks like this:
def can_afford(path, cards):
grays = list()
for segment in path:
if segment.color == 'Gray':
grays.append(segment)
else:
if cards.get(segment.color, 0) >= segment.weight:
cards[segment.color] -= segment.weight
else:
return False
for gray in grays:
# Halp!
pass
return True
("weight" is the length of the segment in train cars.)
I feel like there's a really trivial solution lurking in here that I just can't put my finger on. Any ideas?

As Daniel Brückner says, the problem of finding a way to assign colors of cards to gray segments corresponds to the bin packing problem, with the sets of colored cards corresponding to the bins, and the gray segments corresponding to the objects to be packed.
Now, the bin packing problem is NP-hard, but that's not a disaster in this case, because the problem can be solved in pseudo-polynomial time (that is, in time that's polynomial in the size of the bins) using dynamic programming, and that should be good enough for your application, where the size of the bins is limited by the number of cards in the game. Here's an example implementation of bin packing, using the #functools.lru_cache decorator to memoize it:
from functools import lru_cache
#lru_cache(maxsize=None)
def packing(bins, objects):
"""Return a packing of objects into bins, or None if impossible. Both
arguments are tuples of numbers, and the packing is returned in
the form of a list giving the bin number for each object.
>>> packing((4,5,6), (6,5,4))
[2, 1, 0]
>>> packing((4,5,6), (1,1,2,4,5))
[0, 0, 0, 1, 2]
"""
if not objects:
return []
o = objects[0]
rest = objects[1:]
for i, b in enumerate(bins):
if o <= b:
p = packing(bins[:i] + (b - o,) + bins[i+1:], rest)
if p is not None:
return [i] + p
return None
And this can be used to determine if a path can be followed in Ticket to Ride:
def can_afford(path, cards):
"""Return True if path can be followed using cards, False if not.
cards might be updated, so pass a copy if you don't want that to
happen.
"""
grays = []
for segment in path:
c, w = segment.color, segment.weight
if c == 'Gray':
grays.append(w)
elif cards.get(c, 0) >= w:
cards[c] -= w
else:
return False
return packing(tuple(cards.values()), tuple(grays)) is not None
Note that if you made cards a collection.Counter, then you could just write cards[c] instead of cards.get(c, 0).

This problem bears some similarities with problems like bin packing, subset sum and other similar problems. The mentioned and many related problems are NP-complete and therefore it may turn out that there is no (known) efficient algorithm for this problem but I can not prove this at the moment - it is just an intuition. I will think about it some more and then update this answer.

Another way of approaching this is to build a search tree as follows:
Each node is labelled with a city name, a set of cards, and a number of trains. This corresponds to the starting city of a particular route and the cards and train pieces you have available.
Each child of a node corresponds to a city you can reach from the parent, along with the cards and train pieces that remain in your hand after completing the route from the parent to the node.
For example, the root of the tree could correspond to Montreal with 4 blue, 1 white, and 1 wild card, and 45 train pieces. The children of the root would be:
Toronto, (1 blue, 1 white, 1 wild), 42 # Use 3 blue cards
Toronto, (2 blue, 1 white), 42 # Use 2 blue and a wild card
New York, ( 1 blue, 1 white, 1 wild), 42 # Use 3 blue cards
New York, ( 2 blue, 1 white ), 43 # Use 2 blue and a wild card
Boston, ( 3 blue, 1 white ), 43 # Use 1 blue and a wild card
Boston, ( 2 blue, 1 white, 1 wild ), 43 # Use 2 blue cards
Boston, ( 4 blue ), 43 # Use 1 white and a wild card
Now, you just need to perform a depth-first search in this tree to see if you can reach the destination city. The edges you add to the search tree are limited by the cards in your hand (i.e., you can't go directly from Montreal to Sault St. Marie because you don't have a total of 6 black/wild cards in your hand) and the number of trains remaining (you couldn't go from Calgary to Seattle if you only had 3 cards remaining).