Single vector multiple times in 2d array - python

This is my code:
b = [6 * [1, 3, 4, 2],
4 * [2, 1, 4, 3],
3 * [3, 4, 2, 1],
4 * [4, 2, 1, 3],
4 * [4, 3, 2, 1],
]
Which returns an array which has 6X4=24 elements in the first line 4X4=16 in the second etc...
What i want to achieve is adding the exact same line multiple times like:
1, 3, 4, 2
1, 3, 4, 2
1, 3, 4, 2
1, 3, 4, 2
1, 3, 4, 2
1, 3, 4, 2 #6 tines the first line
2, 1, 4, 3
2, 1, 4, 3
2, 1, 4, 3
2, 1, 4, 3 # 4 times the second
..........
but of course by not copying the same line again and again


Try:
b = [
*[[1, 3, 4, 2] for _ in range(6)],
*[[2, 1, 4, 3] for _ in range(4)],
*[[3, 4, 2, 1] for _ in range(3)],
*[[4, 2, 1, 3] for _ in range(4)],
*[[4, 3, 2, 1] for _ in range(4)],
]
print(b)
Prints:
[
[1, 3, 4, 2],
[1, 3, 4, 2],
[1, 3, 4, 2],
[1, 3, 4, 2],
[1, 3, 4, 2],
[1, 3, 4, 2],
[2, 1, 4, 3],
[2, 1, 4, 3],
[2, 1, 4, 3],
[2, 1, 4, 3],
[3, 4, 2, 1],
[3, 4, 2, 1],
[3, 4, 2, 1],
[4, 2, 1, 3],
[4, 2, 1, 3],
[4, 2, 1, 3],
[4, 2, 1, 3],
[4, 3, 2, 1],
[4, 3, 2, 1],
[4, 3, 2, 1],
[4, 3, 2, 1],
]


You can also put it in one line with
b = 6*[[1, 3, 4, 2]] + 4*[[2, 1, 4, 3]] + 3*[[3, 4, 2, 1]] + 4*[[4, 2, 1, 3]] + 4* [[4, 3, 2, 1]])
print(b)


Only two steps if you don't want to change how b was created.
import numpy as np
B = np.concatenate(b).ravel()
b = np.reshape(B,(21,4))


You can use a list comprehension of a list comprehension to repeat y times your x lists (from bb and aa, respectively)
The notation is a bit strange, the "center" one is the outer element, then it expands on the right.
Advantage: numpy not required, and it will work for any size entries, unlike other answers "harcoding" the rows
aa=[[1, 3, 4, 2],
[2, 1, 4, 3],
[3, 4, 2, 1],
[4, 2, 1, 3],
[4, 3, 2, 1]]
bb=[6,4,3,4,4]
xx = [x for x,y in zip(aa,bb) for _ in range(y) ]
returns:
[[1, 3, 4, 2], [1, 3, 4, 2], [1, 3, 4, 2], [1, 3, 4, 2], [1, 3, 4, 2], [1, 3, 4, 2], [2, 1, 4, 3], [2, 1, 4, 3], [2, 1, 4, 3], [2, 1, 4, 3], [3, 4, 2, 1], [3, 4, 2, 1], [3, 4, 2, 1], [4, 2, 1, 3], [4, 2, 1, 3], [4, 2, 1, 3], [4, 2, 1, 3], [4, 3, 2, 1], [4, 3, 2, 1], [4, 3, 2, 1], [4, 3, 2, 1]]


It can be done systematically by pairing the factors and the sublists and multiplying them together. Then flat the list.
Here a hacky-way
b = [[1, 3, 4, 2],
[2, 1, 4, 3],
[3, 4, 2, 1],
[4, 2, 1, 3],
[4, 3, 2, 1],
]
factors = [6, 4, 3, 4, 4]
print(sum(f*[l] for f, l in zip(factors, b), []))
The flattening part (chaining of lists) is done with sum( , []) but is not performant (and designed for that)... but still useful, in my opinion, to test some small stuffs on the fly. The list chaining have to be done with i.e. itertools.chain. Here an example
import itertools as it
...
list(it.chain.from_iterable(f*[l] for f, l in zip(factors, b)))
# or (for many iterators)
list(it.chain(*(f*[l] for f, l in zip(factors, b))))
# or with repeat
list(it.chain.from_iterable(it.starmap(it.repeat, zip(b, factors))))
# or (another) with repeat
list(it.chain.from_iterable(map(it.repeat, b, factors))
# or ...
list(it.chain.from_iterable(map(list.__mul__, ([i] for i in b) , factors)))
# or with reduce
import functools as fc
list(fc.reduce(lambda i, j: i + j, zip(b, factors)))

Related

checking for elements in rows and columns of lists of lists without using pandas

If I have a list of lists
matrix = [[2, 3, 1, 2],[1, 2, 3, 2],[3, 3, 1, 2], [2, 2, 3, 3]]
how can I check with a for loop if for example element = 1 is present in each column

Using numpy:
np.any(a==1, 1).all()
>>> a = np.array([[2, 3, 1, 2],[1, 2, 3, 2],[3, 3, 1, 2], [2, 2, 3, 3]])
>>> np.any(a==1, 1).all()
False
>>> a = np.array([[2, 3, 1, 2],[1, 2, 3, 2],[3, 3, 1, 2], [2, 1, 3, 3]])
>>> np.any(a==1, 1).all()
True

Using all, in and a list comprehension:
matrix = [[2, 3, 1, 2], [1, 2, 3, 2], [3, 3, 1, 2], [2, 2, 3, 3]]
valid = all(1 in row for row in matrix)
Or, the verbose way:
matrix = [[2, 3, 1, 2], [1, 2, 3, 2], [3, 3, 1, 2], [2, 2, 3, 3]]
valid = True
for row in matrix:
if 1 not in row:
valid = False
break

OpenAI Gym: Walk through all possible actions in an action space

I want to build a brute-force approach that tests all actions in a Gym action space before selecting the best one. Is there any simple, straight-forward way to get all possible actions?
Specifically, my action space is
import gym
action_space = gym.spaces.MultiDiscrete([5 for _ in range(4)])
I know I can sample a random action with action_space.sample() and also check if an action is contained in the action space, but I want to generate a list of all possible action within that space.
Is there anything more elegant (and performant) than just a bunch of for loops? The problem with for loops is that I want it to work with any size of action space, so I cannot hard-code 4 for loops to walk through the different actions.

The actions in a gym environment are usually represented by integers only, this mean if you get the total number of possible actions, then an array of all possible actions can be created.
The way to get the total number of possible actions in a gym environment depends on the type of action space it has, for your case it's a MultiDiscrete action space and thus the attribute nvec can be used as mentioned here by #Valentin Macé like so -:
>> print(env.action_space.nvec)
array([5, 5, 5, 5], dtype=int64)
Note that the attribute nvec stands for n vector, since its output is a multidimensional vector. Also note that the attribute is a numpy array.
Now that we have the array to convert it into a list of lists of actions assuming that since the action_space.sample function returns a numpy array of a random function from each of the dimensions of the MultiDiscrete action_space i.e. -:
>> env.action_space.sample() # This does not return a single action but 4 actions for your case since you have a multi discrete action space of length 4.
array([2, 2, 0, 1], dtype=int64)
So thus to convert the array to a list of lists of possible actions in each dimensions we can use list comprehensions like so -:
>> [list(range(1, (k + 1))) for k in action_space.nvec]
[[1, 2, 3, 4, 5], [1, 2, 3, 4, 5], [1, 2, 3, 4, 5], [1, 2, 3, 4, 5]]
Note that this is scalable to any number of dimensions and is also quite efficient performance wise.
Now you can loop over the possible actions in each dimension using only two loops like so -:
possible_actions = [list(range(1, (k + 1))) for k in action_space.nvec]
for action_dim in possible_actions :
for action in action_dim :
# Find best action.....
pass
For more info about the same I would like you to also visit this thread on github, with a somewhat similar issue being discussed incase you find the same useful.
EDIT:
So as per the comment of yours #CGFoX I assume that you want it such that all the possible combination vectors of the actions can be generated as a list for any number of dimensions, somewhat like so -:
>> get_actions()
[[1, 1, 1, 1], [1, 1, 1, 2] ....] # For all possible combinations.
The same can be achieved like so using recursion and with only two loops, this is also expandable to as many dimensions as provided.
def flatten(actions) :
# This function flattens any actions passed somewhat like so -:
# INPUT -: [[1, 2, 3], 4, 5]
# OUTPUT -: [1, 2, 3, 4, 5]
new_actions = [] # Initializing the new flattened list of actions.
for action in actions :
# Loop through the actions
if type(action) == list :
# If any actions is a pair of actions i.e. a list e.g. [1, 1] then
# add it's elements to the new_actions list.
new_actions += action
elif type(action) == int :
# If the action is an integer then append it directly to the new_actions
# list.
new_actions.append(action)
# Returns the new_actions list generated.
return new_actions
def get_actions(possible_actions) :
# This functions recieves as input the possibilities of actions for every dimension
# and returns all possible dimensional combinations for the same.
# Like so -:
# INPUT-: [[1, 2, 3, 4], [1, 2, 3, 4]] # Example for 2 dimensions but can be scaled for any.
# OUTPUT-: [[1, 1], [1, 2], [1, 3] ... [4, 1] ... [4, 4]]
if len(possible_actions) == 1 :
# If there is only one possible list of actions then it itself is the
# list containing all possible combinations and thus is returned.
return possible_actions
pairs = [] # Initializing a list to contain all pairs of actions generated.
for action in possible_actions[0] :
# Now we loop over the first set of possibilities of actions i.e. index 0
# and we make pairs of it with the second set i.e. index 1, appending each pair
# to the pairs list.
# NOTE: Incase the function is recursively called the first set of possibilities
# of actions may contain vectors and thus the newly formed pair has to be flattened.
# i.e. If a pair has already been made in previous generation like so -:
# [[[1, 1], [2, 2], [3, 3] ... ], [1, 2, 3, 4]]
# Then the pair formed will be this -: [[[1, 1], 1], [[1, 1], 2] ... ]
# But we want them to be flattened like so -: [[1, 1, 1], [1, 1, 2] ... ]
for action2 in possible_actions[1] :
pairs.append(flatten([action, action2]))
# Now we create a new list of all possible set of actions by combining the
# newly generated pairs and the sets of possibilities of actions that have not
# been paired i.e. sets other than the first and the second.
# NOTE: When we made pairs we did so only for the first two indexes and not for
# all thus to do so we make a new list with the sets that remained unpaired
# and the paired set. i.e.
# BEFORE PAIRING -: [[1, 2, 3, 4], [1, 2, 3, 4], [1, 2, 3, 4]]
# AFTER PAIRING -: [[[1, 1], [1, 2] ... ], [1, 2, 3, 4]] # Notice how the third set
# i.e. the index 2 is still unpaired and first two sets have been paired.
new_possible_actions = [pairs] + possible_actions[2 : ]
# Now we recurse the function and call it within itself to make pairs for the
# left out sets, Note that since the first two sets were combined to form a paired
# first set now this set will be paired with the third set.
# This recursion will keep happening until all the sets have been paired to form
# a single set with all possible combinations.
possible_action_vectors = get_actions(new_possible_actions)
# Finally the result of the recursion is returned.
# NOTE: Only the first index is returned since now the first index contains the
# paired set of actions.
return possible_action_vectors[0]
Once we have this function defined it can be used with our previously generated set of possibilities of actions to get all possible combinations like so -:
possible_actions = [list(range(1, (k + 1))) for k in action_space.nvec]
print(get_actions(possible_actions))
>> [[1, 1, 1, 1], [1, 1, 1, 2], [1, 1, 1, 3], [1, 1, 1, 4], [1, 1, 1, 5], `[1, 1, 2, 1], [1, 1, 2, 2], [1, 1, 2, 3], [1, 1, 2, 4], [1, 1, 2, 5], [1, 1, 3, 1], [1, 1, 3, 2], [1, 1, 3, 3], [1, 1, 3, 4], [1, 1, 3, 5], [1, 1, 4, 1], [1, 1, 4, 2], [1, 1, 4, 3], [1, 1, 4, 4], [1, 1, 4, 5], [1, 1, 5, 1], [1, 1, 5, 2], [1, 1, 5, 3], [1, 1, 5, 4], [1, 1, 5, 5], [1, 2, 1, 1], [1, 2, 1, 2], [1, 2, 1, 3], [1, 2, 1, 4], [1, 2, 1, 5], [1, 2, 2, 1], [1, 2, 2, 2], [1, 2, 2, 3], [1, 2, 2, 4], [1, 2, 2, 5], [1, 2, 3, 1], [1, 2, 3, 2], [1, 2, 3, 3], [1, 2, 3, 4], [1, 2, 3, 5], [1, 2, 4, 1], [1, 2, 4, 2], [1, 2, 4, 3], [1, 2, 4, 4], [1, 2, 4, 5], [1, 2, 5, 1], [1, 2, 5, 2], [1, 2, 5, 3], [1, 2, 5, 4], [1, 2, 5, 5], [1, 3, 1, 1], [1, 3, 1, 2], [1, 3, 1, 3], [1, 3, 1, 4], [1, 3, 1, 5], [1, 3, 2, 1], [1, 3, 2, 2], [1, 3, 2, 3], [1, 3, 2, 4], [1, 3, 2, 5], [1, 3, 3, 1], [1, 3, 3, 2], [1, 3, 3, 3], [1, 3, 3, 4], [1, 3, 3, 5], [1, 3, 4, 1], [1, 3, 4, 2], [1, 3, 4, 3], [1, 3, 4, 4], [1, 3, 4, 5], [1, 3, 5, 1], [1, 3, 5, 2], [1, 3, 5, 3], [1, 3, 5, 4], [1, 3, 5, 5], [1, 4, 1, 1], [1, 4, 1, 2], [1, 4, 1, 3], [1, 4, 1, 4], [1, 4, 1, 5], [1, 4, 2, 1], [1, 4, 2, 2], [1, 4, 2, 3], [1, 4, 2, 4], [1, 4, 2, 5], [1, 4, 3, 1], [1, 4, 3, 2], [1, 4, 3, 3], [1, 4, 3, 4], [1, 4, 3, 5], [1, 4, 4, 1], [1, 4, 4, 2], [1, 4, 4, 3], [1, 4, 4, 4], [1, 4, 4, 5], [1, 4, 5, 1], [1, 4, 5, 2], [1, 4, 5, 3], [1, 4, 5, 4], [1, 4, 5, 5], [1, 5, 1, 1], [1, 5, 1, 2], [1, 5, 1, 3], [1, 5, 1, 4], [1, 5, 1, 5], [1, 5, 2, 1], [1, 5, 2, 2], [1, 5, 2, 3], [1, 5, 2, 4], [1, 5, 2, 5], [1, 5, 3, 1], [1, 5, 3, 2], [1, 5, 3, 3], [1, 5, 3, 4], [1, 5, 3, 5], [1, 5, 4, 1], [1, 5, 4, 2], [1, 5, 4, 3], [1, 5, 4, 4], [1, 5, 4, 5], [1, 5, 5, 1], [1, 5, 5, 2], [1, 5, 5, 3], [1, 5, 5, 4], [1, 5, 5, 5], [2, 1, 1, 1], [2, 1, 1, 2], [2, 1, 1, 3], [2, 1, 1, 4], [2, 1, 1, 5], [2, 1, 2, 1], [2, 1, 2, 2], [2, 1, 2, 3], [2, 1, 2, 4], [2, 1, 2, 5], [2, 1, 3, 1], [2, 1, 3, 2], [2, 1, 3, 3], [2, 1, 3, 4], [2, 1, 3, 5], [2, 1, 4, 1], [2, 1, 4, 2], [2, 1, 4, 3], [2, 1, 4, 4], [2, 1, 4, 5], [2, 1, 5, 1], [2, 1, 5, 2], [2, 1, 5, 3], [2, 1, 5, 4], [2, 1, 5, 5], [2, 2, 1, 1], [2, 2, 1, 2], [2, 2, 1, 3], [2, 2, 1, 4], [2, 2, 1, 5], [2, 2, 2, 1], [2, 2, 2, 2], [2, 2, 2, 3], [2, 2, 2, 4], [2, 2, 2, 5], [2, 2, 3, 1], [2, 2, 3, 2], [2, 2, 3, 3], [2, 2, 3, 4], [2, 2, 3, 5], [2, 2, 4, 1], [2, 2, 4, 2], [2, 2, 4, 3], [2, 2, 4, 4], [2, 2, 4, 5], [2, 2, 5, 1], [2, 2, 5, 2], [2, 2, 5, 3], [2, 2, 5, 4], [2, 2, 5, 5], [2, 3, 1, 1], [2, 3, 1, 2], [2, 3, 1, 3], [2, 3, 1, 4], [2, 3, 1, 5], [2, 3, 2, 1], [2, 3, 2, 2], [2, 3, 2, 3], [2, 3, 2, 4], [2, 3, 2, 5], [2, 3, 3, 1], [2, 3, 3, 2], [2, 3, 3, 3], [2, 3, 3, 4], [2, 3, 3, 5], [2, 3, 4, 1], [2, 3, 4, 2], [2, 3, 4, 3], [2, 3, 4, 4], [2, 3, 4, 5], [2, 3, 5, 1], [2, 3, 5, 2], [2, 3, 5, 3], [2, 3, 5, 4], [2, 3, 5, 5], [2, 4, 1, 1], [2, 4, 1, 2], [2, 4, 1, 3], [2, 4, 1, 4], [2, 4, 1, 5], [2, 4, 2, 1], [2, 4, 2, 2], [2, 4, 2, 3], [2, 4, 2, 4], [2, 4, 2, 5], [2, 4, 3, 1], [2, 4, 3, 2], [2, 4, 3, 3], [2, 4, 3, 4], [2, 4, 3, 5], [2, 4, 4, 1], [2, 4, 4, 2], [2, 4, 4, 3], [2, 4, 4, 4], [2, 4, 4, 5], [2, 4, 5, 1], [2, 4, 5, 2], [2, 4, 5, 3], [2, 4, 5, 4], [2, 4, 5, 5], [2, 5, 1, 1], [2, 5, 1, 2], [2, 5, 1, 3], [2, 5, 1, 4], [2, 5, 1, 5], [2, 5, 2, 1], [2, 5, 2, 2], [2, 5, 2, 3], [2, 5, 2, 4], [2, 5, 2, 5], [2, 5, 3, 1], [2, 5, 3, 2], [2, 5, 3, 3], [2, 5, 3, 4], [2, 5, 3, 5], [2, 5, 4, 1], [2, 5, 4, 2], [2, 5, 4, 3], [2, 5, 4, 4], [2, 5, 4, 5], [2, 5, 5, 1], [2, 5, 5, 2], [2, 5, 5, 3], [2, 5, 5, 4], [2, 5, 5, 5], [3, 1, 1, 1], [3, 1, 1, 2], [3, 1, 1, 3], [3, 1, 1, 4], [3, 1, 1, 5], [3, 1, 2, 1], [3, 1, 2, 2], [3, 1, 2, 3], [3, 1, 2, 4], [3, 1, 2, 5], [3, 1, 3, 1], [3, 1, 3, 2], [3, 1, 3, 3], [3, 1, 3, 4], [3, 1, 3, 5], [3, 1, 4, 1], [3, 1, 4, 2], [3, 1, 4, 3], [3, 1, 4, 4], [3, 1, 4, 5], [3, 1, 5, 1], [3, 1, 5, 2], [3, 1, 5, 3], [3, 1, 5, 4], [3, 1, 5, 5], [3, 2, 1, 1], [3, 2, 1, 2], [3, 2, 1, 3], [3, 2, 1, 4], [3, 2, 1, 5], [3, 2, 2, 1], [3, 2, 2, 2], [3, 2, 2, 3], [3, 2, 2, 4], [3, 2, 2, 5], [3, 2, 3, 1], [3, 2, 3, 2], [3, 2, 3, 3], [3, 2, 3, 4], [3, 2, 3, 5], [3, 2, 4, 1], [3, 2, 4, 2], [3, 2, 4, 3], [3, 2, 4, 4], [3, 2, 4, 5], [3, 2, 5, 1], [3, 2, 5, 2], [3, 2, 5, 3], [3, 2, 5, 4], [3, 2, 5, 5], [3, 3, 1, 1], [3, 3, 1, 2], [3, 3, 1, 3], [3, 3, 1, 4], [3, 3, 1, 5], [3, 3, 2, 1], [3, 3, 2, 2], [3, 3, 2, 3], [3, 3, 2, 4], [3, 3, 2, 5], [3, 3, 3, 1], [3, 3, 3, 2], [3, 3, 3, 3], [3, 3, 3, 4], [3, 3, 3, 5], [3, 3, 4, 1], [3, 3, 4, 2], [3, 3, 4, 3], [3, 3, 4, 4], [3, 3, 4, 5], [3, 3, 5, 1], [3, 3, 5, 2], [3, 3, 5, 3], [3, 3, 5, 4], [3, 3, 5, 5], [3, 4, 1, 1], [3, 4, 1, 2], [3, 4, 1, 3], [3, 4, 1, 4], [3, 4, 1, 5], [3, 4, 2, 1], [3, 4, 2, 2], [3, 4, 2, 3], [3, 4, 2, 4], [3, 4, 2, 5], [3, 4, 3, 1], [3, 4, 3, 2], [3, 4, 3, 3], [3, 4, 3, 4], [3, 4, 3, 5], [3, 4, 4, 1], [3, 4, 4, 2], [3, 4, 4, 3], [3, 4, 4, 4], [3, 4, 4, 5], [3, 4, 5, 1], [3, 4, 5, 2], [3, 4, 5, 3], [3, 4, 5, 4], [3, 4, 5, 5], [3, 5, 1, 1], [3, 5, 1, 2], [3, 5, 1, 3], [3, 5, 1, 4], [3, 5, 1, 5], [3, 5, 2, 1], [3, 5, 2, 2], [3, 5, 2, 3], [3, 5, 2, 4], [3, 5, 2, 5], [3, 5, 3, 1], [3, 5, 3, 2], [3, 5, 3, 3], [3, 5, 3, 4], [3, 5, 3, 5], [3, 5, 4, 1], [3, 5, 4, 2], [3, 5, 4, 3], [3, 5, 4, 4], [3, 5, 4, 5], [3, 5, 5, 1], [3, 5, 5, 2], [3, 5, 5, 3], [3, 5, 5, 4], [3, 5, 5, 5], [4, 1, 1, 1], [4, 1, 1, 2], [4, 1, 1, 3], [4, 1, 1, 4], [4, 1, 1, 5], [4, 1, 2, 1], [4, 1, 2, 2], [4, 1, 2, 3], [4, 1, 2, 4], [4, 1, 2, 5], [4, 1, 3, 1], [4, 1, 3, 2], [4, 1, 3, 3], [4, 1, 3, 4], [4, 1, 3, 5], [4, 1, 4, 1], [4, 1, 4, 2], [4, 1, 4, 3], [4, 1, 4, 4], [4, 1, 4, 5], [4, 1, 5, 1], [4, 1, 5, 2], [4, 1, 5, 3], [4, 1, 5, 4], [4, 1, 5, 5], [4, 2, 1, 1], [4, 2, 1, 2], [4, 2, 1, 3], [4, 2, 1, 4], [4, 2, 1, 5], [4, 2, 2, 1], [4, 2, 2, 2], [4, 2, 2, 3], [4, 2, 2, 4], [4, 2, 2, 5], [4, 2, 3, 1], [4, 2, 3, 2], [4, 2, 3, 3], [4, 2, 3, 4], [4, 2, 3, 5], [4, 2, 4, 1], [4, 2, 4, 2], [4, 2, 4, 3], [4, 2, 4, 4], [4, 2, 4, 5], [4, 2, 5, 1], [4, 2, 5, 2], [4, 2, 5, 3], [4, 2, 5, 4], [4, 2, 5, 5], [4, 3, 1, 1], [4, 3, 1, 2], [4, 3, 1, 3], [4, 3, 1, 4], [4, 3, 1, 5], [4, 3, 2, 1], [4, 3, 2, 2], [4, 3, 2, 3], [4, 3, 2, 4], [4, 3, 2, 5], [4, 3, 3, 1], [4, 3, 3, 2], [4, 3, 3, 3], [4, 3, 3, 4], [4, 3, 3, 5], [4, 3, 4, 1], [4, 3, 4, 2], [4, 3, 4, 3], [4, 3, 4, 4], [4, 3, 4, 5], [4, 3, 5, 1], [4, 3, 5, 2], [4, 3, 5, 3], [4, 3, 5, 4], [4, 3, 5, 5], [4, 4, 1, 1], [4, 4, 1, 2], [4, 4, 1, 3], [4, 4, 1, 4], [4, 4, 1, 5], [4, 4, 2, 1], [4, 4, 2, 2], [4, 4, 2, 3], [4, 4, 2, 4], [4, 4, 2, 5], [4, 4, 3, 1], [4, 4, 3, 2], [4, 4, 3, 3], [4, 4, 3, 4], [4, 4, 3, 5], [4, 4, 4, 1], [4, 4, 4, 2], [4, 4, 4, 3], [4, 4, 4, 4], [4, 4, 4, 5], [4, 4, 5, 1], [4, 4, 5, 2], [4, 4, 5, 3], [4, 4, 5, 4], [4, 4, 5, 5], [4, 5, 1, 1], [4, 5, 1, 2], [4, 5, 1, 3], [4, 5, 1, 4], [4, 5, 1, 5], [4, 5, 2, 1], [4, 5, 2, 2], [4, 5, 2, 3], [4, 5, 2, 4], [4, 5, 2, 5], [4, 5, 3, 1], [4, 5, 3, 2], [4, 5, 3, 3], [4, 5, 3, 4], [4, 5, 3, 5], [4, 5, 4, 1], [4, 5, 4, 2], [4, 5, 4, 3], [4, 5, 4, 4], [4, 5, 4, 5], [4, 5, 5, 1], [4, 5, 5, 2], [4, 5, 5, 3], [4, 5, 5, 4], [4, 5, 5, 5], [5, 1, 1, 1], [5, 1, 1, 2], [5, 1, 1, 3], [5, 1, 1, 4], [5, 1, 1, 5], [5, 1, 2, 1], [5, 1, 2, 2], [5, 1, 2, 3], [5, 1, 2, 4], [5, 1, 2, 5], [5, 1, 3, 1], [5, 1, 3, 2], [5, 1, 3, 3], [5, 1, 3, 4], [5, 1, 3, 5], [5, 1, 4, 1], [5, 1, 4, 2], [5, 1, 4, 3], [5, 1, 4, 4], [5, 1, 4, 5], [5, 1, 5, 1], [5, 1, 5, 2], [5, 1, 5, 3], [5, 1, 5, 4], [5, 1, 5, 5], [5, 2, 1, 1], [5, 2, 1, 2], [5, 2, 1, 3], [5, 2, 1, 4], [5, 2, 1, 5], [5, 2, 2, 1], [5, 2, 2, 2], [5, 2, 2, 3], [5, 2, 2, 4], [5, 2, 2, 5], [5, 2, 3, 1], [5, 2, 3, 2], [5, 2, 3, 3], [5, 2, 3, 4], [5, 2, 3, 5], [5, 2, 4, 1], [5, 2, 4, 2], [5, 2, 4, 3], [5, 2, 4, 4], [5, 2, 4, 5], [5, 2, 5, 1], [5, 2, 5, 2], [5, 2, 5, 3], [5, 2, 5, 4], [5, 2, 5, 5], [5, 3, 1, 1], [5, 3, 1, 2], [5, 3, 1, 3], [5, 3, 1, 4], [5, 3, 1, 5], [5, 3, 2, 1], [5, 3, 2, 2], [5, 3, 2, 3], [5, 3, 2, 4], [5, 3, 2, 5], [5, 3, 3, 1], [5, 3, 3, 2], [5, 3, 3, 3], [5, 3, 3, 4], [5, 3, 3, 5], [5, 3, 4, 1], [5, 3, 4, 2], [5, 3, 4, 3], [5, 3, 4, 4], [5, 3, 4, 5], [5, 3, 5, 1], [5, 3, 5, 2], [5, 3, 5, 3], [5, 3, 5, 4], [5, 3, 5, 5], [5, 4, 1, 1], [5, 4, 1, 2], [5, 4, 1, 3], [5, 4, 1, 4], [5, 4, 1, 5], [5, 4, 2, 1], [5, 4, 2, 2], [5, 4, 2, 3], [5, 4, 2, 4], [5, 4, 2, 5], [5, 4, 3, 1], [5, 4, 3, 2], [5, 4, 3, 3], [5, 4, 3, 4], [5, 4, 3, 5], [5, 4, 4, 1], [5, 4, 4, 2], [5, 4, 4, 3], [5, 4, 4, 4], [5, 4, 4, 5], [5, 4, 5, 1], [5, 4, 5, 2], [5, 4, 5, 3], [5, 4, 5, 4], [5, 4, 5, 5], [5, 5, 1, 1], [5, 5, 1, 2], [5, 5, 1, 3], [5, 5, 1, 4], [5, 5, 1, 5], [5, 5, 2, 1], [5, 5, 2, 2], [5, 5, 2, 3], [5, 5, 2, 4], [5, 5, 2, 5], [5, 5, 3, 1], [5, 5, 3, 2], [5, 5, 3, 3], [5, 5, 3, 4], [5, 5, 3, 5], [5, 5, 4, 1], [5, 5, 4, 2], [5, 5, 4, 3], [5, 5, 4, 4], [5, 5, 4, 5], [5, 5, 5, 1], [5, 5, 5, 2], [5, 5, 5, 3], [5, 5, 5, 4], [5, 5, 5, 5]]
EDIT-2 : I have fixed some code which previously was returning a nested list, now the list returned is the one with the pairs and is not nested within another list.
EDIT-3-: Fixed my spelling mistakes.

One could also use the function below, to make an explicit list of all states or actions in the observation or action space respectively.
def get_space_list(space):
"""
Converts gym `space`, constructed from `types`, to list `space_list`
"""
# -------------------------------- #
types = [
gym.spaces.multi_binary.MultiBinary,
gym.spaces.discrete.Discrete,
gym.spaces.multi_discrete.MultiDiscrete,
gym.spaces.dict.Dict,
gym.spaces.tuple.Tuple,
]
if type(space) not in types:
raise ValueError(f'input space {space} is not constructed from spaces of types:' + '\n' + str(types))
# -------------------------------- #
if type(space) is gym.spaces.multi_binary.MultiBinary:
return [
np.reshape(np.array(element), space.n)
for element in itertools.product(
*[range(2)] * np.prod(space.n)
)
]
if type(space) is gym.spaces.discrete.Discrete:
return list(range(space.n))
if type(space) is gym.spaces.multi_discrete.MultiDiscrete:
return [
np.array(element) for element in itertools.product(
*[range(n) for n in space.nvec]
)
]
if type(space) is gym.spaces.dict.Dict:
keys = space.spaces.keys()
values_list = itertools.product(
*[get_space_list(sub_space) for sub_space in space.spaces.values()]
)
return [
{key: value for key, value in zip(keys, values)}
for values in values_list
]
return space_list
if type(space) is gym.spaces.tuple.Tuple:
return [
list(element) for element in itertools.product(
*[get_space_list(sub_space) for sub_space in space.spaces]
)
]
# -------------------------------- #

How to generate all possible k size vectors in a matrix (diagonals included)?

Given this matrix:
x = [[1,2,3,4,5], [1,2,3,4,5], [1,2,3,4,5], [1,2,3,4,5], [1,2,3,4,5], [1,2,3,4,5]]
What is an efficient way to return all the possible 4 by 1 vectors and 1 by 4 matrix in this matrix. As well as any 4 diagonal spaces joined in a line.
For example:
[1,1,1,1] would appear 3 times
Diagonals also need to be addressed so [1,2,3,4] would be included as a row but also a diagonal.

Splitting the problem into two steps:
Step 1 - get all horizontal, vertical and diagonal lines
Diagonals are tackled using the fact either i+j, or respectively i-j, is constant for the indexes i, j
x = [[1,2,3,4,5], [1,2,3,4,5], [1,2,3,4,5], [1,2,3,4,5], [1,2,3,4,5], [1,2,3,4,5]]
pprint.pprint(x)
# [[1, 2, 3, 4, 5],
# [1, 2, 3, 4, 5],
# [1, 2, 3, 4, 5],
# [1, 2, 3, 4, 5],
# [1, 2, 3, 4, 5],
# [1, 2, 3, 4, 5]]
all_lines = (
# Horizontal
[x[i] for i in range(len(x))] +
# Vertical
[[x[i][j] for i in range(len(x))] for j in range(len(x[0]))] +
# Diagonal k = i - j
[[x[k+j][j] for j in range(len(x[0])) if 0 <= k+j < len(x)] for k in range(-len(x[0])+1, len(x))] +
# Diagonal k = i + j
[[x[k-j][j] for j in range(len(x[0])) if 0 <= k-j < len(x)] for k in range(len(x[0])+len(x)-1)]
)
>>> pprint.pprint(all_lines)
[[1, 2, 3, 4, 5],
[1, 2, 3, 4, 5],
[1, 2, 3, 4, 5],
[1, 2, 3, 4, 5],
[1, 2, 3, 4, 5],
[1, 2, 3, 4, 5],
[1, 1, 1, 1, 1, 1],
[2, 2, 2, 2, 2, 2],
[3, 3, 3, 3, 3, 3],
[4, 4, 4, 4, 4, 4],
[5, 5, 5, 5, 5, 5],
[5],
[4, 5],
[3, 4, 5],
[2, 3, 4, 5],
[1, 2, 3, 4, 5],
[1, 2, 3, 4, 5],
[1, 2, 3, 4],
[1, 2, 3],
[1, 2],
[1],
[1],
[1, 2],
[1, 2, 3],
[1, 2, 3, 4],
[1, 2, 3, 4, 5],
[1, 2, 3, 4, 5],
[2, 3, 4, 5],
[3, 4, 5],
[4, 5],
[5]]
Step 2 - for each line select each 4-length slice
ans = [a[i:i+4] for i in range(len(a)-4+1) for a in all_lines if len(a[i:i+4]) == 4]
>>> ans = [a[i:i+4] for i in range(len(a)-4+1) for a in all_lines if len(a[i:i+4]) == 4]
>>> pprint.pprint(ans)
[[1, 2, 3, 4],
[1, 2, 3, 4],
[1, 2, 3, 4],
[1, 2, 3, 4],
[1, 2, 3, 4],
[1, 2, 3, 4],
[1, 1, 1, 1],
[2, 2, 2, 2],
[3, 3, 3, 3],
[4, 4, 4, 4],
[5, 5, 5, 5],
[2, 3, 4, 5],
[1, 2, 3, 4],
[1, 2, 3, 4],
[1, 2, 3, 4],
[1, 2, 3, 4],
[1, 2, 3, 4],
[1, 2, 3, 4],
[2, 3, 4, 5],
[2, 3, 4, 5],
[2, 3, 4, 5],
[2, 3, 4, 5],
[2, 3, 4, 5],
[2, 3, 4, 5],
[2, 3, 4, 5],
[1, 1, 1, 1],
[2, 2, 2, 2],
[3, 3, 3, 3],
[4, 4, 4, 4],
[5, 5, 5, 5],
[2, 3, 4, 5],
[2, 3, 4, 5],
[2, 3, 4, 5],
[2, 3, 4, 5],
[1, 1, 1, 1],
[2, 2, 2, 2],
[3, 3, 3, 3],
[4, 4, 4, 4],
[5, 5, 5, 5]]
Maybe not be the most efficient but it's at least a way of doing it.
There will likely be a way of using itertools combinations to simplify this dramatically.

Removing Element from Lists of List. PYTHON

for i in file:
file = [i.strip() for i in file]
return [file[i:i+1] for i in range(0, len(file),1)]
OUTPUT:
[['1,3,4,5,2'], ['4,2,5,3,1'], ['1,3,2,5,4'], ['1,2,4,3,5'], ['1,3,4,5,2'], ['2,1,3,5,4'], ['1,3,4,5,2'], ['3,5,2,4,1'], ['1,4,5,2,3'], ['5,1,4,3,2'], ['3,2,5,4,1'], ['3,1,2,5,4'], ['2,5,1,4,3'], ['3,2,1,4,5'], ['4,5,3,1,2'], ['1,5,4,3,2'], ['1,5,3,4,2'], ['2,1,4,3,5'], ['4,1,2,5,3']]
This is my List with sub-lists.
How can I take off the ' '? So ['1,3,4,5,2'] -> [1,3,4,5,2]
Thanks! (:

You can try this:
s = [['1,3,4,5,2'], ['4,2,5,3,1'], ['1,3,2,5,4'], ['1,2,4,3,5'], ['1,3,4,5,2'], ['2,1,3,5,4'], ['1,3,4,5,2'], ['3,5,2,4,1'], ['1,4,5,2,3'], ['5,1,4,3,2'], ['3,2,5,4,1'], ['3,1,2,5,4'], ['2,5,1,4,3'], ['3,2,1,4,5'], ['4,5,3,1,2'], ['1,5,4,3,2'], ['1,5,3,4,2'], ['2,1,4,3,5'], ['4,1,2,5,3']]
final_data = [map(int, i[0].split(",")) for i in s]
Output:
[[1, 3, 4, 5, 2], [4, 2, 5, 3, 1], [1, 3, 2, 5, 4], [1, 2, 4, 3, 5], [1, 3, 4, 5, 2], [2, 1, 3, 5, 4], [1, 3, 4, 5, 2], [3, 5, 2, 4, 1], [1, 4, 5, 2, 3], [5, 1, 4, 3, 2], [3, 2, 5, 4, 1], [3, 1, 2, 5, 4], [2, 5, 1, 4, 3], [3, 2, 1, 4, 5], [4, 5, 3, 1, 2], [1, 5, 4, 3, 2], [1, 5, 3, 4, 2], [2, 1, 4, 3, 5], [4, 1, 2, 5, 3]]

try this:
import ast
l = []
s = [['1,3,4,5,2'], ['4,2,5,3,1'], ['1,3,2,5,4'], ['1,2,4,3,5'], ['1,3,4,5,2'], ['2,1,3,5,4'], ['1,3,4,5,2'], ['3,5,2,4,1'], ['1,4,5,2,3'], ['5,1,4,3,2'], ['3,2,5,4,1'], ['3,1,2,5,4'], ['2,5,1,4,3'], ['3,2,1,4,5'], ['4,5,3,1,2'], ['1,5,4,3,2'], ['1,5,3,4,2'], ['2,1,4,3,5'], ['4,1,2,5,3']]
for li in s:
l.append(list(ast.literal_eval(li[0])))
note: ast is a built-in (standard) module if you don't know it, it cames with python installation

One line solution :
list_1=[['1,3,4,5,2'], ['4,2,5,3,1'], ['1,3,2,5,4'], ['1,2,4,3,5'], ['1,3,4,5,2'], ['2,1,3,5,4'], ['1,3,4,5,2'], ['3,5,2,4,1'], ['1,4,5,2,3'], ['5,1,4,3,2'], ['3,2,5,4,1'], ['3,1,2,5,4'], ['2,5,1,4,3'], ['3,2,1,4,5'], ['4,5,3,1,2'], ['1,5,4,3,2'], ['1,5,3,4,2'], ['2,1,4,3,5'], ['4,1,2,5,3']]
print([[int(item_3) for item_3 in item_1 if item_3.isdigit()] for item in list_1 for item_1 in item])
output:
[[1, 3, 4, 5, 2], [4, 2, 5, 3, 1], [1, 3, 2, 5, 4], [1, 2, 4, 3, 5], [1, 3, 4, 5, 2], [2, 1, 3, 5, 4], [1, 3, 4, 5, 2], [3, 5, 2, 4, 1], [1, 4, 5, 2, 3], [5, 1, 4, 3, 2], [3, 2, 5, 4, 1], [3, 1, 2, 5, 4], [2, 5, 1, 4, 3], [3, 2, 1, 4, 5], [4, 5, 3, 1, 2], [1, 5, 4, 3, 2], [1, 5, 3, 4, 2], [2, 1, 4, 3, 5], [4, 1, 2, 5, 3]]
in detailed :
for item in list_1:
for item_1 in item:
new=[]
for item_3 in item_1:
if item_3.isdigit():
new.append(int(item_3))
print(new)

Need to get set of numbers in cyclic order - Python

I am writing some python script to generate JSON. I constructed the jSON successfully. but stuck in getting selecting number in cyclic order.
let us say, i have a list of 1,2,3,4,5. I need to select first 4 numbers (1,2,3,4) here for first first item and 2,3,4,5 for second and 3,4,5,1 for third and it should go on till 30 times.
import json
import random
json_dict = {}
number = []
brokers = [1,2,3,4,5]
json_dict["version"] = version
json_dict["partitions"] = [{"topic": "topic1", "name": i,"replicas":
random.choice(brokers)} for i in range(0, 30)]
with open("output.json", "w") as outfile:
json.dump(json_dict, outfile, indent=4)
Output
"version": "1",
"partitions": [
{
"topic": "topic1",
"name": 0,
"replicas": 1,2,3,4
},
{
"topic": "topic1",
"name": 1,
"replicas": 2,3,4,5
},
{
"topic": "topic1",
"name": 3,
"replicas": 3,4,5,1
Anyway, how can i achieve this?

In order to get a cyclic elements from your brokers list you can use deque from collections module and do a deque.rotation(-1) like this example:
from collections import deque
def grouper(iterable, elements, rotations):
if elements > len(iterable):
return []
b = deque(iterable)
for _ in range(rotations):
yield list(b)[:elements]
b.rotate(-1)
brokers = [1,2,3,4,5]
# Pick 4 elements from brokers and yield 30 cycles
cycle = list(grouper(brokers, 4, 30))
print(cycle)
Output:
[[1, 2, 3, 4], [2, 3, 4, 5], [3, 4, 5, 1], [4, 5, 1, 2], [5, 1, 2, 3], [1, 2, 3, 4], [2, 3, 4, 5], [3, 4, 5, 1], [4, 5, 1, 2], [5, 1, 2, 3], [1, 2, 3, 4], [2, 3, 4, 5], [3, 4, 5, 1], [4, 5, 1, 2], [5, 1, 2, 3], [1, 2, 3, 4], [2, 3, 4, 5], [3, 4, 5, 1], [4, 5, 1, 2], [5, 1, 2, 3], [1, 2, 3, 4], [2, 3, 4, 5], [3, 4, 5, 1], [4, 5, 1, 2], [5, 1, 2, 3], [1, 2, 3, 4], [2, 3, 4, 5], [3, 4, 5, 1], [4, 5,1, 2], [5, 1, 2, 3]]
Also, this is a way how to implement this solution to your final dict:
# in this example i'm using only 5 cycles
cycles = grouper(brokers, 4, 5)
partitions = [{"topic": "topic1", "name": i, "replicas": cycle_elem} for i, cycle_elem in zip(range(5), cycles)]
final_dict = {"version": "1", "partitions": partitions}
print(final_dict)
Output:
{'partitions': [{'name': 0, 'replicas': [1, 2, 3, 4], 'topic': 'topic1'}, {'name': 1, 'replicas': [2, 3, 4, 5], 'topic': 'topic1'}, {'name': 2, 'replicas': [3, 4, 5, 1], 'topic': 'topic1'}, {'name': 3, 'replicas': [4, 5, 1, 2], 'topic': 'topic1'}, {'name': 4, 'replicas': [5, 1, 2, 3], 'topic': 'topic1'}], 'version': '1'}

Here's a purely procedural solution which also adds flexibility of selecting any number of groups, of any size (even bigger than the original 'brokers' list) with any offset:
def get_subgroups(groups, base, size, offset=1):
# cover the group size > len(base) case by expanding the base
# this step is completely optional if your group size will never be bigger
base *= -(-size // len(base))
result = [] # storage for our groups
base_size = len(base) # no need to call len() all the time
current_offset = 0 # tracking current cycle offset
for i in range(groups): # use xrange() on Python 2.x instead
tail = current_offset + size # end index for our current slice
end = min(tail, base_size) # normalize to the base size
group = base[current_offset:end] + base[:tail - end] # get our slice
result.append(group) # append it to our result storage
current_offset = (current_offset + offset) % base_size # increase our current offset
return result
brokers = [1, 2, 3, 4, 5]
print(get_subgroups(5, brokers, 4)) # 5 groups of size 4, with default offset
# prints: [[1, 2, 3, 4], [2, 3, 4, 5], [3, 4, 5, 1], [4, 5, 1, 2], [5, 1, 2, 3]]
print(get_subgroups(3, brokers, 7, 2)) # 3 groups of size 7, with offset 2
# prints: [[1, 2, 3, 4, 5, 1, 2], [3, 4, 5, 1, 2, 3, 4], [5, 1, 2, 3, 4, 5, 1]]
And it does it in a O(N) time with a single loop.
If you're planning to run this on a very large generator, you can turn get_subgroups() function into a generator by forgoing the result collection and doing yield group instead of result.append(group). That way you can call it in a loop as: for group in get_subgroups(30, broker, 4): and store the group in whatever structure you want.
UPDATE
If memory is not an issue, we can optimize (processing-wise) this even more by expanding the whole base (or brokers in your case) to fit the whole set:
def get_subgroups(groups, base, size, offset=1): # warning, heavy memory usage!
base *= -(-(offset * groups + size) // len(base))
result = [] # storage for our groups
current_offset = 0 # tracking current cycle offset
for i in range(groups): # use xrange() on Python 2.x instead
result.append(base[current_offset:current_offset+size])
current_offset += offset
return result
Or we can make it even faster with list comprehension if we don't need the ability to turn it into a generator:
def get_subgroups(groups, base, size, offset=1): # warning, heavy memory usage!
base *= -(-(offset * groups + size) // len(base))
return [base[i:i+size] for i in range(0, groups * offset, offset)]
# as previously mentioned, use xrange() on Python 2.x instead

This is a pretty cool problem, and I think I have a pretty cool solution:
items = [1, 2, 3, 4, 5]
[(items * 2)[x:x+4] for i in range(30) for x in [i % len(items)]]
which gives
[[1, 2, 3, 4],
[2, 3, 4, 5],
[3, 4, 5, 1],
[4, 5, 1, 2],
[5, 1, 2, 3],
[1, 2, 3, 4],
[2, 3, 4, 5],
[3, 4, 5, 1],
[4, 5, 1, 2],
[5, 1, 2, 3],
[1, 2, 3, 4],
[2, 3, 4, 5],
[3, 4, 5, 1],
[4, 5, 1, 2],
[5, 1, 2, 3],
[1, 2, 3, 4],
[2, 3, 4, 5],
[3, 4, 5, 1],
[4, 5, 1, 2],
[5, 1, 2, 3],
[1, 2, 3, 4],
[2, 3, 4, 5],
[3, 4, 5, 1],
[4, 5, 1, 2],
[5, 1, 2, 3],
[1, 2, 3, 4],
[2, 3, 4, 5],
[3, 4, 5, 1],
[4, 5, 1, 2],
[5, 1, 2, 3]]
What this is doing is taking your set of things and appending it to itself (items * 2 -> [1, 2, 3, 4, 5, 1, 2, 3 ,4, 5]), and then picking a starting place (x) by taking our loop iteration (i) and modulating it (probably not the right word) by the number of items we have (i in [x % len(items)]).

Categories