I am trying to implement a Q-Learning algorithm, my state-space contains all possible combinations of numbers 0,1,2 in a vector of a given length.
Now I am trying to initialize a Q-Table full of zeros which would have the same amount of rows as my state-space. And then I want to in each step to run through the state space and check which of all possible state vector is right now. But that means I have to subscript an itertools.product()
How can I do that? because when I try to print it n-th vector from the product it shows an error that product is not subscriptable
I tried this:
import itertools
NUMBER_OF_SECTORS = 6
state_space = itertools.product(*[[0, 1, 2]] * NUMBER_OF_SECTORS)
length = len(list(state_space)) # 729
for obs in range(length):
print(list(state_space[obs]))
Also, is there a possibility, how can I rid the length variable? Because when I define the for loop as: for obs in range(len(list(state_space))) it is not executed at all.
Thank you very much
You can only iterate over an instance of product once: after that, it is consumed. list iterates over the instance in order to produce a list whose length you compute. Once you do that, the state space is gone; all you have left is the length.
You don't need to convert the state space to a list or compute its length; you can just iterate over it directly:
state_space = itertools.product([0,1,2], repeat=NUMBER_OF_SECTORS)
for state in state_space:
print(state)
Related
I have a numpy array total_weights which is an IxI array of floats. Each row/columns corresponds to one of I items.
During my main loop I acquire another real float array weights of size NxM (N, M < I) where each/column row also corresponds to one of the original I items (duplicates may also exist).
I want to add this array to total_weights. However, the sizes and order of the two arrays are not aligned. Therefore, I maintain a position map, a pandas Series with an index of item IDs to their proper index/position in total_weights, called pos_df.
In order to properly make the addition I want I perform the following operation inside the loop:
candidate_pos = pos_df.loc[candidate_IDs] # don't worry about how I get these
rated_pos = pos_df.loc[rated_IDs] # ^^
total_weights[candidate_pos, :][:, rated_pos] += weights
Unfortunately, the above operation must be editing a copy of the orignal total_weights matrix and not a view of it, since after the loop the total_weights array is still full of zeroes. How do I make it change the original data?
Edit:
I want to clarify that candidate_IDs are the N IDs of items and rated_IDs are the M IDs of items in the NxM array called weights. Through pos_df I can get their total order in all of I items.
Also, my guess as to the reason a copy is returned is that candidate_IDs and thus candidate_pos will probably contain duplicates e.g. [0, 1, 3, 1, ...]. So the same rows will sometimes have to be pulled into the new array/view.
Your first problem is in how you are using indexing. As candidate_pos is an array, total_weights[candidate_pos, :] is a fancy indexing operation that returns a new array. When you apply indexing again, i.e. ...[:, rated_pos] you are assigning elements to the newly created array rather than to total_weights.
The second problem, as you have already spotted, is in the actual logic you are trying to apply. If I understand your example correctly, you have a I x I matrix with weights, and you want to update weights for a sequence of pairs ((Ix_1, Iy_1), ..., (Ix_N, Iy_N)) with repetitions, with a single line of code. This can't be done in this way, using += operator, as you'll find yourself having added to weights[Ix_n, Iy_n] the weight corresponding to the last time (Ix_n, Iy_n) appears in your sequence: you have to first merge all the repeating elements in your sequence of weight updates, and then perform the update of your weights matrix with the new "unique" sequence of updates. Alternatively, you must collect your weights as an I x I matrix, and directly sum it to total_weights.
After #rveronese pointed out that it's impossible to do it one go because of the duplicates in candidate_pos I believe I have managed to do what I want with a for-loop on them:
candidate_pos = pos_df.loc[candidate_IDs] # don't worry about how I get these
rated_pos = pos_df.loc[rated_IDs] # ^^
for i, c in enumerate(candidate_pos):
total_weights[c, rated_pos] += weights[i, :]
In this case, the indexing does not create a copy and the assignment should be working as expected...
For example, suppose I had an (n,2) dimensional tensor t whose elements are all from the set S containing random integers. I want to build another tensor d with size (m,2) where individual elements in each tuple are from S, but the whole tuples do not occur in t.
E.g.
S = [0,1,2,3,7]
t = [[0,1],
[7,3],
[3,1]]
d = some_algorithm(S,t)
/*
d =[[2,1],
[3,2],
[7,4]]
*/
What is the most efficient way to do this in python? Preferably with pytorch or numpy, but I can work around general solutions.
In my naive attempt, I just use
d = np.random.choice(S,(m,2))
non_dupes = [i not in t for i in d]
d = d[non_dupes]
But both t and S are incredibly large, and this takes an enormous amount of time (not to mention, rarely results in a (m,2) array). I feel like there has to be some fancy tensor thing I can do to achieve this, or maybe making a large hash map of the values in t so checking for membership in t is O(1), but this produces the same issue just with memory. Is there a more efficient way?
An approximate solution is also okay.
my naive attempt would be a base-transformation function to reduce the problem to an integer set problem:
definitions and assumptions:
let S be a set (unique elements)
let L be the number of elements in S
let t be a set of M-tuples with elements from S
the original order of the elements in t is irrelevant
let I(x) be the index function of the element x in S
let x[n] be the n-th tuple-member of an element of t
let f(x) be our base-transform function (and f^-1 its inverse)
since S is a set we can write each element in t as a M digit number to the base L using elements from S as digits.
for M=2 the transformation looks like
f(x) = I(x[1])*L^1 + I(x[0])*L^0
f^-1(x) is also rather trivial ... x mod L to get back the index of the least significant digit. floor(x/L) and repeat until all indices are extracted. lookup the values in S and construct the tuple.
since now you can represet t as an integer set (read hastable) calculating the inverse set d becomes rather trivial
loop from L^(M-1) to (L^(M+1)-1) and ask your hashtable if the element is in t or d
if the size of S is too big you can also just draw random numbers against the hashtable for a subset of the inverse of t
does this help you?
If |t| + |d| << |S|^2 then the probability of some random tuple to be chosen again (in a single iteration) is relatively small.
To be more exact, if (|t|+|d|) / |S|^2 = C for some constant C<1, then if you redraw an element until it is a "new" one, the expected number of redraws needed is 1/(1-C).
This means, that by doing this, and redrawing elements until this is a new element, you get O((1/(1-C)) * |d|) times to process a new element (on average), which is O(|d|) if C is indeed constant.
Checking is an element is already "seen" can be done in several ways:
Keeping hash sets of t and d. This requires extra space, but each lookup is constant O(1) time. You could also use a bloom filter instead of storing the actual elements you already seen, this will make some errors, saying an element is already "seen" though it was not, but never the other way around - so you will still get all elements in d as unique.
Inplace sorting t, and using binary search. This adds O(|t|log|t|) pre-processing, and O(log|t|) for each lookup, but requires no additional space (other then where you store d).
If in fact, |d| + |t| is very close to |S|^2, then an O(|S|^2) time solution could be to use Fisher Yates shuffle on the available choices, and choosing the first |d| elements that do not appear in t.
I am solving a knapsack problem by using branch and bound algorithm I am working on right now. In the algorithm, I wanted to start selecting the items with the highest density(value/weight). I created a list named "density" and made necessary calculations. I need to pick the maximum value each time from that list. But everytime I try, the order get mixed. I need to update the variable "a" because everytime I delete an item the list gets one smaller. But couldn't figure out how to update it. I need help on selecting the items in the right order.
weight, value, density are lists. capacity and room are integer values given in the problem.
This is the density list.
What I want is, to get the index of the maximum item in this list. Then, subtract the "weight" of it from the "capacity" in order to find how much "room" left. And add the "value" to the "highest" in order the reach the highest value could be added in the knapsack. After I did this for the first item, then iterate it until no or very little room left.
def branch_n_bound(value,weight,capacity):
global highest,size
size=0
room=capacity
density = [0] * len(items)
highest = 0
for i in range(n):
density[i] = val[i] / weight[i]
for i in range(n):
a=density.index(max(density))
if weight[a]<=room:
room-=weight[a]
highest+=value[a]
size+=weight[a]
taken[a]=1
del density[a], weight[a], value[a]
else:
break
I think the problem you try to solve can be solved easier with a change in data structure. Instead of building the density array, you can build an array of tuples [(density, weight, value)...] and base your solution over that array. If you don't want to use so much extra memory and assuming you are ok with changing the input data, you can mark your indices as deleted - for example, you can set the value, weight and density to something negative to know that data was deleted from that index.
You can also take a look at the heapq data structure: https://docs.python.org/3/library/heapq.html . You can work with a heap to extract the maximum, and store indices in that heap.
I am working with the IRIS dataset. I have two sets of data, (1 training set) (2 test set). Now I want to calculate the euclidean distance between every test set row and the train set rows. However, I only want to include the first 4 points of the row.
A working example would be:
dist = np.linalg.norm(inner1test[0][0:4]-inner1train[0][0:4])
print(dist)
***output: 3.034243***
The problem is that I have 120 training set points and 30 test set points - so i would have to do 2700 operations manually, thus I thought about iterating through with a for-loop. Unfortunately, every of my attemps is failing.
This would be my best attempt, which shows the error message
for i in inner1test:
for number in inner1train:
dist = np.linalg.norm(inner1test[i][0:4]-inner1train[number][0:4])
print(dist)
(IndexError: arrays used as indices must be of integer (or boolean)
type)
What would be the best solution to iterate through this array?
ps: I will also provide a screenshot for better vizualisation.
From what I see, inner1test is a tuple of lists, so the i value will not be an index but the actual list.
You should use enumerate, which returns two variables, the index and the actual data.
for i, value in enumerate(inner1test):
for j, number in enumerate(inner1train):
dist = np.linalg.norm(inner1test[i][0:4]-inner1train[number][0:4])
print(dist)
Also, if your lists begin the be bigger, consider using a generator which will execute your calculcations iteration per iteration and return only one value at a time, avoiding to return a big chunk of results which would occupy a lot of memory.
eg:
def my_calculatiuon(inner1test, inner1train):
for i, value in enumerate(inner1test):
for j, number in enumerate(inner1train):
dist = np.linalg.norm(inner1test[i][0:4]-inner1train[number][0:4])
yield dist
for i in my_calculatiuon(inner1test, inner1train):
print(i)
You might also want to investigate python list comprehension which is sometimes more elegant way to handle for loops with lists.
[EDIT]
Here's a probably easier solution anyway, without the need of indexes, which won't fail to enumerate a numpy object:
for testvalue in inner1test:
for testtrain in inner1train:
dist = np.linalg.norm(testvalue[0:4]-testtrain[0:4])
[/EDIT]
This was the final solution with the correct output for me:
distanceslist = list()
for testvalue in inner1test:
for testtrain in inner1train:
dist = np.linalg.norm(testvalue[0:4]-testtrain[0:4])
distances = (dist, testtrain[0:4])
distanceslist.append(distances)
distanceslist
I am trying to create a list of sequential integers from 0 to n, then after picking a randon integer from that list, generate another random integer from the same list that doesn't include the integer previously generated.
n = 10
a = np.arange(1,n) #Creating my initial list
for b=np.random.choice(a): #Generating my first random number
c=np.random.choice(np.arange(1,b)) or np.random.choice(np.arange(b+1,n))
I know that this won't work because my for loop is pretty iffy. I haven't used python in a long time and I am just starting a project and getting myself back into it is proving to be a little tricky!
I think the procedure you are trying to perform is random sampling without replacement.
Let's say you want to pick k numbers:
import numpy as np
n = 10
k = 3
a = np.arange(1,n) #Creating my initial list
numbers = np.random.choice(a, k, replace=False)
Another answer:
1) Generate your initial list.
2) Shuffle the list. If there is no library function to do it, then use the Fisher-Yates shuffle. Hint: there is a big time saver here.
3) Pick the first number from the shuffled list. This is your initial number.
4) Pick the second number from the shuffled list. This is your second number that is both (almost) random and not the same as the first number.
Help on method sample in module random:
sample(self, population, k) method of random.Random instance
Chooses k unique random elements from a population sequence.
Returns a new list containing elements from the population while
leaving the original population unchanged. The resulting list is
in selection order so that all sub-slices will also be valid random
samples. This allows raffle winners (the sample) to be partitioned
into grand prize and second place winners (the subslices).
Members of the population need not be hashable or unique. If the
population contains repeats, then each occurrence is a possible
selection in the sample.
To choose a sample in a range of integers, use xrange as an argument.
This is especially fast and space efficient for sampling from a
large population: sample(xrange(10000000), 60)
Then to pick k random non-repeated numbers in range [0, n] you can do this:
import random
result_list = random.sample(xrange(n + 1), k)