itertools.product for the full range of columns - python

as a part of my code, I'm trying to get a full factorial matrix, this is not a problem since I already have a working code for it. However, I would like to generalize it in a way that it wouldn't matter the number of inputs. This would require modifying the line:
for combination in itertools.product(X[0,:],X[1,:],X[2,:],X[3,:],X[4,:],X[5,:],X[6,:]):
input_list = dfraw.columns[0:n_inputs]
output_list = dfraw.columns[n_inputs:len(dfraw.columns)]
fflvls = 4
lhspoints = 60000
X = np.zeros((n_inputs, fflvls),float)
ii=0
for entrada in input_list:
X[ii] = np.linspace(min(dfraw[entrada]), max(dfraw[entrada]), fflvls)
ii+=1
number=1
i=0
X_fact=np.zeros((int(fflvls**n_inputs),n_inputs),float)
for combination in itertools.product(X[0,:],X[1,:],X[2,:],X[3,:],X[4,:],X[5,:],X[6,:]):
X_fact[i,:] = (combination)
i +=1
number+=1
I thought of writing the input of itertools.product as a string with a loop and then evaluating but it doesn't work and I've also seen it is regarded as bad practice
prodstring = ['X[0,:]']
for ii in range(n_inputs):
prodstring.append(',X[%d,:]'%(ii))
in_products = ''.join(prodstring)
for combination in itertools.product(eval(in_products)):
X_fact[i,:] = (combination)
i +=1
number+=1
what other way is there to inputing the full range of columns in this function? (or similar ones)

who said working harder is working better? im back from lunch and I delved into *args and **kwargs as a form of procrastination cause ive sometimes seen them mentioned and i was curious. It seems like it was just the tool I needed. In case this can help other code rookies like me in the future:
args = ()
for ii in range(n_inputs):
b = (X[ii,:])
args += (b,)
for combination in itertools.product(*args):
X_fact[i,:] = (combination)
i +=1
number+=1
Seems to work properly. Solved in an hour of "not working" what i haven't solved in approx 4 hours of "working"

Related

Extract words from random strings

Below I have some strings in a list:
some_list = ['a','l','p','p','l','l','i','i','r',i','r','a','a']
Now I want to take the word april from this list. There are only two april in this list. So I want to take that two april from this list and append them to another extract list.
So the extract list should look something like this:
extract = ['aprilapril']
or
extract = ['a','p','r','i','l','a','p','r','i','l']
I tried many times trying to get the everything in extract in order, but I still can't seems to get it.
But I know I can just do this
a_count = some_list.count('a')
p_count = some_list.count('p')
r_count = some_list.count('r')
i_count = some_list.count('i')
l_count = some_list.count('l')
total_count = [a_count,p_count,r_count,i_count,l_count]
smallest_count = min(total_count)
extract = ['april' * smallest_count]
Which I wouldn't be here If I just use the code above.
Because I made some rules for solving this problem
Each of the characters (a,p,r,i and l) are some magical code elements, these code elements can't be created out of thin air; they are some unique code elements, that has some uniquw identifier, like a secrete number that is associated with them. So you don't know how to create this magical code elements, the only way to get the code elements is to extract them to a list.
Each of the characters (a,p,r,i and l) must be in order. Imagine they are some kind of chains, they will only work if they are together. Meaning that we got to put p next to and in front of a, and l must come last.
These important code elements are some kind of top secrete stuff, so if you want to get it, the only way is to extract them to a list.
Below are some examples of a incorrect way to do this: (breaking the rules)
import re
word = 'april'
some_list = ['aaaaaaappppppprrrrrriiiiiilll']
regex = "".join(f"({c}+)" for c in word)
match = re.match(regex, text)
if match:
lowest_amount = min(len(g) for g in match.groups())
print(word * lowest_amount)
else:
print("no match")
from collections import Counter
def count_recurrence(kernel, string):
# we need to count both strings
kernel_counter = Counter(kernel)
string_counter = Counter(string)
effective_counter = {
k: int(string_counter.get(k, 0)/v)
for k, v in kernel_counter.items()
}
min_recurring_count = min(effective_counter.values())
return kernel * min_recurring_count
This might sounds really stupid, but this is actually a hard problem (well for me). I originally designed this problem for myself to practice python, but it turns out to be way harder than I thought. I just want to see how other people solve this problem.
If anyone out there know how to solve this ridiculous problem, please help me out, I am just a fourteen-year-old trying to do python. Thank you very much.
I'm not sure what do you mean by "cannot copy nor delete the magical codes" - if you want to put them in your output list you will need to "copy" them somehow.
And btw your example code (a_count = some_list.count('a') etc) won't work since count will always return zero.
That said, a possible solution is
worklist = [c for c in some_list[0]]
extract = []
fail = False
while not fail:
lastpos = -1
tempextract = []
for magic in magics:
if magic in worklist:
pos = worklist.index(magic, lastpos+1)
tempextract.append(worklist.pop(pos))
lastpos = pos-1
else:
fail = True
break
else:
extract.append(tempextract)
Alternatively, if you don't want to pop the elements when you find them, you may compute the positions of all the occurences of the first element (the "a"), and set lastpos to each of those positions at the beginning of each iteration
May not be the most efficient way, although code works and is more explicit to understand the program logic:
some_list = ['aaaaaaappppppprrrrrriiiiiilll']
word = 'april'
extract = []
remove = []
string = some_list[0]
for x in range(len(some_list[0])//len(word)): #maximum number of times `word` can appear in `some_list[0]`
pointer = i = 0
while i<len(word):
j=0
while j<(len(string)-pointer):
if string[pointer:][j] == word[i]:
extract.append(word[i])
remove.append(pointer+j)
i+=1
pointer = j+1
break
j+=1
if i==len(word):
for r_i,r in enumerate(remove):
string = string[:r-r_i] + string[r-r_i+1:]
remove = []
elif j==(len(string)-pointer):
break
print(extract,string)

Can I make python read a string as a parameter?

I would like to make this code somehow work, in this example i used the int() function which obviously gave me the "invalid literal for int() with base 10" error. But is there any way to make python not read the "" around the string and just read "i" in this case as a parameter.
I hope you understand what I mean.
s=0
w=""
for k in range(3):
w+="i"
for i in range(5):
s+=int(w)
I donĀ“t know whats your expected result. Your main problems seems the quotes for w = "" and w+="i" if you want to add numbers instead of textstrings.
Maybe you also mixed up your k and i variable.
Is this your expected result(?):
s = 0
w = 0
for k in range(3):
w += k
for i in range(5):
s += w
print(s)
print(w)
In your current version of code, when you say int(w) you are trying to convert 'iii' into a base 10 number which runs the error. Also, using i as a variable will only work within the for loop 'for i in range(5):'. That is where the variable i has its lifespan.

Cannot understand how this Python code works

I found this question on HackerRank and I am unable to understand the code(solution) that is displayed in the discussions page.
The question is:
Consider a list (list = []). You can perform the following commands:
insert i e: Insert integer at position .
print: Print the list.
remove e: Delete the first occurrence of integer .
append e: Insert integer at the end of the list.
sort: Sort the list.
pop: Pop the last element from the list.
reverse: Reverse the list.
Even though I have solved the problem using if-else, I do not understand how this code works:
n = input()
slist = []
for _ in range(n):
s = input().split()
cmd = s[0]
args = s[1:]
if cmd !="print":
cmd += "("+ ",".join(args) +")"
eval("slist."+cmd)
else:
print slist
Well, the code takes advantage of Python's eval function. Many languages have this feature: eval, short for "evaluate", takes a piece of text and executes it as if it were part of the program instead of just a piece of data fed to the program. This line:
s = input().split()
reads a line of input from the user and splits it into words based on whitespace, so if you type "insert 1 2", s is set to the list ["insert","1","2"]. That is then transformed by the following lines into "insert(1,2)", which is then appended to "slist." and passed to eval, resulting in the method call slist.insert(1,2) being executed. So basically, this code is taking advantage of the fact that Python already has methods to perform the required functions, that even happen to have the same names used in the problem. All it has to do is take the name and arguments from an input line and transform them into Python syntax. (The print option is special-cased since there is no method slist.print(); for that case it uses the global command: print slist.)
In real-world code, you should almost never use eval; it is a very dangerous feature, since it allows users of your application to potentially cause it to run any code they want. It's certainly one of the easier features for hackers to use to break into things.
It's dirty code that's abusing eval.
Basically, when you enter, for example, "remove 1", it creates some code that looks like sList.remove(1), then gives the created code to eval. This has Python interpret it.
This is probably the worst way you could solve this outside of coding competitions though. The use of eval is entirely unnecessary here.
Actually I Find some error in the code, but I came to an understanding of how this code runs. here is it:
input :
3
1 2 3
cmd = 1 + ( 2 + 3)
then eval(cmd) i.e., eval("1 + (2 + 3)") which gives an output 6
another input:
4
4 5 6 2
cmd = 4 + ( 5 + 6 + 2)
eval(cmd)
if __name__ == '__main__':
N = int(raw_input())
lst=[]
for _ in range(N):
cmd, *line = input().split()
ele= list(map(str,line))
if cmd in dir(lst):
exec('lst.'+cmd+'('+','.join(ele)+')')
elif cmd == 'print':
print(lst)
else:
print('wrong command', cmd)

Selecting from multiple variables

I am attempting to find objects on the screen, see if they exist, and if so, select them. Using the Sikuli library to run this little automation.
while True:
if exist("image/one.png", "image/two.png", "image/three.png"):
click ("image/one.png", or "image/two.png", or "image/three.png")
break
I get SyntaxError: mismatched input 'or' expecting RPARENa I've done a quick search but there is nothing I saw relevant to my particular issue.
I've even tried
while True:
if exist("image/one.png", or "image/two.png", or "image/three.png"):
click ("image/one.png", or "image/two.png", or "image/three.png")
break
And that results in the same error.
#Stephan: New code snippet with error.
class gameImages():
imageFiles = ["one.png", "two.png", "three,png"]
for imageFile in imageFiles:
if exists(imageFile):
click(imageFile)
The Error now, :
NameError: name 'imageFiles' is not defined
for imageFile in imageFiles:
if exists(imageFile):
click(imageFile)
Your while loop isn't doing anything, and neither is your break statement. This might do what you want, assuming I understand what you want to do.
After reading a little of the Sikuli docs, I think this might also do what you want.
for impath in ("image/one.png", "image/two.png", "image/three.png"):
match = exists(impath)
if match:
click(match.getTarget())
Even easier, this is a perfect use of filter(ifexist,imageFiles). You then know that all >=0 elements in the return of filter can be used :). And it's more concise and clearly conveys your intent - much nicer to read then a chain of for's and if's
a = range(10)
# [1,2,3,4,5,6,7,8,9]
print filter(lambda x: x > 5, a)
# [6,7,8,9]
Also the or is a logical operator:
e.g.
a = 5
b = 6
c = 5
if( (a==c) or (b==c) ):
print 'c is repeated'
# c is repeated
your use of the or here makes no sense as it doesn't have operands to operate on - these can even be two objects, e.g.
1 or 2 since anything can be cast to a boolean
a concise way to do what you want is:
//imagepaths = your list of imagepaths
map(lambda x: click(x.getTarget()), filter(exists, imagepaths))

What's wrong with my python multiprocessing code?

I am an almost new programmer learning python for a few months. For the last 2 weeks, I had been coding to make a script to search permutations of numbers that make magic squares.
Finally I succeeded in searching the whole 880 4x4 magic square numbers sets within 30 seconds. After that I made some different Perimeter Magic Square program. It finds out more than 10,000,000 permutations so that I want to store them part by part to files. The problem is that my program doesn't use all my processes that while it is working to store some partial data to a file, it stops searching new number sets. I hope I could make one process of my CPU keep searching on and the others store the searched data to files.
The following is of the similar structure to my magic square program.
while True:
print('How many digits do you want? (more than 20): ', end='')
ansr = input()
if ansr.isdigit() and int(ansr) > 20:
ansr = int(ansr)
break
else:
continue
fileNum = 0
itemCount = 0
def fileMaker():
global fileNum, itemCount
tempStr = ''
for i in permutationList:
itemCount += 1
tempStr += str(sum(i[:3])) + ' : ' + str(i) + ' : ' + str(itemCount) + '\n'
fileNum += 1
file = open('{0} Permutations {1:03}.txt'.format(ansr, fileNum), 'w')
file.write(tempStr)
file.close()
numList = [i for i in range(1, ansr+1)]
permutationList = []
itemCount = 0
def makePermutList(numList, ansr):
global permutationList
for i in numList:
numList1 = numList[:]
numList1.remove(i)
for ii in numList1:
numList2 = numList1[:]
numList2.remove(ii)
for iii in numList2:
numList3 = numList2[:]
numList3.remove(iii)
for iiii in numList3:
numList4 = numList3[:]
numList4.remove(iiii)
for v in numList4:
permutationList.append([i, ii, iii, iiii, v])
if len(permutationList) == 200000:
print(permutationList[-1])
fileMaker()
permutationList = []
fileMaker()
makePermutList(numList, ansr)
I added from multiprocessing import Pool at the top. And I replaced two 'fileMaker()' parts at the end with the following.
if __name__ == '__main__':
workers = Pool(processes=2)
workers.map(fileMaker, ())
The result? Oh no. It just works awkwardly. For now, multiprocessing looks too difficult for me.
Anybody, please, teach me something. How should my code be modified?
Well, addressing some things that are bugging me before getting to your asked question.
numList = [i for i in range(1, ansr+1)]
I know list comprehensions are cool, but please just do list(range(1, ansr+1)) if you need the iterable to be a list (which you probably don't need, but I digress).
def makePermutList(numList, ansr):
...
This is quite the hack. Is there a reason you can't use itertools.permutations(numList,n)? It's certainly going to be faster, and friendlier on memory.
Lastly, answering your question: if you are looking to improve i/o performance, the last thing you should do is make it multithreaded. I don't mean you shouldn't do it, I mean that it should literally be the last thing you do. Refactor/improve other things first.
You need to take all of that top-level code that uses globals, apply the backspace key to it, and rewrite functions that pass data around properly. Then you can think about using threads. I would personally use from threading import Thread and manually spawn Threads to do each unit of I/O rather than using multiprocessing.

Categories