I have a very big iterator. due to lack of resources (Network, Memory and Time), It's not possible for me to execute my program at one step.
So I thought it would be nice if I run my program till 10000th element in iterator and then save its state. next time I run program it continues from 10001st element in iterator.
here is the code I use:
import itertools
import requests
POSSIBLE_CHARS = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
URL = "URL?key={code}"
all_possible = itertools.combinations_with_replacement(POSSIBLE_CHARS, 29)
counter = itertools.count(start=1)
for c in all_possible:
print("Try {}:".format(next(counter)), c)
c_url = URL.format(code=c)
resp = requests.get(c_url)
if resp.status_code == 200:
print("C found:", c)
with open(c+".gif", 'b') as f:
f.write(resp.content)
this link shows how to continue iterator inside a single execution. but what I want is to stop program and execute again.
So I thought it would be nice if I run my program till 10000th element in iterator and then save its state. next time I run program it continues from 10001st element in iterator.
You're in luck, because itertools.combinations_with_replacement objects have APIs allowing to set state. This is used by the copy module in Python, but there is no reason you can't also hook in.
For example the 10,001th item would be this:
>>> all_possible = itertools.combinations_with_replacement(POSSIBLE_CHARS, 29)
>>> for i in range(10_000):
... next(all_possible)
...
>>> "".join(next(all_possible))
'aaaaaaaaaaaaaaaaaaaaaaaaaafwI'
To "fast-forward" to this item in a fresh instance, it would be:
>>> new_iterator = itertools.combinations_with_replacement(POSSIBLE_CHARS, 29)
>>> state = (0,)*26 + (5, 22, 33)
>>> new_iterator.__setstate__(state)
>>> "".join(next(new_iterator))
'aaaaaaaaaaaaaaaaaaaaaaaaaafwI'
>>> "".join(next(new_iterator))
'aaaaaaaaaaaaaaaaaaaaaaaaaafwJ'
>>> new_iterator.__setstate__(state) # bonus: rewind iterator!
>>> "".join(next(new_iterator))
'aaaaaaaaaaaaaaaaaaaaaaaaaafwI'
To understand why the combination 10,001 corresponds to some tuple of length 29 like (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 5, 22, 33), I invite you to browse the CPython sources to see how combinations objects allow themselves to be statefully copied and also the relevant section of the pickle docs.
If you don't require security against erroneous or maliciously constructed data, then you might consider to simply pickle such iterators to file rather than hook into the __setstate__ method manually.
Related
I'm learning unit testing and TDD in python and have got the basics mastered with pytest.
I've started a learning project of writing a sudoku solver and one thing I'm realising is that at the start I'm not currently sure how I want to store the data for the sudoku board. Many talks warn about testing behaviour over implementation. Would the board data structure count as implementation?
It's very hard to write a test that is independent of the board data structure.
For example you could store the values of each cell as an integer (eg 1) or a string (eg '1') and the board as a whole can be a dict (using cell label like 'A1' or 'C5' etc as keys), list (flat or 2d) or a 81 character string.
So I might start with
hardboard = [
[0, 0, 0, 0, 0, 0, 4, 0, 0],
[0, 0, 1, 0, 7, 0, 0, 9, 0],
[5, 0, 0, 0, 3, 0, 0, 0, 6],
[0, 8, 0, 2, 0, 0, 0, 0, 0],
[7, 0, 0, 0, 0, 0, 9, 2, 0],
[1, 2, 0, 6, 0, 5, 0, 0, 0],
[0, 5, 0, 0, 0, 0, 0, 4, 0],
[0, 7, 3, 9, 0, 8, 0, 0, 0],
[6, 0, 0, 4, 5, 0, 2, 0, 0]
]
but then decide to use a dict
hardboard = {'A1': 0, 'A2': 0, 'A3': 0, ... 'I9': 0}
So how should I start? The only solution I can think of is to commit to one format, at least in terms how I represent the board in tests, even if the actual code uses a different data structure later and has to translate. But in my TDD noobiness I'm unsure if this is a good idea?
I'm not a python developer, but still:
Don't think about the board in terms of implementation details (like, this is a dictionary, array, slice, whatever).
Instead imagine that the board is an abstraction and it has some behaviors - which are in plain words - things you can do with a board:
place a digit in an index(i,j) that should be in 1 to 9 inclusive boundaries
remove a digit by index (i,j)
So its like you have a "contract" - you create a board and can execute some operations on it. Of course it has some internal implementation, but its hidden from everyone who uses it. Its your responsibility to provide a working implementation but this is what you want to test: The implementation is considered working if it "reacts" on all the operations correctly (behavior = operation = function at the level of actual programming implementation)
In this case your test (schematically might look like this):
Board Tests:
Test 1 : Check that its impossible to put a negative i index:
// setup
board = ...
// when:
board.put(-1, 3, 8) // i-index is wrong
// then:
expect exception to be thrown
Test 2 : Check that its impossible to put a negative j index:
// setup
board = ...
// when:
board.put(0, -1, 5) // j-index is wrong
// then:
expect exception to be thrown
Test 3 : Check that its impossible to put a negative number into cell:
// setup
board = ...
// when:
board.put(0, 0, -7)
// then:
expect exception to be thrown
Test 4 : Check that its impossible to put number that alreay contains another number:
// setup
board = ...
// when:
board.put(0, 0, 6)
board.put(0, 0, 5)
// then:
expect exception to be thrown
... and so on and so forth (also cover with tests for getting a cell, and any other additional behavior you create for the board).
Note, that all these tests indeed check the behavior of your abstraction rather than internal implementation.
At the level of implementation you can create a class for the board, internal data fields will contain the state, whereas the behavior will be represented by the methods exposed to the users of the class.
Please I need help with this code.I want mylist to retain values appended to it next time the function 'no_repeat_rule' is called. I'm pretty new to python. My code is below:
def no_repeat_rule(play_board):
mylist = list()
if seeds_left(play_board) == 2 and sum(play_board[:6])== 1:
mylist.append(play_board)
return mylist
Output of this code (in part) is:
...
[[1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0]]
Player 1 chose cup 0
[[0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0]]
Player 2 chose cup 6
[[0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0]]
...
what I want the function 'no_repeat_rule' to do is to grow mylist each time a player plays. I don't know if this is clear enough to get help?
The simplest thing to do would be to add another parameter in the function defintion, such that it looks like:
def no_repeat_rule(play_board, aList):
Before you call the function, declare a list outside of the function. Set this equal to the result of the function, and pass it as a parameter whenever you call the function. For instance:
x = list()
def no_repeat_rule(play_board, aList):
myList = aList
if seeds_left(play_board) == 2 and sum(play_board[:6])== 1:
myList.append(play_board)
return myList
x = no_repeat_rule(someBoardHere, x)
I believe this should work if I understand what you're asking. If not, please respond and I'll try something else.
what do you need is an object which is associated with the function. It calls attribute. It is very handy in python.
Your code may look like this:
def no_repeat_rule(play_board):
if not hasattr(no_repeat_rule,"mylist"):
no_repeat_rule.mylist = []
if seeds_left(play_board) == 2 and sum(play_board[:6])== 1:
no_repeat_rule.mylist.append(play_board)
return no_repeat_rule.mylist
I couldn't check this code, but it should work for local atributes. BTW it is for python 2.7
I am new to Python and programming in general. I was working on Pyschool exercises Topic 8, Q 11 on converting Dictionary to Spare Vectore.
I was asked to Write a function that converts a dictionary back to its sparese vector representation.
Examples
>>> convertDictionary({0: 1, 3: 2, 7: 3, 12: 4})
[1, 0, 0, 2, 0, 0, 0, 3, 0, 0, 0, 0, 4]
>>> convertDictionary({0: 1, 2: 1, 4: 2, 6: 1, 9: 1})
[1, 0, 1, 0, 2, 0, 1, 0, 0, 1]
>>> convertDictionary({})
[]
I have attempted many times. Below is the latest code I have:
def convertDictionary(dictionary):
k=dictionary.keys()
v=dictionary.values()
result=[]
for i in range(0,max(k)):
result.append(0)
for j in k:
result[j]=v[k.index(j)]
return result
The returned error is:
Traceback (most recent call last):
File "Code", line 8, in convertDictionary
IndexError: list assignment index out of range
Could anyone help me? Thank you so much!
Something like this should suffice:
M = max(dictionary, default=0)
vector = [dictionary.get(i, 0) for i in range(M)]
Translated into a plain old for-loop
M = max(dictionary, default=0)
vector = []
for i in range(M):
vector.append(dictionary.get(i, 0))
The get method lets you provide a default as a second argument in case the key is missing. Once you get more advance you could use a defaultdict
Edit: the default parameter for max requires Python >3.4 . You can either use exception handling (generally prefered) or explicit checks for empty dictionary to deal with that case if you have earlier versions.
Your code works logically fine, but you have a problem of indentation. Your function should be:
def convertDictionary(dictionary):
k=dictionary.keys()
v=dictionary.values()
result=[]
for i in range(0,max(k)):
result.append(0)
for j in k:
result[j]=v[k.index(j)]
return result
The problem is that your second for was inside of the first. What you want is to build a list with max(k) elements and then put the right values into it. Then, the two for loops should be one after the other, rather than one inside of the other.
I'm new to python, and I'm doing a homework with it.
I'm using PyEDA with Python3 on Debian 8.
I created a variable named:
R = exprvars('r', n, n)
with n = 4, this gives me:
farray([[r[0,0], r[0,1], r[0,2], r[0,3]],
[r[1,0], r[1,1], r[1,2], r[1,3]],
[r[2,0], r[2,1], r[2,2], r[2,3]],
[r[3,0], r[3,1], r[3,2], r[3,3]]])
Then, after some logic, I create an CNF boolean function f and a BDD with it using:
f = expr2bdd(f)
Then, the expression:
U = f.satisfy_one()
gives me:
{r[2,1]: 0, r[3,2]: 1, r[1,1]: 1, r[0,2]: 0, r[0,3]: 1, r[2,2]: 0, r[2,3]: 0, r[3,3]: 0, r[3,1]: 0, r[1,2]: 0, r[0,1]: 0, r[1,0]: 0, r[2,0]: 1, r[3,0]: 0, r[0,0]: 0, r[1,3]: 0}
But here is what I can't understand:
I was expecting
U[R[0,0]]
To return a 0, but instead it gives me
KeyError: r[0,0]
What is the problem? R[0,0] gives me r[0,0] and the dictionary do have it as a key.
[edit]
When I said R[0,0] gives me r[0,0], this means that I printed it using pdb, placing a breakpoint right after U = f.satisfy_one():
(Pdb) p R[0,0]
r[0,0]
This needs more details.
How do you say, R[0,0] gives me r[0,0]? Did you print them?
Try to verify if you are looking for same
assert list(U.keys())[0] == R[2,1]
Look the the values
print(list(type(U.keys())[0])
print(type(R[2,1])
See if they match, only if they match, will you be able to gather it.
Also, check if U itself has any methods to do the query for you.
I'm writing a program in Linux which reads and distinguish inputs from two USB devices(two barcode readers) which simulates a keyboard.
I've already can read inputs from USB, but it happens before OS translate keycode in a charactere.
For example, when I read 'a' i got 24, 'b' 25, etc....
For example, when I read 'a' i got 4, 'b' 5, etc....
Is there any way to convert that code in a char without manual mapping?
Some output exemples:
KEYPRESS = a output = array('B', [0, 0, 4, 0, 0, 0, 0, 0])
KEYPRESS = SHIFT + a output = array('B', [2, 0, 4, 0, 0, 0, 0, 0])
KEYPRESS = 1 output = array('B', [0, 0, 30, 0, 0, 0, 0, 0])
KEYPRESS = ENTER output = array('B', [0, 0, 81, 0, 0, 0, 0, 0])
thx!
Use the chr function. Python uses a different character mapping (ASCII) from whatever you're receiving though, so you will have to add 73 to your key values to fix the offset.
>>> chr(24 + 73)
'a'
>>> chr(25 + 73)
'b'
I've already can read inputs from USB, but it happens before OS
translate keycode in a charactere.
The problem seems to me in your interface or the driver program.
In ASCII 'a' is supposed to have ordinal value 97 whose binary representation is 0b1100001, where as what you are receiving is 27 whose binary representation is 0b11000, similarly for 'b' you were supposed to received '0b1100010' instead you received 25 which is 0b11001. Check your hardware to determine if the 1st and the 3rd bit is dropped from the input.
What you are receiving is USB scan code. I do not think there is a third party python library to do the conversion for you. I would suggest you to refer any of the USB Scan Code Table and from it, create a dictionary of USB Scan Code vs the corresponding ASCII.