We Keep Coding

Dummy Encoding for Date(time) in Python 3.7

i wanted to create a function that encodes the month of a date(time) object into a season. So for example, if we have 2 seasons and the fourth month the function would return 0 as encoding. My question is now how to create such a function that it is most efficient but also pythonic (less lines of code). I came up with 3 different approaches, the first one looks like this:
def seasonal_encoding(month: int, seasons):
assert 12 % seasons == 0, "Seasons must be in [1, 2, 3, 4, 6, 12]"
if seasons == 1:
return _one_season()
elif seasons == 2:
return _two_seasons(month)
elif seasons == 3:
return _three_seasons(month)
elif seasons == 4:
return _four_seasons(month)
elif seasons == 6:
return _six_seasons(month)
elif seasons == 12:
return _twelve_seasons(month)
where for example three_seasons looks like this:
def _three_seasons(month):
if month in range(1, 5):
return 0
elif month in range(5, 9):
return 1
elif month in range(9, 13):
return 2
This is so far the fastest approach i came up with, but as you can imagine this may work for
a season encoding, but if we want to encode other date(time) attributes, for example hours, this becomes cumbersome real fast. So i came up with a second solution which is a little slower but has less lines of code:
def for_loop_seasonal_encoding(month: int, seasons: int):
assert 12 % seasons == 0, "Seasons must be in [1, 2, 3, 4, 6, 12]"
if seasons != 12:
stepsize = int(12 / seasons)
for i in range(1, seasons+1):
if month in range((i-1) * stepsize + 1, i * stepsize + 1):
return i-1
else:
return month
I also had another solution with numpy before coming up with the for loop solution which i will add for reasons of completeness, but this was four times slower then the other solutions.
def numpy_seasonal_encoding(month: int, seasons):
assert 12 % seasons == 0, "Seasons must be in [1, 2, 3, 4, 6, 12]"
arr = np.array(range(1, 13))
reshaped_arr = arr.reshape(seasons, int(12 / seasons))
return np.where(reshaped_arr == month)[0][0]
So my Question would be: Has anyone of you an idea how to make this as fast as the if representation but with way less lines of codes in "pure" python (so without resorting to Cython, numba etc., external packages like numpy are ok)?
This is the link to the .py file
Python File that includes the function used for timing and the code. Thank you in advance for any help.

Using the result from a function in a new one (plus more)

I am trying to do the following:
1) calculate the amount of the same numbers in the data list. eg : there are three numbers between and including 10 and 20.
2) represent the value for each number range with the same number of '#'. eg: there are 3 numbers between 10 and 20 = ###.
Ideally ending in having the two values represented next to each other.
Unfortunately I really can't figure out step two and any help would really be appreciated.
My code is below:
def count_range_in_list(li, min, max):
ctr = 0
for x in li:
if min <= x <= max:
ctr += 1
return ctr
def amountOfHashes(count_range_in_list,ctr):
ctr = count_range_in_list()
if ctr == 1:
print ('#')
elif ctr == 2:
print ('##')
elif ctr == 3:
print ('###')
elif ctr == 4:
print ('####')
elif ctr == 5:
print ('#####')
elif ctr == 6:
print ('######')
elif ctr == 7:
print ('#######')
elif ctr == 8:
print ('########')
elif ctr == 9:
print ('#########')
elif ctr == 10:
print ('##########')
data = [90,30,13,67,85,87,50,45,51,72,64,69,59,17,22,23,44,25,16,67,85,87,50,45,51]
print(count_range_in_list(data, 0, 10),amountOfHashes)
print(count_range_in_list(data, 10, 20),amountOfHashes)
print(count_range_in_list(data, 20, 30),amountOfHashes)
print(count_range_in_list(data, 30, 40),amountOfHashes)
print(count_range_in_list(data, 40, 50),amountOfHashes)
print(count_range_in_list(data, 50, 60),amountOfHashes)
print(count_range_in_list(data, 60, 70),amountOfHashes)
print(count_range_in_list(data, 70, 80),amountOfHashes)
print(count_range_in_list(data, 80, 90),amountOfHashes)
print(count_range_in_list(data, 90, 100),amountOfHashes)

I'll start by clearing out some doubts you seem to have.
First, how to use the value of a function inside another one:
You don't need to pass the reference of a method to another here. What I mean is, in amountOfHashes(count_range_in_list,ctr) you can just drop count_range_in_list as a parameter, and just define it like amountOfHashes(ctr). Or better yet, use snake case in the method name instead of camel case, so you end up with amount_of_hashes(ctr). Even if you had to execute count_range_in_list inside amount_of_hashes, Python is smart enough to let you do that without having to pass the function reference, since both methods are inside the same file already.
And why do you only need ctr? Well, count_range_in_list already returns a counter, so that's all we need. One parameter, named ctr. In doing so, to "use the result from a function in a new one", we could:
def amount_of_hashes(ctr):
...
# now, passing the value of count_range_in_list in amount_of_hashes
amount_of_hashes(count_range_in_list(data, 10, 20))
You've figured out step 1) quite well already, so we can go to step 2) right away.
In Python it's good to think of iterative processes such as yours dynamically rather than in hard coded ways. That is, creating methods to check the same condition with a tiny difference between them, such as the ones in amountOfHashes, can be avoided in this fashion:
# Method name changed for preference. Use the name that best fits you
def counter_hashes(ctr):
# A '#' for each item in a range with the length of our counter
if ctr == 0:
return 'N/A'
return ''.join(['#' for each in range(ctr)])
But as noted by Roland Smith, you can take a string and multiply it by a number - that'll do exactly what you think: repeat the string multiple times.
>>> 3*'#'
###
So you don't even need my counter_hashes above, you can just ctr*'#' and that's it. But for consistency, I'll change counter_hashes with this new finding:
def counter_hashes(ctr):
# will still return 'N/A' when ctr = 0
return ctr*'#' or 'N/A'
For organization purposes, since you have a specific need (printing the hashes and the hash count) you may then want to format right what comes into print, you could make a specific method for the printing, that calls both counter_hashes and count_Range_in_list, and gives you a cleaner result afterwards:
def hash_range(data, min, max):
ctr = count_range_in_list(data, min, max)
hashes = counter_hashes(ctr)
print(f'{hashes} | {ctr} items in range')
The use and output of this would then become:
>>> data = [90,30,13,67,85,87,50,45,51,72,64,69,59,17,22,23,44,25,16,67,85,87,50,45,51]
>>> hash_range(data, 0, 10)
N/A | 0 items in range
>>> hash_range(data, 10, 20)
### | 3 items in range
>>> hash_range(data, 20, 30)
#### | 4 items in range
And so on. If you just want to print things right away, without the hash_range method above, it's simpler but more lengthy/repetitive if you want a oneliner:
>>> ctr = count_range_in_list(data, 10, 20)
>>> print(counter_hashes(ctr), ctr)
### 3

Why not just do it like this:
Python 3.x:
def amount_of_hashes(ctr):
while ctr > 0:
print('#', end = '')
ctr = ctr-1
Python 2.x:
def amount_of_hashes(ctr):
while ctr > 0:
print '#',
ctr = ctr-1

Counting the number in a list can be done like this:
def count_range_in_list(li, mini, maxi):
return len([i for i in li if mini <= i <= maxi])
Then making a number of hashes is even simpler. Just multiply a string containing the hash sign with a number.
print(ount_range_in_list(data, 0, 10)*'#')
Example in IPython:
In [1]: data = [90,30,13,67,85,87,50,45,51,72,64,69,59,17,22,23,44,25,16,67,85,87,50,45,51]
In [2]: def count_range_in_list(li, mini, maxi):
...: return len([i for i in li if mini <= i <= maxi])
...:
In [3]: print(count_range_in_list(data, 0, 10)*'#')
In [4]: print(count_range_in_list(data, 10, 20)*'#')
###
In [5]: print(count_range_in_list(data, 20, 30)*'#')
####

There are many ways to do this. One way is to use a for loop with range:
# Most basic
def count_range_in_list(li, min, max):
ctr = 0
hashes = ""
for x in li:
if min <= x <= max:
ctr += 1
hashes += "#"
print("There are {0} numbers = {1}".format(ctr, hashes))
# more declarative
def count_range_in_list(li, min, max):
nums = [x for x in li if min <= x <= max]
hashes = "".join(["#" for n in nums])
print("There are {0} numbers = {1}".format(len(nums), hashes))

Alerter monitoring tool : How to find if a number greater than average of the consecutive subsets of windowsize

The Alerter is a simple monitoring tool, intended to help detect
increases in response time for some process. It does that by computing
a few statistics about the process across a 'window' of a certain
number of runs, and alerting (returning true) if certain thresholds
are met.
It takes the following parameters:
inputs: A list of integer times for the process. This list may be very long
window size: how many runs long a window is, as an integer
allowedIncrease: how far over 'average' a window or value is allowed to be, as a percent.
This is represented as a decimal value based on one, so a 50%
allowable increase would be represented as 1.5
Your Alerter should return true if either of the following conditions
are met:
Any value is more than the allowed increase above the window average in ALL windows in which it appears. For example: alert({1, 2, 100, 2,
2}, 3, 1.5) should alert: the value 100 appears in three windows, and
in all cases is more than 50% over the average value alert({1, 2, 4,
2, 2}, 3, 2) should not alert: the largest outlier is 4, and that
value appears in a window with average value 2.6, less than 100% of
that average
Any window's average is more than the acceptable increase over a previous window's average value For example: alert({1,2,100,2,2}, 2,
2.5) should alert: Even though no individual value causes an alert, there is a window with average 1.5 and a later window with an average
more than 2.5 times larger
Otherwise, you should return false.
This is my solution, but it is not working.
from decimal import *
def alert(inputs, windowSize, allowedIncrease):
average = dict()
increase_average = dict()
val_list = list()
## calculating the average and appending to the dictionary
for i in range(0, len(inputs)):
val = sum(inputs[i:i + windowSize])
avg = Decimal(val) / windowSize
if i == len(inputs) - windowSize + 1:
break
else:
for j in range(0, windowSize-1):
try:
average[inputs[i + j]] = avg
except:
average[inputs[i + j]].append(avg)
increase = Decimal(allowedIncrease - 1)
##appending increase in the average
for key, values in average.items():
data = (Decimal(values) * increase) + Decimal(values)
try:
increase_average[key] = data
except:
increase_average[key].append(data)
##checking if all the average value is greater than key
for key, value in increase_average.items():
if key > value:
return True
##checking if any average value greater than incease*previous average value
for (k, v) in average.items():
val_list.append(v)
for h in range(len(val_list)):
if any(val_list >= (Decimal(x * increase) + Decimal(x)) for x in val_list[:h]):
return True
return False
if __name__ == "__main__":
inputs = [1, 2, 4, 2, 2]
windowSize = 3
allowedIncrease = 2
res = alert(inputs, windowSize, allowedIncrease)
print res

There would be a TypeError in the following line:
if any(val_list >= (Decimal(x * increase) + Decimal(x)) for x in val_list[:h]:
You need to change val_list to val_list[k]

change here, and it will start working.
for h in range(len(val_list)):
if any(val_list[h] >= (Decimal(x * increase) + Decimal(x)) for x in ..

This will work.
def alerter(l,w,inc):
dic = {}
lis = []
for i in range(0,len(l)-w+1):
avg = sum(l[i:i+w])/w
lis.append(avg)
for j in range(0,w):
if l[i+j] in dic.keys():
dic[l[i+j]].append(avg)
else:
dic[l[i+j]] = [avg]
for i in range(len(lis)-1):
if lis[i]*inc < lis[i+1]:
return True
for k,v in dic.items():
if min(v)*inc < k:
return True
return False

8 puzzle using blind search (brute-force) and manhattan distance heuristic

I developed my own program in Python for solving 8-puzzle. Initially I used "blind" or uninformed search (basically brute-forcing) generating and exploring all possible successors and using breadth-first search. When it finds the "goal" state, it basically back-tracks to the initial state and delivers (what I believe) is the most optimized steps to solve it. Of course, there were initial states where the search would take a lot of time and generate over 100,000 states before finding the goal.
Then I added the heuristic - Manhattan Distance. The solutions started coming exponentially quickly and with lot less explored states. But my confusion is that some of the times, the optimized sequence generated was longer than the one reached using blind or uninformed search.
What I am doing is basically this:
For each state, look for all possible moves (up, down, left and right), and generate the successor states.
Check if state is repeat. If yes, then ignore it.
Calculate Manhattan for the state.
Pick out the successor(s) with lowest Manhattan and add at the end of the list.
Check if goal state. If yes, break the loop.
I am not sure whether this would qualify as greedy-first, or A*.
My question is, is this an inherent flaw in the Manhattan Distance Heuristic that sometimes it would not give the most optimal solution or am i doing something wrong.
Below is the code. I apologize that it is not a very clean code but being mostly sequential it should be simple to understand. I also apologize for a long code - I know I need to optimize it. Would also appreciate any suggestions/guidance for cleaning up the code. Here is what it is:
import numpy as np
from copy import deepcopy
import sys
# calculate Manhattan distance for each digit as per goal
def mhd(s, g):
m = abs(s // 3 - g // 3) + abs(s % 3 - g % 3)
return sum(m[1:])
# assign each digit the coordinate to calculate Manhattan distance
def coor(s):
c = np.array(range(9))
for x, y in enumerate(s):
c[y] = x
return c
#################################################
def main():
goal = np.array( [1, 2, 3, 4, 5, 6, 7, 8, 0] )
rel = np.array([-1])
mov = np.array([' '])
string = '102468735'
inf = 'B'
pos = 0
yes = 0
goalc = coor(goal)
puzzle = np.array([int(k) for k in string]).reshape(1, 9)
rnk = np.array([mhd(coor(puzzle[0]), goalc)])
while True:
loc = np.where(puzzle[pos] == 0) # locate '0' (blank) on the board
loc = int(loc[0])
child = np.array([], int).reshape(-1, 9)
cmove = []
crank = []
# generate successors on possible moves - new states no repeats
if loc > 2: # if 'up' move is possible
succ = deepcopy(puzzle[pos])
succ[loc], succ[loc - 3] = succ[loc - 3], succ[loc]
if ~(np.all(puzzle == succ, 1)).any(): # repeat state?
child = np.append(child, [succ], 0)
cmove.append('up')
crank.append(mhd(coor(succ), goalc)) # manhattan distance
if loc < 6: # if 'down' move is possible
succ = deepcopy(puzzle[pos])
succ[loc], succ[loc + 3] = succ[loc + 3], succ[loc]
if ~(np.all(puzzle == succ, 1)).any(): # repeat state?
child = np.append(child, [succ], 0)
cmove.append('down')
crank.append(mhd(coor(succ), goalc))
if loc % 3 != 0: # if 'left' move is possible
succ = deepcopy(puzzle[pos])
succ[loc], succ[loc - 1] = succ[loc - 1], succ[loc]
if ~(np.all(puzzle == succ, 1)).any(): # repeat state?
child = np.append(child, [succ], 0)
cmove.append('left')
crank.append(mhd(coor(succ), goalc))
if loc % 3 != 2: # if 'right' move is possible
succ = deepcopy(puzzle[pos])
succ[loc], succ[loc + 1] = succ[loc + 1], succ[loc]
if ~(np.all(puzzle == succ, 1)).any(): # repeat state?
child = np.append(child, [succ], 0)
cmove.append('right')
crank.append(mhd(coor(succ), goalc))
for s in range(len(child)):
if (inf in 'Ii' and crank[s] == min(crank)) \
or (inf in 'Bb'):
puzzle = np.append(puzzle, [child[s]], 0)
rel = np.append(rel, pos)
mov = np.append(mov, cmove[s])
rnk = np.append(rnk, crank[s])
if np.array_equal(child[s], goal):
print()
print('Goal achieved!. Successors generated:', len(puzzle) - 1)
yes = 1
break
if yes == 1:
break
pos += 1
# generate optimized steps by back-tracking the steps to the initial state
optimal = np.array([], int).reshape(-1, 9)
last = len(puzzle) - 1
optmov = []
rank = []
while last != -1:
optimal = np.insert(optimal, 0, puzzle[last], 0)
optmov.insert(0, mov[last])
rank.insert(0, rnk[last])
last = int(rel[last])
# show optimized steps
optimal = optimal.reshape(-1, 3, 3)
print('Total optimized steps:', len(optimal) - 1)
print()
for s in range(len(optimal)):
print('Move:', optmov[s])
print(optimal[s])
print('Manhattan Distance:', rank[s])
print()
print()
################################################################
# Main Program
if __name__ == '__main__':
main()
Here are some of the initial states and the optimized steps calculated if you would like to check (above code would give this option to choose between blind vs Informed search)
Initial states
- 283164507 Blind: 19 Manhattan: 21
- 243780615 Blind: 15 Manhattan: 21
- 102468735 Blind: 11 Manhattan: 17
- 481520763 Blind: 13 Manhattan: 23
- 723156480 Blind: 16 Manhattan: 20
I have deliberately chosen examples where results would be quick (within seconds or few minutes).
Your help and guidance would be much appreciated.
Edit: I have made some quick changes and managed to reduce some 30+ lines. Unfortunately can't do much at this time.
Note: I have hardcoded the initial state and the blind vs informed choice. Please change the value of variable "string" for initial state and the variable "inf" [I/B] for Informed/Blind. Thanks!

what this python code trying to do

The following python code is to traverse a 2D grid of (c, g) in some special order, which is stored in "jobs" and "job_queue". But I am not sure which kind of order it is after trying to understand the code. Is someone able to tell about the order and give some explanation for the purpose of each function? Thanks and regards!
import Queue
c_begin, c_end, c_step = -5, 15, 2
g_begin, g_end, g_step = 3, -15, -2
def range_f(begin,end,step):
# like range, but works on non-integer too
seq = []
while True:
if step > 0 and begin > end: break
if step < 0 and begin < end: break
seq.append(begin)
begin = begin + step
return seq
def permute_sequence(seq):
n = len(seq)
if n <= 1: return seq
mid = int(n/2)
left = permute_sequence(seq[:mid])
right = permute_sequence(seq[mid+1:])
ret = [seq[mid]]
while left or right:
if left: ret.append(left.pop(0))
if right: ret.append(right.pop(0))
return ret
def calculate_jobs():
c_seq = permute_sequence(range_f(c_begin,c_end,c_step))
g_seq = permute_sequence(range_f(g_begin,g_end,g_step))
nr_c = float(len(c_seq))
nr_g = float(len(g_seq))
i = 0
j = 0
jobs = []
while i < nr_c or j < nr_g:
if i/nr_c < j/nr_g:
# increase C resolution
line = []
for k in range(0,j):
line.append((c_seq[i],g_seq[k]))
i = i + 1
jobs.append(line)
else:
# increase g resolution
line = []
for k in range(0,i):
line.append((c_seq[k],g_seq[j]))
j = j + 1
jobs.append(line)
return jobs
def main():
jobs = calculate_jobs()
job_queue = Queue.Queue(0)
for line in jobs:
for (c,g) in line:
job_queue.put((c,g))
main()
EDIT:
There is a value for each (c,g). The code actually is to search in the 2D grid of (c,g) to find a grid point where the value is the smallest. I guess the code is using some kind of heuristic search algorithm? The original code is here http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/gridsvr/gridregression.py, which is a script to search for svm algorithm the best values for two parameters c and g with minimum validation error.

permute_sequence reorders a list of values so that the middle value is first, then the midpoint of each half, then the midpoints of the four remaining quarters, and so on. So permute_sequence(range(1000)) starts out like this:
[500, 250, 750, 125, 625, 375, ...]
calculate_jobs alternately fills in rows and columns using the sequences of 1D coordinates provided by permute_sequence.
If you're going to search the entire 2D space eventually anyway, this does not help you finish sooner. You might as well just scan all the points in order. But I think the idea was to find a decent approximation of the minimum as early as possible in the search. I suspect you could do about as well by shuffling the list randomly.
xkcd readers will note that the urinal protocol would give only slightly different (and probably better) results:
[0, 1000, 500, 250, 750, 125, 625, 375, ...]

Here is an example of permute_sequence in action:
print permute_sequence(range(8))
# prints [4, 2, 6, 1, 5, 3, 7, 0]
print permute_sequence(range(12))
# prints [6, 3, 9, 1, 8, 5, 11, 0, 7, 4, 10, 2]
I'm not sure why it uses this order, because in main, it appears that all candidate pairs of (c,g) are still evaluated, I think.