Can anyone suggest an efficient way of reshaping a column (in a python pandas dataframe) into multiple columns, with alternating column assignment. I could do this with a loop but wondering if there is a more elegant way. For an example, consider the following example:
Added: does anyone have a solution that will reshape every n values in a single column into n separate columns e.g. reshaping from a single column with n variables to n columns?
Col
1 x1
2 y1
3 z1
4 x2
5 y2
6 z2
7 x3
8 y3
9 z3
..
to
x y z
1 x1 y1 z1
2 x2 y2 z2
3 x3 y3 z3
...
You can just reshape the underlying values, assuming that you have the correct number of values for the given shape and that you only care about ordering the values by shape without respect to the values themselves
s
Col
1 x1
2 y1
3 z1
4 x2
5 y2
6 z2
7 x3
8 y3
9 z3
pd.DataFrame(s.to_numpy().reshape(3, 3))
0 1 2
0 x1 y1 z1
1 x2 y2 z2
2 x3 y3 z3
You can use:
df_final=(pd.DataFrame(df.groupby(df.Col.str[-1])['Col'].apply(list)
.values.tolist(),columns=['x','y','z']))
x y z
0 x1 y1 z1
1 x2 y2 z2
2 x3 y3 z3
You can use auxiliary variables to work as the row and column index, then apply df.pivot
df1['aux'] = df1.Col.str[:-1]
df1['aux_idx'] = df1.Col.str[-1:]
print(df1.pivot(index= 'aux_idx', columns='aux', values='Col'))
Output:
aux x y z
aux_idx
1 x1 y1 z1
2 x2 y2 z2
3 x3 y3 z3
For the same result by just counting the number of elements, use df.index module n as the key
df1['aux_idx'] = (df1.index-1)// 3
df1['aux'] = df1.Col.str[:-1]
print(df1.pivot(index= 'aux_idx', columns='aux', values='Col'))
Output:
aux x y z
aux_idx
0 x1 y1 z1
1 x2 y2 z2
2 x3 y3 z3
Related
If i have 2 z3 Ints for examaple x1 and x2, and a 2d array of numbers for example:
list = [[1,2],[12,13],[45,7]]
i need to right a rule so that x1 and x2 are any of the pairs of numbers in the list for example x1 would be 1 and x2 would be 2 or x1 is 12 and x2 is 13
im guessing it would be something like:
solver = Solver()
for i in range(o,len(list)):
solver.add(And((x1==list[i][0]),(x2==list[i][1])))
but this would obviously just always be unsat, so i need to right it so that x1 and x2 can be any of the pairs in the list. It's worth noting that the number of pairs in the list could be anything not just 3 pairs.
You're on the right track. Simply iterate and form the disjunction instead. Something like:
from z3 import *
list = [[1,2],[12,13],[45,7]]
s = Solver()
x1, x2 = Ints('x1 x2')
s.add(Or([And(x1 == p[0], x2 == p[1]) for p in list]))
while s.check() == sat:
m = s.model()
print("x1 = %2d, x2 = %2d" % (m[x1].as_long(), m[x2].as_long()))
s.add(Or(x1 != m[x1], x2 != m[x2]))
When run, this prints:
x1 = 1, x2 = 2
x1 = 12, x2 = 13
x1 = 45, x2 = 7
Let's say I have two ranges (x1,x2) and (z1,z2). What is fastest way to check if they intersect?
if
#case: (2,4), (3,6)
x1 > z1 < x2 and (z2 < x1 or z2 > x2)
or
#case: (2,4) , (1,3)
x1 > z2 < x2 and (z1 < x1 or z1 > x2)
or
#case: (3,6), (2,4)
............. reverse(1) x <--> z
or
#case: (1,3), (2,4)
............. reverse(2) x <--> z
BTW containing one range in the other is OK, for example
(2,5) , (3,4)
(2,5) , (3,5)
(2,5) , (2,4)
(this makes the question not a duplicate. In addition the conditions are <, > rather than >=, <=. And one more I expect input ranges to be also unordered, for example (x2,x1),(z2,z1) ; (z1,z2),(x1,x2) )
But this, for example, does not count as intersection:
(2,5), (2,5)
What is a faster way with less checks?
NOT a duplicate ... here is the solution :
def olap(x1,x2,z1,z2):
if x1 == x2 or z1 == z2 : return True
if x1 == z1 and x2 == z2 : return True
if x1 > x2 : x1,x2 = x2,x1
if z1 > z2 : z1,z2 = z2,z1
if x1 > z1 or x2 > z2 : z1,z2,x1,x2 = x1,x2,z1,z2
return x1 < z1 < x2 and (z2 < x1 or z2 > x2)
Simplest check is to verify a property of intersecting intervals: None of the leftmost endpoints (smaller) can be higher than a rightmost endpoint (higher).
x1 x2
a ------------ if x1 > y2: a would be to the right of b
b ---------------- if y1 > x2: b would be to the right of a
y1 y2
def overlap(x1, x2, y1, y2):
""" Overlap means neither of the intervals
is "to the right or left" of the other.
"""
return x1 <= y2 and y1 <= x2
I have n rows and n+1 columns matrix and need to construct such system
For example matrix is
x4 x3 x2 x1 result
1 1 0 1 0
1 0 1 0 1
0 1 0 1 1
1 0 1 1 0
Then equation will be (+ is XOR)
x4+x3+x1=0
x4+x2=1
x3+x1=1
x4+x2+x1=0
I need to return answer as list of x1,.....
How can we do it in python?
You could make use of the Python interface pyz3 of Microsoft's Z3 solver:
from z3 import *
def xor2(a, b):
return Xor(a, b)
def xor3(a, b, c):
return Xor(a, Xor(b, c))
# define Boolean variables
x1 = Bool('x1')
x2 = Bool('x2')
x3 = Bool('x3')
x4 = Bool('x4')
s = Solver()
# every equation is expressed as one constraint
s.add(Not(xor3(x4, x3, x1)))
s.add(xor2(x4, x2))
s.add(xor2(x3, x1))
s.add(Not(xor3(x4, x2, x1)))
# solve and output results
print(s.check())
print(s.model())
Result:
sat
[x3 = False, x2 = False, x1 = True, x4 = True]
Learn Gauss, can be also used for XOR. Then write a gauss python program
I'm trying to solve a linear systems of inequations in Python.
My linear system look something like this :
3 * x1 + 2 * x2 + 4 * x3 > 0
x1 - 4 * x2 - 7 * x3 > 0
I've tried to use NumPy but linalg.solve(a, b) is designed for equations (=) and I have inequations (>).
I've thought about adding variables to my problem to trasnform inequations into equation like :
3 * x1 + 2 * x2 + 4 * x3 - x4 + 0 * x5 = 0
x1 + 4 * x2 + 7 * x3 + 0 * x4 - x5 = 0
x4 and x5 being > 0.
But I don't know how many constraints I'm going to have and I don't know if linalg.solve gives only strictly positive value to variables.
I've also look into SciPy linprog.
I could add an objective function like x1 + x2 + x3, that would'nt be a problem.
But with linprog I have only inequations as <= 0, but I want to exclude the value 0. It would be okay if I could have < 0.
I hope my problem is clear.
I've asked Google for some help, but find nothing. I guess I'm missing something since I can't be the only one with this problem.
Thank you for your help.
I would suggest to introduce a tolerance that defines how close you can come to zero and perhaps iterate on the tolerance value
I.e. Rewrite
3 * x1 + 2 * x2 + 4 * x3 > 0
x1 + 4 * x2 + 7 * x3 > 0
as
3 * x1 + 2 * x2 + 4 * x3 >= t
x1 + 4 * x2 + 7 * x3 >= t
Now, you can use spicy.linprog to solve this.
where t > 0.
Perhaps t >= 0.01 is acceptable as a starting point.
Then iterate on t in [0.01, 0.001, 0.0001, ... ]
At some point, your solution might start changing by less than your precision requirement.
Hope this helps.
I am trying to implement a simple quadratic program using CPLEX's Python API. The sample file qpex1 provided with CPLEX discusses this. The problem, as mentioned in qpex.lp is
Maximize
obj: x1 + 2 x2 + 3 x3 + [ - 33 x1 ^2 + 12 x1 * x2 - 22 x2 ^2 + 23 x2 * x3
- 11 x3 ^2 ] / 2
Subject To
c1: - x1 + x2 + x3 <= 20
c2: x1 - 3 x2 + x3 <= 30
Bounds
0 <= x1 <= 40
End
The problem, while being implemented in python, receives a matrix qmat which implements the quadratic portion of the objective function. The matrix is :
qmat = [[[0, 1], [-33.0, 6.0]],
[[0, 1, 2], [6.0, -22.0, 11.5]],
[[1, 2], [11.5, -11.0]]]
p.objective.set_quadratic(qmat)
Can someone explain the structure of this matrix? What are the parts in the data structure that is being used? What are the components and so on.
First list is the set of indeces, the second list the set of the corresponding values, so the qmat matrix is:
-33 6 0
6 -22 11.5
0 11.5 -11
that results in:
| -33 6 0 | x1
x1 x2 x3 | 6 -22 11.5 | x2 = - 33 x1 ^2 + 12 x1 * x2 - 22 x2 ^2 + 23 x2 * x3 - 11 x3 ^2
| 0 11.5 -11 | x3