How to divide a specific number in a list? - python

A Mersenne Prime follows this formula 2^n-1. I have created a new type of factoring method for numbers which do not produce Mersenne primes. It is very abstract. Its premise is if a specific number is applied using modular math and the new number becomes (zero), it is not a Mersenne Prime Number. I submitted a paper to The Journal of Number Theory online, however it was rejected by the journal. I have attached it if you would like to look it over, I still feel my method is promising yet I'm no coding expert. This is a pdf I sent to the Journal of Number Theory My problem is in my new code I don't know how to divide a number in the list. The list enumerates ok yet I want to subtract z=11 from 253 which equals 242 than mod it by 121, however when I create a range from 1-254 I cannot seem to do this math. The reason I'm interested in this is 253//11=23 which is a factor of 2^11-1. I got this idea from a ratio page.
Type 1:11 and the second number is a 22 just add 1 and its 23.
Check it out
https://goodcalculators.com/ratio-calculator/
The formula will target any number in the range and what I'm looking for is a zero.
Additional details for grismar as per request:
Grismar and others,
What I have found is that Mersenne primes will produce fewer zero's below the number 11 vs. a number like 2^11-1. Also when you output the number by subtraction of z and then mod z*z you may find the number with the lowest factor in it after you divide it by z. The range must be large enough as to find that number, yet if is zero simply divide by z. Then for instance when you find 23 by dividing 11 into 253. You can divide 23 into 2047 and you should get 89. More than likely if you use a different number to check this factor you will get a fraction. So when checking using this method when you find a zero for a number which does not produce a Mersenne Prime number like. Lets pick 29. 536870911 รท 233 = 2304167 so you get a factor number not a fraction.
These are all the factors of 536870911
[1, 233, 1103, 256999, 2089, 486737, 2304167, 536870911]
If you would like even more details leave a comment please.
Programmer in learning looking for help here is my program:
1 should be the start range!
while True:
x = int(input("Use 1 for the start range to make this work correctly:
"))
i = int(input("End Range: "))
z = int(input("square of primes multiplied by a number plus z which
does not make a
mersenne prime, this finds its factor of z: "))
fact = [(i + 1, x) for i, x in enumerate(range(x, i))]
print([((int(i)-z) % (z*z)) if isinstance(i, str) else i for i in fact])


Maybe what you are trying is this, the int call is unnecessary since the values are integers from the start. Also, don't use the same variable i for different purposes:
calculations = [
(index + 1, (fact_tuple[0] - z) % (z*z)) for index, fact_tuple in enumerate(fact)
]
print(calculations) # with x = 1, i = 254, z = 11
>>> [(1, 111), (2, 112), (3, 113), (4, 114), (5, 115), (6, 116), (7, 117), (8, 118), (9, 119), (10, 120), (11, 0), (12, 1), (13, 2), (14, 3), (15, 4), (16, 5), (17, 6), (18, 7), (19, 8), (20, 9), (21, 10), (22, 11), (23, 12), (24, 13), (25, 14), (26, 15), (27, 16), (28, 17), (29, 18), (30, 19), (31, 20), (32, 21), (33, 22), (34, 23), (35, 24), (36, 25), (37, 26), (38, 27), (39, 28), (40, 29), (41, 30), (42, 31), (43, 32), (44, 33), (45, 34), (46, 35), (47, 36), (48, 37), (49, 38), (50, 39), (51, 40), (52, 41), (53, 42), (54, 43), (55, 44), (56, 45), (57, 46), (58, 47), (59, 48), (60, 49), (61, 50), (62, 51), (63, 52), (64, 53), (65, 54), (66, 55), (67, 56), (68, 57), (69, 58), (70, 59), (71, 60), (72, 61), (73, 62), (74, 63), (75, 64), (76, 65), (77, 66), (78, 67), (79, 68), (80, 69), (81, 70), (82, 71), (83, 72), (84, 73), (85, 74), (86, 75), (87, 76), (88, 77), (89, 78), (90, 79), (91, 80), (92, 81), (93, 82), (94, 83), (95, 84), (96, 85), (97, 86), (98, 87), (99, 88), (100, 89), (101, 90), (102, 91), (103, 92), (104, 93), (105, 94), (106, 95), (107, 96), (108, 97), (109, 98), (110, 99), (111, 100), (112, 101), (113, 102), (114, 103), (115, 104), (116, 105), (117, 106), (118, 107), (119, 108), (120, 109), (121, 110), (122, 111), (123, 112), (124, 113), (125, 114), (126, 115), (127, 116), (128, 117), (129, 118), (130, 119), (131, 120), (132, 0), (133, 1), (134, 2), (135, 3), (136, 4), (137, 5), (138, 6), (139, 7), (140, 8), (141, 9), (142, 10), (143, 11), (144, 12), (145, 13), (146, 14), (147, 15), (148, 16), (149, 17), (150, 18), (151, 19), (152, 20), (153, 21), (154, 22), (155, 23), (156, 24), (157, 25), (158, 26), (159, 27), (160, 28), (161, 29), (162, 30), (163, 31), (164, 32), (165, 33), (166, 34), (167, 35), (168, 36), (169, 37), (170, 38), (171, 39), (172, 40), (173, 41), (174, 42), (175, 43), (176, 44), (177, 45), (178, 46), (179, 47), (180, 48), (181, 49), (182, 50), (183, 51), (184, 52), (185, 53), (186, 54), (187, 55), (188, 56), (189, 57), (190, 58), (191, 59), (192, 60), (193, 61), (194, 62), (195, 63), (196, 64), (197, 65), (198, 66), (199, 67), (200, 68), (201, 69), (202, 70), (203, 71), (204, 72), (205, 73), (206, 74), (207, 75), (208, 76), (209, 77), (210, 78), (211, 79), (212, 80), (213, 81), (214, 82), (215, 83), (216, 84), (217, 85), (218, 86), (219, 87), (220, 88), (221, 89), (222, 90), (223, 91), (224, 92), (225, 93), (226, 94), (227, 95), (228, 96), (229, 97), (230, 98), (231, 99), (232, 100), (233, 101), (234, 102), (235, 103), (236, 104), (237, 105), (238, 106), (239, 107), (240, 108), (241, 109), (242, 110), (243, 111), (244, 112), (245, 113), (246, 114), (247, 115), (248, 116), (249, 117), (250, 118), (251, 119), (252, 120), (253, 0)]

Related

How can I manually place networkx nodes using the mouse?

I have a fairly large and messy network of nodes that I wish to display as neatly as possible.
This is how it's currently being displayed:
First, I tried playing with the layout to see if it could generate a good output automatically.
I have tried many different nx layouts, but they all display similar results. I have also tried answers from all of these stack exchange questions:
How to increase node spacing for networkx.spring_layout
Drawing a huge graph with networkX and matplotlib
NetworkX - Stop Nodes from Bunching Up - Tried Scale/K parameters
Fix position of subset of nodes in NetworkX spring graph
Here is the code I am using:
import networkx as nx
import matplotlib.pyplot as plt\
def generate_plot(connections, filename):
G=nx.Graph()
G.add_edges_from(connections)
nx.draw(G)
plt.show()
#plt.savefig(filename)
and here is the data that I am trying to display:
connections = [(0, 36), (0, 113), (2, 11), (2, 12), (2, 26), (2, 27), (2, 28), (2, 29), (2, 32), (2, 33), (2, 34), (2, 35), (2, 82), (3, 41), (4, 41), (5, 3), (6, 3), (11, 7), (11, 10), (11, 13), (11, 42), (12, 10), (12, 164), (26, 10), (26, 100), (27, 10), (27, 164), (28, 10), (28, 92), (29, 56), (29, 58), (29, 79), (29, 91), (29, 99), (29, 100), (29, 101), (29, 102), (30, 59), (30, 83), (30, 99), (31, 55), (31, 56), (31, 57), (31, 74), (31, 91), (31, 96), (31, 100), (31, 113), (31, 134), (31, 164), (32, 10), (33, 10), (34, 10), (34, 164), (35, 10), (35, 91), (35, 100), (36, 64), (36, 74), (36, 82), (36, 91), (36, 107), (36, 99), (38, 41), (39, 40), (39, 41), (39, 59), (40, 41), (40, 47), (40, 91), (40, 99), (41, 3), (41, 39), (41, 40), (43, 68), (43, 69), (45, 50), (46, 51), (46, 69), (46, 99), (47, 49), (47, 91), (47, 107), (47, 100), (47, 101), (47, 113), (48, 76), (50, 68), (50, 69), (51, 68), (51, 76), (51, 114), (52, 46), (52, 47), (52, 65), (53, 42), (53, 107), (53, 99), (53, 100), (53, 101), (53, 113), (54, 76), (55, 74), (55, 96), (55, 99), (56, 99), (56, 100), (56, 109), (57, 29), (57, 64), (57, 91), (57, 96), (57, 107), (57, 100), (58, 91), (58, 99), (58, 100), (58, 101), (58, 102), (59, 30), (59, 46), (59, 47), (59, 61), (59, 83), (59, 99), (59, 100), (59, 101), (60, 3), (60, 12), (60, 26), (60, 27), (60, 29), (60, 30), (60, 31), (60, 35), (60, 36), (60, 40), (60, 42), (60, 44), (60, 49), (60, 55), (60, 56), (60, 57), (60, 58), (60, 59), (60, 61), (60, 64), (60, 74), (60, 75), (60, 79), (60, 81), (60, 82), (60, 83), (60, 86), (60, 90), (60, 91), (60, 92), (60, 93), (60, 94), (60, 96), (60, 107), (60, 99), (60, 100), (60, 101), (60, 102), (60, 111), (60, 112), (60, 113), (60, 116), (60, 126), (60, 129), (60, 134), (60, 135), (60, 136), (60, 140), (60, 144), (60, 150), (60, 152), (60, 162), (60, 164), (60, 179), (60, 195), (61, 91), (61, 107), (61, 100), (61, 101), (61, 113), (61, 135), (62, 79), (62, 102), (63, 59), (64, 29), (64, 36), (64, 91), (64, 99), (64, 100), (64, 101), (64, 140), (65, 12), (65, 27), (65, 29), (65, 30), (65, 31), (65, 34), (65, 35), (65, 43), (65, 45), (65, 50), (65, 56), (65, 57), (65, 58), (65, 59), (65, 60), (65, 61), (65, 66), (65, 67), (65, 72), (65, 75), (65, 81), (65, 82), (65, 91), (65, 92), (65, 96), (65, 107), (65, 99), (65, 100), (65, 101), (65, 102), (65, 104), (65, 109), (65, 129), (65, 140), (65, 142), (65, 156), (65, 164), (67, 68), (67, 69), (67, 195), (68, 60), (69, 51), (69, 53), (69, 68), (69, 195), (70, 12), (70, 26), (70, 29), (70, 31), (70, 35), (70, 47), (70, 56), (70, 58), (70, 59), (70, 61), (70, 66), (70, 75), (70, 79), (70, 82), (70, 83), (70, 93), (70, 96), (70, 107), (70, 99), (70, 100), (70, 101), (70, 126), (70, 129), (70, 135), (70, 136), (70, 162), (70, 164), (71, 12), (71, 26), (71, 29), (71, 31), (71, 35), (71, 47), (71, 56), (71, 58), (71, 59), (71, 61), (71, 66), (71, 75), (71, 79), (71, 83), (71, 96), (71, 107), (71, 99), (71, 100), (71, 101), (71, 126), (71, 135), (71, 162), (71, 164), (72, 29), (72, 35), (72, 36), (72, 46), (72, 47), (72, 55), (72, 57), (72, 58), (72, 60), (72, 61), (72, 64), (72, 66), (72, 70), (72, 71), (72, 74), (72, 75), (72, 76), (72, 79), (72, 86), (72, 90), (72, 91), (72, 93), (72, 94), (72, 95), (72, 96), (72, 107), (72, 99), (72, 100), (72, 101), (72, 102), (72, 111), (72, 112), (72, 116), (72, 126), (72, 129), (72, 144), (73, 83), (74, 96), (74, 100), (75, 100), (76, 29), (76, 31), (76, 42), (76, 44), (76, 49), (76, 55), (76, 56), (76, 59), (76, 61), (76, 66), (76, 74), (76, 78), (76, 83), (76, 91), (76, 93), (76, 96), (76, 107), (76, 109), (76, 113), (76, 114), (76, 116), (76, 134), (76, 140), (77, 30), (77, 44), (77, 49), (77, 61), (77, 74), (77, 78), (77, 83), (77, 96), (77, 109), (77, 140), (78, 61), (78, 91), (78, 92), (79, 44), (79, 99), (80, 42), (80, 64), (80, 65), (80, 75), (80, 83), (80, 134), (80, 135), (80, 136), (80, 144), (80, 155), (81, 35), (81, 91), (81, 100), (82, 10), (83, 84), (83, 85), (83, 86), (83, 107), (83, 100), (84, 194), (84, 195), (87, 36), (87, 88), (87, 142), (87, 144), (88, 59), (88, 83), (88, 134), (88, 135), (88, 136), (88, 144), (88, 158), (88, 162), (89, 61), (89, 135), (89, 141), (90, 36), (91, 96), (91, 113), (93, 96), (93, 107), (93, 134), (93, 135), (94, 74), (94, 96), (95, 61), (95, 134), (95, 135), (95, 162), (96, 35), (96, 74), (96, 91), (96, 99), (96, 100), (96, 101), (98, 12), (98, 26), (98, 27), (98, 29), (98, 30), (98, 31), (98, 34), (98, 35), (98, 41), (98, 56), (98, 57), (98, 58), (98, 59), (98, 60), (98, 61), (98, 70), (98, 75), (98, 79), (98, 81), (98, 82), (98, 83), (98, 84), (98, 87), (98, 91), (98, 92), (98, 95), (98, 96), (98, 107), (98, 99), (98, 100), (98, 101), (98, 106), (98, 134), (98, 135), (98, 142), (98, 147), (98, 152), (98, 159), (99, 91), (99, 113), (99, 164), (100, 91), (100, 101), (100, 113), (100, 164), (101, 57), (101, 91), (101, 113), (101, 164), (102, 101), (103, 44), (103, 61), (103, 140), (104, 56), (104, 90), (104, 101), (104, 102), (104, 104), (104, 129), (104, 140), (105, 83), (105, 135), (106, 2), (106, 29), (106, 30), (106, 31), (106, 36), (106, 41), (106, 48), (106, 56), (106, 57), (106, 58), (106, 59), (106, 60), (106, 61), (106, 62), (106, 63), (106, 65), (106, 66), (106, 68), (106, 70), (106, 71), (106, 72), (106, 74), (106, 75), (106, 76), (106, 77), (106, 78), (106, 79), (106, 80), (106, 81), (106, 82), (106, 83), (106, 84), (106, 85), (106, 87), (106, 89), (106, 90), (106, 91), (106, 92), (106, 93), (106, 94), (106, 95), (106, 96), (106, 107), (106, 99), (106, 100), (106, 101), (106, 102), (106, 103), (106, 104), (106, 105), (106, 108), (106, 110), (106, 111), (106, 112), (106, 116), (106, 119), (106, 123), (106, 124), (106, 125), (106, 126), (106, 127), (106, 128), (106, 129), (106, 130), (106, 131), (106, 134), (106, 135), (106, 136), (106, 137), (106, 138), (106, 139), (106, 140), (106, 141), (106, 142), (106, 144), (106, 146), (106, 147), (106, 148), (106, 149), (106, 151), (106, 152), (106, 153), (106, 154), (106, 157), (106, 158), (106, 159), (106, 160), (106, 161), (106, 162), (106, 164), (106, 194), (108, 42), (108, 61), (108, 76), (108, 109), (108, 110), (108, 114), (109, 42), (109, 91), (109, 113), (109, 164), (110, 76), (111, 114), (112, 114), (114, 0), (114, 26), (114, 29), (114, 30), (114, 31), (114, 35), (114, 36), (114, 40), (114, 42), (114, 44), (114, 47), (114, 49), (114, 53), (114, 55), (114, 56), (114, 57), (114, 59), (114, 61), (114, 64), (114, 74), (114, 75), (114, 81), (114, 82), (114, 86), (114, 91), (114, 93), (114, 96), (114, 107), (114, 99), (114, 100), (114, 101), (114, 102), (114, 109), (114, 129), (114, 134), (114, 135), (114, 140), (114, 164), (114, 179), (115, 10), (115, 31), (115, 164), (116, 114), (119, 150), (120, 157), (121, 137), (122, 137), (123, 163), (124, 117), (124, 125), (124, 130), (124, 150), (124, 163), (125, 126), (126, 117), (126, 135), (127, 134), (128, 135), (129, 36), (129, 91), (130, 131), (131, 134), (131, 135), (132, 157), (133, 137), (134, 36), (134, 74), (134, 109), (134, 135), (135, 36), (135, 74), (135, 136), (135, 156), (136, 36), (137, 134), (138, 139), (139, 135), (140, 107), (140, 134), (140, 136), (141, 118), (141, 135), (141, 144), (142, 36), (142, 61), (142, 64), (142, 83), (142, 118), (142, 129), (142, 134), (142, 135), (142, 136), (142, 140), (142, 141), (142, 143), (142, 144), (142, 150), (142, 152), (142, 156), (142, 158), (142, 162), (142, 164), (142, 195), (143, 135), (144, 36), (144, 134), (144, 135), (144, 136), (144, 162), (145, 134), (146, 134), (146, 135), (146, 155), (147, 29), (147, 44), (147, 46), (147, 47), (147, 118), (147, 134), (147, 135), (147, 136), (147, 140), (147, 144), (147, 156), (147, 158), (147, 164), (148, 136), (148, 150), (149, 134), (149, 135), (149, 136), (149, 141), (150, 134), (150, 164), (151, 150), (152, 134), (152, 135), (152, 156), (152, 162), (153, 134), (153, 135), (153, 162), (154, 150), (155, 134), (156, 135), (157, 134), (158, 136), (158, 140), (159, 135), (160, 134), (160, 135), (160, 155), (162, 36), (162, 61), (162, 74), (162, 96), (162, 134), (162, 135), (162, 144), (167, 171), (168, 83), (169, 99), (170, 83), (170, 85), (171, 2), (172, 195), (173, 78), (174, 78), (175, 60), (175, 66), (175, 70), (175, 142), (176, 94), (176, 120), (176, 121), (176, 122), (176, 123), (176, 132), (176, 133), (176, 137), (176, 138), (176, 145), (176, 157), (177, 12), (177, 26), (177, 27), (177, 28), (177, 29), (177, 30), (177, 31), (177, 33), (177, 34), (177, 35), (177, 36), (177, 44), (177, 55), (177, 56), (177, 57), (177, 58), (177, 59), (177, 61), (177, 62), (177, 63), (177, 64), (177, 74), (177, 75), (177, 78), (177, 79), (177, 81), (177, 82), (177, 86), (177, 89), (177, 90), (177, 91), (177, 93), (177, 94), (177, 96), (177, 107), (177, 99), (177, 100), (177, 101), (177, 102), (177, 109), (177, 113), (177, 118), (177, 128), (177, 129), (177, 134), (177, 135), (177, 136), (177, 139), (177, 140), (177, 141), (177, 144), (177, 148), (177, 149), (177, 150), (177, 151), (177, 152), (177, 153), (177, 154), (177, 159), (177, 162), (178, 35), (180, 37), (182, 31), (183, 31), (184, 185), (185, 29), (185, 30), (185, 31), (185, 44), (185, 79), (185, 166), (185, 187), (185, 190), (187, 185), (188, 164), (188, 187), (188, 190), (189, 108), (190, 185), (192, 193), (195, 164)]
I am hoping to be able to spread out the nodes more and display them in a hierarchical manner.
I would prefer to be able to move the nodes around manually , similar to this animation, but with the nodes staying where they are placed instead of snapping back. However, if you can display them neatly without the need for manual interaction, that would work too!

Here is an MWE using netgraph on a networkx graph object.
import numpy as np
import matplotlib.pyplot as plt; plt.ion()
import networkx
import netgraph # pip install netgraph
# Construct sparse, directed, weighted graph
total_nodes = 20
weights = np.random.rand(total_nodes, total_nodes)
connection_probability = 0.1
is_connected = np.random.rand(total_nodes, total_nodes) <= connection_probability
graph = np.zeros((total_nodes, total_nodes))
graph[is_connected] = weights[is_connected]
# construct a networkx graph
g = networkx.from_numpy_array(graph, networkx.DiGraph)
# decide on a layout
pos = networkx.layout.spring_layout(g)
# Create an interactive plot.
# NOTE: you must retain a reference to the object instance!
# Otherwise the whole thing will be garbage collected after the initial draw
# and you won't be able to move the plot elements around.
plot_instance = netgraph.InteractiveGraph(graph, node_positions=pos)
######## drag nodes around #########
# To access the new node positions:
node_positions = plot_instance.node_positions

How can I place networkx nodes with the most connections closer to the top?

I have a fairly large and messy network of nodes that I wish to display as neatly as possible.
This is how it's currently being displayed:
First, I tried playing with the layout to see if it could generate a good output automatically.
I have tried many different nx layouts, but they all display similar results. I have also tried answers from all of these stack exchange questions:
How to increase node spacing for networkx.spring_layout
Drawing a huge graph with networkX and matplotlib
NetworkX - Stop Nodes from Bunching Up - Tried Scale/K parameters
Fix position of subset of nodes in NetworkX spring graph
Here is the code I am using:
import networkx as nx
import matplotlib.pyplot as plt\
def generate_plot(connections, filename):
G=nx.Graph()
G.add_edges_from(connections)
nx.draw(G)
plt.show()
#plt.savefig(filename)
and here is the data that I am trying to display:
connections = [(0, 36), (0, 113), (2, 11), (2, 12), (2, 26), (2, 27), (2, 28), (2, 29), (2, 32), (2, 33), (2, 34), (2, 35), (2, 82), (3, 41), (4, 41), (5, 3), (6, 3), (11, 7), (11, 10), (11, 13), (11, 42), (12, 10), (12, 164), (26, 10), (26, 100), (27, 10), (27, 164), (28, 10), (28, 92), (29, 56), (29, 58), (29, 79), (29, 91), (29, 99), (29, 100), (29, 101), (29, 102), (30, 59), (30, 83), (30, 99), (31, 55), (31, 56), (31, 57), (31, 74), (31, 91), (31, 96), (31, 100), (31, 113), (31, 134), (31, 164), (32, 10), (33, 10), (34, 10), (34, 164), (35, 10), (35, 91), (35, 100), (36, 64), (36, 74), (36, 82), (36, 91), (36, 107), (36, 99), (38, 41), (39, 40), (39, 41), (39, 59), (40, 41), (40, 47), (40, 91), (40, 99), (41, 3), (41, 39), (41, 40), (43, 68), (43, 69), (45, 50), (46, 51), (46, 69), (46, 99), (47, 49), (47, 91), (47, 107), (47, 100), (47, 101), (47, 113), (48, 76), (50, 68), (50, 69), (51, 68), (51, 76), (51, 114), (52, 46), (52, 47), (52, 65), (53, 42), (53, 107), (53, 99), (53, 100), (53, 101), (53, 113), (54, 76), (55, 74), (55, 96), (55, 99), (56, 99), (56, 100), (56, 109), (57, 29), (57, 64), (57, 91), (57, 96), (57, 107), (57, 100), (58, 91), (58, 99), (58, 100), (58, 101), (58, 102), (59, 30), (59, 46), (59, 47), (59, 61), (59, 83), (59, 99), (59, 100), (59, 101), (60, 3), (60, 12), (60, 26), (60, 27), (60, 29), (60, 30), (60, 31), (60, 35), (60, 36), (60, 40), (60, 42), (60, 44), (60, 49), (60, 55), (60, 56), (60, 57), (60, 58), (60, 59), (60, 61), (60, 64), (60, 74), (60, 75), (60, 79), (60, 81), (60, 82), (60, 83), (60, 86), (60, 90), (60, 91), (60, 92), (60, 93), (60, 94), (60, 96), (60, 107), (60, 99), (60, 100), (60, 101), (60, 102), (60, 111), (60, 112), (60, 113), (60, 116), (60, 126), (60, 129), (60, 134), (60, 135), (60, 136), (60, 140), (60, 144), (60, 150), (60, 152), (60, 162), (60, 164), (60, 179), (60, 195), (61, 91), (61, 107), (61, 100), (61, 101), (61, 113), (61, 135), (62, 79), (62, 102), (63, 59), (64, 29), (64, 36), (64, 91), (64, 99), (64, 100), (64, 101), (64, 140), (65, 12), (65, 27), (65, 29), (65, 30), (65, 31), (65, 34), (65, 35), (65, 43), (65, 45), (65, 50), (65, 56), (65, 57), (65, 58), (65, 59), (65, 60), (65, 61), (65, 66), (65, 67), (65, 72), (65, 75), (65, 81), (65, 82), (65, 91), (65, 92), (65, 96), (65, 107), (65, 99), (65, 100), (65, 101), (65, 102), (65, 104), (65, 109), (65, 129), (65, 140), (65, 142), (65, 156), (65, 164), (67, 68), (67, 69), (67, 195), (68, 60), (69, 51), (69, 53), (69, 68), (69, 195), (70, 12), (70, 26), (70, 29), (70, 31), (70, 35), (70, 47), (70, 56), (70, 58), (70, 59), (70, 61), (70, 66), (70, 75), (70, 79), (70, 82), (70, 83), (70, 93), (70, 96), (70, 107), (70, 99), (70, 100), (70, 101), (70, 126), (70, 129), (70, 135), (70, 136), (70, 162), (70, 164), (71, 12), (71, 26), (71, 29), (71, 31), (71, 35), (71, 47), (71, 56), (71, 58), (71, 59), (71, 61), (71, 66), (71, 75), (71, 79), (71, 83), (71, 96), (71, 107), (71, 99), (71, 100), (71, 101), (71, 126), (71, 135), (71, 162), (71, 164), (72, 29), (72, 35), (72, 36), (72, 46), (72, 47), (72, 55), (72, 57), (72, 58), (72, 60), (72, 61), (72, 64), (72, 66), (72, 70), (72, 71), (72, 74), (72, 75), (72, 76), (72, 79), (72, 86), (72, 90), (72, 91), (72, 93), (72, 94), (72, 95), (72, 96), (72, 107), (72, 99), (72, 100), (72, 101), (72, 102), (72, 111), (72, 112), (72, 116), (72, 126), (72, 129), (72, 144), (73, 83), (74, 96), (74, 100), (75, 100), (76, 29), (76, 31), (76, 42), (76, 44), (76, 49), (76, 55), (76, 56), (76, 59), (76, 61), (76, 66), (76, 74), (76, 78), (76, 83), (76, 91), (76, 93), (76, 96), (76, 107), (76, 109), (76, 113), (76, 114), (76, 116), (76, 134), (76, 140), (77, 30), (77, 44), (77, 49), (77, 61), (77, 74), (77, 78), (77, 83), (77, 96), (77, 109), (77, 140), (78, 61), (78, 91), (78, 92), (79, 44), (79, 99), (80, 42), (80, 64), (80, 65), (80, 75), (80, 83), (80, 134), (80, 135), (80, 136), (80, 144), (80, 155), (81, 35), (81, 91), (81, 100), (82, 10), (83, 84), (83, 85), (83, 86), (83, 107), (83, 100), (84, 194), (84, 195), (87, 36), (87, 88), (87, 142), (87, 144), (88, 59), (88, 83), (88, 134), (88, 135), (88, 136), (88, 144), (88, 158), (88, 162), (89, 61), (89, 135), (89, 141), (90, 36), (91, 96), (91, 113), (93, 96), (93, 107), (93, 134), (93, 135), (94, 74), (94, 96), (95, 61), (95, 134), (95, 135), (95, 162), (96, 35), (96, 74), (96, 91), (96, 99), (96, 100), (96, 101), (98, 12), (98, 26), (98, 27), (98, 29), (98, 30), (98, 31), (98, 34), (98, 35), (98, 41), (98, 56), (98, 57), (98, 58), (98, 59), (98, 60), (98, 61), (98, 70), (98, 75), (98, 79), (98, 81), (98, 82), (98, 83), (98, 84), (98, 87), (98, 91), (98, 92), (98, 95), (98, 96), (98, 107), (98, 99), (98, 100), (98, 101), (98, 106), (98, 134), (98, 135), (98, 142), (98, 147), (98, 152), (98, 159), (99, 91), (99, 113), (99, 164), (100, 91), (100, 101), (100, 113), (100, 164), (101, 57), (101, 91), (101, 113), (101, 164), (102, 101), (103, 44), (103, 61), (103, 140), (104, 56), (104, 90), (104, 101), (104, 102), (104, 104), (104, 129), (104, 140), (105, 83), (105, 135), (106, 2), (106, 29), (106, 30), (106, 31), (106, 36), (106, 41), (106, 48), (106, 56), (106, 57), (106, 58), (106, 59), (106, 60), (106, 61), (106, 62), (106, 63), (106, 65), (106, 66), (106, 68), (106, 70), (106, 71), (106, 72), (106, 74), (106, 75), (106, 76), (106, 77), (106, 78), (106, 79), (106, 80), (106, 81), (106, 82), (106, 83), (106, 84), (106, 85), (106, 87), (106, 89), (106, 90), (106, 91), (106, 92), (106, 93), (106, 94), (106, 95), (106, 96), (106, 107), (106, 99), (106, 100), (106, 101), (106, 102), (106, 103), (106, 104), (106, 105), (106, 108), (106, 110), (106, 111), (106, 112), (106, 116), (106, 119), (106, 123), (106, 124), (106, 125), (106, 126), (106, 127), (106, 128), (106, 129), (106, 130), (106, 131), (106, 134), (106, 135), (106, 136), (106, 137), (106, 138), (106, 139), (106, 140), (106, 141), (106, 142), (106, 144), (106, 146), (106, 147), (106, 148), (106, 149), (106, 151), (106, 152), (106, 153), (106, 154), (106, 157), (106, 158), (106, 159), (106, 160), (106, 161), (106, 162), (106, 164), (106, 194), (108, 42), (108, 61), (108, 76), (108, 109), (108, 110), (108, 114), (109, 42), (109, 91), (109, 113), (109, 164), (110, 76), (111, 114), (112, 114), (114, 0), (114, 26), (114, 29), (114, 30), (114, 31), (114, 35), (114, 36), (114, 40), (114, 42), (114, 44), (114, 47), (114, 49), (114, 53), (114, 55), (114, 56), (114, 57), (114, 59), (114, 61), (114, 64), (114, 74), (114, 75), (114, 81), (114, 82), (114, 86), (114, 91), (114, 93), (114, 96), (114, 107), (114, 99), (114, 100), (114, 101), (114, 102), (114, 109), (114, 129), (114, 134), (114, 135), (114, 140), (114, 164), (114, 179), (115, 10), (115, 31), (115, 164), (116, 114), (119, 150), (120, 157), (121, 137), (122, 137), (123, 163), (124, 117), (124, 125), (124, 130), (124, 150), (124, 163), (125, 126), (126, 117), (126, 135), (127, 134), (128, 135), (129, 36), (129, 91), (130, 131), (131, 134), (131, 135), (132, 157), (133, 137), (134, 36), (134, 74), (134, 109), (134, 135), (135, 36), (135, 74), (135, 136), (135, 156), (136, 36), (137, 134), (138, 139), (139, 135), (140, 107), (140, 134), (140, 136), (141, 118), (141, 135), (141, 144), (142, 36), (142, 61), (142, 64), (142, 83), (142, 118), (142, 129), (142, 134), (142, 135), (142, 136), (142, 140), (142, 141), (142, 143), (142, 144), (142, 150), (142, 152), (142, 156), (142, 158), (142, 162), (142, 164), (142, 195), (143, 135), (144, 36), (144, 134), (144, 135), (144, 136), (144, 162), (145, 134), (146, 134), (146, 135), (146, 155), (147, 29), (147, 44), (147, 46), (147, 47), (147, 118), (147, 134), (147, 135), (147, 136), (147, 140), (147, 144), (147, 156), (147, 158), (147, 164), (148, 136), (148, 150), (149, 134), (149, 135), (149, 136), (149, 141), (150, 134), (150, 164), (151, 150), (152, 134), (152, 135), (152, 156), (152, 162), (153, 134), (153, 135), (153, 162), (154, 150), (155, 134), (156, 135), (157, 134), (158, 136), (158, 140), (159, 135), (160, 134), (160, 135), (160, 155), (162, 36), (162, 61), (162, 74), (162, 96), (162, 134), (162, 135), (162, 144), (167, 171), (168, 83), (169, 99), (170, 83), (170, 85), (171, 2), (172, 195), (173, 78), (174, 78), (175, 60), (175, 66), (175, 70), (175, 142), (176, 94), (176, 120), (176, 121), (176, 122), (176, 123), (176, 132), (176, 133), (176, 137), (176, 138), (176, 145), (176, 157), (177, 12), (177, 26), (177, 27), (177, 28), (177, 29), (177, 30), (177, 31), (177, 33), (177, 34), (177, 35), (177, 36), (177, 44), (177, 55), (177, 56), (177, 57), (177, 58), (177, 59), (177, 61), (177, 62), (177, 63), (177, 64), (177, 74), (177, 75), (177, 78), (177, 79), (177, 81), (177, 82), (177, 86), (177, 89), (177, 90), (177, 91), (177, 93), (177, 94), (177, 96), (177, 107), (177, 99), (177, 100), (177, 101), (177, 102), (177, 109), (177, 113), (177, 118), (177, 128), (177, 129), (177, 134), (177, 135), (177, 136), (177, 139), (177, 140), (177, 141), (177, 144), (177, 148), (177, 149), (177, 150), (177, 151), (177, 152), (177, 153), (177, 154), (177, 159), (177, 162), (178, 35), (180, 37), (182, 31), (183, 31), (184, 185), (185, 29), (185, 30), (185, 31), (185, 44), (185, 79), (185, 166), (185, 187), (185, 190), (187, 185), (188, 164), (188, 187), (188, 190), (189, 108), (190, 185), (192, 193), (195, 164)]
I am hoping to be able to spread out the nodes more and display them in a hierarchical manner, with the nodes that have the most connections closer to top, and the ones with few connections at the bottom.

This is a bit roundabout, but maybe useful:
The idea is to use the degree (i.e., its number of edges) of the node as y-coordinate, and then assign x-coords based on frequency of the given degree:
import networkx as nx
import matplotlib.pyplot as plt
import numpy as np
G=nx.Graph()
G.add_edges_from(connections)
# check how many times a given degree occurs:
degrees = [G.degree[a] for a in G.nodes]
# generate unique x-coordinates. divide by 2 for zero-centering:
degrees = {degree: [a for a in degrees.count(degree)/2. - np.arange(degrees.count(degree))] for degree in set(degrees)}
# build positioning dicitonary:
positions = {a : (degrees[G.degree[a]].pop(), G.degree[a]) for a in G.nodes}
Fixed distances in Y-direction, and slightly less compressed code:
degrees = [G.degree[a] for a in G.nodes]
degrees_unique = sorted(list(set(degrees)))
y_positions = {degrees_unique[i] : i for i in range(len(degrees_unique))}
x_positions = {}
for degree in degrees_unique:
x_positions[degree] = [a for a in degrees.count(degree) / 2. - np.arange(degrees.count(degree))]
positions = {}
for node in G.nodes:
deg = G.degree[node]
positions[node] = (x_positions[deg].pop(), y_positions[deg])
nx.draw(G, pos=positions, node_size=10)
alternatively, a mirror variable can be included in the last loop, s.t. nodes are plotted above and below the x-axis, every other iteration, somewhat decluttering everything:
mirror = 1
for node in G.nodes:
deg = G.degree[node]
positions[node] = (x_positions[deg].pop(), mirror*y_positions[deg])
mirror *= -1
nx.draw(G, pos=positions, node_size=10)

How to determine two 2 Dimensional lists are exactl same?

This is a part of a large program. I have a list like
cnfn=[(1, -3), (2, -3), (-1, -2, 3), (-1, 4), (-2, 4), (1, 2, -4), (-4, -5), (4, 5), (-3, 6), (-5, 6), (3, 5, -6), (7, -8), (6, -8), (-7, -6, 8), (-6, 9), (-7, 9), (6, 7, -9), (-9, -10), (9, 10), (-8, 11), (-10, 11), (8, 10, -11), (7, -12), (4, -12), (-7, -4, 12), (-12, 13), (-3, 13), (12, 3, -13), (14, -16), (15, -16), (-14, -15, 16), (-16, -17), (16, 17), (-14, 18), (-15, 18), (14, 15, -18), (17, -19), (18, -19), (-17, -18, 19), (13, -20), (19, -20), (-13, -19, 20), (-20, -21), (20, 21), (-19, 22), (-13, 22), (19, 13, -22), (21, -23), (22, -23), (-21, -22, 23), (13, -24), (18, -24), (-13, -18, 24), (-24, 25), (-16, 25), (24, 16, -25), (26, -28), (27, -28), (-26, -27, 28), (-28, -29), (28, 29), (-26, 30), (-27, 30), (26, 27, -30), (29, -31), (30, -31), (-29, -30, 31), (25, -32), (31, -32), (-25, -31, 32), (-32, -33), (32, 33), (-31, 34), (-25, 34), (31, 25, -34), (33, -35), (34, -35), (-33, -34, 35), (25, -36), (30, -36), (-25, -30, 36), (-36, 37), (-28, 37), (36, 28, -37), (38, -40), (39, -40), (-38, -39, 40), (-40, -41), (40, 41), (-38, 42), (-39, 42), (38, 39, -42), (41, -43), (42, -43), (-41, -42, 43), (37, -44), (43, -44), (-37, -43, 44), (-44, -45), (44, 45), (-43, 46), (-37, 46), (43, 37, -46), (45, -47), (46, -47), (-45, -46, 47), (37, -48), (42, -48), (-37, -42, 48), (-48, 49), (-40, 49), (48, 40, -49), (-50, -51), (50, 51), (-51, 53), (-52, 53), (51, 52, -53), (-52, -54), (52, 54), (-54, 55), (-50, 55), (54, 50, -55), (53, -56), (55, -56), (-53, -55, 56), (-56, -57), (56, 57), (58, -59), (57, -59), (-58, -57, 59), (52, -60), (50, -60), (-52, -50, 60), (-59, 61), (-60, 61), (59, 60, -61), (56, -62), (58, -62), (-56, -58, 62), (-58, -63), (58, 63), (57, -64), (63, -64), (-57, -63, 64), (-62, 65), (-64, 65), (62, 64, -65), (-66, -67), (66, 67), (-67, 69), (-68, 69), (67, 68, -69), (-68, -70), (68, 70), (-70, 71), (-66, 71), (70, 66, -71), (69, -72), (71, -72), (-69, -71, 72), (-72, -73), (72, 73), (61, -74), (73, -74), (-61, -73, 74), (68, -75), (66, -75), (-68, -66, 75), (-74, 76), (-75, 76), (74, 75, -76), (72, -77), (61, -77), (-72, -61, 77), (-61, -78), (61, 78), (73, -79), (78, -79), (-73, -78, 79), (-77, 80), (-79, 80), (77, 79, -80), (-81, -82), (81, 82), (-82, 84), (-83, 84), (82, 83, -84), (-83, -85), (83, 85), (-85, 86), (-81, 86), (85, 81, -86), (84, -87), (86, -87), (-84, -86, 87), (-87, -88), (87, 88), (76, -89), (88, -89), (-76, -88, 89), (83, -90), (81, -90), (-83, -81, 90), (-89, 91), (-90, 91), (89, 90, -91), (87, -92), (76, -92), (-87, -76, 92), (-76, -93), (76, 93), (88, -94), (93, -94), (-88, -93, 94), (-92, 95), (-94, 95), (92, 94, -95), (-96, -97), (96, 97), (-97, 99), (-98, 99), (97, 98, -99), (-98, -100), (98, 100), (-100, 101), (-96, 101), (100, 96, -101), (99, -102), (101, -102), (-99, -101, 102), (-102, -103), (102, 103), (91, -104), (103, -104), (-91, -103, 104), (-104, -105), (104, 105), (-104, 106), (-105, 106), (104, 105, -106), (102, -107), (91, -107), (-102, -91, 107), (-91, -108), (91, 108), (103, -109), (108, -109), (-103, -108, 109), (-107, 110), (-109, 110), (107, 109, -110), (-1, 50), (1, -50), (-2, 52), (2, -52), (-7, 58), (7, -58), (-14, 66), (14, -66), (-15, 68), (15, -68), (-26, 81), (26, -81), (-27, 83), (27, -83), (-38, 96), (38, -96), (-39, 98), (39, -98), (-11, -65, -111), (-11, 65, 111), (11, -65, 111), (11, 65, -111), (-23, -80, -112), (-23, 80, 112), (23, -80, 112), (23, 80, -112), (-35, -95, -113), (-35, 95, 113), (35, -95, 113), (35, 95, -113), (-47, -106, -114), (-47, 106, 114), (47, -106, 114), (47, 106, -114), (-49, -110, -115), (-49, 110, 115), (49, -110, 115), (49, 110, -115), (111, 112, 113, 114, 115)]
And there is another list
cnfb=[(1, -3), (2, -3), (-1, -2, 3), (-1, 4), (-2, 4), (1, 2, -4), (4, 5), (-4, -5), (-3, 6), (-5, 6), (3, 5, -6), (7, -8), (6, -8), (-7, -6, 8), (-6, 9), (-7, 9), (6, 7, -9), (9, 10), (-9, -10), (-8, 11), (-10, 11), (8, 10, -11), (7, -12), (4, -12), (-7, -4, 12), (-12, 13), (-3, 13), (12, 3, -13), (14, -16), (15, -16), (-14, -15, 16), (16, 17), (-16, -17), (-14, 18), (-15, 18), (14, 15, -18), (17, -19), (18, -19), (-17, -18, 19), (13, -20), (19, -20), (-13, -19, 20), (20, 21), (-20, -21), (-19, 22), (-13, 22), (19, 13, -22), (21, -23), (22, -23), (-21, -22, 23), (13, -24), (18, -24), (-13, -18, 24), (-24, 25), (-16, 25), (24, 16, -25), (26, -28), (27, -28), (-26, -27, 28), (28, 29), (-28, -29), (-26, 30), (-27, 30), (26, 27, -30), (29, -31), (30, -31), (-29, -30, 31), (25, -32), (31, -32), (-25, -31, 32), (32, 33), (-32, -33), (-31, 34), (-25, 34), (31, 25, -34), (33, -35), (34, -35), (-33, -34, 35), (25, -36), (30, -36), (-25, -30, 36), (-36, 37), (-28, 37), (36, 28, -37), (38, -40), (39, -40), (-38, -39, 40), (40, 41), (-40, -41), (-38, 42), (-39, 42), (38, 39, -42), (41, -43), (42, -43), (-41, -42, 43), (37, -44), (43, -44), (-37, -43, 44), (44, 45), (-44, -45), (-43, 46), (-37, 46), (43, 37, -46), (45, -47), (46, -47), (-45, -46, 47), (37, -48), (42, -48), (-37, -42, 48), (-48, 49), (-40, 49), (48, 40, -49), (50, 51), (-50, -51), (-51, 53), (-52, 53), (51, 52, -53), (52, 54), (-52, -54), (-54, 55), (-50, 55), (54, 50, -55), (53, -56), (55, -56), (-53, -55, 56), (56, 57), (-56, -57), (58, -59), (57, -59), (-58, -57, 59), (52, -60), (50, -60), (-52, -50, 60), (-59, 61), (-60, 61), (59, 60, -61), (56, -62), (58, -62), (-56, -58, 62), (58, 63), (-58, -63), (57, -64), (63, -64), (-57, -63, 64), (-62, 65), (-64, 65), (62, 64, -65), (66, 67), (-66, -67), (-67, 69), (-68, 69), (67, 68, -69), (68, 70), (-68, -70), (-70, 71), (-66, 71), (70, 66, -71), (69, -72), (71, -72), (-69, -71, 72), (72, 73), (-72, -73), (61, -74), (73, -74), (-61, -73, 74), (68, -75), (66, -75), (-68, -66, 75), (-74, 76), (-75, 76), (74, 75, -76), (72, -77), (61, -77), (-72, -61, 77), (61, 78), (-61, -78), (73, -79), (78, -79), (-73, -78, 79), (-77, 80), (-79, 80), (77, 79, -80), (81, 82), (-81, -82), (-82, 84), (-83, 84), (82, 83, -84), (83, 85), (-83, -85), (-85, 86), (-81, 86), (85, 81, -86), (84, -87), (86, -87), (-84, -86, 87), (87, 88), (-87, -88), (76, -89), (88, -89), (-76, -88, 89), (83, -90), (81, -90), (-83, -81, 90), (-89, 91), (-90, 91), (89, 90, -91), (87, -92), (76, -92), (-87, -76, 92), (76, 93), (-76, -93), (88, -94), (93, -94), (-88, -93, 94), (-92, 95), (-94, 95), (92, 94, -95), (96, 97), (-96, -97), (-97, 99), (-98, 99), (97, 98, -99), (98, 100), (-98, -100), (-100, 101), (-96, 101), (100, 96, -101), (99, -102), (101, -102), (-99, -101, 102), (102, 103), (-102, -103), (91, -104), (103, -104), (-91, -103, 104), (104, 105), (-104, -105), (-104, 106), (-105, 106), (104, 105, -106), (102, -107), (91, -107), (-102, -91, 107), (91, 108), (-91, -108), (103, -109), (108, -109), (-103, -108, 109), (-107, 110), (-109, 110), (107, 109, -110), (35, 95, -111), (-35, -95, -111), (-35, 95, 111), (35, -95, 111), (23, 80, -112), (-23, -80, -112), (-23, 80, 112), (23, -80, 112), (49, 106, -113), (-49, -106, -113), (-49, 106, 113), (49, -106, 113), (47, 110, -114), (-47, -110, -114), (-47, 110, 114), (47, -110, 114), (11, 65, -115), (-11, -65, -115), (-11, 65, 115), (11, -65, 115), [111, 112, 113, 114, 115], (-26, 83), (26, -83), (-2, 50), (2, -50), (-38, 98), (38, -98), (-27, 81), (27, -81), (-39, 96), (39, -96), (-7, 58), (7, -58), (-14, 68), (14, -68), (-15, 66), (15, -66), (-1, 52), (1, -52)]
If I check with plane eye the look like having same values but if I put them in the same function the result is different. How can I determine those two have exactly same type and same value?

The two lists are NOT the same. That is why a function may be giving you a different result for the different lists.
To check if 2 lists are identical, you can do:
list1 == list2
So to give some examples:
>>> [1, 2, 3, 4, 5] == [1, 2, 3, 4, 5]
True
>>> [1, 2, 3, 4, 5] == [1, 2, 3, 4, 3]
False
>>> [1, 2, 3, 4, 5] == [5, 4, 3, 2, 1]
False
>>> [(1, 2), (3, 4)] == [(1, 2), (3, 4)]
True
>>> [(1, 2), (3, 4)] == [(1, 2), (3, 5)]
False
If you want to find what the differences are, you can do the following:
[e for e in list1 if e not in list2] + [e for e in list2 if e not in list1]
which I think is actually very readable for what it is.
So we could put that inside a function:
def comp(list1, list2):
return [e for e in list1 if e not in list2] + [e for e in list2 if e not in list1]
and some examples:
>>> comp([1, 2, 3], [1, 2, 3]) #should be empty as no differnence
[]
>>> comp([(1, 2), (3, 4)], [(1, 2), (3, 5)])
[(3, 4), (3, 5)]
>>> comp([(1, 2), (3, 4)], [(1, 2), (3, 5), (6, 7)])
[(3, 4), (3, 5), (6, 7)]

partitionBy assigns partitions, but WHERE in each partition

With hash function:
balanceLoad = lambda x: bisect.bisect_left(boundary_array, -keyfunc(x))
Where boundary_array is [-64, -10, 35]
The folowing tells me which partition to assign each element to
rdd.partitionBy(numPartitions, balanceLoad)
However, is there a way to determine /control WHERE in each partition they are assigned / placed? {1,2,3} vs {3,2,1}.
For example when I do this:
rdd = CleanRDD(sc.parallelize(range(100), 4).map(lambda x: (x *((-1) ** x) , x)))
sortByKey(rdd, keyfunc=lambda key: key, ascending=False).collect()
Elements in each partition are in reverse order:
[(64, 64),
(66, 66),
(68, 68),
(70, 70),
(72, 72),
(74, 74),
(76, 76),
(78, 78),
(80, 80),
(82, 82),
(84, 84),
(86, 86),
(88, 88),
(90, 90),
(92, 92),
(94, 94),
(96, 96),
(98, 98),
(10, 10),
(12, 12),
(14, 14),
(16, 16),
(18, 18),
(20, 20),
(22, 22),
(24, 24),
(26, 26),
(28, 28),
(30, 30),
(32, 32),
(34, 34),
(36, 36),
(38, 38),
(40, 40),
(42, 42),
(44, 44),
(46, 46),
(48, 48),
(50, 50),
(52, 52),
(54, 54),
(56, 56),
(58, 58),
(60, 60),
(62, 62),
(-35, 35),
(-33, 33),
(-31, 31),
(-29, 29),
(-27, 27),
(-25, 25),
(-23, 23),
(-21, 21),
(-19, 19),
(-17, 17),
(-15, 15),
(-13, 13),
(-11, 11),
(-9, 9),
(-7, 7),
(-5, 5),
(-3, 3),
(-1, 1),
(0, 0),
(2, 2),
(4, 4),
(6, 6),
(8, 8),
(-99, 99),
(-97, 97),
(-95, 95),
(-93, 93),
(-91, 91),
(-89, 89),
(-87, 87),
(-85, 85),
(-83, 83),
(-81, 81),
(-79, 79),
(-77, 77),
(-75, 75),
(-73, 73),
(-71, 71),
(-69, 69),
(-67, 67),
(-65, 65),
(-63, 63),
(-61, 61),
(-59, 59),
(-57, 57),
(-55, 55),
(-53, 53),
(-51, 51),
(-49, 49),
(-47, 47),
(-45, 45),
(-43, 43),
(-41, 41),
(-39, 39),
(-37, 37)]
Notice that elements in each of the three groups are in reverse order.
How can I correct this?

Determine no, because an order of the shuffle is nondeterministic.
You can control the order but not as a part of the partitioning process or at least not in PySpark. Instead you can take a similar approach like sortByKey and enforce the order per partition afterwards:
def applyOrdering(iter):
"""Takes an itertools.chain object
and returns iterable with specific ordering"""
...
rdd.partitionBy(numPartitions, balanceLoad).mapPartitions(applyOrdering)
Note that iter may be to large fit into memory so you should either increase granularity or use sorting mechanism which doesn't require reading all data at once.

Append values of two strings into pairs

I start with two numpy arrays, the "x values" and the "y values":
import numpy as np
x = np.arange(100)
y = np.arange(100)
The output is
[ 0 1 2 3 4 ..... 96 97 98 99]
[ 0 1 2 3 4 ..... 96 97 98 99]
I would like to append these values together into an array of len() = 100 such that the output is
[ (0,0) (1,1) (2,2) (3,3) .... (98,98) (99,99) ]
How does one use indexing to both (A) put the pairs in the correct order and (B) put the paratheses ( and comma , in the correct order?

For your particular requirement, you can use the built-in zip function, which combines multiple lists at their corresponding indexes (that is ith index of all lists that are parameter to it in combined in the returned iterator).
Example -
import numpy as np
x = np.arange(100)
y = np.arange(100)
print(list(zip(x,y)))
>>> [(0, 0), (1, 1), (2, 2), (3, 3), (4, 4), (5, 5), (6, 6), (7, 7), (8, 8), (9, 9), (10, 10), (11, 11), (12, 12), (13, 13), (14, 14), (15, 15), (16, 16), (17, 17), (18, 18), (19, 19), (20, 20), (21, 21), (22, 22), (23, 23), (24, 24), (25, 25), (26, 26), (27, 27), (28, 28), (29, 29), (30, 30), (31, 31), (32, 32), (33, 33), (34, 34), (35, 35), (36, 36), (37, 37), (38, 38), (39, 39), (40, 40), (41, 41), (42, 42), (43, 43), (44, 44), (45, 45), (46, 46), (47, 47), (48, 48), (49, 49), (50, 50), (51, 51), (52, 52), (53, 53), (54, 54), (55, 55), (56, 56), (57, 57), (58, 58), (59, 59), (60, 60), (61, 61), (62, 62), (63, 63), (64, 64), (65, 65), (66, 66), (67, 67), (68, 68), (69, 69), (70, 70), (71, 71), (72, 72), (73, 73), (74, 74), (75, 75), (76, 76), (77, 77), (78, 78), (79, 79), (80, 80), (81, 81), (82, 82), (83, 83), (84, 84), (85, 85), (86, 86), (87, 87), (88, 88), (89, 89), (90, 90), (91, 91), (92, 92), (93, 93), (94, 94), (95, 95), (96, 96), (97, 97), (98, 98), (99, 99)]
For Python 2.x , please note you do not need list(zip(...)) , since zip itself would return a list , but for Python 3.x , zip returns an iterator, and to print it we would need to convert it into a list.

You can use np.dstack to get the columns :
>>> np.dstack((x,y))
array([[[ 0, 0],
[ 1, 1],
[ 2, 2],
[ 3, 3],
[ 4, 4],
[ 5, 5],
[ 6, 6],
[ 7, 7],
[ 8, 8],
[ 9, 9],
...
[99, 99]]])
And if you want to get tuple instead of list you can use map to convert it to tuple:
>>> map(tuple,np.dstack((x,y))[0])
[(0, 0), (1, 1), (2, 2), (3, 3), (4, 4), (5, 5), (6, 6), (7, 7), (8, 8), (9, 9), (10, 10), (11, 11), (12, 12), (13, 13), (14, 14), (15, 15), (16, 16), (17, 17), (18, 18), (19, 19), (20, 20), (21, 21), (22, 22), (23, 23), (24, 24), (25, 25), (26, 26), (27, 27), (28, 28), (29, 29), (30, 30), (31, 31), (32, 32), (33, 33), (34, 34), (35, 35), (36, 36), (37, 37), (38, 38), (39, 39), (40, 40), (41, 41), (42, 42), (43, 43), (44, 44), (45, 45), (46, 46), (47, 47), (48, 48), (49, 49), (50, 50), (51, 51), (52, 52), (53, 53), (54, 54), (55, 55), (56, 56), (57, 57), (58, 58), (59, 59), (60, 60), (61, 61), (62, 62), (63, 63), (64, 64), (65, 65), (66, 66), (67, 67), (68, 68), (69, 69), (70, 70), (71, 71), (72, 72), (73, 73), (74, 74), (75, 75), (76, 76), (77, 77), (78, 78), (79, 79), (80, 80), (81, 81), (82, 82), (83, 83), (84, 84), (85, 85), (86, 86), (87, 87), (88, 88), (89, 89), (90, 90), (91, 91), (92, 92), (93, 93), (94, 94), (95, 95), (96, 96), (97, 97), (98, 98), (99, 99)]
>>>

You could use vstack
In [36]: xy = np.vstack((x,y)).T
In [37]: xy.shape
Out[37]: (100, 2)
In [38]: xy[0]
Out[38]: array([0, 0])
In [39]: xy[1]
Out[39]: array([1, 1])

Categories