We Keep Coding

Writing and routing over ethernet without IP

I have a project in which I need to setup a network that is essentially a bunch of Raspberry Pis connected through a router over ethernet, and have them talk to each other without using IP.
My challenge here is two folds, first, how can I write raw binary data to ethernet to pass my own custom payloads and have a custom parser on each end picking up and deserialising that data?
Second, and this is secondary for this post, if anyone has any ideas as to how I could use a router without using IP (aka setting up my own simple addressing protocol) this would be very welcomed. So far what I've sketched out is to procure myself a router than can be flashed, then have custom software on there running a custom protocol. However I'm not sure if this is even doable with off the shelf routers. Pointers are welcome.
Ideally I'd like to do all of this in python.

For your first question: asyncio comes as a standard library with Python. It can handle most of your communication needs, essentially acting as your communication stream for your binary data. Example implementation here.
For your second question: you can't go wrong with using IPv4. You could potentially implement something new but you'd probably go down a rabbit hole in doing so.

Sending custom frame / packet in Python

I read many articles and found how to send custom packet based on IP using socket(AF_INET, SOCK_RAW, IPPROTO_RAW). But I want to send completely custom packet starting from Ethernet header. I can't send ARP packet if I can't form Ethernet header cause ARP don't based IP. Please, help!
P.S. I am on Windows 7, not Linux :(

In python, the easiest way is to use the cross-platform scapy library. It’s well known for that
Scapy
You can sniff, send.... lots of packets, add your own protocols, use existing ones... and it works on nearly all platforms. (On windows, it uses Npcap/Winpcap)
You can then build an ARP packet using
from scapy.all import *
pkt = ARP()
pkt.show()
sendp(Ether(dst=..., src=...)/pkt)
Which will create such packets
###[ ARP ]###
hwtype= 0x1
ptype= 0x800
hwlen= 6
plen= 4
op= who-has
hwsrc= 00:50:56:00:1e:3d
psrc= 212.83.148.19
hwdst= 00:00:00:00:00:00
pdst= 0.0.0.0
To build the packet, use the / operator
ether = Ether()
ether.src = “00:00:00:00:00:00”
ether.dst = “ff:ff:ff:ff:ff:ff”
arp = ARP()
[edit arp.psrc, arp.pdst, arp.hwsrc, arp.hwdst]
packet = ether/arp
sendp(packet) # sens packet on layer 2
Have a look at its Scapy documentation

There's no cross-platform way to do what you want, of course.
Python is just passing these values through to the underlying C API. So, on a platform with a complete BSD sockets API including the packet interface, you can just use AF_PACKET and the other appropriate flags. (I think you'd want ETH_P_ALL or ETH_P_802_2 rather than IPPROTO_RAW, or you might want SOCK_DGRAM… anyway, read your platform's man packet and figure it out based on what you actually need to do.) On Linux, at least most of these flags should be available on the SOCKET module; on other Unixes, they often don't get picked up, so you have to manually look them up in the system headers and use hardcoded constant ints in your code.
Unfortunately, if you're on Windows, this doesn't do any good. While WinSock has a feature they call TCP/IP Raw Sockets, accessed via SOCK_RAW, and recent versions of Python do expose this, it's just an emulation of a small subset of what actual BSD sockets implementations can do, and doesn't offer any way to go below the IP level (hence the name of the feature).
Microsoft's solution to this used to be that you'd write a TDI provider with the DDK, which would implement whatever protocol you wanted to expose as another WinSock protocol, and then your application-level code could just use that protocol the same way it would use, e.g., TCP. From the linked document above, it looks like this is obsolete, but the replacement seems like the same idea but with different acronyms (and, presumably, different APIs).
On the other hand, I'm pretty sure Windows already comes with protocols for ARP, ICMP, and anything other protocols needed for its usermode tools (because they obviously can't be written around raw packets). I'm just not sure how to access them.
As far as I know, the usual alternative is to use WinPcap.
While this was originally designed to be a packet capture library, it also implements a complete link-level socket interface that you can use for sending and receiving raw frames.
And there are Python wrappers for it, like WinPcapy.
So, as long as you can require that the WinPcap driver be installed, you can write ARP code, etc., on Windows in Python. It's just different from doing it on Unix.
In fact, one of the examples on the front page of WinPcapY, "Easy Packet sending", should get you started:
from winpcapy import WinPcapUtils
# Build a packet buffer
# This example-code is built for tutorial purposes, for actual packet crafting use modules like dpkt
arp_request_hex_template = "%(dst_mac)s%(src_mac)s08060001080006040001" \
"%(sender_mac)s%(sender_ip)s%(target_mac)s%(target_ip)s" + "00" * 18
packet = arp_request_hex_template % {
"dst_mac": "aa"*6,
"src_mac": "bb"*6,
"sender_mac": "bb"*6,
"target_mac": "cc"*6,
# 192.168.0.1
"sender_ip": "c0a80001",
# 192.168.0.2
"target_ip": "c0a80002"
}
# Send the packet (ethernet frame with an arp request) on the interface
WinPcapUtils.send_packet("*Ethernet*", packet.decode("hex"))

only accept certain ip/mac/ethtype packets in a socket

I;m a completely new in network programming and a starter in python.
I want so set a socket in Python to just accept certain packages.
I have the following values at my disposal:
destination address and port
source MAC address
ethtype (own)
how can I set a filter to my socket class to only accept packets for me (at dest address,port)
and/or the right ethtype, and/or send by the device with the known mac address ?
Any help would be greatly appreciated, I tried to look via Google, but the amount of hits is incredible, and I haven't been able to find an answer that solves my question so far.
Thanx,
Arthur

I would recommend you to use scapy. It is a great tool for crafting custom packets and to do lot of other stuff.
You can add filters in the scapy's sniff() to capture the packets you desire. You can also use scapy with your own python programm.
More over you'll find ton of tutorials on the internet on how to use scapy.

Is it possible to Inject packets into an existing tcp connection using Python?

As the title says, I would like to send data using an existing tcp connection. Said connection has already been established by a 3rd party program. I haven't been able to find much information about this, and it's safe to say I don't know how this will work at all.
The operating system is Windows. My preferred programming language is python - I'd prefer not to use 3rd party python modules, but I will if they make my life easier.
Just to clarify, in case you aren't sure what I want to do: I want to send data as if it were sent by a different program; pretty much like WPE pro's send function does.
Update:
Technically, couldn't I manually design the TCP packet and then tell the network device (or operating system) to send that packet? Wouldn't that be exactly the same thing an injected socket would do?
Edit: Wikipedia says the receiving host acknowledges packets it receives, which makes this a bit more difficult. But if can drop that acknowledge-packet before the 3rd party program receives it, then this should work. Right?

Scapy/Pcapy are pretty powerful tools for monitoring and injecting packets into a live network interface. I've used them for several projects. These tools are ideal for stimulus/response low-level network protocols (ie DHCP, DNS, etc) and anything non-stateful sent over simple UDP.
Unfortunately, the TCP layer is very complicated and stateful. So injecting something that makes sense into the stream will be more difficult. Moreover, Scapy/Pcapy do not currently have support for tcp.

A TCP session is not intended to be a many-to-one connection. Its a point-to-point stateful protocol which keeps track of packets that have been sent versus those that have been received by the other end. I don't believe you can inject yourself into an already-established session. Your best bet, as was pointed out previously, is to create a proxy and act as a man-in-the-middle interloper. Still not a trivial thing but doable.

Stop packets at the network card

This is the problem I'm trying to solve,
I want to write an application that will read outbound http request packets on the same machine's network card. This would then be able to extract the GET url from it.On basis of this information, I want to be able to stop the packet, or redirect it , or let it pass.
However I want my application to be running in promiscuous mode (like wireshark does), and yet be able to eat up (stop) the outbound packet.
I have searched around a bit on this..
libpcap / pcap.h allows to me read packets at the network card, however I haven't yet been able to figure out a way to stop these packets or inject new ones into the network.
Certain stuff like twisted or scapy in python, allows me set up a server that is listening on some local port, I can then configure my browser to connect to it, using proxy configurations. This app can then do the stuff.. but my main purpose of being promiscuous is defeated here..
Any help on how I could achieve this would be greatly appreciated ..

I'd suggest that you approach this at the application layer and use a transparent proxy (e.g. squid) and iptables based interception of outbound port-80 traffic.
The reason I suggest this is that that it will avoid issues with the request being split between packets.
However, if you still want to go ahead with packet interception, you can do it in userspace using netfilters in netlink. I believe there are python wrappers for libnl around.
Essentially you create an iptables rule pointing to "QUEUE" for the traffic you want to intercept and write a program using a netlink library to process the queue, accepting, rejecting and/or modifying packets.

Using pcap you cannot stop the packets, if you are under windows you must go down to the driver level... but you can stop only packets that your machine send.
A solution is act as a pipe to the destination machine: You need two network interfaces (without address possibly), when you get a packet that you does not found interesting on the source network card you simply send it on the destination network card. If the packet is interesting you does not send it, so you act as a filter. I have done it for multimedia performance test (adding jitter, noise, etc.. to video streaming)

You are confusing several things here:
"Promiscuous" usually refers to a mode of a hardware ethernet network card where it delivers all packets in its collision domain up to the kernel network stack and have it sort out delivery (vs. just unicast to given MAC, subscribed multicast, and broadcast in normal operating mode of the card). This is on the receive path.
All the data outbound from your machine will go through (one of) the network cards on the machine, so "promiscuous" does not at all apply here.
You are working on filtering TCP-based protocol (HTTP), but talk in terms of packets. This is wrong. TCP connection is a stream that could be (as far as socket readers and writers are concerned) arbitrarily split into IP datagrams. That URL from HTTP request header could be split across multiple link-layer frames. You would have to stitch them back together and parse the stream anyway. Then you have no chance even at that if SSL is in use.
If you are interested in HTTP filtering then read HTTP RFCs, and read existing open-source code, e.g. squid, nginx, etc.
If you are digging through network stack for better understaning then read W. Richard Stevens books, look into existing code in open-source operating systems, check out BPF and netlink.
Hope this clears it a little.

I have implemented this module in Windows by using two separate NICs and using a socket/pipe(whatever you like) between them in this thread