I'm currently trying to implement a sendall() function in a server written in C, and a recvall() function on the corresponding client written in python.
I can get the server and the client to work together when they're both written in the same language (i.e. both in c or both in python), but I can't get it to work with the server in c and the client in python. currently, i want to send a simple string from the c server to the python client.
Server sendall() implementation in C as follows (referenced from Beej's guide):
int sendall(int socket, char * buf, int *len){
int total = 0; // how many bytes we've sent
int bytesleft = *len; // how many we have left to send
int n;
while(total < *len) {
n = send(socket, buf + total, bytesleft, 0);
if (n == -1) { break; }
total += n;
bytesleft -= n;
}
*len = total; // return number actually sent here
return n==-1?-1:0; // return -1 on failure, 0 on success
}
Calling the function:
char buf[10] = "test";
int len;
len = strlen(buf);
if (sendall(command_socket, buf, &len) == -1) {
perror("sendall");
printf("We only sent %d bytes because of the error!\n", len);
}
printf("Bytes sent: %d\n", len);
Client recvall() implementation in Python (referenced from http://stupidpythonideas.blogspot.com/2013/05/sockets-are-byte-streams-not-message.html):
def recv_one_message(sock):
lengthbuf = recvall(sock, 4)
length, = struct.unpack('!I', lengthbuf)
return recvall(sock, length)
def recvall(sock, count):
buf = ''
while count:
newbuf = sock.recv(count)
print newbuf
if not newbuf:
return None
buf += newbuf
count -= len(newbuf)
return buf
Called as:
buf = recv_one_message(command_socket)
print buf
Whenever I send a message from the C-server to the Python-client, I get a return of "None." I've traced the the response coming in on the client side -- it is getting the message I sent, but the final response is always "none" and the message won't be printed. I've also tried just returning the message instead of having the None return, which also results in nothing getting printed. Can't see where I'm going wrong, any thoughts? Thanks.
Say you have a perfectly fluent English speaker and a perfectly fluent French speaker. They will not be able to communicate with each other very well. Who is at fault? -- Whoever put the two of them together expecting them to be able to communicate without first agreeing on how they would communicate.
Your send function and your receive function implement totally different protocols. The send function requires the receiver to already know the length of the data to receive. Your receive function requires the sender to send the length prior to the data. So you cannot mix and match them because they do not use the same wire format
Here's some valuable advice gained over decades of experience: Never attempt to use TCP without first documenting the protocol you're going to use on top of TCP at the byte level. As it is, there's no way to know which function is right and which is wrong because nobody has any idea what bytes are supposed to be sent over the wire.
TCP is a byte-stream protocol, and for two programs to work correctly with a TCP connection between them, each must send precisely the stream of bytes the other expects to receive. The best way to make sure this happens is to first document that stream of bytes.
If you have no idea where to start with such documentation, have a look at the simplest existing protocol that's analogous to what you're trying to do. Possible protocols to look at it include HTTP, SMTP, IRC, and DNS.
Related
I had been learning how to use raw sockets in python.
Raw sockets in python can be used to send any arbitrary binary data, to any particular interface, without any other details such as MAC address or IP address.
For example, consider the following piece of code:
import socket
sock = socket.socket(socket.AF_PACKET, socket.SOCK_RAW, socket.htons(3))
sock.bind(('lo',0))
sock.send('This is a sample Raw Socket Code'.encode())
sock.close()
When this is script is executed as a root user, we can capture a packet in Wireshark (or any sniffer) in the loopback interface (since I've bound the socket to 'lo'), that contains the binary string of the above text, and nothing else (of course, this is a malformed packet).
Thing is, I can't find out an implementation of this use case in C. Every implementation of raw sockets in C that I can find, deals with some kind of address. The sendto() requires the argument (struct sockaddr*), which is a generic version of (struct sockaddr_ll*) or (struct sockaddr_in*), which eventually requires an address.
How can we make an implementation of the above use case in C? I know we can straight away use python implementation, as it is available. But I'm curious to know how to implement in C.
You can use following code for that
int send(int fd, int ifindex, const uint8_t* buf, size_t len) {
struct sockaddr_ll to;
struct sockaddr* to_ptr = (sockaddr*)&to;
to.sll_family = AF_PACKET;
to.sll_ifindex = ifindex;
to.sll_halen = 6;
memcpy(&to.sll_addr, buf, 6);
int ret = sendto(fd, buf, len, 0, to_ptr, sizeof(to));
return ret;
}
where
fd - socket id
ifindex - ifindex of the interface you want to use for sending
buf - is a buffer with a formed ethernet frame
len - len of the buffer
I need to have a python GUI communicating with an mbed (LPC1768) board. I am able to send a string from the mbed board to python's IDLE but when I try to send a value back to the mbed board, it does not work as expected.
I have written a very basic program where I read a string from the mbed board and print it on Python's IDLE. The program should then ask for the user's to type a value which should be sent to the mbed board.
This value should set the time between LED's flashing.
The python code
import serial
ser = serial.Serial('COM8', 9600)
try:
ser.open()
except:
print("Port already open")
out= ser.readline()
#while(1):
print(out)
time=input("Enter a time: " )
print (time)
ser.write(time.encode())
ser.close()
and the mbed c++ code
#include "mbed.h"
//DigitalOut myled(LED1);
DigitalOut one(LED1);
DigitalOut two(LED2);
DigitalOut three(LED3);
DigitalOut four(LED4);
Serial pc(USBTX, USBRX);
float c = 0.2;
int main() {
while(1) {
pc.printf("Hello World!\n");
one = 1;
wait(c);
two=1;
one = 0;
wait(c);
two=0;
c = float(pc.getc());
three=1;
wait(c);
three=0;
four=1;
wait(c);
four=0;
}
}
The program waits for the value to be entered in IDLE and sent to the mbed board and begins to use the value sent to it but suddenly stops working and I cannot figure out why.
You need to take this line:
c = float(pc.getc());
out of your loop.
The reason your program stops working is that line is holding until you send something again. If you only send once it waits forever.
If you want to dynamically set wait time after the program enters the while loop, I would suggest attaching a callback function to a serial RX interrupt.
RawSerial pc(USBTX, USBRX);
void callback() {
c = float(pc.getc());
}
Serial uses mutex and cannot be used in ISR on mbed OS5. Use RawSerial instead.
int main() {
pc.attach(&callback, Serial::RxIrq);
while(1) {
// your code for LED flashing
// no need to call pc.getc() in here
one = 1;
wait(c);
one = 0;
wait(c);
}
}
This way, the LED continues to blink and you can update c whenever the mbed receives a value.
Also, it looks like you are sending ASCII characters. ASCII 1 is 49 in decimal. So, pc.get() returns 49 when you send '1'. I don't think that is what you want. If you are always sending a single digit (1~9), an easy fix is pc.getc() - 48. But you better parse string to int and do error handling on python side.
I wrote this snippet of Python code (using pybluez) to send raw BNEP Bluetooth packet over L2CAP. The purpose is to do some fuzzing-like testing.
BNEP_PSM = 0x000F
btSock = bluetooth.BluetoothSocket(bluetooth.L2CAP)
btSock.connect(('<some BDADDR>', BNEP_PSM))
for i in range(10):
btSock.send('<some payload>')
This is working quite fine and as expected creating multiple BNEP packet even if the payload is malformed.
Now, I'm trying to write the same function in C++ using Qt, but it is not working the same way. An excerpt of the code is the following:
QBluetoothSocket btSock(QBluetoothServiceInfo::L2capProtocol);
btSock.connectToService(QBluetoothAddress("<some BDADDR>"), QBluetoothUuid::Bnep);
QObject::connect(&btSock, &QBluetoothSocket::connected, [&btSock](){
int i = 10;
while (i--)
btSock.write("<some payload>");
});
Running it with i = 1 works just fine sending a single packet with the specified payload.
Running it with i = 10 will results in a single packet with the payload equals to ten times the specified payload.
For instance setting a payload of "AAAA" in a loop of 3 will result in the first case using Python in
+------------+----+ +------------+----+ +------------+----+
|L2CAP Header|AAAA| --> |L2CAP Header|AAAA| --> |L2CAP Header|AAAA|
+------------+----+ +------------+----+ +------------+----+
In the second case using Qt in
+------------+------------+
|L2CAP Header|AAAAAAAAAAAA|
+------------+------------+
How could I force Qt socket's write to behave like Python socket's send?
UPDATE:
Looking at the documentation it says that
The bytes are written when control goes back to the event loop
How could I force buffer to flush before going back to the event loop?
How could I force buffer to flush before going back to the event loop?
You can't, because the sending can only be done asynchronously, not synchronously.
But we can queue a flush the same way the packets are queued. Namely: send each packet after the previous one has been sent. Thus we shall send it every time the event loop has processed all other work. The idiom for that is zero-duration timers - note that this has nothing at all to do with timers, it's a weird overloading of the timer concept that really makes no sense otherwise.
int i = 10;
while (i--)
QTimer::singleShot(0, this, [this]{ m_btSocket.write("<some payload>"); });
m_btSocket must be a member of the class, and must be a value member - otherwise the code will be unsafe.
If you wish to ensure that stale packets are dumped in case of a disconnection and won't affect any subsequent connections, keep track of their generation and send only if it's current:
class Foo : public QObject {
unsigned int m_generation = {}; // unsigned: modulo math w/o overflows
QBluetoothSocket m_btSocket{QBluetoothServiceInfo::L2CAP};
...
bool isBtConnected() const { return m_btSocket::state() == QBluetoothSocket::ConnectedState; }
void sendSinglePacket(const QByteArray & data) {
if (!isBtConnected()) return;
auto gen = m_generation;
QTimer::singleShot(0, this, [this, gen, data] {
if (m_generation == gen)
m_btSocket.write(data);
});
}
Foo(QObject * parent = {}) : QObject(parent) {
connect(&m_btSocket, &QBluetoothSocket::Disconnected, this, [this]{
m_generation++; // drops all in-flight packets
});
...
}
};
I did not found a proper solution using QBluetoothSocket's methods, but I made it work with a little hack.
Just used C header sys/socket.h (I need to support POSIX compliant only OSs) and changed
btSock.write("<some payload>");
to
send(btSock.socketDescriptor(), "<some payload>", <payload length>);
Objective: understand strange behavior regarding two classes initialized with identical arguments, and producing different results.
Background:
My project involves a Raspberry Pi 3 communicating with various sensors on an Arduino MEGA via serial (through the USB port) communication. The communication protocol is simple.
The Raspberry Pi sends two bytes:
The address of the sensor (e.g. '\x04')
A second empty byte that can contain additional commands (e.g. '\x00')
The MEGA waits for two bytes, and then replys with the requested information based on a large case/switch tree.
I have encapsulated the code for handling various sensors withing Python classes. The one that is giving me trouble is the Encoder() class. Here is a minimal example:
import serial
class Encoder(object):
def __init__(self, hex_name):
self.hex_name = hex_name
def value(self):
temp = self.hex_name + '\x00'
arduino.write(temp)
return int(arduino.readline())
if __name__ == '__main__':
arduino = serial.Serial('/dev/ttyACMO', 115200)
encoder_one = Encoder('\x03')
encoder_two = Encoder('\x04')
print encoder_one.value()
print encoder_two.value()
The Arduino handles the request as shown below:
if(Serial.available() >= 2){
temp1 = Serial.read();
temp2 = Serial.read();
switch(temp1){
case 1:
...
case 3:
Serial.println(positionLeft);
break;
case 4:
Serial.println(positionRight);
break;
case 5:
...
}
}
Problem:
I get nonsensical values from the encoders. Particularly concerning is that when I initialize both encoder with the same hex_name (i.e. '\x04'), I get different values from encoder_one.value() and encoder_two.value().
Hypothesis:
Something about the class structure is wrong.
I had this working alright before I encapsulated it in a class. I wondered if I was assigning encoder_one and encoder_two to the same object or something silly like that.
I added a line just before return in value() that would print the hex_name (i.e. print self.hex_name). When I set both encoders to '\x04' I get the same non-alphanumeric character printed. When I set one encoder to '\x03' and the other to '\x04' I get two different non-alphanumeric characters--one being the character from the previous test.
Frame-shift in the communication.
The MEGA is expecting two bytes. It waits for two bytes, sends the requested information, then clears the buffer. Is is possible that storing the hex values as string in my Python program, a extra byte is getting added when it performs arduino.write(). Something like a \n or other non-printable character.
Conclusion:
I've put about three hours into this bug, and I think solving it requires some information about how classes work that I do not understand.
It turned out to be the second hypothesis. There is apparently some non-printable character being included in the concatenation step of the hex bytes. Switching to the following order solved the problem.
EDIT: I discovered that performing a readline() immediately after write() was giving intermitent errors. I added a 10 ms delay between the two operations.
Old:
def value(self):
temp = self.hex_name + '\x00'
arduino.write(temp)
return int(arduino.readline())
New:
def value(self):
arduino.write(self.hex_name)
arduino.write('\x00')
time.sleep(0.010)
return int(arduino.readline())
The complete (now working) example is shown below.
import serial
class Encoder(object):
def __init__(self, hex_name):
self.hex_name = hex_name
def value(self):
arduino.write(self.hex_name)
arduino.write('\x00')
time.sleep(0.010)
return int(arduino.readline())
if __name__ == '__main__':
arduino = serial.Serial('/dev/ttyACMO', 115200)
encoder_one = Encoder('\x03')
encoder_two = Encoder('\x04')
print encoder_one.value()
print encoder_two.value()
Situation:
I have tcp client made with Python and tcp server made with Qt. I try to send bytes with my client but I can't get Qt server to read these bytes.
Using Python made client and server, everything works fine. Also I can get my Python client work with C# server with no problems.
Code for Python client:
import socket
import sys
HOST = 'localhost'
PORT = 50505
try:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except socket.error as msg:
sys.stderr.write("[ERROR] %s\n" % msg)
sys.exit(1)
try:
sock.connect((HOST, PORT))
except socket.error as msg:
sys.stderr.write("[ERROR] %s\n" % msg)
sys.exit(2)
sock.send(b'Hello World!\r\n')
I have tried e.q fortuneserver/fortuneclient code examples but they didn't work. Bytesavailable() is always 0.
So the question is how can I read in my Qt application that "Hello World!" line ? I just need that function which starts when a server emits newConnection() signal.
connect(tcpServer, SIGNAL(newConnection()), this, SLOT(startRead()));
UPDATE:
Part of the code for Qt server:
void Server::startRead()
{
QDataStream in(tcpSocket);
in.setVersion(QDataStream::Qt_4_0);
QString ipAddress;
if (blockSize == 0) {
if (tcpSocket->bytesAvailable() < (int)sizeof(quint16))
return;
in >> blockSize;
}
if (tcpSocket->bytesAvailable() < blockSize)
return;
QString nextFortune;
in >> nextFortune;
statusLabel->setText(nextFortune);
ABOVE IS FROM FORTUNE CLIENT EXAMPLE.
BELOW IS FROM ANOTHER EXAMPLE.
/*
char buffer[128] = {0};
QTcpSocket *client = tcpServer->nextPendingConnection();
qDebug() << client->bytesAvailable();
for (int i=0; i<100; i++)
{
client->read(buffer, client->bytesAvailable());
qDebug() << buffer;
std::string sString(buffer);
QString qString(sString.c_str());
statusLabel->setText(qString);
}
*/
}
That last part is rather bad. I tried to make something but I have no clue what I'm doing with Qt :)
You need to arrange for your code to read when there is data available. From your description, there is not data available yet when startRead() runs.
I assume you called QTcpServer::nextPendingConnection to get your tcpSocket in startRead()? If not, you need to do so.
Just connect the readyRead signal from your tcpSocket or client to a doRead() slot, and check bytesAvailable() and read in that slot.
This slot will get called whenever new data arrives.
You might also be interested in the disconnected() signal from your TCP socket to know when it is done sending data.
The server code you attached works this way: Every 'packet' (logical one) is consisted of a size field, and following that size worth of data.
Basically it goes like this:
Read a quint16, if there aren't
enough bytes to be read, wait for
more.
quint16 read, wait until we have
atleast the amount of the quint16
to be read.
Once we have enough, we can read in
the data.
After we understood that, even if you write the packet length like the server expects, what you're trying to do still won't work. This is because the way the server reads the QString from the socket. The client is writing a raw python string, I'm guessing it's just writing a plain old ASCII to the socket. The server is using a QDataStream to read from the socket (this is the standard way to serialize in Qt). If you look at the implementation of QString's operator<< here, it's doing much more than reading an ASCII string.
If you can't change your server, change your client. You can see the implementation of the fortune client in the examples that come with PyQt.