We Keep Coding

Cannot read data sent from TCP client on python to TCP server in C#

I have a windows service written in C# which has opens a TCP socket. I've connected and tested this TCP socket through telnet, and I am able to send and receive messages to the socket through putty with the telnet protocol. When connecting with a small python script, the script is able to receive messages, but unable to send them such that they are received by the C# server. The code for the server is:
IPAddress ipAddress = IPAddress.Parse("127.0.0.1");
TcpListener listener = new TcpListener(ipAddress, 5555);
listener.Start();
Thread whileThread = new Thread(() =>
{
while (true)
{
TcpClient client = listener.AcceptTcpClient();
Thread childThread = new Thread(() =>
{
NetworkStream stream = client.GetStream();
StreamReader streamreader = new StreamReader(client.GetStream(), Encoding.ASCII);
string line = null;
WriteToNetworkStream(stream, "Connected to the service");
eventLog1.WriteEntry("TCP Socket opened");
eventLog1.WriteEntry((GetState(client) == TcpState.Established).ToString()); //this does return True.
while ((line = streamreader.ReadLine()) != "<EOF>" && GetState(client) == TcpState.Established)
{
eventLog1.WriteEntry(line);
}
eventLog1.WriteEntry("TCP Socket closed");
stream.Close();
client.Close();
});
childThread.Start();
}
});
whileThread.Start();
The python script on the other side is:
import socket
import time
TCP_IP = '127.0.0.1'
TCP_PORT = 5555
BUFFER_SIZE = 1024
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((TCP_IP, TCP_PORT))
time.sleep(1)
s.send("this is a test")
time.sleep(10)
data = s.recv(BUFFER_SIZE)
print(data)
The message "this is a test" is never received by the other side, but "TCP Socket closed" isn't logged until after after the time.sleep(10) is done, showing the C# does recognize some sort of prolonged connection. Also, the data variable does contain "Connected to the service" as expected, showing the server can send data to the client. Like I said earlier, when trying to send and receive messages using telnet with putty, this does work correctly.

Have you tried reading the bytes from the network stream? I can't say whether it will make a difference but it's worth a try to check if actual bytes are coming in:
NetworkStream stream = client.GetStream();
Byte[] bytes = new Byte[256]; //whatever size is appropriate
int bytesRead;
string data;
while ((bytesRead = stream.Read(bytes, 0, bytes.Length)) != 0)
{
data = System.Text.Encoding.ASCII.GetString(bytes, 0, bytesRead);
// ......
}

How to get address of sender in PYZMQ socket? (Multiple client, 1 server)

I have created a server that has a socket bound to 2 ports. Now there are 2 clients each of which is connected to 1 port. The server receives random numbers from both clients, but I want the server to know from which port it is receiving the data.
I know it maybe a pretty stupid question and I am new to this. Please help !
Server
import zmq
context = zmq.Context()
socket = context.socket(zmq.REP)
socket.bind("tcp://*:5555") # A single port is used for the communication
socket.bind("tcp://*:5556") # A single port is used for the communication
# Wait for next request from client
while True:
try:
message = socket.recv()
msg = message.decode("utf-8") # decode the bytes to string
print("Random Integer Received: %s" % msg) # display received string
# Send reply back to client
socket.send_string("Random Integer Received from client 1")
#sock_add = socket.getaddrinfo()
except zmq.ZMQError as e:
print('Unable to receive: %s', e)
Client 1 (client 2 has identical code with port address 5556)
import zmq
import time
import numpy as np
#import pandas as pd
context = zmq.Context()
num = np.random.randint(150) # Generates a random integer
# Socket to talk to server
print("Connecting to server…")
socket = context.socket(zmq.REQ)
socket.connect("tcp://localhost:5555")
# Do 100 requests, waiting each time for a response
for request in range(100):
k = np.random.randint(150) # Generates a random integer
print("Sending request %s …" % request, "sent no is %s" % k)
socket.send_string(str(k))
# Get the reply.
message = socket.recv()
print("Received reply %s [ %s ]" % (request, message))
time.sleep(5) # delay of 5 seconds
Please help !!

I would simply add the information that you need (port) into the message itself and use the zmq.SNDMORE...
Something like:
socket.send_string("Port 5555", flags=zmq.SNDMORE)
socket.send_string(str(k)) # or 5556
And then, on the server side, you can see which port the messages is coming from based on the message itself.

Python, MQTT broker, publish message

I have configured MQTT Broker, receiving published messages from another piece of code (not written or accessible by me). I added another topic to the broker configuration and am now trying to publish data to this new topic from a piece of python code. I get the feedback that the message is published by the callback function, but no actual data is received.
Is there anything I am missing?
I am using the following code:
import paho.mqtt.client as mqtt
import time
#=========================================================================
def on_connect(client, userdata, flags, rc) :
print "on_connect()"
#=========================================================================
def on_publish(client, userdata, mid) :
print "on_publish()"
#=========================================================================
def send() :
mqttc = mqtt.Client()
mqttc.on_connect = on_connect
mqttc.on_publish = on_publish
#host = "localhost"
host = "127.0.0.1"
port = 1883
keepalive = 60
print "\nConnect to '%s', port '%s', keepalive '%s'" % (host, port, keepalive)
mqttc.connect(host=host, port=port, keepalive=keepalive)
time.sleep(3)
mqttc.loop_start()
time.sleep(3)
topic = "data/MY/TOPIC"
msg = "MY_MESSAGE"
print "Publish to '%s' msg '%s'" % (topic, msg)
mqttc.publish(topic, msg, qos=2)
time.sleep(3)
mqttc.loop_stop()
# end send()
#=========================================================================
if __name__ == "__main__":
send()
# end if
Getting the stdout
Connect to '127.0.0.1', port '1883', keepalive '60'
on_connect()
Publish to 'data/MY/TOPIC' msg 'MY MESSAGE'
on_publish()
I am not sure if the loop() functions are necessary, but if I do not embed the publishing in the loop_start() and loop_stop(), I do not get a on_connect callback.

The loop functions are necessary as these are where all the network traffic is processed.
Manually setting up a connection to the broker to just send a single message like this is not a good idea, it would be better to start the client, leave it running (by calling loop_start() and not calling loop_stop()) in the background and then just call the publish method on the mqttc client object.
If you don't want to keep a instance of the client running then you should use the single message publish helper method provided by the paho python library (https://pypi.python.org/pypi/paho-mqtt/1.1#single):
import paho.mqtt.publish as publish
publish.single("paho/test/single", "payload", hostname="iot.eclipse.org")

Use of pyzmq's logging handler in python

I want to introduce a zmq based logging into a Python program. As I was facing ZMQError: Address in use errors I decided to boil it down to a simple proof of concept. I was able to run the boiled down version, but not to receive any log entries. This is the code I used:
Log Publisher:
import time
import logging
from zmq.log import handlers as zmqHandler
logger = logging.getLogger('myapp')
logger.setLevel(logging.ERROR)
zmqH=zmqHandler.PUBHandler('tcp://127.0.0.1:12344')
logger.addHandler(zmqH)
for i in range(50):
logger.error('error test...')
print "Send error #%s" % (str(i))
time.sleep(1)
Result
Send error #0
Send error #1
Send error #2
Send error #3
Send error #4
...
Log Subscriber:
import time
import zmq
def sub_client():
port = "12344"
context = zmq.Context()
socket = context.socket(zmq.SUB)
socket.connect("tcp://127.0.0.1:%s" % port)
# Generate 30 entries
for i in range (30):
print "Listening to publishers..."
message = socket.recv()
print "Received error #%s: %s" % (str(i), message)
time.sleep(1)
sub_client()
Result
Listening to publishers...
So the subscriber is locked at the call of socket.recv(). I started publisher and subscriber in different consoles. Both processes appear when I use netstat:
C:\>netstat -a -n -o | findstr 12344
TCP 127.0.0.1:12344 0.0.0.0:0 LISTEN 1336
TCP 127.0.0.1:12344 127.0.0.1:51937 ESTABLISHED 1336
TCP 127.0.0.1:51937 127.0.0.1:12344 ESTABLISHED 8624
I fail to see my mistake here, any ideas?
In addition to the problem at hand, how do I use this zmq listener in general.
Do I have to create one instance of the PUBHandler per process and then add it to all instances of logger (logging.getLogger('myapp') creates a own logger instance, right?) or do I have to create an own PUBHandler for all of the different classes I use? As the PUBHandlerclass has a createLock() I assume that it is not thread save...
For completness I want to mention the doc of the PUBHandler class
I am using a python(x,y) distribution at Win7 with python 2.7.10 and pyzmq 14.7.0-14
[update]
I ruled out the the windows firewall as the source of the missing packages

The problem is on the subscriber side. Initially a subscriber filters out all messages, until a filter is set. Use socket.setsockopt(opt, value) function to archive this.
The pyZMQ description is not very clear about the use of this function:
getsockopt(opt) get default socket options for new sockets created by
this Context
But the documentation of the zmq_setsockopt function is quite clear (see here):
int zmq_setsockopt (void *socket, int option_name, const void *option_value, size_t option_len)
...
ZMQ_SUBSCRIBE: Establish message filter The ZMQ_SUBSCRIBE option shall establish a new message filter on a ZMQ_SUB socket.
Newly created ZMQ_SUB sockets shall filter out all incoming
messages, therefore you should call this option to establish an
initial message filter.
So the solution is to set a filter with socket.setsockopt(zmq.SUBSCRIBE,filter), where filter is the string you want to filter for. Use filter='' to show all messages. A filter like filter='ERROR' will only display the error messages and suppress all other types like WARNING,INFO or DEBUG.
With this the sub_client() function looks like this:
import time
import zmq
def sub_client():
port = "12344"
context = zmq.Context()
socket = context.socket(zmq.SUB)
socket.connect("tcp://127.0.0.1:%s" % port)
socket.setsockopt(zmq.SUBSCRIBE,'')
# Process 30 updates
print "Listening to publishers..."
for i in range (30):
print "Listening to publishers..."
message = socket.recv()
print "Received error #%s: %s" % (str(i), message)
time.sleep(1)
sub_client()

I know it is an older post, but if someone lands here this is what the subscriber looks like
def sub_client():
port = "12345"
context = zmq.Context()
socket = context.socket(zmq.SUB)
socket.connect("tcp://localhost:%s" % port)
socket.subscribe("")
# Process 30 updates
for i in range (30):
print("Listening to publishers...")
message = socket.recv()
print("Received error #%s: %s",str(i), message)
time.sleep(1)
sub_client()
And Publisher looks like
import zmq
import logging
import time
from zmq.log.handlers import PUBHandler
port = "12345"
context = zmq.Context()
pub = context.socket(zmq.PUB)
pub.bind("tcp://*:%s" % port)
handler = PUBHandler(pub)
logger = logging.getLogger()
logger.setLevel(logging.ERROR)
logger.addHandler(handler)
for i in range(50):
logger.error('error test...')
print("publish error",str(i))
time.sleep(1)

I guess you missed to set PUB socket in server.
something like this should do,
publisher = context.socket(zmq.PUB)
publisher.bind('tcp://127.0.0.1:12344')
zmqh = PUBHandler(publisher)
logger = logging.getLogger('myapp')
logger.setLevel(logging.ERROR)
logger.addHandler(zmqh)
I hope this helps.

Python: Socket Programming: accept() and connect calls

I have been self-learning python since few months now , and finally learning Socket programming. As an text book exercise, I am supposed to design a half-duplex chat system . Below is the code. The first request and response are just fine , but everytime I try sending a second message from client, the server seems to be hanging. The program is TCP based.
I am suspecting that since ss.accept() is being called everytime a new message has to be sent, a new thread is being created but since I have made only 1 call to sc.connect() from client , may be my new connection at the server end is hanging there for infinite time.
As a trail : I called ss.accept() outside the while loop, ie making only 1 connection and listening to data over and over on while loop, the conversations works just fine
Can someone please have a look a the code and help me understand where exactly is the issue.
Since, I am learning, I have not moved to twisted yet. I want to learn all the basics first before I move to frameworks.
!bin/usr/env python
import socket, sys
HOST =''
PORT = 1060
ADDR =(HOST,PORT)
def userinput(sock):
usermessage = input('>')
sock.sendall(str(len(usermessage)))
return usermessage
def server():
s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)
s.bind(ADDR)
s.listen(1)
print 'the server is listening at',s.getsockname()
while True:
ss,sockname = s.accept()
#listen to determine the bytes sent by client
msglen = ss.recv(4096)
#accept the complete message
msg = ss.recv(int(msglen))
print 'client:', repr(msg)
servermsg = userinput(ss)
ss.sendall(servermsg)
print " ---------------"
ss.close()
def client():
sc = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
sc.connect(ADDR)
while True:
message = userinput(sc)
sc.sendall(message)
replylen = sc.recv(4096)
reply = sc.recv(int(replylen))
print "reply:",reply
print "----------------"
sc.close()
if sys.argv[1:] == ['server']:
server()
elif sys.argv[1:] == ['client']:
client()
else:
print >> sys.stderr,'usage:tcp_2_7.py server|client[host]'

Your trial - accepting once and then receiving multiple messages - is how you should do this. Calling accept is waiting for a new connection - you don't need to do this every time you want to send or receive a message, just as you don't want to call connect every time you want to send or receive.
Think of it this way:
When you connect to a chat server, do you connect, send a message, then disconnect immediately? No - you have a constant open connection which messages are sent through, and the connection is only closed at the end of a chat session.
From the docs on accept:
socket.accept()
Accept a connection. The socket must be bound to an
address and listening for connections. The return value is a pair
(conn, address) where conn is a new socket object usable to send and
receive data on the connection, and address is the address bound to
the socket on the other end of the connection.