We Keep Coding

DB2 connectivity from Python - ibm_db.connect running continuously

I have been searching for an answer for this for hours, but unfortunately the closest thing I can find is 1 unanswered question. This is a similar issue, but it unfortunately did not have a resolution.
I had a working connection to a IBM DB2 database, but the web console was erroring out, so I was forced to delete the instance and make a new one. I changed nothing regarding the code to connect other than the values used to connect. When I changed these values the ibm_db.connect function runs continuously. There are no output errors as I have left it running for 10 minutes and nothing happens at all. I do change the values to force an error and it will error out saying the values are not correct. I have no clue what the problem is as I have no information to go off of. My only thought is the SSL could have something to do with it.
dsn_driver = connection_data['dsn_driver']
dsn_database = connection_data['dsn_database']
dsn_hostname = connection_data['dsn_hostname']
dsn_port = connection_data['dsn_port']
dsn_protocol = connection_data['dsn_protocol']
dsn_uid = connection_data['dsn_uid']
dsn_pwd = connection_data['dsn_pwd']
dsn = (
"DRIVER={0};"
"DATABASE={1};"
"HOSTNAME={2};"
"PORT={3};"
"PROTOCOL={4};"
"UID={5};"
"PWD={6};").format(dsn_driver, dsn_database, dsn_hostname,
dsn_port, dsn_protocol, dsn_uid, dsn_pwd)
try:
connection = ibm_db.connect(dsn, "", "")
print("Connected to database: ", dsn_database,
"as user: ", dsn_uid, "on host: ", dsn_hostname)
return connection
except:
print("Unable to connect: ", ibm_db.conn_errormsg())
The breakpoint is at connection = ibm_db.connect(dsn, "", "")
This data is loaded from a local JSON file with the following values (except for sensitive information).
{
"dsn_driver": "{IBM DB2 ODBC DRIVER}",
"dsn_database":"BLUDB",
"dsn_hostname": "hostname",
"dsn_port": "port",
"dsn_protocol": "TCPIP",
"dsn_uid": "uid",
"dsn_pwd": "pwd"
}
I have tried everything I can think of, but since nothing outputs I unfortunately do not know where to start. If someone has experience with this please let me know.
Thank you.
Edit: I did end up getting this error message returned from the ibm_db.connect method
Unable to connect: [IBM][CLI Driver] SQL30081N A communication error has been detected. Communication protocol being used: "TCP/IP". Communication API being used: "SOCKETS". Location where the error was detected: "xxx.xx.xxx.xxx". Communication function detecting the error: "recv". Protocol specific err SQLCODE=-30081054", "*", "0". SQLSTATE=08001

A couple of points for clarification:
When you say "the ibm_db.connect function runs continuously" do you mean you see the CPU spinning or just that the python process doesn't progress past the connect?
What type of database are you connecting to? DB2 LUW or z/OS?
Have you tried to make sure that the connectivity is still working? i.e. did you try the suggestion from the other linked post? This:
To verify that there is network connectivity between you and the database you can try telnet xxxxxx 1234 (or nc xxxxxx 1234, where xxxxxx and 1234 are the service hostname and port, respectively
From a debugging point of view I'd be looking at the logs of the intervening processes:
Db2 Connect log if you are using it
DB2 target logs
TCPIP and z/OS Connect address spaces if z/os. BAQ region ? (not sure if that would just be my site)
Firewall - I know that you had a working connection but always best to check the obvious as well
As you've pointed out, without an error message it's hard to know where to start

python socket programming for transferring a photo

I'm new to socket programming in python. Here is an example of opening a TCP socket in a Mininet host and sending a photo from one host to another. In fact I changed the code that I had used to send a simple message to another host (writing the received data to a text file) in order to meet my requirements. Although when I implement this revised code, there is no error and it seems to transfer correctly, I am not sure whether this is a correct way to do this transmission or not. Since I'm running both hosts on the same machine, I thought it may have an influence on the result. I wanted to ask you to check whether this is a correct way to transfer or I should add or remove something.
mininetSocketTest.py
#!/usr/bin/python
from mininet.topo import Topo, SingleSwitchTopo
from mininet.net import Mininet
from mininet.log import lg, info
from mininet.cli import CLI
def main():
lg.setLogLevel('info')
net = Mininet(SingleSwitchTopo(k=2))
net.start()
h1 = net.get('h1')
p1 = h1.popen('python myClient2.py')
h2 = net.get('h2')
h2.cmd('python myServer2.py')
CLI( net )
#p1.terminate()
net.stop()
if __name__ == '__main__':
main()
myServer2.py
import socket
import sys
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(('10.0.0.1', 12345))
buf = 1024
f = open("2.jpg",'wb')
s.listen(1)
conn , addr = s.accept()
while 1:
data = conn.recv(buf)
print(data[:10])
#print "PACKAGE RECEIVED..."
f.write(data)
if not data: break
#conn.send(data)
conn.close()
s.close()
myClient2.py:
import socket
import sys
f=open ("1.jpg", "rb")
print sys.getsizeof(f)
buf = 1024
data = f.read(buf)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('10.0.0.1',12345))
while (data):
if(s.sendall(data)):
#print "sending ..."
data = f.read(buf)
print(f.tell(), data[:10])
else:
s.close()
s.close()

This loop in client2 is wrong:
while (data):
if(s.send(data)):
print "sending ..."
data = f.read(buf)
As the send
docs say:
Returns the number of bytes sent. Applications are responsible for checking that all data has been sent; if only some of the data was transmitted, the application needs to attempt delivery of the remaining data. For further information on this topic, consult the Socket Programming HOWTO.
You're not even attempting to do this. So, while it probably works on localhost, on a lightly-loaded machine, with smallish files, it's going to break as soon as you try to use it for real.
As the help says, you need to do something to deliver the rest of the buffer. Since there's probably no good reason you can't just block until it's all sent, the simplest thing to do is to call sendall:
Unlike send(), this method continues to send data from bytes until either all data has been sent or an error occurs. None is returned on success. On error, an exception is raised…
And this brings up the next problem: You're not doing any exception handling anywhere. Maybe that's OK, but usually it isn't. For example, if one of your sockets goes down, but the other one is still up, do you want to abort the whole program and hard-drop your connection, or do you maybe want to finish sending whatever you have first?
You should at least probably use a with clause of a finally, to make sure you close your sockets cleanly, so the other side will get a nice EOF instead of an exception.
Also, your server code just serves a single client and then quits. Is that actually what you wanted? Usually, even if you don't need concurrent clients, you at least want to loop around accepting and servicing them one by one.
Finally, a server almost always wants to do this:
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
Without this, if you try to run the server again within a few seconds after it finished (a platform-specific number of seconds, which may even depend whether it finished with an exception instead of a clean shutdown), the bind will fail, in the same way as if you tried to bind a socket that's actually in use by another program.

First of all, you should use TCP and not UDP. TCP will ensure that your client/server has received the whole photo properly. UDP is more used for content streaming.
Absolutely not your use case.

Pymodbus RTU connection troubles

I am a beginner with both python and modbus and I'm trying to control a fan connected to a serial port with pymodbus for hours now. Using a proprietary software from the manufacturer I was able to control the fan, so the connection itself works. Yet, my own code does not.
According to the manual the fan is set to the following values:
mode: RTU
baudrate: 19200
parity: even
timeout: 1
slave ID: 247 (F7h)
databits: 8
I modified the pymodbus example code from the Pymodbus Library Examples. This is my code:
from pymodbus3.client.sync import ModbusSerialClient as ModbusClient
Fan = ModbusClient(method='rtu',port ='/dev/ttymxc1', parity = 'E', baudrate='19200', timeout=1)
connection = Fan.connect()
a = Fan.write_register(2,1, unit=0xF7)
b = Fan.read_holding_registers(2,1, unit = 0xF7)
Fan.close()
Both the read and write command result in the following error:
pymodbus3.exceptions.ModbusIOException: Modbus Error: [Input/Output] Server responded with bad response
The same message I get even when the cable is not plugged in.
Does anyone have a suggestion what's wrong?

Timeout is the time for the port to wait for an answer.
Set the timeout to something like ~2000, 1 is 1ms and they probably meant 1s.
And try also address 1 (if the above is not working), check also if the Modbus address can be configured to something else (it might be some switchs on the fan).

Best way to handle no-blocking socket service result

I want to talk with a proxy soft, here is official socket example.
This soft would send traffic statistic to connected cli every 10 seconds. I also need to send commands and get command results on same connection.
It cost me some time to figure out that I can't create two socket clients and connect to same server to get different results.
My problem: how do I manage user config while recording traffic statistic ?
Test:
When add a port, would get ok after several cli.recv(1506) (because the stat send every 10 second, it I don't read all the time, ok would behind of many stat) :
>>> cli.send(b'add: {"server_port":8003, "password":"123123"}')
46
>>> print(cli.recv(1506))
stat: {"8002":164}
>>> print(cli.recv(1506))
stat: {"8002":336}
>>> print(cli.recv(1506))
ok
>>> print(cli.recv(1506))
stat: {"8002":31}
>>> print(cli.recv(1506))
# hang here wait for next result
So if I send many command, I can't recognise the which result map which command.
The solution I came up with is:
# client main code
while True:
# need open another sock port to recieve command
c = self.on_new_command()
if c:
self.cli.send(c)
# does python socket server ensure response is FIFO when `setblocking(False)`
self.queue.append(c)
ret = self.cli.recv(1524)
if 'stat' in ret:
self.record(ret)
else :
self.log(self.queue.pop(), ret)
I need open another sock port to receive command in this client, and have to write another client which sends commands to this... I just don't feel it is good.
Because it is my first time to play with socket programming, is my solution the best for this situation? Is there any better way to achieve my goal?

Python for loop slows and evenutally hangs

I'm totally new to Python (as of half an hour ago) and trying to write a simple script to enumerate users on an SMTP server.
The users file is a simple list (one per line) of usernames.
The script runs fine but with each iteration of the loop it slows until, around loop 14, it seems to hang completely. No error - I have to ^c.
Can anyone shed some light on the problem please?
TIA,
Tom
#!/usr/bin/python
import socket
import sys
if len(sys.argv) != 2:
print "Usage: vrfy.py <username file>"
sys.exit(0)
#open user file
file=open(sys.argv[1], 'r')
users=[x.strip() for x in file.readlines()]
file.close
#Just for debugging
print users
# Create a Socket
s=socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Connect to the Server
connect=s.connect(('192.168.13.222',25))
for x in users:
# VRFY a user
s.send('VRFY ' + x + '\r\n')
result=s.recv(1024)
print result
# Close the socket
s.close()

Most likely your SMTP server is tarpitting your client connection. This is a defense against runaway clients, or clients which submit large volumes of "junk" commands. From the manpage for Postfix smtpd:
smtpd_junk_command_limit (normal: 100, stress: 1)
The number of junk commands (NOOP, VRFY, ETRN or RSET) that a
remote SMTP client can send before the Postfix SMTP server
starts to increment the error counter with each junk command.
The smtpd daemon will insert a 1-second delay before replying after a certain amount of junk is seen. If you have root access to the smtp server in question, try an strace to see if nanosleep syscalls are being issued by the server.
Here is a trace from running your script against my local server. After 100 VRFY commands it starts sleeping between commands. Your server may have a lower limit of ~15 junk commands:
nanosleep({1, 0}, 0x7fffda9a67a0) = 0
poll([{fd=9, events=POLLOUT}], 1, 300000) = 1 ([{fd=9, revents=POLLOUT}])
write(9, "252 2.0.0 pat\r\n", 15) = 15
poll([{fd=9, events=POLLIN}], 1, 300000) = 1 ([{fd=9, revents=POLLIN}])
read(9, "VRFY pat\r\n", 4096) = 10

s.recv blocks so if you have no more data on the socket then it will block forever.
You have to keep track of how much data you are receiving. You need to know this ahead of time so the client and the server can agree on the size.

Solving the exact same problem I also ran into the issue.
I'm almost sure #samplebias is right. I found I could work around the "tarpitting" by abusing the poor system even more, tearing down and rebuilding every connection:
#[ ...Snip... ]
import smtplib
#[ ...Snip... ]
for USER in open(opts.USERS,'r'):
smtpserver = smtplib.SMTP(HOST,PORT)
smtpserver.ehlo()
verifyuser = smtpserver.verify(USER)
print("%s %s: %s") % (HOST.rstrip(), USER.rstrip(), verifyuser)
smtpserver.quit()
I'm curious whether this particular type of hammering would work in a live environment, but too certain it would make some people very unhappy.
PS, python: batteries included.

In a glance, your code has no bugs. However, you shall notice that TCP isn't a "message" oriented protocol. So, you can't use socket.send in a loop assuming that one message will be actually sent through the medium at every call. Thus, if some calls starts to get buffered in the output buffer, and you just call socket.recv after it, your program will stuck in a deadlock.
What you should do is a threaded or asynchronous code. Maybe Twisted Framework may help you.