We Keep Coding

Read loop that does not crash when file is recreated

I have a python script that needs to read bytes from a file and publish them in MQTT room. But the file could be recreated during the process (the python script have to be running 24/24).
I tried this code, but when the file is being recreated os.path.isfile always return false.
def mqttToTtyOnMessage(client, userdata, message):
tty = openFile(filename, os.O_WRONLY)
print("%s # %s : Message from %s to %s\n" %
(threadName, time.ctime(time.time()), message.topic, filename))
try:
os.write(tty, message.payload)
except:
print("%s # %s : Unable to write message to file %s\n%s" %
(threadName, time.ctime(time.time()), filename, sys.exc_info()))
while True:
if os.path.isfile(filename):
print("%s # %s : File exist, come back to main loop." % (threadName, time.ctime(time.time())))
subTty = openFile(filename, os.O_WRONLY)
os.write(subTty, message.payload)
break
else:
print("%s # %s : File still not exist, wait few seconds and check again." % (threadName, time.ctime(time.time())))
time.sleep(1)
os.close(tty)
The function openFile looks like this :
def openFile(filename, permission):
try:
fd = os.open(filename, permission)
except:
print("%s # %s : Unable to write message to file %s\n%s" %
(threadName, time.ctime(time.time()), filename, sys.exc_info()))
return (fd)
I hope you have ideas about an error in my code or another way to achieve this.
Thanks
EDIT: Solve this problem by changing the way I checked if the file exist :
def mqttToTtyOnMessage(client, userdata, message):
try:
tty = openFile(filename, os.O_WRONLY)
os.write(tty, message.payload)
print("%s # %s : Message from %s to %s\n" %
(threadName, time.ctime(time.time()), message.topic, filename))
except OSError:
print("%s # %s : Unable to write message to file %s\n%s\n" %
(threadName, time.ctime(time.time()), filename, sys.exc_info()))
while True:
try:
subTty = openFile(filename, os.O_WRONLY)
os.write(subTty, message.payload)
break
except:
print("%s # %s : Can't open file for the moment, try again in 5s.\n%s\n" % ("mqttToTty", time.ctime(time.time()), sys.exc_info()))
time.sleep(5)
os.close(tty)

Waiting for the previous function to be completed in Python

is there any solution in python which lets a function execute after the previous one was finished?
Here is one of the ideas I'm using now. But it is not solving the problem when files are larger and the program needs more time.
def copy_to_jumphost(self):
try:
if self.connect():
stdin, stdout, stderr = self.client.exec_command('sshpass -p %s scp -r %s#%s:%s/' % (self.password,self.username,self.hostname,self.log_path) + self.lista[self.file_number].rstrip() + ' ' + '/home/%s/' % (self.username) + self.lista[self.file_number].rstrip())
except (AttributeError, TypeError) as e:
print("Error occurred:", e)
try:
if self.connect():
if self.copy_to_jumphost():
ftp_client = self.client.open_sftp()
ftp_client.get(filepath, self.localpath)
print("Success! \nFile coppied to %s" %(self.localpath))
else:
time.sleep(5)
ftp_client = self.client.open_sftp()
ftp_client.get(filepath, self.localpath)
print("Success but needed some time! \nFile coppied to %s" %(self.localpath))
except (AttributeError, TypeError) as e:
print("Error occurred:", e)
Perfect situation for me will be if in else statement there is a solution to wait for finishing the copy_to_jumphost() function, because time.sleep(5) will fail if I will need to copy larger files.

Checking condition while Exception handling in python

This is part of my code in python. I want to check the error message and if HTTPError() then I want to add the host to the file ok.txt. But it doesn't work. what is the problem here?
except urllib2.URLError, e:
print '%-15s\t%15r' % (url.strip(), e)
if e == 'HTTPError()':
OK.write('%-15s' % (url.strip()) + '\n')
OK.flush()
When I run whole script the output is something like this:
http://a.com HTTPError()
http://b.com URLError(timeout('timed out',),)
http://c.com URLError(timeout('timed out',),)
http://d.com URLError(error(111, 'Connection refused'),)
http://e.com 200

Use isinstance() to check whether or not your error is of type HTTPError:
except urllib2.URLError as e: # use the "as e" instead of the old style comma delimitation.
print '%-15s\t%15r' % (url.strip(), e)
if isinstance(e, HTTPError):
OK.write('%-15s' % (url.strip()) + '\n')
OK.flush()

add_header expecting 3 arguments instead of just key/value

I'm encountering this error message:
TypeError: add_header() takes exactly 3 arguments (2 given)
when using these parameters:
testService("SomeServiceName", "POST", "[redacted valid url]", ('Content-type','application/json'), [redacted valid json])
Normally this error means I'm not passing "self" as a parameter, but seeing as this method is not being called in a class, I'm not sure what to do. I've tried passing self in as a parameter in both the parameters and inside the method. And I've tried wrapping the header in brackets and parentheses. When I pass "self" I get the error message that self is undefined, and when I use the brackets instead of parentheses, I get the same error as above.
Anyone with magical Python debugging skills out there? Thanks so much for taking the time to check this out!
def testService(name, verb, url, header="", requestBody=""):
#Log out the name of the request we're testing
if (name is not None) or (name.strip() is not ""):
print "Checking " + name + "\n\n"
# Make URL with StoreNumber
if (url is not None) or (url is not ""):
testUrl = url
# If specified verb is GET
if verb.strip().upper() == "GET":
# Create request
req = urllib2.Request(testUrl)
print "Making request with URL: " + testUrl + "\n\n"
# Send request
try:
response = urllib2.urlopen(req)
# If service returns 200 Okay
print "Connection to " + name + " Service successful. Returned with code " + str(response.code) + "\n\n"
# Log response
print "Response: " + response.read() + "\n\n"
# Handle exceptions
# If HTTP Error
except HTTPError as e:
if hasattr(e, 'reason'):
print name + ' failed to reach a server.'
print 'Reason: ', e.reason
elif hasattr(e, 'code'):
print e.code
elif hasattr(e, 'message'):
print e.message
pass
# If URL was the problem
except URLError as e:
if hasattr(e, 'reason'):
print name + ' failed to reach a server.'
if str(e.reason) == "[Errno 11004] getaddrinfo failed":
print "[Errno 11004] getaddrinfo failed with bad url: " + testUrl + "\n\n"
else:
print 'Reason: ', e.reason
elif hasattr(e, 'code'):
print 'Error code: ', e.code
elif hasattr(e, 'message'):
print e.message
pass
# If specified verb was POST
elif verb.strip().upper() == "POST":
# Check for None requestBody
if (requestBody is not None) or (requestBody.strip() is not ""):
data = urllib.urlencode(requestBody)
# Create request
req = urllib2.Request(testUrl, data)
# Check for header
if (header is not None) or (header.strip() is not ""):
req.add_header(header)
# YO YO THE BELOW CODE IS INCOMPLETE PLEASE FINISH
# Log request with URL and Data
print "Making request with URL: " + testUrl + " and data: THIS PART IS UNFINISHED PLEASE FINISH ME \n\n"
try:
response = urllib2.urlopen(req)
# If service returns 200 Okay
print "Connection to " + name + " Service successful. Returned with code " + str(response.code) + "\n\n"
# Log response
print "Response: " + response.read() + "\n\n"
# Handle exceptions
# If HTTP Error
except HTTPError as e:
if hasattr(e, 'code'):
print e.code
elif hasattr(e, 'message'):
print e.message
elif hasattr(e, 'reason'):
print name + ' failed to reach a server.'
print 'Reason: ', e.reason
pass
except URLError as e:
if hasattr(e, 'reason'):
print name + ' failed to reach a server.'
if str(e.reason) == "[Errno 11004] getaddrinfo failed":
print "[Errno 11004] getaddrinfo failed with bad url: " + url + "\n\n"
else:
print 'Reason: ', e.reason
elif hasattr(e, 'code'):
print 'Error code: ', e.code
elif hasattr(e, 'message'):
print e.message
pass
# Header non-existent in testService call
else:
print "Service header not provided. Exiting program"
sys.exit()
# Requesty Body not present in testService call
else:
print "Service request body not provided in code. Exiting program"
sys.exit()
# If specified verb is not supported (Currently only GET and POST are supported)
else:
print name + " Service written with HTTP verb other than GET or POST. Exiting program"
sys.exit()
else:
print "Service url not provided in code. Exiting program"
sys.exit()
else:
print "Service name not provided in code. Exiting program"
sys.exit()

From the documentation, add_header takes two arguments. You are calling it with one argument, a tuple with two values.
What you should do:
req.add_header(key, value)
What you are currently doing because you are getting the header as a tuple:
req.add_header((key, value,)) # aka passing a tuple with both arguments to the key parameter
You need to unpack the tuple:
req.add_header(header[0], header[1])
Or even better, using the splat operator (*):
req.add_header(*header) # Does the same thing as above
Also, you are using an empty string as the default argument for header, when when it is supplied it is a tuple. You should probably change the default value to a tuple or None.

Your header is a 2-tuple:
('Content-Type', 'application/json')
You're trying to do this:
req.add_header('Content-Type', 'application/json')
But in reality you're doing this:
req.add_header(('Content-Type', 'application/json'))
Notice that you're only passing one argument - a tuple - instead of two, a key and a value.
To fix, unpack your header when you pass it with the * (informally, 'splat') operator:
req.add_header(*header)

Take a look at the documentation: http://docs.python.org/2/library/urllib2.html#urllib2.Request.add_header.
While the function expects a key and a value, you're passing only a single object. Since you're calling this on the req object, that is the implicit "self" that's being passed as well.
You could call this function in two ways:
req.add_header(key, value)
urllib2.Request.add_header(req, key, value) # explicitly passing the reference instead of self
I'm not sure whether you're expecting the string you pass to be treated as the key or the value, but adding another paramater (or making the header parameter take a dict and then splitting appropriately in a for loop) should solve the issue. For example (with irrelevant code removed):
def testService(name, verb, url, header=None, requestBody=""):
if header is None:
header = {}
for key, value in header.iteritems():
req.add_header(key, value)

Printing Ascii Text from using Telnet in Python 3.0

So I'm unable to actually print all of the information that I see after issuing a "help" command. Do i need to change the length of the skt.receive()? Or is there a way to simply print all of the data that comes through? It seems like there has to be a way to account for a data that you want to print of an unknown length? Or am I approaching this in the wrong way.
Thanks.
#!/usr/bin/python
host = '192.168.1.50'
port = 23
msg = "help\r"
msg2 = "y\r"
import socket
import sys
import time
try:
skt = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except socket.error, e:
print("Error creating socket: %s" % e)
sys.exit(1)
try:
skt.connect((host,port))
except socket.gaierror, e:
print("Address-related error connecting to server: %s" % e)
sys.exit(1)
except socket.error, e:
print("Error connecting to socket: %s" % e)
time.sleep(15)
skt.connect((host,port))
sys.exit(1)
try:
print(skt.send(msg))
skt.send('help\r')
print("SEND: %s" % msg)
except socket.error, e:
print("Error sending data: %s" % e)
sys.exit(1)
while 1:
try:
buf = skt.recv(50000000000)
if(len(buf)):
print(buf)
if 'AMX' in buf:
print("Length buff")
if 'AMX' in buf:
print(skt.send(msg))
#print("first wait")
#print("RECV: %s" % buf)
#time.sleep(9)
#print("second wait")
sys.exit(1)
except socket.error, e:
print("Error receiving data: %s" % e)
sys.exit(1)
if not len(buf):
break
sys.stdout.write(buf)

Have you considered using telnetlib, rather than re-inventing the wheel? :)
Example:
import telnetlib
HOST = "192.168.1.50"
tn = telnetlib.Telnet(HOST)
tn.write("help\n")
print tn.read_all()

So the telnetlib def makes things easier and streamlines the process. No sense in reinventing the wheel.