We Keep Coding

JRC JJ1000 + dronekit >> ERROR:dronekit.mavlink:Exception in MAVLink input loop

I am trying to connect JRC JJ1000 drone using dronekit + python.
when executing the connect command:
dronekit.connect('com3', baud=115200, heartbeat_timeout=30)
I am getting the following error:
ERROR:dronekit.mavlink:Exception in MAVLink input loop
Traceback (most recent call last):
File "C:\Python37\lib\site-packages\dronekit\mavlink.py", line 211, in mavlink_thread_in
fn(self)
File "C:\Python37\lib\site-packages\dronekit\__init__.py", line 1371, in listener
self._heartbeat_error)
dronekit.APIException: No heartbeat in 5 seconds, aborting.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python37\lib\site-packages\dronekit\__init__.py", line 3166, in connect
vehicle.initialize(rate=rate, heartbeat_timeout=heartbeat_timeout)
File "C:\Python37\lib\site-packages\dronekit\__init__.py", line 2275, in initialize
raise APIException('Timeout in initializing connection.')
dronekit.APIException: Timeout in initializing connection.
I left no store unturned but no progress. I also tried both Python 2.7 and 3.7 with same result.

I have been getting the same error. I am using some custom code in a docker container to run simulations with dronekit and ArduPilot. The error is intermittent. So far it seems like the only way to get the error to stop is to:
Close all docker containers.
Open windows task manager and wait for vmmem to lower memory usage (5-10m).
Try again.
Maybe the problems are related somehow. To me it seems like the connection might be in use by a previous instance and it was not properly close. Since waiting for vmmem to free up resources appears to fix it. I would prefer a better solution if anyone finds one!
We are using python code like this to connect:
from dronekit import connect
...
# try to connect 5 times
while connected == False and fails < 5:
try:
vehicle = connect(connection_string, wait_ready=True)
except:
fails += 1
time.sleep(3)
print("Failed to connect to local mavlink sleeping for 3 seconds")
else:
connected = True
Where the connection_string is of the form:
"tcp:host:port"
Also, the documentation states "If the baud rate is not set correctly, connect may fail with a timeout error. It is best to set the baud rate explicitly." Are you sure that you have the correct baud rate?

Establishing Serial Communication b/t Beaglebone Black (RevC, Deb) and Arduino Mega

I'm trying to establish serial communications between a Beaglebone Black and Arduino Mega, but I'm having issues getting this to work, particularly on the Beagle's side. I keep getting this error message:
Traceback (most recent call last):
File "/var/lib/cloud9/IBID 2.0 /data stream test (1).py", line 35, in <module>
sensorValue += ser.read('UART1') #add more for more pins
File "/usr/lib/python2.7/dist-packages/serial/serialposix.py", line 449, in read
buf = os.read(self.fd, size-len(read))
TypeError: unsupported operand type(s) for -: 'str' and 'int'
in response to trying to run this code:
import Adafruit_BBIO.UART as UART
import serial
UART.setup('UART1')
name = raw_input('name your file: ')
final_name = os.path.join(/sequence of files and folders/, name + '.txt')
data = open(name + '.txt', 'a+')
ser = serial.Serial(port = "/dev/ttyO1", baudrate=9600, timeout = 1000)
sensorValue = 0
header = 'Sensor 1 output'
data.write(str(header))
data.write(str('\n'))
while True:
ser.open()
sensorValue += ser.read('UART1')
data.write(sensorValue)
I'm using the cloud 9 IDE to program the Beaglebone to receive incoming data from a sensor hooked up to the Arduino (via logic converter.) The error code is mystifying me, to say the least. The links it provides aren't leading me to anything (no files found) in the IDE. I haven't been able to find much [on how to resolve this error.]

On this line
sensorValue += ser.read('UART1')
you are calling serial.Serial.read(size=1) with type str as an argument. The method takes an int.

UART in python on raspberry pi doesn't receive data

I am trying to implement data sending in Python 3 on a raspberryPi (as a part of a bigger project) and cannot receive data when I connect the Rx and Tx pins. Regardless of using Python 2 or 3 (as far as I understand this API allows Python 3 programming) I either get Received: b'\n' response or such an exception:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/serial/serialposix.py", line 471, in write
n = os.write(self.fd, d)
OSError: [Errno 5] Input/output error
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "./uart.py", line 12, in <module>
port.write(bytearray(input_data, 'utf-8'))
File "/usr/lib/python3/dist-packages/serial/serialposix.py", line 485, in write
raise SerialException('write failed: %s' % (v,))
serial.serialutil.SerialException: write failed: [Errno 5] Input/output error
I can't write anything except from buffered reader though.
The code I've made is here:
#!/usr/bin/env python3
import serial
port = serial.Serial("/dev/ttyAMA0", baudrate=9600)
while True:
input_data = input("Say sth: ")
if input_data != 'exit':
port.write(bytearray(input_data, 'utf-8'))
print('Sent: {0}'.format(bytearray(input_data, 'ASCII')))
output_data = port.readline()
print('Received: {0}\n'.format(str(output_data)))
else:
break
port.close()
I want to use ASCII encoding since it will be further connected to an microcontroller with code in C. I've also checked whether any data is written into the buffer (and it is), I've tried out laying the programme to sleep for a second after sending data, I've tried using port.read(port.inWaiting()) and port.read(in_waiting) (no attribute found in the latter case) and nothing seems to be helpful.
I've also tried this example; I am sure correct pins are connected and I have updated and upgraded my raspbian by using sudo apt-get update and sudo apt-get upgrade and when I typed sudo apt-get install python3-serial I was told that I already have newest version installed.

I am posting this answear to close the topic and to help anyone who might come across similar difficulties.
Since the processor is of different architecture trying to set up ports with setserial was pointles, however this was exactly the problem.
pi#raspberrypi ~ $ sudo setserial -g /dev/ttyAMA0
/dev/ttyAMA0, UART: undefined, Port: 0x0000, IRQ: 83
The answear I found here solved all the problems.

pyserial error - cannot open port

I have seen simple code in stackoverflow using pyserial in USB ports with Python 3.3 but I can't get this to work on my new installation of pyserial 2.7 [in Windows 7, 64 bit, with 3 USB ports]. Installation of pyserial went smoothly, I can import without error and methods are recognized in the Pyscripter IDE which boosts confidence in a good installation, however:
The code stripped down to its error producing essentials is:
import serial
def main():
ser = serial.Serial(port='COM2')
ser.close()
if __name__ == '__main__':
main
From this I receive a dialog box with the error "SerialException: could not open port 'COM2': FileNotFoundError(2,'The system cannot find the file specified.',None,2)"
The Traceback states:
*** Remote Interpreter Reinitialized ***
>>>
Traceback (most recent call last):
File "<string>", line 420, in run_nodebug
File "C:\Python33\Lib\site-packages\scanport2.py", line 19, in <module>
main()
File "C:\Python33\Lib\site-packages\scanport2.py", line 15, in main
ser = serial.Serial(port='COM2')
File "C:\Python33\Lib\site-packages\serial\serialwin32.py", line 38, in __init__
SerialBase.__init__(self, *args, **kwargs)
File "C:\Python33\Lib\site-packages\serial\serialutil.py", line 282, in __init__
self.open()
File "C:\Python33\Lib\site-packages\serial\serialwin32.py", line 66, in open
raise SerialException("could not open port %r: %r" % (self.portstr, ctypes.WinError()))
serial.serialutil.SerialException: could not open port 'COM2': FileNotFoundError(2, 'The system cannot find the file specified.', None, 2)
And the code segment in the imported module which raises the SerialException is:
# the "\\.\COMx" format is required for devices other than COM1-COM8
# not all versions of windows seem to support this properly
# so that the first few ports are used with the DOS device name
port = self.portstr
try:
if port.upper().startswith('COM') and int(port[3:]) > 8:
port = '\\\\.\\' + port
except ValueError:
# for like COMnotanumber
pass
self.hComPort = win32.CreateFile(port,
win32.GENERIC_READ | win32.GENERIC_WRITE,
0, # exclusive access
None, # no security
win32.OPEN_EXISTING,
win32.FILE_ATTRIBUTE_NORMAL | win32.FILE_FLAG_OVERLAPPED,
0)
if self.hComPort == win32.INVALID_HANDLE_VALUE:
self.hComPort = None # 'cause __del__ is called anyway
raise SerialException("could not open port %r: %r" % (self.portstr, ctypes.WinError()))
I do have an active device connected to COM2 as identified in the Windows device manager. I also have tried scanning all the ports, but the code stops on the first use of serial.Serial
This appears that something may be going on with win32?
I am a newbie for interfacing Python with hardware.

I would try the following:
Unplug and replug the device.
Reboot.
Run WinObj and look in the GLOBAL?? folder; you should see COM2 there as a symbolic link to something more driver-specific.
What type of device do you have connected to COM2? If it uses usbser.sys, you might have better luck substituting \\.\USBSER000 for COM2 in your code, but remember to escape those backslashes properly.
On some machines there are strange problems with low COM port numbers that I can't explain. Try reassigning the device to COM6 in the Device Manager.

It looks like the pyserial download page only contains links for 32 bit python? This unofficial page seems to have links for 64 bit installations, however be cautious installing from unknown sources.
This answer also suggests installing it using pip: https://stackoverflow.com/a/8491164/66349

Python script freezes infinitely during a socket connection

I have a simple python script that updates that statuses of justin.tv streams in my database. It's a Django based web application. This script worked perfectly before I moved it to my production server, but now it has issues with timing out or freezing. I've solved the time out problem by adding try/except blocks and making the script retry, but I still can't figure out the freezing problem.
I know it freezes on the line streamOnline = manager.getStreamOnline(stream.name, LOG). That's the same point where the socket.timeout exception occurs. Some times however, it just locks up for ever. I just can't picture a scenario where python would freeze infinitely. Here is the code for the script that freezes. I'm linking website.networkmanagers below, as well as oauth and the justin.tv python library that I'm using.
import sys, os, socket
LOG = False
def updateStreamInfo():
# Set necessary paths
honstreams = os.path.realpath(os.path.dirname(__file__) + "../../../")
sys.path.append(honstreams)
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
# Import necessary moduels
from website.models import Stream, StreamInfo
from website.networkmanagers import get_manager, \
NetworkManagerReturnedErrorException
# Get all streams
streams = Stream.objects.all()
try:
# Loop through them
for stream in streams:
skipstream = False
print 'Checking %s...' % stream.name,
# Get the appropriate network manager and
manager = get_manager(stream.network.name)
# Try to get stream status up to 3 times
for i in xrange(3):
try:
streamOnline = manager.getStreamOnline(stream.name, LOG)
break
except socket.error as e:
code, message = e
# Retry up to 3 times
print 'Error: %s. Retrying...'
# If this stream should be skipped
if(skipstream):
print 'Can\'t connect! Skipping %s' % stream.name
continue
# Skip if status has not changed
if streamOnline == stream.online:
print 'Skipping %s because the status has not changed' % \
stream.name
continue
# Save status
stream.online = streamOnline
stream.save()
print 'Set %s to %s' % (stream.name, streamOnline)
except NetworkManagerReturnedErrorException as e:
print 'Stopped the status update loop:', e
if(__name__ == "__main__"):
if(len(sys.argv) > 1 and sys.argv[1] == "log"):
LOG = True
if(LOG): print "Logging enabled"
updateStreamInfo()
networkmanagers.py
oauth.py
JtvClient.py
Example of the script freezing
foo#bar:/.../honstreams/honstreams# python website/scripts/updateStreamStatus.py
Checking angrytestie... Skipping angrytestie because the status has not changed
Checking chustream... Skipping chustream because the status has not changed
Checking cilantrogamer... Skipping cilantrogamer because the status has not changed
| <- caret sits here blinking infinitely
Interesting update
Every time it freezes and I send a keyboard interrupt, it's on the same line in socket.py:
root#husta:/home/honstreams/honstreams# python website/scripts/updateStreamStatus.py
Checking angrytestie... Skipping angrytestie because the status has not changed
Checking chustream... Skipping chustream because the status has not changed
^CChecking cilantrogamer...
Traceback (most recent call last):
File "website/scripts/updateStreamStatus.py", line 64, in <module>
updateStreamInfo()
File "website/scripts/updateStreamStatus.py", line 31, in updateStreamInfo
streamOnline = manager.getStreamOnline(stream.name, LOG)
File "/home/honstreams/honstreams/website/networkmanagers.py", line 47, in getStreamOnline
return self.getChannelLive(channelName, log)
File "/home/honstreams/honstreams/website/networkmanagers.py", line 65, in getChannelLive
response = client.get('/stream/list.json?channel=%s' % channelName)
File "/home/honstreams/honstreams/website/JtvClient.py", line 51, in get
return self._send_request(request, token)
File "/home/honstreams/honstreams/website/JtvClient.py", line 90, in _send_request
return conn.getresponse()
File "/usr/lib/python2.6/httplib.py", line 986, in getresponse
response.begin()
File "/usr/lib/python2.6/httplib.py", line 391, in begin
version, status, reason = self._read_status()
File "/usr/lib/python2.6/httplib.py", line 349, in _read_status
line = self.fp.readline()
File "/usr/lib/python2.6/socket.py", line 397, in readline
data = recv(1)
KeyboardInterrupt
Any thoughts?

Have you tried using another application to open that connection? Given that it's an issue in production, perhaps you don't have some firewall issues.

Down in JtvClient.py it uses httplib to handle the connection. Have you tried changing this to use httplib2 instead?
Other than that stab in the dark, I would add a lot of logging statements to this code in order to track what actually happens and where it gets stuck. Then I would make sure that the point where it gets stuck can timeout on the socket (which usually involves either monkeypatching or forking the codebase) so that stuff fails instead of hanging.
You said:
I know it freezes on the line streamOnline = manager.getStreamOnline(stream.name, LOG). That's the same point where the socket.timeout exception occurs.
Wrong. It doesn't freeze on that line because that line is a function call which calls lots of other functions through several levels of other modules. So you do not yet know where the program freezes. Also, that line is NOT the point where the socket timeout occurs. The socket timeout will only occur on a low level socket operation like select or recv which is being called several times in the chain of activity triggered by getStreamOnline.
You need to trace your code in a debugger or add print statements to track down exactly where the hang occurs. It could possibly be an infinite loop in Python but is more likely to be a low-level call to an OS networking function. Until you find the source of the error, you can't do anything.
P.S. the keyboard interrupt is a reasonable clue that the problem is around line 90 in JtvClient.py, so put in some print statements and find out what happens. There may be a stupid loop in there that keeps calling getresponse, or you may be calling it with bad parameters or maybe the network server really is borked. Narrow it down to fewer possibilities.

It turns out this HTTP connection isn't passed a timeout in jtvClient.py
def _get_conn(self):
return httplib.HTTPConnection("%s:%d" % (self.host, self.port))
Changed the last line to
return httplib.HTTPConnection("%s:%d" % (self.host, self.port), timeout=10)
Which solved it