We Keep Coding

Using Python and pyserial to run python scripts on a pico (RP2040)

My ultimate goal is to control a number of peripheral chips on a PCB from a GUI interface on a PC. To do so, my plan was to incorporate a RP2040 (and memory) on the PCB in order to hold all the python scripts and to program/monitor all the peripheral chips. Then, using a PC to interface with the RP2040, send commands over the serial port to execute specific python files on the pico.
I realize that is a bit confusing, so the attached block diagram should help.
Block diagram
Starting on the left of the block diagram, I have a PC running a tkinter GUI. I am currently running the tkinter gui in Thonny. (eventually i would like it to be an executable, but that is beyond the scope of this post) The gui has a number of buttons to choose which python scripts to run. The PC is connected to the PCB through the USB cable. The USB data lines are routed to the RP2040's USB inputs (pins 47,48). The memory on the PCB holds a number of python scrips that correspond to the buttons in the GUI. Ideally, pressing a button on the PC would execute the corresponding py file on the pcb.
What I've got working so far:
My real expertise lies in peripheral chips and PCB design, in this case the front end for a 2-18GHz transceiver, so bare with me if some of my python questions seem basic or misinformed. I've written and tested all the .py files on the pico's memory. To test those scripts I used Thonny to connect to my pico and simply ran (f5) the scripts with the peripherals connected to the right GPIOs. I was also able to get tkinter working and create functioning buttons that can execute commands. Using the pyserial module, I am also able to connected to the pico through the USB and write... strings. Not very useful, but a start.
import serial
ser = serial.Serial('COM3', 38400, timeout=1, parity=serial.PARITY_EVEN, rtscts=1)
s = ser.read(100) # read up to one hundred bytes or as much is in the buffer
print(ser.name) # check which port was really used
ser.write(b'ToggleLED.py') # write a string
ser.close() # close port
Remaining task: The final task I have been failing miserably at the past 2 days has been actually trying to execute the .py files located on the pico's memory through the serial port. My unexperienced/naïve notion was to simply send a string with the files name, obviously not correct. Any thoughts on how to execute those py files using that pyserial module?
BTW, if there is a better solution, please feel free to share! Perhaps the files should be located on the PC and i send 1 command at time?

I can't say anything about your serial problems until you clarify just what is running on the pi (see my comment: I'll update this answer when you do), but re 'is there a better way': possibly.
Since the Pi is running a full operating system, there are a few options. You are basically creating a network connection to the Pi. Whilst this can be done over serial (with the Pi, presumably, acting as a fake USB serial device), it can also be done more conventionally over wifi or ethernet. Lastly, you could host your interface on the Pi and interact with it in a webbrowser, cutting the second computer out of the picture. Exactly which option you decide to take is up to you, and is really off topic here (though it might be on topic elsewhere on SE).
Sending commands to the pi and having it run scripts is Remote Procedure Call. You might want to lookup some of the protocols (like JSON-RPC) generally used to do that, but the basic approach will have code running on the pi:
def do_something():
pass
def do_something_else():
pass
functions = {"something": do_something, "something_else": do_something_else}
while True:
cmd = get_input() # blocks until input comes
if cmd in functions:
reply(f"Running {cmd}")
output = functions[cmd]()
reply(f"{cmd} returned with output {output}")
else:
reply(f"Invalid command {cmd}")
This is schematic: exactly what get_input() on the pi is will depend on how you end up connecting, and what protocol (if any) you end up using. Notice that I have built in confirmation: you want to know if things work or fail.
Whilst you could store these commands in separate scripts and invoke them, if they are just python scripts there is no reason not to call the functions directly from the code running on the pi.

I've seen two different solutions to the question:
With the Pi pico running the REPL (python shell), text strings can simply be sent from a python program running on the host (Windows or Raspberry Pi, etc., connected to the USB com port, with Thonny window closed), and sent to the pico as python commands. This approach is detailed in:
https://blog.rareschool.com/2021/01/controlling-raspberry-pi-pico-using.html
On the pico side, one just need to define a number of functions that will execute from the REPL. E.g.,:
from machine import Pin
# use onboard LED which is controlled by Pin 25
led = Pin(25, Pin.OUT)
# Turn the LED on
def on():
led.value(1)
# Turn the LED off
def off():
led.value(0)
Any string sent to the pico, such as
on()
will be executed accordingly.
In the 2nd approach, the pico will read each line from the USB serial port and parse the line into command and additional parameters, to be compared against a list of predefined commands and executed accordingly. I did that using the Arduino Due, and I'm in the process of porting it to the pico (= no code to show yet :-().

Using Python to wait for a specific Serial Input before sending a serial output

Here's a little background on this project. Im trying to write a script that will run at the startup of the raspberry pi and run continuously as long as the pi is on. The script needs to output a text file over the rs232 port, but only when it receives chr17 (ascii) from a cnc machine. chr17 is also the xon code for software flow control I believe.
So as soon as the machine requests an input, the raspberry pi will output whatever file the script is pointed to. When the machine requests an input it outputs chr17. And when it gets the last line of the file it outputs chr18 to signal it is finished.
As far as the memory on the pi, I don't know the most efficient way of executing this script, I think a while loop might take up too much memory on the pi. I'm not sure though.
If anyone can help me come up with this script that would be very much appreciated.
As of now this code works if i execute it after the machine has requested an input. But i need it to run continuously and then as soon as it sees the machine request an input, it should output the file.
Here's what I have, feel free to modify whatever you need or completely start over if its not helpful, lol.
import serial
port = serial.Serial("/dev/ttyAMA0", baudrate=4800, bytesize=7, parity='E')
machinenumber = "BND-2"
cncfile = "/home/pi/myNAS/myShare/e-matix/Machines/%s/Upload/9937.cnc" % machinenumber
try:
port.open()
except Exception, e:
print "error opening serial port:" + str(e)
if port.isOpen():
port.write(open(cncfile, "rb").read())
print ("worked")
print (open(cncfile, "rb").read())
else:
print ("cannot open port")
time.sleep(5)
port.close()

1) put script in crontab, set so it starts at boot time:
#reboot python /path/to/script.py
http://www.cyberciti.biz/faq/linux-execute-cron-job-after-system-reboot/
2) put above script in a while loop, so it runs then starts again

Program execution stopping on atml atmega16 microcontroller upon terminal connection

I am attempting to receive data from the qu-bot at http://www.qu-bot.com. The robot has an ATML atmega16 microcontroller. I have written a program that runs on the robot which outputs data to its serial port. The program however stops whenever I connect to the controller. I tested this by adding a beep statement. The robot beeps as long as the program is running. The beeping stops when I connect to the robot. I tried qu-bo support and they suggested disabling the dtr flag on the serial port. I did that but no joy.
Is there anything else I can try?
[start of code running on the qu-bot]
Note:
This is written in some kind of proprietary variant of C which they call quick c.
// This code displays the uart functions.
int main(void)
{
INIT();
UART_INIT(57600);
UART_PRINT("HELLO!!\n");
DELAYS(1);
UART_PRINT("MY NAME IS QU-BOT.\n");
DELAYS(1);
UART_PRINT("HELLO!!\n");
UART_PRINT("YOU ARE USING UART SAMPLE CODES.\n");
while(1)
{
UART_PRINT("test\n");
BEEP();
DELAYS(60);
}
}
now for my python serial port reading program. I have tried this program both on raspbian and on windows 7 64bit. I am pasting the windows version. The raspbian version has a different name for the linux.
import serial
import time
ser=serial.Serial()
ser.port=8
ser.baudrate=57600
ser.setDsrDtr(False)
print 'initialized'
flag = ser.isOpen()
if flag:
print 'port already open.'
pass
else:
ser.open() # opening the port 'ser' that was just created to receive data
time.sleep(0.5)
print 'ready to read'
print ser
ser.write('a')
s=ser.read(4)
print s
ser.close()
Pranav
P.S. I have consulted the following links amongst others.
<https://learn.sparkfun.com/tutorials/terminal-basics/all>
<http://www.plainlystated.com/2013/06/raspberry-pi-serial-console/>
<http://elinux.org/RPi_Serial_Connection>
<https://learn.sparkfun.com/tutorials/terminal-basics/all>

Based on my experience in serial communications with microcontrollers the most likely cause of this problem is that when you connect the serial cable to the robot it causes a phantom byte (due to electrical noise that happens when you make the connection) to look like it's coming from the controller - either this or the controller is sending a legitimate byte to the robot. In either case it is likely that the arrival of a byte at the robot's serial port (even if it was only electrical noise mistaken to be a data byte - this is a very common occurrence) caused the robot's microcontroller to invoke the UART receive interrupt. If you don't have an appropriate UART receive handler (ISR - Interrupt Service Routine) written and linked into the correct interrupt vector then your robot's microcontroller can go off into "deep space" upon the detection of the first incoming serial data byte - and make your robot appear to "hang". If you intend to do "polled" serial communications (your code manually checks for received bytes in its main loop) instead of interrupt-driven (hardware detection of an incoming byte causes your UART Rx ISR to be invoked) then all you have to do is to disable UART interrupts and your problem should go away.

Echo Program in between Arduino and Python

I want to send some data to an Arduino through pyserial in Python. All I want to the Arduino to do is read the variable length string data from the serial port, and write it back so that Python can read it. Since I've been unable to do that, the code below only has Python sending on character. Here's the Python code:
import serial
import sys
import pywapi
import time
def main():
ser = serial.Serial(3, 9600, timeout=1)
print "Conn established"
print "Sending: %s" % "z".__repr__()
print ser.write('z'.encode("ascii"))
time.sleep(2)
print "Received: %s" % ser.read(10).__repr__()
ser.close()
Here's the Arduino code:
void setup(){
analogReference(DEFAULT);
Serial.begin(9600);
}
void loop(){
if(Serial.available() > 0)
Serial.println("x");
while(Serial.available() > 0){
Serial.print(Serial.read(), BYTE);
}
}
The output:
Conn established
Sending: 'z'
1
Received: ''
I know the code for the Arduino works because it works when data is being sent from the Arduino terminal. However, the moment I try to send anything from Python it fails. I've been struggling with this all day. Any help would be greatly appreciated.

Try increasing or removing the timeout, and set read's size to 1. You may also want to increase the sleep delay, or even implement a simple read loop.
Something like:
try:
while True:
data = ser.read(1).__repr__()
if data:
print "Received: %s." % data
else:
print "Looping."
except KeyboardInterrupt:
print "Done."
except:
raise
finally:
ser.close()
print "Closed port."
Then just use ctrl-c to stop it.

I would recommend verifying the two parts independently, using a separate serial port and serial comms software on the PC.
E.g. if your PC has two serial ports, then use a null-modem (loopback) cable to connect them. Or use com0com to make a pair of linked virtual serial ports. Run your Python software on one serial port, and a terminal program (Hyperterminal or RealTerm) on the other serial port. Manually verify the Python program's operation that way.
Then, connect your PC directly to the Arduino as usual, and use the terminal software to manually verify the Arduino software operation.
That process will allow you to narrow down the problem. Once you've verified them both, they should work well together.
Serial Port Monitor
Another method you can use is software that hooks into the PC's serial port driver, and allows you to monitor traffic on the serial port. I've used the Free Serial Port Monitor software from HHD Software in the past, and it worked well for our purposes. It allows you to monitor any of the PC's serial ports, and shows you a log (hex and text) of the serial data going over the port in both directions.

Do you need to flush the sent character out of any held serial buffer?
It may be that your character is not actually leaving the COM port and arriving at the Arduino. When you test this with the Arduino Terminal (I assume you mean the UI terminal in the development environment) you are actually sending your string + a carriage return I think, not just the character. (i.e. do you hit return after you type 'z' in your test?)
Try ser.flush() or perhaps also send a \r character. From your testing the Arduino works just fine, it's the python program that doesn't seem to be sending anything.

The reason you may have to send twice is that, if you're connecting via the USB, the first serial connection will reset the Arduino.

Pyserial problem with Arduino - works with the Python shell but not in a program

All right, so I am positive my Arduino circuit is correct and the code for it. I know this because when I use the serial monitor built into the Arduino IDE and send 'H' an LED lights up, when I send 'L' that LED turns off.
Now I made a Python program
import serial
ser = serial.Serial("COM4",9600)
ser.write("H")
When I run the code the LED blinks on for a second then goes back off.
However when I do each of these lines separately in the shell it works just like it is supposed to.
Any ideas?

When you open the serial port, this causes the Arduino to reset. Since the Arduino takes some time to bootup, all the input goes to the bitbucket (or probably to the bootloader which does god knows what with it). If you insert a sleep, you wait for the Arduino to come up so your serial code. This is why it works interactively; you were waiting the 1.5 seconds needed for the software to come up.
I confirmed that opening the serial port resets my Arduino Uno; I flashed a program which will blink the LED from the setup() routine -- calling open("/dev/ttyACM0") was sufficient to trigger the reset. This is IMHO a confusing and undocumented wrinkle in the serial support.

I had the same problem and it works if I add a delay of about 2 seconds from opening the serial connection to writing on it, 1 second was not enough.

Just to make it a bit more clear I'll modify the code so everyone can see what needs to be added!
import serial
import time
ser = serial.Serial("COM4",9600)
time.sleep(3)
ser.write("H")
Adding in a sleep statment helps to let the serial open up without any problems!

Assuming you are using an Arduino Uno
The USB port and the Uno serial bus exposed on pins 1 and 0 share the same RX/TX lines. I suggest getting a USB to TTL adapter like the one here so that you can communicate to the Arduino without using the USB port. The Arduino IDE has its own method for disengaging from the USB driver such that a virtual serial port can be created. Have your Ardunio use SoftwareSerial instead.
Here is an example I found on the internet where somebody had clashing bus issues.