I have a custom python script that depends on MinimalModbus and the pySerial library. I am trying to deploy it to a router which runs a python interpreter.
MinimalModbus is just a single .py file which is trivial to deploy. However, the pySerial library appears to be much more robust. It looks like several python files that work together to "automatically select the appropriate backend".
Does one have to "install" pySerial in order to use it? Or is there some way to extract just the pertinent files/dependencies for a given OS?
I don't know what all is performed when you run pySerial's setup.py (e.g. files copied?). I don't know if it will work for this particular type of deployment. I was hoping to just include specific files.
Any help will be appreciated.
We are using Python version 2.6.
Update:
I basically took the "installed" files from the /site-packages/serial folder on my development box and uploaded them to the device. This got me a bit further; however, I am now getting the following error:
Line ~273 of serialposix.py , it's calling:
self.fd = os.open(self.portstr, os.O_RDWR|os.O_NOCTTY|os.O_NONBLOCK)
Why would it not be able to find the os.open routine?
Update 2:
Further simplifying the problem, my script now consists of something as simple as the following, and it still fails with the same error:
import os
serfd = os.open("/com/0", os.O_RDWR | os.O_NONBLOCK)
Under Python Standard Modules with Digi-Specific behavior, they make the following comment about the os module:
Use of the os module in Digi devices is currently very limited. The primary purpose in exposing it is to allow access to the serial ports, which are presented as nodes in the file system. Serial ports are available as files with the path in the form /com/0 with the zero replaced by the zero-based index of the serial port to control.
In addition, both of their sample applications use the os.open routine for serial communication.
I would have expected to maybe see an error such as: OSError: [Errno 2] No such file or directory: '/com/0', but this is not the case. Python can't even locate the os.open routine.
Would you expect the os.py file to have a def open(...) routine defined?
After opening a support case with the manufacturer of this router, it turns out that the os.open function is not supported on this device. However, the device does support io.open which I believe is similar. More importantly, I learned that the manufacturer provides their own "pyserial" implementation specifically designed to work on the device's operating system. Internally, it looks like they've switched out the calls to use io.open equivalent.
having had a look through the source files, i believe that it is entirely python based, and when it installs it copies the files across to the site-packages folder.
having said that it appears that you would need several of the source files in order for it to work, and if you were simply copying them across you may need to modify their imports to ensure they work properly.
eg for linux you would need the serialposix.py file and the serialutil.py
you may need more than just this, but i have only had a quick look through.
but at the top of serialposix.py there is a line:
from serial.serialutil import *
this would need to be changed to:
from serialutil import *
and there may be other such changes to make.
but ultimately this seems to use ctypes to do the hard work talking to the underlying OS, so you should be able to make it work.
UPDATE:
to explain why it is calling os.open, on most platforms serial ports are treated almost like files, in tthat the return a "file like" handle, the idea behind pyserial is to abstract operating system level differences away to create a single easy to interface to these, but ultimately it is still treated as a file like handle by the OS.
just for clarification could you let us know what version of pyserial you are using? as the line number you quoted and what i'm looking at don't match.
i suspect the main reason you are having difficulty is the zip nature of the python deployment you are using, but i do find it hard to believe that OS is not included in it, have you checked the OS python file in the ZIP or is it a compiled python file?
UPDATE 2:
haveing looked at the documentation for the distro you are using i would suggest you have a read through the following:
Python Standard Modules with Digi-Specific Behavior
where is states that some funtionality is limited on these devices, it also gives a method by which you can test if it is supported by telnet/SSH into the device and trying it on the python command line.
also, while it is anot as neat and easy to use as the pyserial module, i suggest you give this a read too:
Digi Serial Port Access
Related
I have encountered a rather funny situation: I work in a big scientific collaboration whose major software package is based on C++ and python (2.7.15 still). This collaboration also has multiple servers (SL6) to run the framework on. Since I joined the collaboration recently, I received instructions on how to set up the software and run it. All works perfectly on the server. Now, there are reasons not to connect to the server to do simple tasks or code development, instead it is preferrable to do these kind of things on your local laptop. Thus, I set up a virtual machine (docker) according to a recipe I received, installed a couple of things (fuse, cvmfs, docker images, etc.) and in this way managed to connect my MacBook (OSX 10.14.2) to the server where some of the libraries need to be sourced in order for the software to be compiled and run. And after 2h it does compile! So far so good..
Now comes the fun part: you run the software by executing a specific python script which is fed as argument another python script. Not funny yet. But somewhere in this big list of python scripts sourcing one another, there is a very simple task:
import logging
variable = logging.DEBUG
This is written inside a script that is called Logging.py. So the script and library only are different by the first letter: l or L. On the server, this runs perfectly smooth. On my local VM set up, I get the error
AttributeError: 'module' object has no attribute 'DEBUG'
I checked the python versions (which python) and the location of the logging library (print logging.__file__), and in both set ups I get the same result for both commands. So the same python version is run, and the same logging library is sourced but in one case there is a mix up with the name of the file that sources the library.
So I am wondering, if there is some "convention file" (like a .vimrc for vi) sourced somewhere where this issue could be resolved by setting some tolerance parameter to some other value...?
Thanks a lot for the help!
conni
as others have said, OSX treats names as case-insensitive by default, so the Python bundled logging module will be confused with your Logging.py file. I'd suggest the better fix would be to get the Logging.py file renamed, as this would improve compatibility of the code base. otherwise, you could create a "Case-sensitive" APFS file system using "Disk Utility"
if you go with creating a file system, I'd suggest not changing the root/system partition to case-sensitive as this will break various programs in subtle ways. you could either repartition your disk and create a case-sensitive filesystem, or create an "Image" (this might be slower, not sure how much) and work in there. Just make sure you pick the "APFS (Case-sensitive)" format when creating the filesystem!
print "Deleting ", temp_dir
try:
os.chmod(temp_dir, stat.S_IRWXU)
shutil.rmtree(temp_dir)
except Exception, e:
print(e)
raise
throws error "[Error 5] Access is denied: 'c:\temp\metabuild\common\build\build.cmd' "
owner has only read permission on file build.cmd, I added chmod line as shown in the code to change it, but it didn't do anything, permissions are still same, any help? thanks.
On Windows, there are a wide range of reasons you may not be able to delete a file or directory even though os.stat says you have write access to it:
You may not have write access to the parent directory.
The filesystem may be read-only.
The file may be currently open in some other process which didn't use the FILE_SHARE_DELETE sharing mode (which most programs don't—in particular, any program using the POSIX-style APIs that Python uses for files will never use this sharing mode).
The actual ACLs on the file may be too complex to be represented as POSIX-style permissions, so you don't actually have permission to delete the file even though both it and its parent directly show up as +w in Unix-style functions like stat.
Unless it's the first one, there's really no way to figure out exactly what's going on via the POSIX-y APIs that Python uses; it doesn't understand Windows sharing modes, ACLs, or anything else that could be a problem. You will need to use tools that work via the native Win32 APIs. You can do this from Python via pywin32 (or ctypes or similar to the DLLs, if you really prefer), but you probably want to figure out the problem with Explorer or the various Sysinternals tools first.
If you don't know all the crazy ways in which Win32 is not like POSIX, read Creating and Opening Files at MSDN.
Also, if you're running this code under a Cygwin Python rather than a native Python, you will have to switch; inside Cygwin, you can only access the Cygwin layer, which emulates a real POSIX environment on top of Windows, and doesn't give you access to the Win32 stuff underneath.
Anyway, once you know what the problem is, most likely the only way to work around it in your code will be to again drop to the Win32 APIs via pywin32. Or, if you're trying to do something like override sharing modes, you have to drop even lower, to the NT layer, which you'll have to access via ctypes. And since most of the stuff you need isn't really documented (for the parts that are documented, search for "driver development") you'll probably have to dig through the C source to the Sysinternals tools to figure out how to force another process to close a file, or delete the file even though the other process has it closed.
Here's the scenario:
I am writing a program that will run inside a Telit HE910 GSM module, which has an on-board python interpreter.
I would like my program to be able to read and store some parameters inside a configuration file onboard the GSM module.
The file may occasionally be transferred into a separate machine for viewing and changing, so it should be both machine-readable and writeable on multiple platforms.
So far, there are multiple solutions for this issue. Here's the kicker though:
The GSM module does not have any python modules for parsing / writing configuration files (so I can't simply use import yaml, import json, import configparser, or even import csv)
The GSM module does not allow the creation of subdirectories. From my limited understanding, this prevents me from simply dumping the contents of say, the PyYAML python module into the GSM module and calling it from my program.
I found a similar question here, but I don't even know where in the GSM module's filesystem I am. import os doesn't seem to work, which is strange (contrary to documentation).
I know I can use a Python file to store some read-only configurations, but I also want to be able to write to the config file (redesigning the system to avoid this is really undesirable).
I think my best bet so far seems to be to write a simple csv parser / writer myself, unless someone has a better idea (or know how to utilise Python modules without any subdirectories).
PS: The documentation below has a list of supported modules. None of the config-related modules seem to be available however.
GSM Module homepage: http://www.telit.com/en/products/umts-hsdpa.php?p_ac=show&p=108
Python interpreter manual : http://www.telit.com/module/infopool/download.php?id=4378
EDIT: I should have mentioned, the configuration file needs to be readable / writeable from a c# .NET application, not another python interpreter on the desktop.
You have _ast (§5.2.36), so it should be possible to reimplement ast.literal_eval(). At that point reading and writing becomes mostly trivial.
I wouldn't recommend it in normal Python usage, but a possible option would be the marshal module mentioned in 5.2.34 of the manual you posted, and description/limitations/warnings here: http://docs.python.org/library/marshal.html
Why don't you just open a txt file and store all info as a text file? I read the Easy Script Manual for your module, there is a posix module which supports methods like open, close, unlink.
Recently I have become a fan of storing various settings used for my testing scripts in the OSX defaults system as it allows me to keep various scripts in git and push them to github without worrying about leaving passwords/settings/etc hardcoded into the script.
When writing a shell script using simple bash commands, it is easy enough to use backticks to call the defaults binary to read the preferences and if there is an error reading the preference, the script stops execution and you can see the error and fix it. When I try to do a similar thing in Python or Ruby it tends to be a little more annoying since you have to do additional work to check the return code of defaults to see if there is an error.
I have been attempting to search via google off and on for a library to use the OSX defaults system which ends up being somewhat difficult when "defaults" is part of your query string.
I thought of trying to read the plist files directly but it seems like the plist libraries I have found (such as the built in python one) are only able to read the XML ones (not the binary ones) which is a problem if I ever set anything with the defaults program since it will convert it back to a binary plist.
Recently while trying another search for a Python library I changed the search terms to something using something like NSUserDefaults (I have now forgotten the exact term) I found a Python library called userdefaults but it was developed for an older version of OSX (10.2) with an older version of Python (2.3) and I have not had much luck in getting it to compile on OSX 10.6 and Python 2.6
Ideally I would like to find a library that would make it easy to read from (and as a bonus write to) the OSX defaults system in a way similar to the following python psudo code.
from some.library.defaults import defaults
settings = defaults('com.example.app')
print settings['setting_key']
Since I am also starting to use Ruby more, I would also like to find a Ruby library with similar functionality.
It may be that I have to eventually just 'give up' and write my own simple library around the defaults binary but I thought it wouldn't hurt to try to query others to see if there was an existing solution.
You´ll want to use PyObjC: have a look at this article at mactech.com (in specific: scroll down to "Accessing plists Via Python"). And this article from oreilly on PyObjC.
Run this, for example:
from Foundation import *
standardUserDefaults = NSUserDefaults.standardUserDefaults()
persistentDomains = standardUserDefaults.persistentDomainNames()
persistentDomains.objectAtIndex_(14)
aDomain = standardUserDefaults.persistentDomainForName_(persistentDomains[14])
aDomain.keys()
This is something that I think would be very useful. Basically, I'd like there to be a way to edit Python source programmatically without requiring human intervention. There are a couple of things I would like to do with this:
Edit the configuration of Python apps that use source modules for configuration.
Set up a "template" so that I can customize a Python source file on the fly. This way, I can set up a "project" system on an open source app I'm working on and allow certain files to be customized.
I could probably write something that can do this myself, but I can see that opening up a lot of "devil's in the details" type issues. Are there any ways to do this currently, or am I just going to have to bite the bullet and implement it myself?
Python's standard library provides pretty good facilities for working with Python source; note the tokenize and parser modules.
Most of these kinds of things can be determined programatically in Python, using modules like sys, os, and the special _file_ identifier which tells you where you are in the filesystem path.
It's important to keep in mind that when a module is first imported it will execute everything in the file-scope, which is important for developing system-dependent behaviors. For example, the os module basically determines what operating system you're using on import and then adjusts its implementation accordingly (by importing another module corresponding to Linux, OSX, Windows, etc.).
There's a lot of power in this feature and something along these lines is probably what you're looking for. :)
[Edit] I've also used socket.gethostname() in some rare, hackish instances. ;)
I had the same issue and I simply opened the file and did some replace: then reload the file in the Python interpreter. This works fine and is easy to do.
Otherwise AFAIK you have to use some conf objects.