On a server I am working on, I need to run the following commands to ensure the xlsxwriter is available to import from python:
module load swdev
module load python/xlsxwriter_py3.4.2/0.7.2
However, I would like this to be done automatically when the python script that needs it is run, from within the python script. Running os.system or subprocess.call doesn't work. How do I do this?
You can call module from a Python script. The module command is provided by the environment-modules software, which also provides a python.py initialization script.
Evaluating this script in a Python script enables the module python function. If environment-modules is installed in /usr/share/Modules, you can find this script at /usr/share/Modules/init/python.py.
Following code enables module python function:
import os
exec(open('/usr/share/Modules/init/python.py').read())
Thereafter you can load your modules:
module('load', 'swdev')
module('load', 'python/xlsxwriter_py3.4.2/0.7.2')
Related
I am using the VS code interactive window and using some python packages that make use of other programs. In the interactive window I am e.g. getting error messages like:
OSError: No command "mcflirt" found on host login2.nan.kcl.ac.uk. Please check that the corresponding package is installed.
If I was just running this from the command line I would simply ensure that I loaded the package which contains 'mcflirt' into the shell before starting the python interpreter and the interpreter would then be able to find it. Is anyone aware of how to tell the interactive window to load the package (e.g. loading the mcflirt containing package by module load fsl)before starting the python interpreter?
One way to access system modules is to use the python 'init' file e.g.:
sys.path.insert(0,'/software/system/modules/4.5.1/init')
import python as mod
mod.module('unload', 'fsl')
mod.module('load', 'fsl/6.0.4')
mod.module('load', 'ants/2.3.5')
I would like to use python scripts in Jmeter, so I can check the Load and the performance of an application...for that I've created a Thread with JSR223 Sampler, and I put the content of python script, and after the run, Jmeter throws error like : File "", line 8, in
ImportError: No module named opcua, ModuleNotFoundError: No module named 'dateutil'
see snapshot:
I don't know how to install this modules (package)in Jmeter, with PyCharm, I installed the package in the Project Interpreter under the setting see the snapshot...
You can do something like:
Install required module(s)
jython -m pip install foo
Add the next line(s) to the beginning of your script:
import sys
sys.path.append('\\location\\of\\jython\\site-packages')
However from performance perspective it will be much better to re-write your script in Groovy as calling Jython interpreter is not the best choice when it comes to high loads.
Also be aware that you can invoke Python interpreter as "external" process using JMeter's OS Process Sampler (if amount of the existing Python code is too big or too complex to rewrite in Groovy), see How to Run External Commands and Programs Locally and Remotely from JMeter for more details.
In a python script running with Python 3.x, importing another python script running with Python 2.7, is it possible to tell the program to use Python 2.7 interpreter when importing that specific script?
In this "mainfile.py" running with Python 3.x
import anotherfile27
As soon as "anotherfile27.py" is importing, everything will automatically run, no functions in "anotherfile27.py" need to be reused in "mainfile.py". Only several .csv scripts will be generated in order to be used later.
Is it possible to write a line of code within "anotherfile27.py" or within "mainfile.py" to tell the program to change the interpreter from Python 3 to Python 2.7 only when running "anotherfile27.py"?
Here's a basic example:
import subprocess
import platform
print("Running ", platform.python_version())
old_version = subprocess.check_output("python2 --version", encoding="utf-8")
print("Old version", old_version)
I'm currently using Python 2.7 on a unix environment.
I need to run R scripts in my python scripts but I can't manage to make it work because my R module needs to be loaded before (using "module load")
Here's my python script :
import os
import subprocess as sp
os.system('module load R/3.2.3')
out = sp.check_output(['Rscript','test.R'], universal_newlines=True)
I keep having the same error : "[Errno 2] No such file or directory"
Any idea ?
I looked here and here but couldn't make it work.
Thank you for your help !
So what "module load" actually does is set some environment variables in the calling shell. So when you do this:
os.system('module load R/3.2.3')
Python creates a process, runs /bin/sh in it, and passes that command to the shell. The module environment variables are set in that shell. Then that shell exits--job done!
The environment variables do not--and cannot--propagate back to the Python process. So when you do this:
sp.check_output(['Rscript','test.R'])
It's totally irrelevant that you ran module load before.
So how can you fix this? Well, one possibility would be to explicitly specify the path to Rscript:
sp.check_output(['/your/full/path/to/Rscript','test.R'])
Another would be to combine your commands:
sp.check_output('module load R/3.2.3 && Rscript test.R', shell=True)
Finally, you could simply run module load before running your Python script in the first place. The environment variables it sets can propagate all the way to the R invocation within Python.
By the way, it is possible to invoke R directly from Python: http://rpy.sourceforge.net/rpy2/doc-dev/html/introduction.html
I am using a Windows server (2012)'s Task Scheduler and Python 2.7
I am scheduling a task to be run using python. Specifically: C:\Anaconda\python.exe and calling a script as the argument located in another directory.
However, all my import statements from that script do not get imported. I tried a bare minimum script using only standard libraries and all was fine (i.e. the script worked flawlessly and ran as intended). However, when I start importing my own user-created packages (located in the same hierarchy but different folders as the script), they fail to import. I think I am misunderstanding something about how to import packages in Python.
My user defined packages are imported at the start of the script, like so:
import yolo
import wisconsin
When I run the script in Eclipse (or whatever editor I am using), the import statements work fine. This incident is isolated to running the script using Task Scheduler in Windows. Can anyone surmise what the problem may be? Do I need to be more explicit with my import statements like maybe including a path somehow?
Let me know if something is unclear.
Could it be that when I run the script using Windows Task Scheduler, it isolates the script in its own environment, whereas if I run it in Eclipse, I have all my packages in the same environment? How would I fix this if I intend to schedule a script that relies on these packages?
Can you try to put imports under __name__ part and use it in it?
Like this:
...
if __name__ == '__main__':
import yolo
import wisconsin
# use yolo here
...