We run many Python scripts for data processing tasks. We have a modeling computer that has been upgraded to provide the best performance for these tasks, but it is shared by many people that all need to run different scripts on it at the same time.
Is it possible for me to run a Python script remotely on that machine from my laptop while others are either directly logged into it or also remotely running a script?
Is SSH a possibility? I haven't ever run any scripts remotely aside from logging in via remote desktop. Ideally, I could start the Python script on that remote machine, but all the messages would be visible to me on my laptop. Does this sound doable?
EDIT:
I forgot to mention all machines are running Windows 7.
SSH is definitely the way to go and also have a look at Fabric.
Regarding your edit. You can use Fabric on Windows. And I think that using SSH on Windows will be a bit easier than dancing with their Powershell's remoting capabilities.
SSH does seem like it should meet your needs.
You could also consider setting up an iPython notebook server that everyone could use.
Its got nice parallel processing capabilities if you are doing some serious number crunching.
Related
I have a Djnago web application on IIS 6 on one server. Is there a way that from this website I call another python script that is on another server in such a way that that script just run itself there?
Calling or Runnig that script in the usual way as internet says, is not working.
I always get the error of os.getcwd() and it also doesn't allow to change that directory.
I just want to run that python script there on that server from this server.
Can anyone help?
Normally, I would recommend using a framework like fabric or winrm if you want to run a python script on another server. Those frameworks use ssh or windows remoting functionality, respectively, to allow a python program to execute other commands (including python scripts) on other systems. If the target machine is a windows machine, be forewarned that you can run into all sorts of UAC issues doing normal operations.
I've written a lot of python scripts. Now I want to run it on another computer which running non-stop to crawling, analyzing data and update to an sql database.
Normally I open a command prompt and run the scripts:
python [script directory]
But with many scripts I have to open many cmd and every script call an python interpreter, so It end up with huge mess using a lot of memory.
What should I do to manage these scripts.
You haven't specified what OS your server is, but assuming that it's a Linux server you should probably research a process management tool such as Supervisord or Systemd. These are tools designed to run and monitor your program automatically, and even restart it if it crashes.
If you're using Ubuntu 16.04 then it comes with Systemd out of the box, however I personally find Supervisord easier to configure and use for simple tasks.
These programs won't necessarily help with your memory consumption issues however. Sure you can place caps on memory use for a process, but that's not really going to help you if it stops your program from working. You're probably best to re-evaluate your code and look for ways to reduce its memory footprint or use a server with more ram.
EDIT:
You've just added that the OS is Windows 10, which makes the above irrelevant. You can use the Windows Task Scheduler to automatically execute long running tasks.
you can use pythonw *.py , and it will run in background.
As my title goes, I tried to look for something can help me to write the script to get CPU usage from another machine which will be running Virtual Machine(VM), yes I want to get the CPU usage of the VM which runs ubuntu and runs as FTP server. The thing is, I'm still trying to write the script in Python which will be run on Raspberry Pi 2.
I found a few solution that needs me to use bash scripting, but I need to compile with my script of sending ICMP request to the other machine. The thing is I need to write the script instead of using another software. Thank you in advance.
You could maybe try it with VNC but that won't give you a script that will just give you a window into the other machine.
I'm familiar with python within the 3D application I use (OSX platform), but am struggling with its usage in a client/server relationship. I've written a simple distributed rendering script which breaks my 3D render script into smaller OSX bash shell scripts and saves them to a directory on my machine. The remote machines in the room then look at my local folder with these smaller bash shell scripts and execute them one by one until they are all gone. It is a rudimentary solution to distributed rendering, but it works. What I would like to do is have the remote machines listen for a command from my local machine (the local machine would need to send an OSX bash command to the remote machines). I have been looking and this site: http://www.tutorialspoint.com/python/python_networking.htm
This seems to be what I'm looking for, but not knowing much about how python works with the network, I'm not sure whether its secure, and I am not sure how to send a Bash command rather than a message.
If anyone has any suggestions it'd be much appreciated.
I am tyring to find a solution for running a python script of mine 24/7.
Research led me to the conclusion that I probably need a CGI hosting.
But suppose that I have one (and which type of hosts should I look for?) :
how will I make the script run all the time?
In pseudo-code it should be something like this:
if (time_since_last_run(script.py)>100):
run(script.py)
Please give me a direction.
CGI has nothing to do with running a script continuously. Also your problem seems to be to run your script on a regular schedule. Depending on your operating system, you can look into Scheduled Tasks on Windows (http://support.microsoft.com/kb/814596) or cron on other systems (https://en.wikipedia.org/wiki/Cron).
You still need a computer that runs continuously, either at home or with some hosting enterprise.
You can also try to use supervisord (http://supervisord.org/)
CGI is used for creating websites with e.g. forms, but not for scripts which run in 24/7 and do tasks A,B, etc.
You need a physical PC, dedicated server or at least a virtual machine.
Unlike Linux, which is widely used today, Unix is a less popular operating system. Servers operating 24 x 7 primarily use Unix operating systems.
A unix server is one of the cheapest way for your solution.