We have developed code to compile calculations into tex files and then convert to pdf's. We are now trying to instead of generating the pdf on local PC's to simply send the tex file to a hosting PC and prompting this PC to run a python file (using its own Python.exe instance) that will then generate the PDF and send to a server folder.
It might seem like a silly approach (since it can be generated on the local PC) but we are trying to remove the step where people have to install software like MikTex and Strawberry on local PC's completely and not use an online converter like Overleaf.
Is there a way that this is possible?
Related
i am new to the community and new in using servers and could use some help.
I am trying to setup an automatic JSON parser to another server using http post calls. The idea is as follows:
I manually put JSON files into a folder Input on the server
A python script that is always running on the server reads the files located within the folder
It reads the JSON files, posts all objects to another server, and moves the files to a "Processed" folder one file at a time.
I have been given a Windows Server, with Windows Server Manager 2016, and have managed to do the following:
installed python 3.8.2. on the windows server
able to run a python script using powershell
Installed NSSM to create a windows service
Now the windows server manager says i cannot resume or start the service that i tried to install via NSSM.
I am very new to servers, as well as python itself. Can somebody help me to get a python script running 24/7 on a windows server with windows server manager 2016?
Edit:
I managed to create a python script that can read files, upload them and move them to a processed folder, but i still have to run it by myself while i want it to always run on the server
I am an astronomy student working with large data set. I have 80 TB of .fits file on a supercomputer that I am trying to process using python script. I could process the data stored on the supercomputer by submitting a job(which stays in queue
for ages) or I could process the data in my local desktop(without downloading all the data). However, I cannot download all(80TB) data to my local desktop due to storage issue. I was wondering if there is a way to run the processing python script on my local desktop but it reads data from the supercomputer using secure shell.
Thanks.
Check out Perform commands over ssh with Python and Download files over SSH using Python How to list all the folders and files in the directory after connecting through sftp in python
You could put it in a loop and get a file > Parse it > get next file and so on.
I am new to Pig and would like to call a local python file from Pig after I have connected to the server via PuTTy. Below are the codes I have tried and the error message I received:
REGISTER ‘myudf.py’ using jython as my_udf
The error message is below and I don't know how to tell Pig the path to the py file.
File myudf.py does not exist
Another code i tried is:
DEFINE mycommand `python myudf.py` ship(‘C:\Users\myname\Documents\code\myudf.py’)；
The error message is
unexpected character ’S’
This may sound super easy but I have spent hours on it and failed. Any advice will be highly appreciated.
It sounds like the Python file is saved on your own computer, but you're running Pig on a server. The server doesn't have access to files on your computer.
You can transfer the file using software like WinSCP (assuming you're on Windows), or start a text editor on the server and copy/paste the text from your Python file into the text editor. For example, vi myudf.py or emacs myudf.py in PuTTy will start a text editor and create a file named myudf.py on the server once you've saved.
Once you've created the Python file, you might want to include the full path to your file on the server in your REGISTER statement to avoid confusion. (pwd shows your current directory on the server.)
I have linux executable file, classify input text when executed. To classify the input, it need to read 114MB model file.
Source of package : www.hlt.utdallas.edu/~altaf/cherrypicker.html
Though it is opensouce, they have not provided the source code for executable file, So I can not modify it.
Is it possible that I create server socket for this executable which keep 114mb model file in loaded mode.
So that when I make request as client socket it does not take time and gives instant reply?
I am working on linux and using python programming.
I would like to do an os.walk in python 2.7 in windows 7 but on a computer that is on the same network. Something like os.walk('\192.168.0.2') but this doesn't work. Is this possible?
Try net use or a UNC path.
Or try Linux, you'd probably be better off with it than Windows.
http://pcsupport.about.com/od/commandlinereference/p/net-use-command.htm
http://www.uwplatt.edu/oit/terms/uncpath.html
In python, using Windows paths is often easier if you use, for example, r'\\hostname\share' rather than '\\\\hostname\\share'.
My thought is also to go for Linux, or at least, Windows with SSH service.
Set up public key authentication over SSH, and make sure you can ssh to that Windows machine (with SSH service on, of course) without password and password phrase.
Make use of Fabric, a Python (2.5 or higher) library and command-line tool for streamlining the use of SSH for application deployment or systems administration tasks.
For Windows, what I did was opened File Explorer, and navigated to the (sub)folder of interest on the network drive, and right clicked on Properties. Copy the Location information, and paste that into your Python file.
Ex.
for root, dirs, files in os.walk("//<network name>/<folder>/<sub-folder1>/<sub-folder2>/.../<sub-folderN>"):
for file in files:
print(file)