I have a boot controller which runs a boot.py file contained in each folder of each tool i am trying to deploy. I want my boot controller to run all of these boot files simultaneously. The config file has the tool names and the versions desired, which help to generate the path to the boot.py.
def run_boot():
config_file = get_config_file()
parse_config_file.init(config_file)
tools = parse_config_file.get_tools_to_deploy()
#tools is now a list of tool names
top_dir = os.getcwd()
for tool in tools:
ver = parse_config_file.get_tool_version(tool).strip()
boot_file_path = "{0}\\Deploy\\{1}\\{2}".format(os.getcwd(),tool,ver)
try:
subprocess.Popen('boot.py', shell=True, cwd=boot_file_path)
except:
print ("{0} failed to open".format(tool))
print(tool, boot_file_path)
os.chdir(top_dir)
The first time i run this, the print(tool, boot_file_path) executes but the processes do not. the second time it is run the processes do open. I cannot find a reason for this.
Related
I am trying to create a program for a project of mine where I needed to copy some files from my local machine to some remote desktop pc's desktop directory. It is a part of automating the whole connecting and copying files to all the remote PCs. I have used subprocess to connect to a remote desktop. As I am a beginner of using subprocess or cmd commands, I am unable to find a way to do this. This is the code:
from pynput.keyboard import Key,Controller
import time
import subprocess
def process_exists(process_name):
call = 'TASKLIST', '/FI', 'imagename eq %s' % process_name
output = subprocess.check_output(call)
last_line = output.decode("utf-8").strip().split('\r\n')[-1]
return last_line.lower().startswith(process_name.lower())
#os.system('cmd /k "mstsc /v:192.168.0.107"')
#os.system('cmd /c "notepad"')
from subprocess import Popen
p = Popen('mstsc /v:192.168.0.107')
time.sleep(1)
keyboard = Controller()
# Password is given here
for i in range(10):
if process_exists('mstsc.exe'):
time.sleep(1)
keyboard.type("**********")
keyboard.press(Key.enter)
time.sleep(0.12)
break
time.sleep(3)
Here I used subprocess just to connect to that remote PC. I tried
Popen(r'copy /Y "E:\test.jar" "\\192.168.0.107\c\"')
But it didn't help. Though I didn't know the basic of this line. The error it showed was:
FileNotFoundError: [WinError 2] The system cannot find the file specified
This code is only for test, that's why I used only time delay, didn't check whether it ran correctly.
The remote desktop started successfully. But unable to start the copying process.
Sorry for my bad English. Thanks in advance
Currently I have a program that uses subprocess.Popen to open an executable file and pass an argument - equivalent to running ./path/to/file args on Linux.
This works very well but I have to execute this file over 1000 times and currently it is done one at a time, on a single processor. I want to be able to execute this file in sets of 8 for example, as I have an 8-core PC.
I have tried the following:
bolsig = ("/home/rdoyle/TEST_PROC/BOLSIG/bolsigminus")
infile_list = glob.glob(str(cwd)+"/BOLSIG Run Files/run*")
cmds_list = [[bolsig, infile] for infile in infile_list]
procs_list = [Popen(cmd) for cmd in cmds_list]
for proc in procs_list:
proc.wait()
But this tries to execute all 1000 commands at the same time.
Anyone have any suggestions?
I like concurrent.futures for simple cases like this, it's so simple and yet so effective.
import os
import glob
from concurrent import futures
from subprocess import Popen
optim = ("/usr/bin/jpegoptim")
img_path = os.path.join(os.path.abspath(os.path.curdir), 'images')
file_list = glob.glob(img_path+'/*jpg')
def compress(fname):
Popen([optim, fname, '-d', 'out/', '-f'])
ex = futures.ThreadPoolExecutor(max_workers=8)
ex.map(compress, file_list)
A great intro at Doug Hellman's PyMOTW. https://pymotw.com/3/concurrent.futures/
You can use the Python multiprocessing module and its multiprocessing.pool.Pool -> https://docs.python.org/3/library/multiprocessing.html#module-multiprocessing
I'm trying to execute a python scrip from SSIS Execute Process Task. I followed all the tutorials of how to do this an still the script is failing from the start. when i execute the python script out of SSIS it runs perfectly.
This is my Python scrip:
import sys
import gender_guesser.detector as gender
import xml.etree.cElementTree as ET
from xml.etree.ElementTree import ParseError
try:
input("Press Enter to continue...")
except SyntaxError:
pass
tree = ET.parse('user.xml')
root = tree.getroot()
for child_of_root in root:
for attr in child_of_root:
if attr.tag == 'first_name':
upperName = "%s%s" % (attr.text[0].upper(), attr.text[1:])
print attr.tag,upperName
d = gender.Detector()
gen = d.get_gender(upperName)
print gen
attr.text= gen
tree = ET.ElementTree(root)
tree.write("user1.xml")
this is an image of the SSIS Execute Process Task:
error message:
[Execute Process Task] Error:
In Executing "C:\Python27\python.exe" "C:\Users\bla\blalba\bla\gender-guesser-0.4.0\test\genderTest.py " at "", The process exit code was "1" while the expected was "0".
Did you have spaces in your file path to your python script? I had the same error when trying to pass a path with spaces as my argument in Execute Script Process. The resolution was to enter the argument with quotes.
When the process is launched from Execute Process Task step of SSIS Package, it's not being run from the same folder as the executable file (.bat, .py, .exe and so on) located.
What is different from the direct file execution.
And it can be especial critical in case when your executable file working with some other files in the same folder.
So, it is necessary additionally specify working folder property of Execute Process Task step of SSIS Package.
On your screenshot Working directory property value is empty. Put there the
C:\Users\bla\blalba\bla\gender-guesser-0.4.0\test\
I first posted an answer in this post, but it didn't conform to the forum standards. I hope this time te answer fits the forum standards. This code should be more clear and easy to read.
In Python 3+ I have the following class that I use to build a Windows Service (it does nothing, just writes a log file):
#MyWindowsService.py
import win32serviceutil
import servicemanager
import win32service
import win32event
import sys
import logging
import win32api
class MyWindowsService(win32serviceutil.ServiceFramework):
_svc_name_ = 'ServiceName'
_svc_display_name_ = 'Service Display Name'
_svc_description_ = 'Service Full Description'
logging.basicConfig(
filename = 'c:\\Temp\\{}.log'.format(_svc_name_),
level = logging.DEBUG,
format = '%(levelname)-7.7s # %(asctime)s: %(message)s'
)
def __init__(self, *args):
self.log('Initializing service {}'.format(self._svc_name_))
win32serviceutil.ServiceFramework.__init__(self, *args)
self.stop_event = win32event.CreateEvent(None, 0, 0, None)
def SvcDoRun(self):
self.ReportServiceStatus(win32service.SERVICE_START_PENDING)
try:
self.log('START: Service start')
self.ReportServiceStatus(win32service.SERVICE_RUNNING)
self.start()
win32event.WaitForSingleObject(self.stop_event, win32event.INFINITE)
except Exception as e:
self.log('Exception: {}'.format(e))
self.SvcStop()
def SvcStop(self):
self.log('STOP: Service stopping...')
self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
self.stop()
win32event.SetEvent(self.stop_event)
self.ReportServiceStatus(win32service.SERVICE_STOPPED)
def log(self, msg):
servicemanager.LogInfoMsg(str(msg)) #system log
logging.info(str(msg)) #text log
def start(self):
self.runflag = True
while self.runflag:
win32api.Sleep((2*1000), True)
self.log('Service alive')
def stop(self):
self.runflag = False
self.log('Stop received')
if __name__ == '__main__':
win32serviceutil.HandleCommandLine(MyWindowsService)
In the script I use a log file to check if it's working properly. I'm running python3.6 (also tried with python3.4) on Windows 7 and I'm experiencing the following problem. When I run python MyWindowsService.py install the prompt says that the service has been installed (but nothing is written in the log file). If I try to start the service, I get Service Error: 1 - More info NET HELPMSG 3547 which doesn't say much about the error. If I run python MyWindowsService.py debug, the program runs just fine (the log file is written), but still I don't have any control over the service: if I open another prompt and try to stop/start the service I still got the same results as stated above.
I also tryed to insert some debug code inside the init function, and when I run python MyWindowsService.py install it seems it doesn't get called. Is it possible?
I've checked for multiple solution and workarounds around the net, but I didn't find anything suitable. What am I missing?
As pointed out by eriksun in the comment to the first post, the problem came from the location of the python script, that was in a drive mapped with an UNC path - I'm working with a virtual machine. Moving the python script in the same drive as the python installation did the job.
To sum it up for future uses, if the service fails to start and you're pretty sure about your code, these are helpful actions to try and solve your issues:
use sc start ServiceName, sc query ServiceName and sc stop ServiceName to get info about the service.
check if your file is in a physical drive or in a UNC-mapped drive. If the latter try to run the script using the UNC path (for example python \\Server\share\python\your-folder\script.py) or move your script in the same drive as the python installation
make sure that "python36.dll", "vcruntime140.dll", and "pywintypes36.dll" are either symlink'd to the directory that has PythonService.exe; or symlink'd to the System32 directory; or that the directories with these DLLs are in the system (not user) Path
Check the system register with command reg query HKLM\System\CurrentControlSet\Services\your_service_name /s to get more information about the script
PLease, feel free to complete, change, modify the last so that it can be usefull for anyone that like me encounder this issue.
EDIT: One more thing... My project was thought to work actually with network folders (and UNC-mapped drive) and it failed when I tried to make it run as service. One very useful (day-saving) resource that I used to make it work is the SysinternalsSuite by Mark Russinovich that I found in this post. Hope this helps.
Original Question
I've got some python scripts which have been using Amazon S3 to upload screenshots taken following Selenium tests within the script.
Now we're moving from S3 to use GitHub so I've found GitPython but can't see how you use it to actually commit to the local repo and push to the server.
My script builds a directory structure similar to \images\228M\View_Use_Case\1.png in the workspace and when uploading to S3 it was a simple process;
for root, dirs, files in os.walk(imagesPath):
for name in files:
filename = os.path.join(root, name)
k = bucket.new_key('{0}/{1}/{2}'.format(revisionNumber, images_process, name)) # returns a new key object
k.set_contents_from_filename(filename, policy='public-read') # opens local file buffers to key on S3
k.set_metadata('Content-Type', 'image/png')
Is there something similar for this or is there something as simple as a bash type git add images command in GitPython that I've completely missed?
Updated with Fabric
So I've installed Fabric on kracekumar's recommendation but I can't find docs on how to define the (GitHub) hosts.
My script is pretty simple to just try and get the upload to work;
from __future__ import with_statement
from fabric.api import *
from fabric.contrib.console import confirm
import os
def git_server():
env.hosts = ['github.com']
env.user = 'git'
env.passowrd = 'password'
def test():
process = 'View Employee'
os.chdir('\Work\BPTRTI\main\employer_toolkit')
with cd('\Work\BPTRTI\main\employer_toolkit'):
result = local('ant viewEmployee_git')
if result.failed and not confirm("Tests failed. Continue anyway?"):
abort("Aborting at user request.")
def deploy():
process = "View Employee"
os.chdir('\Documents and Settings\markw\GitTest')
with cd('\Documents and Settings\markw\GitTest'):
local('git add images')
local('git commit -m "Latest Selenium screenshots for %s"' % (process))
local('git push -u origin master')
def viewEmployee():
#test()
deploy()
It Works \o/ Hurrah.
You should look into Fabric. http://docs.fabfile.org/en/1.4.1/index.html. Automated server deployment tool. I have been using this quite some time, it works pretty fine.
Here is my one of the application which uses it, https://github.com/kracekumar/sachintweets/blob/master/fabfile.py
It looks like you can do this:
index = repo.index
index.add(['images'])
new_commit = index.commit("my commit message")
and then, assuming you have origin as the default remote:
origin = repo.remotes.origin
origin.push()