I'm trying to install CDH5 parcels on Hadoop-cluster using Cloudera Manager Python API. I'm doing this using following code:
test_cluster = ... # configuring cluster
# adding hosts ...
for parcel in test_cluster.get_all_parcels():
if parcel.product == 'CDH' and 'cdh5':
parcel.start_download().wait()
parcel.start_distribution().wait()
success = parcel.activate().wait().success
But I catch such error:
cm_api.api_client.ApiException: Parcel for CDH : 5.8.0-1.cdh5.8.0.p0.42 is not available on UBUNTU_TRUSTY. (error 400)
The CDH 5.8.0-1.cdh5.8.0.p0.42 was in AVAILABLE_REMOTELY, as we can see if print a string representation on this parcel:
<ApiParcel>: CDH-5.8.0-1.cdh5.8.0.p0.42 (stage: AVAILABLE_REMOTELY) (state: None) (cluster: TestCluster)
After the execution of code, parcel changes its stage to DOWNLOADED.
It seems, I should add a new parcel repository, compatible with Ubuntu Trusty (14.04). But I don't know of doing this using Cloudera Manager API.
How I can specify the new repository for installing correct CDH?
You may want to be more specific about the parcel you are acting on. I use something like this for the same purpose, the important part for your question is the combined check on parcel.version and parcel.product. After that (yes I am verbose in my output) I print the list of parcels to verify I am trying to only install the 1 parcel I want.
I'm sure you've been here, but if not the cm_api github site has some helpful examples too.
cdh_version = "CDH5"
cdh_version_number = "5.6.0"
# CREATE THE LIST OF PARCELS TO BE INSTALLED (CDH)
parcels_list = []
for parcel in cluster.get_all_parcels():
if parcel.version.startswith(cdh_version_number) and parcel.product == "CDH":
parcels_list.append(parcel)
for parcel in parcels_list:
print "WILL INSTALL " + parcel.product + ' ' + parcel.version
# DISTRIBUTE THE PARCELS
print "DISTRIBUTING PARCELS..."
for p in parcels_list:
cmd = p.start_distribution()
if not cmd.success:
print "PARCEL DISTRIBUTION FAILED"
exit(1)
# MAKE SURE THE DISTRIBUTION FINISHES
for p in parcels_list:
while p.stage != "DISTRIBUTED":
sleep(5)
p = get_parcel(api, p.product, p.version, cluster_name)
print p.product + ' ' + p.version + " DISTRIBUTED"
# ACTIVATE THE PARCELS
for p in parcels_list:
cmd = p.activate()
if not cmd.success:
print "PARCEL ACTIVATION FAILED"
exit(1)
# MAKE SURE THE ACTIVATION FINISHES
for p in parcels_list:
while p.stage != "ACTIVATED":
p = get_parcel(api, p.product, p.version, cluster_name)
print p.product + ' ' + p.version + " ACTIVATED"
Related
I was working on automate windows patches which filter approved patchs from sever and download and install in local machine. but i am getting Result Code: 4 for both download and installation,
input patch example: 2022-03 Cumulative Update for Windows 11 for x64-based Systems (KB5011493)
from vb2py.vbfunctions import *
from vb2py.vbdebug import *
import sys
updateSession = CreateObject('Microsoft.Update.Session')
updateSession.ClientApplicationID = 'MSDN Sample Script'
#Get update title to search for
print('Enter the title of the update: ' + '(for example, Update for Windows Rights
Management client 1.0)')
updateTitle = input('Enter title')
print('Searching for: ' + updateTitle + '...')
updateSearcher = updateSession.CreateupdateSearcher()
#Search for all software updates, already installed and not installed
searchResult = updateSearcher.Search('Type=\'Software\'')
updateToInstall = CreateObject('Microsoft.Update.UpdateColl')
updateIsApplicable = False
#Cycle through search results to look for the update title
for i in range(0, searchResult.Updates.Count - 1):
update = searchResult.Updates.Item(i)
if update.Title == updateTitle:
#Update in list of applicable updates
#Determine if it is already installed or not
if update.IsInstalled == False:
print('Result: Update applicable, not installed.')
updateIsApplicable = True
updateToInstall.Add(update)
else:
#Update is installed so notify user and quit
print('Result: Update applicable, already installed.')
updateIsApplicable = True
sys.exit()
if updateIsApplicable == False:
print('Result: Update is not applicable to this machine.')
print('Would you like to install now? (Y/N)')
stdInput = input('Enter Choose:')
if stdInput == 'N' or stdInput == 'n' :
sys.exit()
elif stdInput == 'Y' or stdInput == 'y':
#Download update
downloader = updateSession.CreateUpdateDownloader()
downloader.Updates = updateToInstall
print('Downloading...')
downloadResult = downloader.Download()
print('Download Result: ', downloadResult.ResultCode)
#Install Update
installer = updateSession.CreateUpdateInstaller()
print('Installing...')
installer.Updates = updateToInstall
installationResult = installer.Install()
#Output the result of the installation
print('Installation Result:' ,installationResult.ResultCode)
print('Reboot Required: ', installationResult.RebootRequired)
In Terminal:
Enter the title of the update: (for example, Update for Windows Rights Management
client 1.0)
Enter title2022-03 Cumulative Update for Windows 11 for x64-based Systems (KB5011493)
Searching for: 2022-03 Cumulative Update for Windows 11 for x64-based Systems
(KB5011493)...
Result: Update applicable, not installed.
Would you like to install now? (Y/N)
Enter Choose:y
Downloading...
Download Result: 4
Installing...
Installation Result: 4
Reboot Required: False
PS C:\Users\Somesh\Desktop\salt>
I have written python code to retrieve information about build. I prints a summary of successful and unsuccessful builds.
from prettytable import PrettyTable
t = PrettyTable(['Job name','Successful','Failed','Unstable','Aborted','Total Builds','Failure Rate'])
t1 = PrettyTable(['Status', 'Job name','Build #','Date','Duration','Node','User'])
aggregation ={}
jobs = server.get_all_jobs(folder_depth=None)
for job in jobs:
print(job['fullname'])
aggregation[job['fullname']] = {"success" : 0 , "failure" : 0 , "aborted" : 0, "unstable":0}
info = server.get_job_info(job['fullname'])
# Loop over builds
builds = info['builds']
for build in builds:
information = server.get_build_info(job["fullname"],
build['number'])
if "SUCCESS" in information['result']:
aggregation[job['fullname']]['success'] = str(int(aggregation[job['fullname']]['success']) + 1)
if "FAILURE" in information['result']:
aggregation[job['fullname']]['failure'] = str(int(aggregation[job['fullname']]['failure']) + 1)
if "ABORTED" in information['result']:
aggregation[job['fullname']]['aborted'] = str(int(aggregation[job['fullname']]['aborted']) + 1)
if "UNSTABLE" in information['result']:
aggregation[job['fullname']]['unstable'] = str(int(aggregation[job['fullname']]['unstable']) + 1)
t1.add_row([ information['result'], job['fullname'],information["id"],datetime.fromtimestamp(information['timestamp']/1000),information["duration"],"master",information["actions"][0]["causes"][0]["userName"]])
total_build = int(aggregation[job['fullname']]['success'])+int(aggregation[job['fullname']]['failure'])
t.add_row([job["fullname"], aggregation[job['fullname']]['success'],aggregation[job['fullname']]['failure'],aggregation[job['fullname']]['aborted'],aggregation[job['fullname']]['unstable'],total_build,(float(aggregation[job['fullname']]['failure'])/total_build)*100])
with open('result', 'w') as w:
w.write(str(t1))
w.write(str(t))
This is what the output looks like:
And this is what Windows batch execute command looks like:
cd E:\airflowtmp
conda activate web_scraping
python hello.py
hello.py prints hello world. If I add print counter =100 or something like this then how do I return it and print it in this resultant table.
Edit:
I am trying to get some kind of variable from code to display. For instance if Im scraping pages and scraper ran successfully then I want to know the number of pages that it scraped. You can think of it as a simple counter. Is there any way to return a variable from Jenkins to python
I'm trying to configure a Virtual Machine(with Vagrant and Ansible), that needs a file.py to the full correct configuration of this machine (according to the book that I'm studying),I'm was using the DigitalOcean API V2, but as I have no a valid credit card my account is bloked,so I had to change DigitalOcean to AWS,as the company where I work have an account with AWS,now I take the 'client id' and 'api key' from AWS VM,so the foregoing problems returned...when I try use the "python file.py" command the output says again:
dopy.manager.DoError: Unable to authenticate you.
**the file.py:**
"""
dependencias:
sudo pip install dopy pyopenssl ndg-httpsclient pyasn1
"""
import os
from dopy.manager import DoManager
import urllib3.contrib.pyopenssl
urllib3.contrib.pyopenssl.inject_into_urllib3()
api_version = os.getenv("DO_API_VERSION")
api_token=os.getenv("DO_API_KEY")
#do = DoManager(cliend_id, api_key)
do = DoManager(None, api_token, api_version=2)
keys = do.all_ssh_keys()
print "ssh key name\tid"
for key in keys:
print "%s\t%d" % (key["name"], key["id"])
print "Image name\tid"
imgs = do.all_images()
for img in imgs:
if img["slug"] == "ubuntu-14-04-x64":
print "%s\t%d" % (img["name"], img["id"])
print "Region name\tid"
regions = do.all_regions()
for region in regions:
if region["slug"] == "nyc2":
print "%s\t%d" % (region["slug"], region["id"])
print "Size name\tid"
sizes = do.sizes()
for size in sizes:
if size["slug"] == "512mb":
print "%s\t%d" % (size["slug"], size["id"])
I appreciate any help.
Try to remove quotes from api_token:
do = DoManager(None, api_token, api_version=2)
Otherwise your token is always literal string api_token, not a variable api_token.
I am looking to figure out both the current Battery Capacity and the Design Capacity.
So far what I could get to work is using the Win32_Battery() class which doesn't give all the information I need (at least not on my system). I used the pure-python wmi library for that.
On the other hand I found this which works In Python, how can I detect whether the computer is on battery power?, but unfortunately it doesn't provide any information on Capacity neither.
The Battery Information structure and the Battery Status structure seem perfect for this. Now I know that I have to use the DeviceIoControl function to do so and I found this C++ code that explains it a little.
I would prefer something that simply uses ctypes and not the python win32api provided by pywin32.
If you have an idea how to do this in python please let me know!
Thanks in advance.
The most reliable way to retrieve this information is by using GetSystemPowerStatus instead of WMI. psutil exposes this information under Linux, Windows and FreeBSD:
>>> import psutil
>>>
>>> def secs2hours(secs):
... mm, ss = divmod(secs, 60)
... hh, mm = divmod(mm, 60)
... return "%d:%02d:%02d" % (hh, mm, ss)
...
>>> battery = psutil.sensors_battery()
>>> battery
sbattery(percent=93, secsleft=16628, power_plugged=False)
>>> print("charge = %s%%, time left = %s" % (battery.percent, secs2hours(battery.secsleft)))
charge = 93%, time left = 4:37:08
The relevant commit is here.
Tim Golden's excellent wmi module will, I believe, give you everything you want. You'll just have to do several queries to get everything:
import wmi
c = wmi.WMI()
t = wmi.WMI(moniker = "//./root/wmi")
batts1 = c.CIM_Battery(Caption = 'Portable Battery')
for i, b in enumerate(batts1):
print 'Battery %d Design Capacity: %d mWh' % (i, b.DesignCapacity or 0)
batts = t.ExecQuery('Select * from BatteryFullChargedCapacity')
for i, b in enumerate(batts):
print ('Battery %d Fully Charged Capacity: %d mWh' %
(i, b.FullChargedCapacity))
batts = t.ExecQuery('Select * from BatteryStatus where Voltage > 0')
for i, b in enumerate(batts):
print '\nBattery %d ***************' % i
print 'Tag: ' + str(b.Tag)
print 'Name: ' + b.InstanceName
print 'PowerOnline: ' + str(b.PowerOnline)
print 'Discharging: ' + str(b.Discharging)
print 'Charging: ' + str(b.Charging)
print 'Voltage: ' + str(b.Voltage)
print 'DischargeRate: ' + str(b.DischargeRate)
print 'ChargeRate: ' + str(b.ChargeRate)
print 'RemainingCapacity: ' + str(b.RemainingCapacity)
print 'Active: ' + str(b.Active)
print 'Critical: ' + str(b.Critical)
This is certainly not cross-platform, and it requires a 3rd party resource, but it does work very well.
import psutil
battery = psutil.sensors_battery()
plugged = battery.power_plugged
percent = str(battery.percent)
The variable percent will have the battery percent.
I have written a small piece of code using python to notify when the battery is fully charged
https://github.com/Mitzzzzz/Battery-Full-Indicator
Kindly check!!
**Checking the battery percentage in python can be done using the psutil module as
follows **
# Installing the psutil module
from pip._internal import main
main(["install", "psutil"])
import psutil
battery = psutil.sensors_battery();
# checking if the charger is plugged
if battery.power_plugged:
print("Charging: ", battery.percent)
else:
print("Not Charging", battery.percent ,"%");
print( "Discharge time ", int(battery.secsleft)," sec_lft")
The Most Easiest Form to Know the Battery With Python is using Psutil Module...
import psutil #pip install psutil
battery_detecting = psutil.sensors_battery()
plugged = battery_detecting.power_plugged
percent_battery = str(battery_detecting.percent)
plugged = "Plugged In" if plugged else "Not Plugged In"
print(percent_battery+'% | '+plugged)
Try This Code
import psutil
import time
battery = psutil.sensors_battery()
while True:
print("Battery percentage : ", battery.percent)
print("Power plugged in : ", battery.power_plugged)
if battery.percent < 35 and battery.power_plugged == False:
print("Your battry is low")
time.sleep(5)
But this code will run continously until it get's interrupted
If you are talking about Full Charge Capacity of a battery try this:
I AM USING POWERSHELL NONINTERACTIVE SHELL TO USE THIS COMMAND. THIS IS ONE OF THE EASIEST WAYS TO KNOW BATTERY INFO.
systeminfo = subprocess.run(["Powershell", "-NonInteractive", "-Command", "Get-WmiObject", "-Namespace", "'root\wmi'", "-Query", "'select FullChargedCapacity from BatteryFullChargedCapacity'"], capture_output=True).stdout.decode().strip()
print(systeminfo[292:-21])
NOTE: THIS ONLY WORKS ON WINDOWS
I'm having some issues with the EC2 bit of Boto (Boto v2.8.0, Python v2.6.7).
The first command returns a list of S3 Buckets - all good! The second command to get a list of EC2 instances blows up with a 403 with "Query-string authentication requires the Signature, Expires and AWSAccessKeyId parameters"
s3_conn = S3Connection(AWSAccessKeyId, AWSSecretKey)
print s3_conn.get_all_buckets()
ec2_conn = EC2Connection(AWSAccessKeyId, AWSSecretKey)
print ec2_conn.get_all_instances()
Also, my credentials are all good (Full admin) - I tested them using the Ruby aws-sdk, both EC2 and S3 work fine.
I also noticed that the host attribute in the ec2_conn object is s3-eu-west-1.amazonaws.com, "s3"...? Surely thats wrong? I've tried retro fixing it to the correct endpoint but no luck.
Any help would be great appreciate
Thanks
Here's some working code I use to list all my instances across potentially multiple regions.
Its doing a lot more than you need, but maybe you can pare it down to what you want.
#!/usr/bin/python
import boto
import boto.ec2
import sys
class ansi_color:
red = '\033[31m'
green = '\033[32m'
reset = '\033[0m'
grey = '\033[1;30m'
def name(i):
if 'Name' in i.tags:
n = i.tags['Name']
else:
n = '???'
n = n.ljust(16)[:16]
if i.state == 'running':
n = ansi_color.green + n + ansi_color.reset
else:
n = ansi_color.red + n + ansi_color.reset
return n
def pub_dns( i ):
return i.public_dns_name.rjust(43)
def pri_dns( i ):
return i.private_dns_name.rjust(43)
def print_instance( i ):
print ' ' + name(i) + '| ' + pub_dns(i) + ' ' + pri_dns(i)
regions = sys.argv[1:]
if len(regions)==0:
regions=['us-east-1']
if len(regions)==1 and regions[0]=="all":
rr = boto.ec2.regions()
else:
rr = [ boto.ec2.get_region(x) for x in regions ]
for reg in rr:
print "========"
print reg.name
print "========"
conn = reg.connect()
reservations = conn.get_all_instances()
for r in reservations:
# print ansi_color.grey + str(r) + ansi_color.reset
for i in r.instances:
print_instance(i)
There is the connect_to_region command:
import boto.ec2
connection = boto.ec2.connect_to_region('eu-west-1', aws_access_key_id=AWSAccessKeyId,
aws_secret_access_key=AWSSecretKey)
The Boto tutorial gives another way. That method would basically work like this:
import boto.ec2
for region in boto.ec2.regions():
if region.name == 'my-favorite-region':
connection = region.connect()
break
This has not been working on older versions of Boto.
Do you have your IAM credentials in order? The given access key should have rights for EC2. If you're not sure, you can add the policy AmazonEC2FullAccess to test, and later tune this down.