I wrote my first python script which read a ini file using configparser.
First my ini file is define like:
;ini
[hypervisor1]
name = hyp1
how = ssh
vms = vm1 vm2 vm3
[hypervisor2]
name = hyp2
how = vpn
vms = vm4 vm5 vm6
Second my script:
import configparser
import os.path
import argparse
file_path = os.path.join(os.path.dirname(__file__),"config.ini")
parser = configparser.SafeConfigParser()
parser.read(file_path)
def get_args():
parser = argparse.ArgumentParser()
parser.add_argument("--vm", help="display info")
args = parser.parse_args()
vm = args.vm
return vm
vm = get_args()
def print_hyp(name_s):
for info in parser.sections():
for key, value, in parser.items(info):
for value in value.split(): # split whitespace vms value
if (value == name_s):
global info_s
info_s = info
print(name_s, "is deployed on", info)
if (print_hyp):
print_hyp(vm)
name = parser.get(info_s, 'name')
how = parser.get(info_s, 'how')
print("Name:", name)
print("How:", how)
So I need to get section which belongs to value: function print_hyp. And when I get it, I have to print other key with their value.
I'm sure my script is ugly, how can I improve it ?
And I have not errors management, for exemple the output with a vm that does not exist:
./script.py --vm stackoverflow
Traceback (most recent call last): File "script.py", line 43, in <module>
name = parser.get(info_s, 'name')
NameError: global name 'info_s' is not defined
Same thing without argument.
Could you help me?
Many thanks!
I would add
vm = get_args()
print('vm', vm) # to verify vm arg is what you want
This if doesn't make sense:
if (print_hyp):
print_hyp(vm)
...
Do you want something like if vm is not None:?
print_hyp(name_s) doesn't return anything. In prints something(s). It's use of global info_s is unwise. I use global so little that I don't even remember whether it has to also be initialized outside the function. I think your function should should return it, or a list of its values, rather than use global.
You should also be using the if __name__=='__main__:`. If you don't know what that is for, I'd suggest reviewing some basic script layout documentation.
Related
I have a function that parses a given string with specific rules. I would like to design a CLI interface for this function. But the problem is I want that a user should be able to call this function via CLI using a READER & WRITER function of its own. To make it clear, here is a sample code and a demonstration of what I'm trying to explain.
# mylib.py
# piece of code that belongs to my lib
def parser(_id, text):
# parse the text & do some magic
return (_id, parsed_text)
# user-side code
def reader():
# read from a database
# or file or network or who knows where
yield (_id, text)
# user-side code
def writer(_id, text):
# write to somewhere
return True # or false depends on write action
A sample call should be something like this:
$ python mylib.py --reader <something-that-I-dont-know>
I don't want to use eval tricks but also I want that the user should be flexible while passing data to my library. Does this possible? Or should I try another approach?
With the help of #AlexHall, I've come up with the following solution:
import pathlib
import importlib.util
def load_module(filepath):
module_path = pathlib.Path(filepath)
abs_path = module_path.resolve()
module_name = module_path.stem
spec = importlib.util.spec_from_file_location(module_name, abs_path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
Using this function, I am be able to import any valid python module exists in the filesystem even if the module is not in the path.
Here is a sample usage:
parser = make_parser(prog="tokenizer")
args = parser.parse_args()
module = load_module(args.writer) # if nothing is passed, default action defined in the parser
writer = module.writer
module = load_module(args.reader)
reader = module.reader
# do what you want to do with them
This is the part of the mailer.py script:
config = pyfig.Pyfig(config_file)
svnlook = config.general.svnlook #svnlook path
sendmail = config.general.sendmail #sendmail path
From = config.general.from_email #from email address
To = config.general.to_email #to email address
what does this config variable contain? Is there a way to get the value for config variable without pyfig?
In this case config = a pyfig.Pyfig object initialised with the contents of the file named by the content of the string config_file.
To find out what that object does and contains you can either look at the documentation and/or the source code, both here, or you can print out, after the initialisation, e.g.:
config = pyfig.Pyfig(config_file)
print "Config Contains:\n\t", '\n\t'.join(dir(config))
if hasattr(config, "keys"):
print "Config Keys:\n\t", '\n\t'.join(config.keys())
or if you are using Python 3,
config = pyfig.Pyfig(config_file)
print("Config Contains:\n\t", '\n\t'.join(dir(config)))
if hasattr(config, "keys"):
print("Config Keys:\n\t", '\n\t'.join(config.keys()))
To get the same data without pyfig you would need to read and parse at the content of the file referenced by config_file within your own code.
N.B.: Note that pyfig seems to be more or less abandoned - no updates in over 5 years, web site no longer exists, etc., so I would strongly recommend converting the code to use a json configuration file instead.
I'm trying to make a game engine in Python as a "weekend project" but I have a problem.
I'm trying to make it that a user can declare a keybind by typing the key and the function into a text file what they want to run but when I run the code from that sheet with exec it runs the function and I have no idea on how to call the function through the variable. (Also I don't want it to run the function when it runs the code.)
Here is the code I use for executing the code from binds.zdata
for line in bind:
try:
exec line
except:
try:
error.write(localtime + " : " + "ERROR: Could not bind key from the bind file : " + line)
except:
pass
Here's the text in binds.zdata
_w = Functions.motion.move().forward()
_a = Functions.motion.move().left()
_s = Functions.motion.move().back()
_d = Functions.motion.move().right()
You want to lose the () at the end of each line:
_w = Functions.motion.move().forward
_a = Functions.motion.move().left
_s = Functions.motion.move().back
_d = Functions.motion.move().right
now you can call the function through the variable by simply applying parenthesis, such as:
_w()
I'm not sure I recommend "exec" since somebody could put any code in there.
But here is your problem.
_w = Functions.motion.move().forward()
Calls the function 'forward' and puts the results in _w.
_w = Functions.motion.move().forward
Assigns the function 'forward' to the variable '_w'.
Since you asked what I would do, I would create a set of tokens that represent the various functions and then let them do the mapping in side a config file (see: ConfigParser). Then parse the config file. It is a little more work, but a lot more secure.
I'd first change the lines in the binds.zdata file like this so they assign a function to each of the variables:
_w = Functions.motion.move().forward
_a = Functions.motion.move().left
_s = Functions.motion.move().back
_d = Functions.motion.move().right
Then, assuming that Functions is defined, I'd execute all the lines in the file at once with something like:
try:
execfile('binds.zdata')
except Exception as exc:
error.write(localtime + " : " + "ERROR: executing binds file")
Afterwards, the _w, _a, _s, _d variables will have the desired function assigned to them, which you can call as usual — i.e. _w(), _a(), etc.
Reading the config like that with exec is going to open you to all kinds of pain at runtime, as the user's config file is going to be executable code that you are just trusting. Malicious or unintentionally just bad commands will get execute. You could template it like this:
def read_config(config_file):
user_button_map = {}
with open(config_file, 'r') as fh:
for line in fh.readlines():
key,func_key = line.strip().split(',')
assert(key in BUTTON_MAP_FUNCTIONS),"no such action"
user_button_map[key] = BUTTON_MAP_FUNCTIONS[func_key]
return user_button_map
BUTTON_MAP_FUNCTIONS = {
"forward" : Functions.motion.move().forward,
"left" : Functions.motion.move().left,
# etc...
}
# sample config file
# w, forward
# a, left
# etc...
I'm developping a PyQT application. This application is able to load some data from a database, and then do various analysis on these data. All of this works. But as the analyses can be quite complicated, and as a will not be the only user, I had to develop a system with users defined script.
Basically, there's a text editor where the user can program his own small python script (with functions). Then the user can save the script or execute it, by loading the file as a module (within the application).
Here, there a simplified version of my application.
The core of the application is in My_apps.py
and the plugins are here in the same folder i.e. Plugin_A.py
this is the code of My_apps.py:
import sys,os
class Analysis(object):
def __init__(self):
print "I'm the core of the application, I do some analysis etc..."
def Analyze_Stuff(self):
self.Amplitudes_1=[1,2,3,1,2,3]
class Plugins(object):
def __init__(self):
newpath = "C:\Users\Antoine.Valera.NEUROSECRETION\Desktop\Model" #where the file is
sys.path.append(newpath)
Plugin_List=[]
for module in os.listdir(newpath):
if os.path.splitext(module)[1] == ".py":
module=module.replace(".py","")
Plugin_List.append(module)
for plugin in Plugin_List:
a=__import__(plugin)
setattr(self,plugin,a)
def Execute_a_Plugin(self):
Plugins.Plugin_A.External_Function(self)
if __name__ == "__main__":
Analysis=Analysis()
Plugins=Plugins()
Plugins.Execute_a_Plugin()
and here is an example of the code of Plugin_A.py
def External_Function(self):
Analysis.Analyze_Stuff()
print Analysis.Amplitudes_1
why do I get :
Traceback (most recent call last):
File "C:\Users\Antoine.Valera.NEUROSECRETION\Desktop\Model\My_Apps.py", line 46, in <module>
Plugins.Execute_a_Plugin()
File "C:\Users\Antoine.Valera.NEUROSECRETION\Desktop\Model\My_Apps.py", line 37, in Execute_a_Plugin
Plugins.Plugin_A.External_Function(self)
File "C:\Users\Antoine.Valera.NEUROSECRETION\Desktop\Model\Plugin_A.py", line 8, in External_Function
Analysis.Analyze_Stuff()
NameError: global name 'Analysis' is not defined
but if I add the 2 lines following lines instead of Plugins.Execute_a_Plugin()
Analysis.Analyze_Stuff()
print Analysis.Amplitudes_1
then, it works.
How could I indicate to every dynamically loaded plugins that he has to use the variables/objects already existing in Analysis? Why can't I print Analysis.Amplitudes_1 from within the plugin?
Thank you!!
The error message seems perfectly clear: the name "Analysis" doesn't exist in the namespace of the Plugin_A module you imported, and so External_Function cannot access it.
When you import the Plugin_A module, it doesn't get access to the names in the namespace of the importing module, My_apps. So it cannot "see" the instance of the Analysis class that you created there.
A simple solution to this is to change the signature of External_Function (and other related functions), so that it can take an instance of the Analysis class:
Plugin_A.py:
def External_Function(self, analysis):
analysis.Analyze_Stuff()
print analysis.Amplitudes_1
My_apps.py:
...
def Execute_a_Plugin(self):
plugins.Plugin_A.External_Function(self, analysis)
if __name__ == "__main__":
analysis = Analysis()
plugins = Plugins()
plugins.Execute_a_Plugin()
Note that I have altered the naming so that the instance names don't shadow the class names.
You have to import your module. Add the following on top of Plugin_A.py
from My_apps import Analysis
A = Analysis()
A.Analyze_Stuff()
print A.Amplitudes_1
I want to change the env.hosts dynamically because sometimes I want to deploy to one machine first, check if ok then deploy to many machines.
Currently I need to set env.hosts first, how could I set the env.hosts in a method and not in global at script start?
Yes you can set env.hosts dynamically. One common pattern we use is:
from fabric.api import env
def staging():
env.hosts = ['XXX.XXX.XXX.XXX', ]
def production():
env.hosts = ['YYY.YYY.YYY.YYY', 'ZZZ.ZZZ.ZZZ.ZZZ', ]
def deploy():
# Do something...
You would use this to chain the tasks such as fab staging deploy or fab production deploy.
Kind of late to the party, but I achieved this with ec2 like so (note in EC2 you do not know what the ip/hostname may be, generally speaking - so you almost have to go dynamic to really account for how the environment/systems could come up - another option would be to use dyndns, but then this would still be useful):
from fabric.api import *
import datetime
import time
import urllib2
import ConfigParser
from platform_util import *
config = ConfigParser.RawConfigParser()
#task
def load_config(configfile=None):
'''
***REQUIRED*** Pass in the configuration to use - usage load_config:</path/to/config.cfg>
'''
if configfile != None:
# Load up our config file
config.read(configfile)
# Key/secret needed for aws interaction with boto
# (anyone help figure out a better way to do this with sub modules, please don't say classes :-) )
global aws_key
global aws_sec
aws_key = config.get("main","aws_key")
aws_sec = config.get("main","aws_sec")
# Stuff for fabric
env.user = config.get("main","fabric_ssh_user")
env.key_filename = config.get("main","fabric_ssh_key_filename")
env.parallel = config.get("main","fabric_default_parallel")
# Load our role definitions for fabric
for i in config.sections():
if i != "main":
hostlist = []
if config.get(i,"use-regex") == 'yes':
for x in get_running_instances_by_regex(aws_key,aws_sec,config.get(i,"security-group"),config.get(i,"pattern")):
hostlist.append(x.private_ip_address)
env.roledefs[i] = hostlist
else:
for x in get_running_instances(aws_key,aws_sec,config.get(i,"security-group")):
hostlist.append(x.private_ip_address)
env.roledefs[i] = hostlist
if config.has_option(i,"base-group"):
if config.get(i,"base-group") == 'yes':
print "%s is a base group" % i
print env.roledefs[i]
# env["basegroups"][i] = True
where get_running_instances and get_running_instances_by_regex are utility functions that make use of boto (http://code.google.com/p/boto/)
ex:
import logging
import re
from boto.ec2.connection import EC2Connection
from boto.ec2.securitygroup import SecurityGroup
from boto.ec2.instance import Instance
from boto.s3.key import Key
########################################
# B-O-F get_instances
########################################
def get_instances(access_key=None, secret_key=None, security_group=None):
'''
Get all instances. Only within a security group if specified., doesnt' matter their state (running/stopped/etc)
'''
logging.debug('get_instances()')
conn = EC2Connection(aws_access_key_id=access_key, aws_secret_access_key=secret_key)
if security_group:
sg = SecurityGroup(connection=conn, name=security_group)
instances = sg.instances()
return instances
else:
instances = conn.get_all_instances()
return instances
Here is a sample of what my config looked like:
# Config file for fabric toolset
#
# This specific configuration is for <whatever> related hosts
#
#
[main]
aws_key = <key>
aws_sec = <secret>
fabric_ssh_user = <your_user>
fabric_ssh_key_filename = /path/to/your/.ssh/<whatever>.pem
fabric_default_parallel = 1
#
# Groupings - Fabric knows them as roledefs (check env dict)
#
# Production groupings
[app-prod]
security-group = app-prod
use-regex = no
pattern =
[db-prod]
security-group = db-prod
use-regex = no
pattern =
[db-prod-masters]
security-group = db-prod
use-regex = yes
pattern = mysql-[d-s]01
Yet another new answer to an old question. :) But I just recently found myself attempting to dynamically set hosts, and really have to disagree with the main answer. My idea of dynamic, or at least what I was attempting to do, was take an instance DNS-name that was just created by boto, and access that instance with a fab command. I couldn't do fab staging deploy, because the instance doesn't exist at fabfile-editing time.
Fortunately, fabric does support a truly dynamic host-assignment with execute. (It's possible this didn't exist when the question was first asked, of course, but now it does). Execute allows you to define both a function to be called, and the env.hosts it should use for that command. For example:
def create_EC2_box(data=fab_base_data):
conn = boto.ec2.connect_to_region(region)
reservations = conn.run_instances(image_id=image_id, ...)
...
return instance.public_dns_name
def _ping_box():
run('uname -a')
run('tail /var/log/cloud-init-output.log')
def build_box():
box_name = create_EC2_box(fab_base_data)
new_hosts = [box_name]
# new_hosts = ['ec2-54-152-152-123.compute-1.amazonaws.com'] # testing
execute(_ping_box, hosts=new_hosts)
Now I can do fab build_box, and it will fire one boto call that creates an instance, and another fabric call that runs on the new instance - without having to define the instance-name at edit-time.