My problem is i can't find a default value for FlagConvert if the previous parameter of FlagConvert has a another default value.
For example:
class ClearFlag(FlagConverter, prefix='--', delimiter=':'):
has: str = None
match: str = None
by: Tuple[Union[User, Member], ...] = None
check_all: bool = True
#command('clear', aliases=['cls', 'clean'])
async def clear_messages(ctx: Context, limit: int = 10, flags: ClearFlag = ???):
...
In docs examples on commands.html,
always FlagConvert is required and I can't find any answer.
I use discord.py v2.0, so you need these:
Docs
Installation (install the development version).
discord.ext.commands.FlageConvert on API Refrence and Commands
I can use ClearFlags object. because has all of attributes I need. For example:
#command('clear', aliases=['cls', 'clean'])
async def clear_messages(ctx: Context, limit: int = 10, flags: ClearFlag = ClearFlag()):
...
But should never be a type
When adding a SubProblem to a Group which itself was added to another Group, an error occurred (Python3, OpenMDAO 1.7.3). I solved this by overloading the _get_relname_map(self, parent_proms) method of SubProblem. Only pkey = '.'.join((self.name, key)) was changed to pkey = '.'.join((self.pathname, key)):
from openmdao.api import Problem, Group, Component, IndepVarComp, ExecComp, \
ScipyOptimizer, SubProblem
from collections import OrderedDict
class MySubProblem(SubProblem):
def __init__(self, problem, params=(), unknowns=()):
super().__init__(problem, params, unknowns)
def _get_relname_map(self, parent_proms):
umap = OrderedDict()
for key in self._prob_unknowns:
pkey = '.'.join((self.pathname, key)) # self.name -> self.pathname
if pkey in parent_proms:
umap[parent_proms[pkey]] = key
return umap
def setup_sub():
# create subproblem example
sub = Problem(root=Group())
sub.root.add('indep', IndepVarComp([
('x', 0.0), ('y', 2.0)]))
# here's the actual function we're minimizing
sub.root.add("comp", ExecComp("fx = cos(x)-y"))
sub.driver = ScipyOptimizer()
sub.driver.options['optimizer'] = 'COBYLA'
sub.driver.add_desvar("indep.x", lower=-3, upper=3)
sub.driver.add_objective("comp.fx")
sub.root.connect("indep.x", "comp.x")
sub.root.connect("indep.y", "comp.y")
return sub
###
### Subproblem added to 'root'-Group -> working
prob0 = Problem(root=Group())
sp = prob0.root.add(
"subprob",
MySubProblem(
setup_sub(),
params=['indep.x', 'indep.y'],
unknowns=['comp.fx']))
prob0.setup(check=False)
prob0.run()
print("prob0 done")
###
### Subproblem added to Group which is added to 'root'-Group -> not working
prob1 = Problem(root=Group())
prob1.root.add('main', Group())
sp = prob1.root.main.add(
"subprob",
MySubProblem(
setup_sub(),
params=['indep.x', 'indep.y'],
unknowns=['comp.fx']))
prob1.setup(check=False)
prob1.run()
print("prob1 done")
The example above works, but when you comment the _get_relname_map definition in MySubProblem, error is raised while prob1.run().
My question is now: Is there any reason to use self.name instead of self.pathname? Could my solution cause any other problem?
Thank you very much!
Best regards,
Jerome
OpenMDAO 1.X is not actively supported any more, and sub-problems were always a bit of an experimental feature in it anyway. If your fix works for you, it should be fine.
My head is probably in the wrong place with this, but I want to put a variable within a variable.
My goal for this script is to compare current versions of clients software with current software versions that are available from the vendor. At this stage I just want to print out what's available.
I have some def's setup with:
def v80(program_1 = '80.24', program_2 = '80.5', program_3 = '80.16'):
pass
def v81(program_1 = '81.16', program_2 = '81.7', program_3 = '81.14'):
pass
def v82(program_1 = '82.15', program_2 = '82.4', program_3 = '82.9'):
pass
def v83(program_1 = '83.01', program_2 = '83.0', program_3 = '83.1'):
pass
I'm then reading all of the clients versions from a text file and doing comparisons.
One of the vars I'm generating is "program_main", currently I'm doing something like:
If program_main == "83":
if program_1:
if v83['program_1'] > float(program_1):
print ("Update available", program_1, "-->", v83[program_1])
if program_2:
if v83['program_2'] > float(program_2):
print ("Update available", program_2, "-->", v83[program_2])
if program_main == "82"
if program_1:
if v82['program_1'] > float(program_1):
print ("Update available", program_1, "-->", v82[program_1])
etc etc
My train of though would be something like
if program_1:
if v[program_main] > float(program_1):
print('Update available", program_1, "-->", v[program_main])
etc etc
I'm sure there's a much better way to do this entire setup, but this is one of my first proper python scripts so I'm happy to chalk it up to noobish-ness, just wanted to know what the right way of doing what I'm trying to achieve is.
You can put your functions into a dictionary:
per_version = {
'83': v83,
'82': v82,
}
and simply use that to map string to function:
per_version[program_main]('program_1')
However, you may want to instead parameterise your version functions; make one function that takes the version as a parameter:
def program_check(version, program_1=None, program_2=None, program_3=None):
# ...
which then looks up default values per program_x parameter based no the version, again from a dictionary perhaps.
I created EXE file with Python (PySide) + PyInstaller. Once I try to use
print QtGui.QApplication.applicationVersion()
I don't see valid version in x.x.x.x format of the application.
Are there any built-in functions in PySide instead of this, or maybe should I use other library for it?
PS. I don't believe that Python doesn't have any methods to extract information about EXE :)
Have a look here.
There's everything you're looking for !
Hope this helps.
Found one way to do it:
import win32api
def get_version_info():
try:
filename = APP_FILENAME
return get_file_properties(filename)['FileVersion']
except BaseException, err:
log_error(err.message)
return '0.0.0.0'
def get_file_properties(filename):
"""
Read all properties of the given file return them as a dictionary.
"""
propNames = ('Comments', 'InternalName', 'ProductName',
'CompanyName', 'LegalCopyright', 'ProductVersion',
'FileDescription', 'LegalTrademarks', 'PrivateBuild',
'FileVersion', 'OriginalFilename', 'SpecialBuild')
props = {'FixedFileInfo': None, 'StringFileInfo': None, 'FileVersion': '0.0.0.0'}
try:
# backslash as parm returns dictionary of numeric info corresponding to VS_FIXEDFILEINFO struc
print filename
fixedInfo = win32api.GetFileVersionInfo(filename, '\\')
props['FixedFileInfo'] = fixedInfo
props['FileVersion'] = "%d.%d.%d.%d" % (fixedInfo['FileVersionMS'] / 65536,
fixedInfo['FileVersionMS'] % 65536, fixedInfo['FileVersionLS'] / 65536,
fixedInfo['FileVersionLS'] % 65536)
# \VarFileInfo\Translation returns list of available (language, codepage)
# pairs that can be used to retreive string info. We are using only the first pair.
lang, codepage = win32api.GetFileVersionInfo(filename, '\\VarFileInfo\\Translation')[0]
# any other must be of the form \StringfileInfo\%04X%04X\parm_name, middle
# two are language/codepage pair returned from above
strInfo = {}
for propName in propNames:
strInfoPath = u'\\StringFileInfo\\%04X%04X\\%s' % (lang, codepage, propName)
## print str_info
strInfo[propName] = win32api.GetFileVersionInfo(filename, strInfoPath)
props['StringFileInfo'] = strInfo
except BaseException, err:
log_error(err.message)
return props
I'd like to get PyYAML's loader to load mappings (and ordered mappings) into the Python 2.7+ OrderedDict type, instead of the vanilla dict and the list of pairs it currently uses.
What's the best way to do that?
Python >= 3.6
In python 3.6+, it seems that dict loading order is preserved by default without special dictionary types. The default Dumper, on the other hand, sorts dictionaries by key. Starting with pyyaml 5.1, you can turn this off by passing sort_keys=False:
a = dict(zip("unsorted", "unsorted"))
s = yaml.safe_dump(a, sort_keys=False)
b = yaml.safe_load(s)
assert list(a.keys()) == list(b.keys()) # True
This can work due to the new dict implementation that has been in use in pypy for some time. While still considered an implementation detail in CPython 3.6, "the insertion-order preserving nature of dicts has been declared an official part of the Python language spec" as of 3.7+, see What's New In Python 3.7.
Note that this is still undocumented from PyYAML side, so you shouldn't rely on this for safety critical applications.
Original answer (compatible with all known versions)
I like #James' solution for its simplicity. However, it changes the default global yaml.Loader class, which can lead to troublesome side effects. Especially, when writing library code this is a bad idea. Also, it doesn't directly work with yaml.safe_load().
Fortunately, the solution can be improved without much effort:
import yaml
from collections import OrderedDict
def ordered_load(stream, Loader=yaml.SafeLoader, object_pairs_hook=OrderedDict):
class OrderedLoader(Loader):
pass
def construct_mapping(loader, node):
loader.flatten_mapping(node)
return object_pairs_hook(loader.construct_pairs(node))
OrderedLoader.add_constructor(
yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG,
construct_mapping)
return yaml.load(stream, OrderedLoader)
# usage example:
ordered_load(stream, yaml.SafeLoader)
For serialization, you could use the following funcion:
def ordered_dump(data, stream=None, Dumper=yaml.SafeDumper, **kwds):
class OrderedDumper(Dumper):
pass
def _dict_representer(dumper, data):
return dumper.represent_mapping(
yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG,
data.items())
OrderedDumper.add_representer(OrderedDict, _dict_representer)
return yaml.dump(data, stream, OrderedDumper, **kwds)
# usage:
ordered_dump(data, Dumper=yaml.SafeDumper)
In each case, you could also make the custom subclasses global, so that they don't have to be recreated on each call.
2018 option:
oyaml is a drop-in replacement for PyYAML which preserves dict ordering. Both Python 2 and Python 3 are supported. Just pip install oyaml, and import as shown below:
import oyaml as yaml
You'll no longer be annoyed by screwed-up mappings when dumping/loading.
Note: I'm the author of oyaml.
The yaml module allow you to specify custom 'representers' to convert Python objects to text and 'constructors' to reverse the process.
_mapping_tag = yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG
def dict_representer(dumper, data):
return dumper.represent_dict(data.iteritems())
def dict_constructor(loader, node):
return collections.OrderedDict(loader.construct_pairs(node))
yaml.add_representer(collections.OrderedDict, dict_representer)
yaml.add_constructor(_mapping_tag, dict_constructor)
2015 (and later) option:
ruamel.yaml is a drop in replacement for PyYAML (disclaimer: I am the author of that package). Preserving the order of the mappings was one of the things added in the first version (0.1) back in 2015. Not only does it preserve the order of your dictionaries, it will also preserve comments, anchor names, tags and does support the YAML 1.2 specification (released 2009)
The specification says that the ordering is not guaranteed, but of course there is ordering in the YAML file and the appropriate parser can just hold on to that and transparently generate an object that keeps the ordering. You just need to choose the right parser, loader and dumper¹:
import sys
from ruamel.yaml import YAML
yaml_str = """\
3: abc
conf:
10: def
3: gij # h is missing
more:
- what
- else
"""
yaml = YAML()
data = yaml.load(yaml_str)
data['conf'][10] = 'klm'
data['conf'][3] = 'jig'
yaml.dump(data, sys.stdout)
will give you:
3: abc
conf:
10: klm
3: jig # h is missing
more:
- what
- else
data is of type CommentedMap which functions like a dict, but has extra information that is kept around until being dumped (including the preserved comment!)
Note: there is a library, based on the following answer, which implements also the CLoader and CDumpers: Phynix/yamlloader
I doubt very much that this is the best way to do it, but this is the way I came up with, and it does work. Also available as a gist.
import yaml
import yaml.constructor
try:
# included in standard lib from Python 2.7
from collections import OrderedDict
except ImportError:
# try importing the backported drop-in replacement
# it's available on PyPI
from ordereddict import OrderedDict
class OrderedDictYAMLLoader(yaml.Loader):
"""
A YAML loader that loads mappings into ordered dictionaries.
"""
def __init__(self, *args, **kwargs):
yaml.Loader.__init__(self, *args, **kwargs)
self.add_constructor(u'tag:yaml.org,2002:map', type(self).construct_yaml_map)
self.add_constructor(u'tag:yaml.org,2002:omap', type(self).construct_yaml_map)
def construct_yaml_map(self, node):
data = OrderedDict()
yield data
value = self.construct_mapping(node)
data.update(value)
def construct_mapping(self, node, deep=False):
if isinstance(node, yaml.MappingNode):
self.flatten_mapping(node)
else:
raise yaml.constructor.ConstructorError(None, None,
'expected a mapping node, but found %s' % node.id, node.start_mark)
mapping = OrderedDict()
for key_node, value_node in node.value:
key = self.construct_object(key_node, deep=deep)
try:
hash(key)
except TypeError, exc:
raise yaml.constructor.ConstructorError('while constructing a mapping',
node.start_mark, 'found unacceptable key (%s)' % exc, key_node.start_mark)
value = self.construct_object(value_node, deep=deep)
mapping[key] = value
return mapping
Update: the library was deprecated in favor of the yamlloader (which is based on the yamlordereddictloader)
I've just found a Python library (https://pypi.python.org/pypi/yamlordereddictloader/0.1.1) which was created based on answers to this question and is quite simple to use:
import yaml
import yamlordereddictloader
datas = yaml.load(open('myfile.yml'), Loader=yamlordereddictloader.Loader)
On my For PyYaml installation for Python 2.7 I updated __init__.py, constructor.py, and loader.py. Now supports object_pairs_hook option for load commands. Diff of changes I made is below.
__init__.py
$ diff __init__.py Original
64c64
< def load(stream, Loader=Loader, **kwds):
---
> def load(stream, Loader=Loader):
69c69
< loader = Loader(stream, **kwds)
---
> loader = Loader(stream)
75c75
< def load_all(stream, Loader=Loader, **kwds):
---
> def load_all(stream, Loader=Loader):
80c80
< loader = Loader(stream, **kwds)
---
> loader = Loader(stream)
constructor.py
$ diff constructor.py Original
20,21c20
< def __init__(self, object_pairs_hook=dict):
< self.object_pairs_hook = object_pairs_hook
---
> def __init__(self):
27,29d25
< def create_object_hook(self):
< return self.object_pairs_hook()
<
54,55c50,51
< self.constructed_objects = self.create_object_hook()
< self.recursive_objects = self.create_object_hook()
---
> self.constructed_objects = {}
> self.recursive_objects = {}
129c125
< mapping = self.create_object_hook()
---
> mapping = {}
400c396
< data = self.create_object_hook()
---
> data = {}
595c591
< dictitems = self.create_object_hook()
---
> dictitems = {}
602c598
< dictitems = value.get('dictitems', self.create_object_hook())
---
> dictitems = value.get('dictitems', {})
loader.py
$ diff loader.py Original
13c13
< def __init__(self, stream, **constructKwds):
---
> def __init__(self, stream):
18c18
< BaseConstructor.__init__(self, **constructKwds)
---
> BaseConstructor.__init__(self)
23c23
< def __init__(self, stream, **constructKwds):
---
> def __init__(self, stream):
28c28
< SafeConstructor.__init__(self, **constructKwds)
---
> SafeConstructor.__init__(self)
33c33
< def __init__(self, stream, **constructKwds):
---
> def __init__(self, stream):
38c38
< Constructor.__init__(self, **constructKwds)
---
> Constructor.__init__(self)
here's a simple solution that also checks for duplicated top level keys in your map.
import yaml
import re
from collections import OrderedDict
def yaml_load_od(fname):
"load a yaml file as an OrderedDict"
# detects any duped keys (fail on this) and preserves order of top level keys
with open(fname, 'r') as f:
lines = open(fname, "r").read().splitlines()
top_keys = []
duped_keys = []
for line in lines:
m = re.search(r'^([A-Za-z0-9_]+) *:', line)
if m:
if m.group(1) in top_keys:
duped_keys.append(m.group(1))
else:
top_keys.append(m.group(1))
if duped_keys:
raise Exception('ERROR: duplicate keys: {}'.format(duped_keys))
# 2nd pass to set up the OrderedDict
with open(fname, 'r') as f:
d_tmp = yaml.load(f)
return OrderedDict([(key, d_tmp[key]) for key in top_keys])