Good day.
I set up a separate project from the main one called myproject-celery. It is a buildout based project which contains the async part of my project. For convenience I want to have a file, that will be containing this machine's configuration. I know that celery provides the python config file, but I do not like this configuration style.
Let's say I have a configuration in a Yaml config file named myproject.yaml
What I want to achieve:
./bin/celery worker --config /absolute/path/to/project/myproject.yaml --app myproject.celery
The problem really is that I want to specify the file's location, because it can change. I tried writing a custom loader class, but I failed, cause I do not even know why and when the many custom methods of this class are called (the only doc that I found is http://docs.celeryproject.org/en/latest/reference/celery.loaders.base.html?highlight=loader#id1 and It's no help for me). I tried to do something on import phase for the app module, but I can not pass the filepath to that module's code... The only solution that I came up with was using a custom ENV param that will contain the path, but I do not see why can't it be a launch param like in most apps, that I use(refering to pyramid with it's paster serve myproject.ini)
So the question:
What do I have to do to set up the config from a file that I could specify by an absolute path?
EDIT:
The question was not answered, sow I posted an issue on celery's github. Will wait for a response.
https://github.com/celery/celery/issues/1100
Looking at celery.loaders.base it looks like the method you want to override is read_configuration:
from celery.datastructures import DictAttr
from celery.loaders.base import BaseLoader
class YAMLLoader(BaseLoader):
def read_configuration():
# Load YAML file here and return a DictAttr instance
Related
I am creating a piece of code that will have multiple files that need to reference a single config. The reason is that our IT department will use puppet to manage this config file in case any changes are required in the future. We don't want to do a release to change the configuration. I've seen a few projects that have multiple configs in different places but I really do not like this idea and I'd prefer to have a single source. My thought was to create a specific config.py file that can be called anywhere in my code that will ask for the user to input the location of the file.
Is this a good way or is there a better way to do this?
import configparser
class Config(object):
def __init__(self,conf):
self._cfg = configparser.ConfigParser()
self._cfg.read(conf)
def get_conf_value(self,property):
if property not in self._cfg.sections:
return None
return self._cfg.sections[property]
If so, if I have a Main.py while, what's the best way to have the scheduler pass the config location and then reference it across all of my files in my Python Package?
You can create a config.json file in json format. You can read the contents at startup using the json.load function from the json library.
My Revit Add-in reads at some point a text file, that could be located anywhere. In my current implementation, the path to the text file is hardcoded. I'd like to avoid that, so that when I distribute the Add-in to other people, it doesn't simply crash.
Ideally I'd like to give them the ability of specifying their own location for that file on their computer, and that they don't need to re-specify it every time they re-launch the Add-in!
In other words, I'd like to store once and for all this information. And if you close and re-open Revit, the location is still stored somewhere when you re-use the Addin.
This question is actually similar to this one, except that I'd need a solution when developing in Python (pyRevit). Any help?
if you're developing you addon in pyRevit, then you can use the pyrevit.script module to get the configuration for that script.
Ask user for the file location (pyrevit.forms.save_file helps) and then save the file path in the script configuration. pyRevit handles this automatically and saves the information inside its master configuration file at %appdata%/pyRevit
from pyrevit import script
config = script.get_config()
config.filelocation = 'path/to/your/file'
script.save_config()
And then later, read the configuration like this:
from pyrevit import script
config = script.get_config()
print(config.filelocation)
# or to get the config safely
print(config.get_option('filelocation', None)
I implemented two other ways to store Revit add-in settings in the HoloLens Escape Path Waypoint JSON Exporter:
Store add-in option settings in XML using the .NET System.Configuration.ApplicationSettingsBase class
Store add-in option settings in JSON using custom solution and JavaScriptSerializer class
Both solutions are well suited for what you need.
Check them out in the ExportWaypointsJson GitHub repository.
I have a flask application which is utilizing Fabric to deploy code to certain machines.
One particular method, fabric.contrib.files.upload_template, requires a filename which is the path of a file I would like to upload to my remote machines.
I have a file, index.php, which I want to place on my remote machines. I want to use upload_template to accomplish this task.
Where should I place 'index.php' in my flask folder? How will I be able to obtain the path of index.php for upload_template?
Please note that:
My application is hosted on Heroku so the typical /home/Sparrowcide/flaskapp/path/to/file won't work.
I know I can probably get away by invoking a method which writes to /tmp/path/to/file, but I'd rather not as this is not a very elegant solution.
Well the short answer is that you can pretty much put it wherever you want, I don't think flask will enforce any sort of structure on you here. However I personally would create a subdirectory called data and then in the flask app I might do something like this:
path = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'data', 'index.php')
Or, if I felt I was likely to re-use this information (eg if i had multiple files to upload), I might break it out into multiple variables:
SRCDIR = os.path.dirname(os.path.abspath(__file__))
DATADIR = os.path.join(SRCDIR, 'data')
whatever_upload_function(os.path.join(DATADIR, 'index.php'))
paster serve has --reload option to auto-restart serving wsgi application when any of Python source files or the CONFIG_FILE changes.
How to make paster initiate auto-restart also when some other file (not Python source file) changes?
UPDATE
watch_file() function suggested by mksh looks like the solution to the problem. However mksh suggested adding its invocation to the application's entry point which seems to be more invasive than it should. Can I (non-intrusively) extend Paste's serve command adding new option which would result in invocation of watch_file() with filenames read from the app's section in CONFIG_FILE?
See Paster source link
So, you can watch your non-source files as simple as putting such lines at bottom of your application`s entry points:
from paste.reloader import watch_file
#
# logic that puts list of your non-source file names suitable
# for open() into iterable non_source_file_list
#
for non_source_file in non_source_file_list:
watch_file(non_source_file_name)
In general, try to rely more on source code than documentation when working with such modern and really written in pythonic style frameworks as Paste, their code is mostly well documented and futhermore self-documenting .
I have a project with 10 different python files. It has classes and functions - pretty much the lot.
I do want to share specific data that will represent the settings in the project between all the project files.
I came up with creating a settings.py file:
settings = {}
settings['max_bitrate'] = 160000
settings['dl_dir'] = r"C:\Downloads"
and then I import the class from every file.
Is there a more suitable way to do it?
I'm probably a little old-school in this regard, but in my latest project, I created a config file in /etc, then created a config module that uses ConfigParser to read it in and make it available, and import that config module wherever I need to read settings.
Your method sounds good to me, and has the advantage that you can easily change the implementation of the settings module, for example to use configuration files or the windows registry, or to provided a read only API.