We Keep Coding

read variables from a blank file in python

If I have a folder structure like the attached picture. The test.config.py is an empty file.
The default.py contains name variables indicate default folder root and addin information
local_root = r'c:\temp\project\cache'
local_input = local_root + r'inputs'
local_output = local_root + r'outputs'
addin_location = r'c:\user\...
addin_name = r'project_addin'
addin_version = r'1.1'
The setting.py contains name variables to overwrite addin information for testing.
addin_location = r'd:\user\...
addin_name = r'project_addin'
addin_version = r'2.1'
I want to import all variables from default.py and all variables from setting.py to init.py. Then overwrite variables with the same names imported from default.py use setattr(). i.e the addin_location, addin_name, and addin_version in default.py share the same name as variables in setting.py, thus overwrite those with setting.py.
Lastly, for any test.py files in test folder, it cannot refers to any of the variables using default.names or setting.names, but instead use config.names (basically the config.py should contain all variable names from default.py with overwritten information from setting.py, so that the codes in test.py only refer to the variable names in config.py). I have manually updated all reference to config.py but don't know how to put all variable names to config.py as it is an empty file. I think need to write some functions in init.py to dump those variable names to config.py
Thanks for the help.

Do you need to keep a track of those config in a file or do you just want to access all information from the combine default.py + any setting.py overwrite ?
For the latter, I think you can use dataclasses:
# default.py
from dataclasses import dataclass
# Here I am defining the default parameters for the class instance
#dataclass
class Config:
local_root: str = r'c:\temp\project\cache'
local_input: str = local_root + r'inputs'
local_output: str = local_root + r'outputs'
addin_location: str = r'c:\user\...'
addin_name: str = r'project_addin'
addin_version: str = r'1.1'
Then in setting you can instantiate the class by replacing by placing all overwriting values in a dict.
# setting.py
from .default import Config
overwrite_settings = {
"addin_location": r'd:\user\...',
"addin_name": r'project_addin',
"addin_version": r'2.1'
}
config = Config(**overwrite_settings)
Then in your test file you can import the config object and access the variable as follow:
# test.py
from .setting import config
my_path_root = config.local_root
And if you want to use init just instantiate the class in the init.py file and import the config with it:
#init.py
from .setting import overwrite_settings
from .default import Config
config = Config(**overwrite_settings)
For the former I am not sure how to save directly into a ready to use python file. Especially at run time, it might be tricky.
But if you want to keep a track of the config that you run you can add a
__post_init__ method to your class in order to save it to a json for example:
# default.py
from dataclasses import dataclass
#dataclass
class Config:
local_root: str = r'c:\temp\project\cache'
local_input: str = local_root + r'inputs'
local_output: str = local_root + r'outputs'
addin_location: str = r'c:\user\...'
addin_name: str = r'project_addin'
addin_version: str = r'1.1'
def __post_init__(self):
with open('run_config.json', 'r') as file:
json.dump(self.__dict__, file)
Hope this helps.

Python: always import the last revision in the directory

Imagine that we have the following Data Base structure with the data stored in python files ready to be imported:
data_base/
foo_data/
rev_1.py
rev_2.py
bar_data/
rev_1.py
rev_2.py
rev_3.py
In my main script, I would like to import the last revision of the data available in the folder. For example, instead of doing this:
from data_base.foo_data.rev_2 import foofoo
from data_base.bar_data.rev_3 import barbar
I want to call a method:
import_from_db(path='data_base.foo_data', attr='foofoo', rev='last')
import_from_db(path='data_base.bar_data', attr='barbar', rev='last')
I could take a relative path to the Data Base and use glob.glob to search the last revision, but for this, I should know the path to the data_base folder, which complicates things (imagine that the parent folder of the data_base is in sys.path so the from data_base.*** import will work)
Is there an efficient way to maybe retrieve a full path knowing only part of it (data_base.foo_data)? Other ideas?

I think it's better to install the last version.
but going on with your flow, you may use getattr on the module:
from data_base import foo_data
i = 0
while True:
try:
your_module = getattr(foo_data, f'rev_{i}')
except AttributeError:
break
i += 1
# Now your_module is the latest rev

#JohnDoriaN 's idea led me to a quite simple solution:
import os, glob
def import_from_db(import_path, attr, rev_id=None):
"""
"""
# Get all the modules/folders names
dir_list = import_path.split('.')
# Import the last module
exec(f"from {'.'.join(dir_list[:-1])} import {dir_list[-1]}")
db_parent = locals()[dir_list[-1]]
# Get an absolute path to corresponding to the db_parent folder
abs_path = db_parent.__path__._path[0]
rev_path = os.path.join(abs_path, 'rev_*.py')
rev_names = [os.path.basename(x) for x in glob.glob(rev_path)]
if rev_id is None:
revision = rev_names[-1]
else:
revision = rev_names[rev_id]
revision = revision.split('.')[0]
# import attribute
exec(f'from {import_path}.{revision} import {attr}', globals())
Some explanations:
Apparently (I didn't know this), we can import a folder as a module; this module has a __path__ attribute (found out using the built-in dir method).
glob.glob allows us to use regex expressions to search for a required pattern for files in the directory.
using exec without parameters will import only in the local namespace (namespace of the method) so without polluting the global namespace.
using exec with globals() allows us to import in the global namespace.

Possible to generate auto-complete from a namespace object?

Imagine I have a .env file that looks like this.
EARTH_SYNONYM1 = "World"
EARTH_SYNONYM2 = "Planet"
EARTH_SYNONYM3 = "Globe"
I've managed to load it into a namespace.
import json
from pathlib import Path
from types import SimpleNamespace
from dotenv.main import dotenv_values # dotenv package needs to be installed.
def json_to_python(json_str):
return json.loads(json_str, object_hook=lambda d: SimpleNamespace(**d))
dotenv_path = Path(".") / ".env"
dotenv_vars_list = dotenv_values(dotenv_path)
dotenv_vars_as_json = json.dumps(dotenv_vars_list)
dotenv_vars = json_to_python(dotenv_vars_as_json)
print(dotenv_vars)
Which prints
namespace(EARTH_SYNONYM1='World', EARTH_SYNONYM2='Planet', EARTH_SYNONYM3='Globe')
Now I can do things like
print(f"Hello {dotenv_vars.EARTH_SYNONYM1}")
Which prints
Hello World
What I would love to accomplish is turning this into a class module (I think).
I'd like to
import dotenv_vars
And then type
dotenv_vars.
and be presented with a list of auto-complete options.
Continuing this example, each of the EARTH_SYNONYMx would show as auto-complete options.
How can I make the namespace object provide auto-complete?

I always thought dotenv is a pretty silly library and overcomplicating something that is so simple. Load your vars into a module namespace with Python's import system.
import imp
dotenv_vars = imp.load_source('dotenv_vars', '.env')

Reference another class in python

I'm just starting out with Python for Google App Engine. I have a file notifications.py, and in here, I will be creating User entities, which are specified in users.py. How can I do this? I've tried import users, but I get an error: NameError: global name 'User' is not defined

Oh, I just had this problem too! After you do:
import users
to get User you have to type users.User
Alternatively you could import it like:
from users import User
then reference it as just User but if you do it this way you'll have to list every bit from users that you want in the following format:
from users import User, Somthingelse, Somthing
If you're feeling super lazy and you don't want to type in any prefixes or list all the things you want, just type
from users import *

Instead of
import users
do
from users import User

# module.py
foo = "bar"
# main.py
import module
print foo # This will cause error because foo is not located in the current namespace
print module.foo # this will print "bar"
from module import foo # But you can import contents of module "module" in the current namespace
http://docs.python.org/tutorial/modules.html

Django - importing dynamically

I'm trying to do a dynamic import of a python module in django. I have two different apps that I want to import from, and I want to replace these import statements:
from app1.forms import App1ProfileForm
from app2.forms import App2ProfileForm
I am dynamically able to create the strings App1ProfileForm and App2ProfileForm and then instantiate them like so:
globals()[form]()
I tried following some of the instructions in this post: Dynamically import class by name for static access
and so I tried doing this:
theModule = __import__("app1.forms.App1ProfileForm")
but I'm getting an error that says No module named App1ProfileForm
EDIT:::
Ok I tried this code:
theModule = __import__("app1")
print theModule
theClass = getattr(theModule,'forms')
print theClass
theForm = getattr(theClass,'App1ProfileForm')
print theForm
theForm.initialize()
but I get an error that type object 'App1ProfileForm' has no attribute 'initialize'

You don't want to do this. Imports are done when the relevant code is first executed - in the case of module-level imports, it's when the module itself is imported. If you're depending on something in the request, or some other run-time element, to determine what class you want, then this will not work.
Instead, just import them both, and get the code to choose which one you need:
from app1.forms import App1ProfileForm
from app2.forms import App2ProfileForm
forms = {'app1': App1ProfileForm,
'app2': App2ProfileForm}
relevant_form = forms[whatever_the_dependent_value_is]

I don't quite know how you're generting the string to import. I'll assume you generate the whole "path". Try this:
def import_from_strings(paths):
ret = []
for path in paths:
module_name, class_name = path.rsplit('.', 1)
module = __import__(module_name, globals(), locals(), [class_name], -1)
ret.append(getattr(module, class_name))
return ret

Aren't you trying to import a class, and not a module ? I'm not an expert, but I think you must import the module using __import__, then select it's App1ProfileForm class with something like yourmodule.App1ProfileForm

I figured it out. Here's how to do it:
theModule = __import__(module_name+".forms") # for some reason need the .forms part
theClass = getattr(theModule,'forms')
theForm = getattr(theClass,form_name)
then to initialize:
theForm() or theForm(request.POST)