We Keep Coding

Python Logging - only show logs from my libraries

I have a relatively complex ecosystem of applications and libraries that are scheduled to run in my environment.
I am trying to improve my logging and in particular I'd like to write debug information to a logging file, and I'd like that log to contain the logger.debug("string") lines from all the imported libraries I wrote, but not from libraries I import from pypi.
example:
import sys
import numpy
from bs4 import BeautifulSoup
import logging
import mylibrary
import myotherlibrary
logger = logging.getLogger(application_name) # I don't use _ _ name _ _ in all of them, but I can change this line as necessary
so in this case when I set logger level to debug, I'd like to see debug information from the current script, from mylibrary and from myotherlibrary , but not from bs4,numpy, etc.
bonus: Ideally I would like to not have to hardcode every time the name of the libraries, but just have the script "know" it (from naming convention maybe?)
If anyone has any ideas it'd be greatly appreciated!

Python doesn't really have a concept of "libraries I wrote" vs "libraries imported with pypi" - a library is a library unfortunately.
However, depending on how your libraries are set up, you may be able to get a realllly hacky custom logger?
By default, Python libraries installed with pip go to a central location - usually something like /usr/local/lib or %APPDATA% on windows. In contrast, local libraries are usually within the same directory as the calling script. We can use this to our advantage!
The following code demonstrates a kinda proof-of-concept - I've left a few methods needing implementing as an exercise ;)
#CustomLogger.py
import __main__
import logging
import os
#create a custom log class, inheriting the current logger class
class CustomLogger(logging.getLoggerClass()):
custom_lib = False
def __init__(self, name):
#initialise the base logger
super().__init__(name)
#get the directory we are being run from
current_dir = os.path.dirname(__main__.__file__)
permutations = ['/', '.py', '.pyc']
#check if we are a custom library, or if we are one installed via pip etc
self.custom_lib = self.checkExists(current_dir, permutations)
self.propagate = not self.custom_lib
def checkExists(self, current_dir, permutations):
#loop through each permutation and see if a file matching that spec exists
#currently looks for .py/.pyc files and directories
for perm in permutations:
file = os.path.join(current_dir, self.name + perm)
if os.path.exists(file):
return True
return False
def isEnabledFor(self, level):
if self.custom_lib:
return super().isEnabledFor(level)
return False
#the hackiest part :)
#these are two sample overrides that only log if we're a custom
#library (i.e. one we've written, not installed)
#there are a few more methods that I've not implemented, a full
#list is available at https://docs.python.org/3/library/logging.html#logging.Logger
def debug(self, msg, *args, **kwargs):
if self.custom_lib:
return super().debug(msg, args, kwargs)
def info(self, msg, *args, **kwargs):
if self.custom_lib:
return super().info(msg, args, kwargs)
#most important part - also override the logger class
#this means that any calls to logging.getLogger() will use our new subclass
logging.setLoggerClass(CustomLogger)
You could then use it like this:
import CustomLogger #needs importing first so it ensures the logger is setup
import sys
import numpy
from bs4 import BeautifulSoup
import logging
import mylibrary
import myotherlibrary
logger = logging.getLogger(application_name) #returns type CustomLogger

Use Python to manage content in "Enterprise Architect" software from "sparx systems"

Currently i am working with enterprise architect software for creating packages, diagrams.
Is it possible to work in Enterprise architect software using python script ? Some example like Deleting and creating packages and diagrams etc. If so refer example code or link.

Sure, that's no problem.
import win32com.client
from singleton import Singleton
#Singleton
class Repository:
def __init__(self):
try:
self.eaRep = win32com.client.Dispatch("EA.App").Repository
models = self.eaRep.models
done = True
except Exception as e:
print (e)
done = False
(The #Singleton can be found on the net but you can work without it.)
Then in your main program you can access the repository like
rep = repository.Repository.Instance()
print rep.modules.getAt(0).name
etc. Have fun

import win32com.client
def open_repository(path, login, password):
eaApp = win32com.client.Dispatch("EA.App")
eaRep = eaApp.Repository
if login:
eaRep.SuppressSecurityDialog = True
eaRep.OpenFile2(path, login, password)
else:
eaRep.OpenFile(path)
return eaRep
please use OpenFile for open your model. (OpenFile2 if your model has enabled security)

How are Python command line arguments related to methods?

Everyone at Class too big and hard to add new features is completely unphased by the question, which somehow connects command line options to methods, but I can find no documentation for this. It's not optparse, or argparse, or sys.argv - the question implies some kind of direct relationship between methods and command line options. What am I missing?

There isn't any set-in-stone link between them. The question you link to appears to be a program that can do one of several different things, with command-line arguments switching between them. These things happen to be implemented in the program using methods.
It is implied by the question that they have used something like argparse to write the glue between these; but the use of methods is just an implementation detail of the particular program.

I simply use the class like this, what seems not to be a very good idea, because it is very hard to maintain once u got many commands.
class myprogram(object):
def __init__(self)
self.prepare()
def prepare(self):
# some initializations
self.prepareCommands()
def prepareCommands(self):
self.initCommand("--updateDatabase", self.updateDatabase)
self.initCommand("--getImages", self.getImages)
# and so on
def initCommand(self, cmd, func):
options = sys.argv
for option in options:
if option.find(cmd)!=-1:
return func()
# my commands
def updateDatabase(self):
#...
def getImages(self):
#...
if __name__ == "__main__":
p = myprogram()
EDIT1:
Here a cleaner way I just implemented:
myprogram.py:
from config import * # has settings
from commands import *
from logsys import log
import filesys
class myprogram(object):
def __init__(self):
log(_class=self.__name__, _func='__init__', _level=0)
log(_class=self.__name__, _func='__init__', text="DEBUG LEVEL %s" % settings["debug"], _level=0)
self.settings = settings
self.cmds = commands
def prepare(self):
log(_class=self.__name__, _func='prepare', _level=1)
self.dirs = {}
for key in settings["dir"].keys():
self.dirs[key] = settings["dir"][key]
filesys.checkDir(self.dirs[key])
def initCommands(self):
log(_class=self.__name__, _func='initCommands', _level=1)
options = sys.argv
for option in options:
for cmd in self.cmds.keys():
if option.find(cmd) != -1:
return self.cmds[cmd]()
if __name__ == '__main__':
p = myprogram()
p.prepare()
p.initCommands()
commands.py:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
commands = {}
#csv
import csvsys
commands["--getCSV"] = csvsys.getCSV
#commands["--getCSVSplitted"] = csvsys.getCSVSplitted
# update & insert
import database
commands["--insertProductSpecification"] = database.insertProductSpecification
# download
import download
commands["--downloadProductSites"] = download.downloadProductSites
commands["--downloadImages"] = download.downloadImages
# parse
import parse
commands["--parseProductSites"] = parse.parseProductSites
EDIT2: I have now updated my question you linked to your question with a more complete example Class too big and hard to add new features

Python envisage framework

I just started with envisage framework. In the 4.x version I saw a few example, but I need a good documentation: link.
How can I add custom buttons to the envisage workbench, or how can I create a similar one?

The best place look for documentation is the Acmelab example in the Envisage source tree.
I'm assuming when you talk about custom buttons you mean buttons on a toolbar. First you need to create a WorkbenchActionSet, add your toolbar there, and then define your actions and assign them a button image. Here is the (slightly modified) Acmelab example with non-relevant parts taken out:
test_action_set.py
# Enthought library imports.
from envisage.ui.action.api import Action, Group, Menu, ToolBar
from envisage.ui.workbench.api import WorkbenchActionSet
class TestActionSet(WorkbenchActionSet):
""" An action test useful for testing. """
#### 'ActionSet' interface ################################################
tool_bars = [
ToolBar(name='Fred', groups=['AToolBarGroup']),
ToolBar(name='Wilma'),
ToolBar(name='Barney')
]
actions = [
Action(
path='ToolBar',
class_name='acme.workbench.action.new_view_action:NewViewAction'
),]
new_view_action.py
""" An action that dynamically creates and adds a view. """
# Enthought library imports.
from pyface.api import ImageResource
from pyface.action.api import Action
from pyface.workbench.api import View
class NewViewAction(Action):
""" An action that dynamically creates and adds a view. """
#### 'Action' interface ###################################################
# A longer description of the action.
description = 'Create and add a new view'
# The action's name (displayed on menus/tool bar tools etc).
name = 'New View'
# A short description of the action used for tooltip text etc.
tooltip = 'Create and add a new view'
image = ImageResource(Your Image File Name Goes Here)
###########################################################################
# 'Action' interface.
###########################################################################
def perform(self, event):
""" Perform the action. """
# You can give the view a position... (it default to 'left')...
view = View(id='my.view.fred', name='Fred', position='right')
self.window.add_view(view)
# or you can specify it on the call to 'add_view'...
view = View(id='my.view.wilma', name='Wilma')
self.window.add_view(view, position='top')
return
#### EOF ######################################################################

Sending object C from class A to class B

I can't figure out how to design classes in my system.
In classA I create object selenium (it simulates user actions at website).
In this ClassA I create another objects like SearchScreen, Payment_Screen and Summary_Screen.
# -*- coding: utf-8 -*-
from selenium import selenium
import unittest, time, re
class OurSiteTestCases(unittest.TestCase):
def setUp(self):
self.verificationErrors = []
self.selenium = selenium("localhost", 5555, "*chrome", "http://www.someaddress.com/")
time.sleep(5)
self.selenium.start()
def test_buy_coffee(self):
sel = self.selenium
sel.open('/')
sel.window_maximize()
search_screen=SearchScreen(self.selenium)
search_screen.choose('lavazza')
payment_screen=PaymentScreen(self.selenium)
payment_screen.fill_test_data()
summary_screen=SummaryScreen(selenium)
summary_screen.accept()
def tearDown(self):
self.selenium.stop()
self.assertEqual([], self.verificationErrors)
if __name__ == "__main__":
unittest.main()
It's example SearchScreen module:
class SearchScreen:
def __init__(self,selenium):
self.selenium=selenium
def search(self):
self.selenium.click('css=button.search')
I want to know if there is anything ok with a design of those classes?

Your approach is fine. You have a set of tool classes, each of which needs to know its target. Then you have a toolkit class that coordinates these tools on a particular target.
class AgreePrice:
def __init__(self, connection): ...
class PlaceOrder:
def __init__(self, connection): ...
class ConfirmAvailability:
def __init__(self, connection): ...
class BookingService:
def __init__(self, connection): ...
def book(self):
for Command in (ConfirmAvailability, AgreePrice, PlaceOrder):
command = Command(self.connection)
command.run()
assert command.success()
There's nothing at all wrong with those kinds of class structures, in fact they come up all the time, and are a reasonably good design when the individual 'tool' classes can't be placed conveniently in one function.
If ever you find yourself with a class that has tens of methods in it, of which many can be grouped according to specific tasks, this is a good refactor.
As a general rule you want to make sure that your 'tool' classes (SearchScreen, etc) are at a conceptually lower level than your controller (your test cases). Which they are for you.
At their simplest these tool classes are a form of the Function Object design pattern. Although in your case, you are calling more than just one method on each object, so they are a little more sophisticated.
Or, in short. Your design is fine, and very common.

If SearchScreen/PaymentScreen/SummaryScreen only performs test logic, it looks to me like you might as well just put that logic in utility methods of OurSiteTestCases.
Possible design for the test_buy_coffee method (depending on what you actually do in SearchScreen et al):
def test_buy_coffee(self):
sel = self.selenium
sel.open('/')
sel.window_maximize()
# Replace SearchScreen
self.__choose_search()
# Replace PaymentScreen
self.__fill_payment_data()
# Replace SummaryScreen
self.__accept_summary()
Edit:
If you need to factor out the test logic in __choose_search, __fill_payment_data and __accept_summary, to share it between tests, you may write corresponding utility test functions in a common module/package. Or, you could write a test baseclass which contains the selenium object (self.selenium) and has the "protected" utility methods _choose_search, _fill_payment_data and _accept_summary. It all depends on what's practical in your case :)

Maybe you can use Chain of Responsibility pattern:
Definition:
Avoid coupling the sender of a request to its receiver by giving more than one object a chance to handle the request. Chain the receiving objects and pass the request along the chain until an object handles it.
The example is in c# the:
Handler h1 = new ConcreteHandler1();
Handler h2 = new ConcreteHandler2();
Handler h3 = new ConcreteHandler3();
h1.SetSuccessor(h2);
h2.SetSuccessor(h3);
// Generate and process request
int[] requests = { 2, 5, 14, 22, 18, 3, 27, 20 };
foreach (int request in requests)
{
h1.HandleRequest(request);
}
Here is you can see the complete documentation: http://www.dofactory.com/Patterns/PatternChain.aspx#_self1