We Keep Coding

Says 'AnimalShelter' object has no attritbute 'create' but everything is there?

I am trying to test my AnimalShelter.py code in Jupyter Notebook but I keep getting the error that the 'create' attribute is not there. Did I miss something? I have it defined in the code but no matter what I change it still says it is not there.
AnimalShelter.py code:
import pymongo
from pymongo import MongoClient
from bson.objectid import ObjectId
class AnimalShelter(object):
""" CRUD operations for Animal collection in MongoDB """
def __init__(self, username, password):
#Initializing the MongoClient. This helps to access the MongoDB databases and collections.
self.client = MongoClient('mongodb://%s:%s#localhost:45344' % (username, password))
#where xxxx is your unique port number
self.database = self.client['AAC']
#Complete this create method to implement the C in CRUD.
def create(self, data):
if data is not None:
insert = self.database.animals.insert(data) #data should be dictionary
else:
raise Exception("Nothing to save, because data parameter is empty")
#Create method to implement the R in CRUD.
def read(self, searchData):
if searchData:
data = self.database.animals.find(searchData, {"_id": False})
else:
data = self.database.animals.find({}, {"_id": False})
return data
#Create method to implement U in CRUD.
def update(self, searchData, updateData):
if searchData is not None:
result = self.database.animals.update_many(searchData, {"$set": updateData})
else:
return "{}"
return result.raw_result
#Create method to implement D in CRUD.
def delete(self, deleteData):
if deleteData is not None:
result = self.database.animals.delete_many(deleteData)
else:
return "{}"
return result.raw_result
The test code I am trying to use to see that CRUD is functioning properly:
from AnimalShelter import AnimalShelter
data = {}
query = {}
test_class = AnimalShelter()
#test each function in the AnimalShelter class
response = test_class.create(data)
assert response
response = test_class.read(data)
assert response
response = test_class.update(data)
assert response
response = test_class.delete(data)
assert response
I am at a loss. I am new to Jupyter Notebook with the whole testing thing and I figured this was the simplest way to test the attributes before making a different test for specific data from the python code but either way I still get that the create attribute doesn't exist!

I created a slightly altered version of your code for easy testing and got the error that no username and password were being passed to the AnimalShelter constructor.
class AnimalShelter(object):
""" CRUD operations for Animal collection in MongoDB """
def __init__(self, username, password):
print("constructor")
#Complete this create method to implement the C in CRUD.
def create(self, data):
print("create")
#Create method to implement the R in CRUD.
def read(self, searchData):
print("read")
#Create method to implement U in CRUD.
def update(self, searchData, updateData):
print("update")
#Create method to implement D in CRUD.
def delete(self, deleteData):
print("delete")
test_class = AnimalShelter("user", "pass")
data = {}
test_class.create(data)
test_class.read(data)
test_class.update(data, {})
test_class.delete(data)
Could your error be because you didn't pass these variables when constructing the test_class object? Upon passing "user" and "pass", I got the expected output:
constructor
create
read
update
delete

Generate a site dynamically using Twisted

I want to generate all the pages of a site using twisted. It has to be similar with generating a page dynamically.
I came up with this :
class Home(Resource):
isLeaf = False
def __init__(self, pathname):
Resource.__init__(self)
self.pathname = pathname
def getChild(self, name, request):
if name == '':
return self
return Resource.getChild(self, name, request)
def render_GET(self, request):
path = "/var/www/html/books.toscrape.com/catalogue/"
fname = path + self.pathname
if ".html" in self.pathname:
f = open(fname)
s=f.read()
return s
else:
fname = fname + "/index.html"
f = open(fname)
s=f.read()
return s
class ElseSite(Resource):
def getChild(self,name,request):
return Home(name)
resource = ElseSite()
factory = Site(resource)
I'm able to generate pages with the url localhost:8080/foo, but how can I add more slashes to it, i.e. something like localhost:8080/foo/bar?

Children themselves can have children:
from twisted.web.resource import Resource
class Foo(Resource):
def getChild(self, name, request):
return Bar(name)
class Bar(Resource):
def getChild(self, name, request):
return Data(name)
site = Site(Foo())
...
You may also want to take a look at Klein which provides for a different style of defining your hierarchy. From the Klein docs:
from klein import Klein
app = Klein()
#app.route('/')
def pg_root(request):
return 'I am the root page!'
#app.route('/about')
def pg_about(request):
return 'I am a Klein application!'
app.run("localhost", 8080)
The native Twisted Web style is nice for very dynamic resource hierarchies. The Klein style is nice for relatively fixed hierarchies.

This answer helped me : https://stackoverflow.com/a/37689813/217088.
I defined a single resource with isLeaf = True, and then used request.postpath to get the query after http://localhost:8080/.
My code looks like this now :
class Home(Resource):
isLeaf = True
def __init__(self):
Resource.__init__(self)
def render_GET(self, request):
path = "/var/www/html/books.toscrape.com/"
filepath = '/'.join(request.postpath)
fname = path + filepath
f = open(fname)
s=f.read()
return s
resource = Home()
factory = Site(resource)

Pyramid: resource tree in URL Dispatch(hybrid) application

I am trying to implement dynamic ACLs(including row-level security) in URL Dispatch app.
Defining Root factory only doesn't seem to be sufficient as I need to perform individual authz checks for each secured endpoint. My setup looks as follows(I was using pyramid docs and mmerickel's tutorials as a guide):
config.py
...
settings = config.registry.settings
config = Configurator(settings=settings, root_factory=RootPermissionFactory)
config.set_authentication_policy(CustomAuthenticationPolicy(settings))
config.set_authorization_policy(ACLAuthorizationPolicy())
...
views.py
#import ...
#view_defaults(renderer='json', permission='secured')
class RecordsView(object):
...
#view_config(request_method='GET', route_name='records_by_id')
def get(self):
record = self.request.context.data
if not record:
return HTTPNotFound()
return record
#view_config(request_method='GET', route_name='records')
def get_by_owners(self):
owner_uids = self.request.params.mixed()['owner_uids']
return records_service.get_records(owner_uids=owner_uids)
def includeme(config):
config.add_route('records', '/records', factory=RecordsResource)
config.add_route('records_by_id', 'records/{record_id}', factory=RecordFactory, traverse='{record_id}')
authorization.py
class RootPermissionFactory(object):
__name__ = None
__parent__ = None
def __acl__(self):
return [
(Allow, 'authenticated_principal', 'secured'),
]
def __init__(self, request):
self.request = request
class RecordFactory(object):
def __init__(self, request):
self.request = request
def __getitem__(self, key):
record_data = records_service.get_record(key)
owner = record_data.get('owner_uid')
return RecordContext(self.request, owner, record_data)
class RecordContext(object):
def __acl__(self):
owner_principal = 'u:{owner}'.format(owner=self.owner)
return [
(Allow, owner_principal, 'secured'),
]
def __init__(self, request, owner, record_data):
self.request = request
self.owner = owner
self.data = record_data
class RecordsResource(object):
def __acl__(self):
request_params = self.request.params.mixed()
request_owner_uids = request_params['owner_uids']
authorized_owner_uids = owners_api_service.get_authorized_owners(self.request.user['auth_data'])
return [(Allow, 'authenticated_principal', 'secured')]\
if set(authorized_owner_uids) == set(request_owner_uids) else []
def __init__(self, request):
self.request = request
My questions are following:
is it acceptable to utilize row-level security checks without having Model layer? i.e. there is no ORM set up for Records data and there is also no plain Model to put persisted data into, so I have to use 'fake' RecordContext class to attach __acl__ rules and pass data to the view
is it acceptable to treat /records endpoint as a Resource despite the fact it is not a resource from Traversal perspective and relies on query parameters rather than path sections?

I think the answer to both of your question is: if it works for you then it's totally acceptable. I've found a lot of success treating URLs as resources as a general pattern to the extent that I have a some tooling to avoid using route_name. For example:
config.add_route('records', '/records', factory=RecordsResource, use_global_views=True)
config.add_route('records_by_id', 'records/{record_id}', factory=RecordFactory, traverse='{record_id}', use_global_views=True)
#view_config(context=RecordsResource, renderer='json')
def records_view(request):
return {}
#view_config(context=RecordContext, renderer='json')
def record_view(request):
return {}

Python Script Connecting To MySQL Database - Code ran as procedural script, fails after OOP refactor

I have a script that can run a number of different reports, based on what input it receives via the command line. All reports read from a database and return the results as a Pandas Dataframe object.
Here is the super-class, (omitting a large number of property getter and setter functions):
import mysql.connector
import pandas as p
import config
class Report(object):
_connection = None
_cursor = None
def __init__(self):
self._user = config.user
self._password = config.password
self._host = config.host
self._database = config.database
self._port = config.port
self._body_text = "Hello,\n\nPlease find attached these reports:\n\n"
self._connection = mysql.connector.connect(user=self._user, password=self._password, db=self._database,
host=self._host, port=self._port)
self._cursor = self._connection.cursor()
#property
def user(self):
return self._user
#user.setter
def user(self, value):
self._user = value
. . .
#property
def body_text(self):
return self._body_text
#body_text.setter
def body_text(self, value):
self._body_text = value
def append_body_text(self, value):
self._body_text += value
def get_data(self, server_cursor, query, columns):
server_cursor.execute(self, query)
results = server_cursor.fetchall()
data = p.DataFrame(data=results, columns=[columns])
return data
def get_today(self):
return self.today
def close(self):
self._connection_web.close()
self._connection_raw.close()
#staticmethod
def insert_variables_into_sql_statement(query, external_data):
final_query = query % external_data
return final_query
#staticmethod
def create_string_from_column(serial):
created_list = serial.tolist()
string = ', '.join(map(str, created_list))
return string
#staticmethod
def write_to_csv(data_frame, file_name):
data_frame.to_csv(config.baseDirectory + file_name, sep=',', index=False)
def generate_report(self):
data = self.get_data(self._cursor_web, self._query, self._columns)
self.write_to_csv(data, self._filename)
self.close()
Here is how my subclasses are structured:
class ExampleReport(Report):
def __init__(self):
Report.__init__(self)
self._query = """
SELECT
u.first_name AS 'First Name',
u.last_name AS 'Last Name'
FROM users AS u
"""
self._columns = "'FirstName', 'LastName'"
self._filename = "example_report.csv"
self.append_body_text("* All users")
In my main method I call the method like this:
report = Reports.ExampleReport()
report.generate_report()
When I do this, I get the following error:
AttributeError: 'ExampleReport' object has no attribute 'encode'
My database connections worked without a problem when it was terribly constructed procedural code (a working version is currently in production). It has broken now that I've tried to make it object oriented. Does anyone have any idea what I've done wrong?
EDIT: SOLVED MY OWN PROBLEM! In the get_data function in the super-class, the second line contained an erroneous self argument passed into server_cursor.execute(query) line. Once it was taken out, the error goes away.

SOLVED MY OWN PROBLEM! In the get_data function in the super-class, the second line contained an erroneous self argument passed into server_cursor.execute(query) line. Once it was taken out, the error goes away.

What is the most pythonic way/trick to support two database backends and keep my code DRY?

I have the following example code which uses either MongoEngine and Peewee as DB backends.
import mongoengine, peewee
from mongomodels import *
from mysqlmodels import *
class Parser(object):
def __init__(self, line, dbBackend):
if dbBackend in ["MongoDB","MySQL"]:
self.line = line
self.DB = dbBackend
user = self.createUser()
car = self.createCar(user)
parking = self.createParking(car)
else:
raise Exception()
def createUser(self):
if self.DB == "MongoDB":
newUserID = self._createMongoUser(self.line['firstname'], self.line['lastname'], '...')
else:
newUserID = self._createMySQLUser(self.line['firstname'], self.line['lastname'], '...')
return newUserID
def _createMongoUser(self, firstname, lastname, '...'):
try:
_user = MongoUserModel.objects.get(firstname=firstname, lastname=lastname)
except mongoengine.errors.DoesNotExist as e:
user = MongoUserModel(firstname=firstname, password)
_user = user.save()
finally:
return _user
def _createMySQLUser(self, firstname, lastname, '...'):
try:
_user = MySQLUserModel.get(MySQLUserModel.fistname == firstname, MySQLUserModel.lastname == lastname )
except Exception as e:
user = MySQLUserModel(fistname=fistname, lastname=lastname)
_user = user.save()
finally:
return _user
def createCar(self, user):
pass
def createParking(self, car):
pass
Is there any good practice / trick / module to keep my code DRY and to avoid redefining two methods to create my Models?
Should I can create a new abstraction class 'UserModel' as does PDO in PHP?

This is something I went through recently - I swapped from a mongo backend to postgres. When I set up the original project I had some models and a DataLayer. The datalayer (dl) had quite a simple interface that I used throughout my app.
# note: this is half python / half pseudocode
class Model1(object):
__collection__ = 'model1'
__tablename__ = 'model1'
# field definitions etc
class MongoDataLayer(object):
def __init__(self, mongo_db_connection):
self.conn = mongo_db_connection
def load(self, model, conditions):
raw = self.conn[model.__collection__].find(...)
return model(**raw)
def persist(self, obj):
self.conn[obj.__collection__].save(obj.as_dict())
class SQLDataLayer(object):
def __init__(self, sa_session_factory):
self.Session = sa_session_factory
self.session = self.Session()
def load(self, model, conditions):
return self.session.query(model).find_by(conditions).one() # ...etc
def persist(self, obj):
self.conn[obj.__collection__].save(obj)
# connections - mongo and postgres (I use SQLAlchemy)
dl_mongo = MongoDataLayer(db...)
dl_sql = SQLDataLayer(Session...)
# using them - you don't care which one you have
m = dl_mongo.load(models.Model1)
dl_mongo.persist(m)
m = dl_sql.load(models.Model1)
dl_sql.persist(m)
In my app I load up the dl in the initial load and then inject it into the app whenever data access needs to happen. The app itself then knows about models but not the details of how to load / save them.
Maybe not the best way to do it but it's worked well for me. Would be interested to hear how other people deal with it.