We Keep Coding

How do I make my search case insensitive?

I have a web app that searches through a few databases, some of the data saved is in uppercase and some a mix of upper and lowercase, however when searching the keyword I want it to ignore the case and just bring up results that match the word. for example I want to search "document_reference" without having to write the proper way it is saved which is "Document_Reference"
I was told to add case insensitivity in my index hwoever not sure what to do or add in there,
I tried this (found it in whoosh documentation)
class CaseSensitivizer(analysis.Filter):
def __call__(self, tokens):
for t in tokens:
yield t
if t.mode == "index":
low = t.text.lower()
if low != t.text:
t.text = low
yield t
this what my index and query parser looks like
def open_index(indexDirectory):
# open index and return a idex object
ix = index.open_dir(indexDirectory)
return ix
def search_index(srch, ix):
# Search the index and print results
# ix = open_index(indexDirectory)
results = ''
lst = []
qp = MultifieldParser(['Text', 'colname',
'tblname', 'Length', 'DataType', 'tag_name'],
schema=ix.schema, group=qparser.OrGroup)
# qp = QueryParser('Text', schema=ix.schema)
q = qp.parse(srch)
with ix.searcher() as s:
results = s.search(q, limit=None)
for r in results:
print('\n', r)
lst.append(r.fields())
if(DEBUG):
print('Search Results:\n', lst)
print('\nFinished in search.py')
return lst
currently it only ever gives results that exactly match what I typed in search bar, so If I type "document" but the source is actually stored as "DOCUMENT" I wouldnt get any results

I know this is an older issue but thought would reply in case somebody like me came here looking for a solution.
The CaseSensitivizer class needs to be used when you define your schema. This is how you would use it to create the schema from the quickstart example from the docs
>>> from whoosh.index import create_in
>>> from whoosh.fields import *
>>> from whoosh import analysis
>>> class CaseSensitivizer(analysis.Filter):
def __call__(self, tokens):
for t in tokens:
yield t
if t.mode == "index":
low = t.text.lower()
if low != t.text:
t.text = low
yield t
>>> myanalyzer = analysis.RegexTokenizer() | CaseSensitivizer()
>>> schema = Schema(title=TEXT(stored=True), path=ID(stored=True), content=TEXT(analyzer=myanalyzer))
Now you can use this schema to create your index and do what you were doing before to search. That worked for me.

instead of using lower() or upper(), you can use casefold() for string comparison.
A very good example given here.
In short, example is:
s1 = 'Apple'
s3 = 'aPPle'
s1.casefold() == s3.casefold()
returns True.

Convert JSON to .ics (Python)

I am trying to convert a JSON file to an iCalendar file. My supervisor suggested using two functions convertTo(data) (which converts a JSON to a String) and convertFrom(data) (which converts a String to a JSON; I am not sure of the purpose of this function).
My current approach uses a lot of refactoring and multiple functions.
#returns a String
def __convert(data):
convStr = __convertTo(data)
convStr = __fields(convStr)
return convStr
#convert JSON to a String
def __convertTo(data):
str = "" + data
return str
#takes string arg (prev converted from JSON) to split it into useful info
def __fields(data)
#########
iCalStr = __iCalTemplate(title, dtStart, UID, remType, email)
return iCalStr
#
def __iCalTemplate(title, dtStart, UID, remType, email):
icsTempStr = "BEGIN:VEVENT\n
DTSTART:" + dtStart + "\nUID:" + UID + "\nDESCRIPTION:" + desc + "\nSUMMARY:" + title
if remType is not None
icsTempStr += "\nBEGIN:VALARM\nACTION:" + remType + "DESCRIPTION:This is an event reminder"
if remType is email
icsTempStr += "\nSUMMARY:Alarm notification\nATTENDEE:mailto:" + email
icsTempStr += "\nEND:VALARM"
return icsTempStr
Any hints or suggestions would be very helpful. I am fully aware that this code needs a LOT of work.

This isn't intended to be a complete answer, but as a longer tip.
There's a Python idiom that will be very helpful to you in building strings, especially potentially large ones. It's probably easier to see an example than explain:
>>> template = 'a value: {a}; b value: {b}'
>>> data = {'a': 'Spam', 'b': 'Eggs'}
>>> template.format(**data)
'a value: Spam; b value: Eggs'
This idiom has a number of advantages over string concatenation and could eliminate the need for a function altogether if you write the template correctly. Optional inserts could, for example, be given values of ''. Once you format your iCal template correctly, it's just a matter of retrieving the right data points from JSON... and if you name your template insert points the same as what you have in JSON, you might even be able to do that conversion in one step. With a bit of planning, your final answer could be something as simple as:
import json
template = 'full iCal template with {insert_point} spec goes here'
data = json.JSONDecoder().decode(your_json_data)
ical = template.format(**data)
To do a quick (and slightly different) interpreter example:
>>> import json
>>> decoder = json.JSONDecoder()
>>> json_example = '{"item_one" : "Spam", "item_two" : "Eggs"}'
>>> template = 'Item 1: {item_one}\nItem 2: {item_two}'
>>> print template.format(**decoder.decode(json_example))
Item 1: Spam
Item 2: Eggs

I ended up using a completely different, more efficient approach to accomplish this. In summary, my method traverses through a JSON, extracting each value from each field and manually places it in the appropriate place in an iCalendar template. It returns a string. Something like this...
def convert(self, json):
template = 'BEGIN:VEVENT\n'
template += 'DTSTART:%s\n' % json['event-start']
...
return template

How to turn a dictionary into attribute = value

I have the following dictionary in Python:
myDic = {}
myDic['username'] = 'smith44'
myDic['password'] = '123'
myDic['email'] = 'smith#gmail.com'
I want to turn this into:
username = 'smith44'
password = '123'
email = 'smith#gmail.com'
so that I can directly use username, password and email directly in one of my latter methods. I know I can use the dictionary but I need to do it this way for the sake of compatibility.
Again, what I need to do here is turn every dictionary key into a seperate attribute that will take the value of its corresponding dictionary value.
UPDATE
To further support what I need to do, here is what I've done so far:
for key, value in myDic.iteritems():
exec("%s = " "'""%s""'" % (key, value))

If you really really need to do this:
>>> d = {}
>>> d['test'] = "foo"
>>> locals().update(d)
>>> test
'foo'
But if I were you, I'd try to find a different way of solving the problem you're trying to solve instead of automatically adding variables to your local namespace. There's almost always a way around this mess.

you will get string in attribute = value format using str.join
In [12]: ", ".join(["%s = %s" % (k, v) for k,v in myDic.items()])
Out[12]: 'username = smith44, password = 123, email = smith#gmail.com'
In [13]: myDic.items()
Out[13]: [('username', 'smith44'), ('password', '123'), ('email', 'smith#gmail.com')]

(Python) Creating a list with a unique automatically generated name [duplicate]

This question already has answers here:
How can you dynamically create variables? [duplicate]
(8 answers)
Closed 8 years ago.
I am trying to automate populating a town by randomly generating households. I generate the name of the town, generate the number of households, the last name of each household and number of occupants in each. That much is fine. I am now, however, trying to create each individual, to generate a first name, a sex, an age and an occupation, and I'd like to store this data in a list as well, one list containing the attributes of each person. The problem I'm running into is that I want to use a for loop, something like:
#houseArray[currentFam][1] is the number of members in the current house.
for currentFam in range(houseArray[currentFam][1]):
uniquelyNamedArray[0] = genSex()
uniquelyNamedArray[1] = genFirstName()
uniquelyNamedArray[2] = genAge()
So... look at the data of the first household, use a for loop to iterate through each member assigning stats, then go to the next household and do the same, progressing through each household. My problem lies in not knowing how to assign a unique name to each array created by the for loop. It doesn't really matter what the name is, it could be anything as long as each person has their own uniquely named array storing their attributes.

Use a dictionary with the person's name as the key. Like:
people = {}
people["Billy Bloggs"] = ['23','Male','263 Evergreen Tce'] # store to dict
print ( people["Billy Bloggs"] ) # get stuff out of dict
Better still, give the attributes names by storing those as a dict as well:
people["Billy Bloggs"] = { 'Age':23, 'Gender':'M', 'Address':'263 Evergreen Tce' }
print ( people["Billy Bloggs"]['Age'] ) # Get billy's age
You can loop through the elements of a dictionary using the following syntax:
>>> mydict = {'a':'Apple', 'b':'Banana', 'c':'Cumquat'}
>>> for key, value in mydict.iteritems():
... print ('Key is :' + key + ' Value is:' + value)
...
Key is :a Value is:Apple
Key is :c Value is:Cumquat
Key is :b Value is:Banana
Note that there is no guarantee on the order of the data. You may insert data in the order A, B, C and get A, C, B back.
Note: The keys of a dict, in this case the person's name, are constrained to be unique. So if you store data to the same name twice, then the first key:value pair will be overwritten.
mydict["a"] = 5
mydict["a"] = 10
print (mydict["a"]) # prints 10
Sidenote: some of your gen*() functions could almost certainly be replaced by random.choice():
import random
first_names = ['Alice','Bob','Charlie','Dick','Eliza']
random_first_name = random.choice(first_names)

Keep data out of your variable names and just store them in a dict.

First, while you haven't shown us the surrounding code, you are probably relying too much on global variables. Rather than trying to create uniquely named arrays for each family member simply do something like this:
Don't really do this (I'll tell you why in a minute)
#houseArray[currentFam][1] is the number of members in the current house.
for currentFam in range(houseArray[currentFam][1]):
family_member_info = []
family_member_info[0] = genSex()
family_member_info[1] = genFirstName()
family_member_info[2] = genAge()
# Pretend 2 is where we are storing the family member information list
houseArray[currentFam][2].append(family_member_info)
A better way
Don't use an array for this sort of thing - it gets very difficult very quickly to tell what is actually stored in which index. Even in your example you have to note that houseArray[currentFam][1] is storing the number of members in the current house.
I would use either a dictionary or a named tuple and store your information in there. That way you can do something like this:
from collections import namedtuple
# Create a class called "household"
# with three fields, "owner", "size" and "members"
household = namedtuple("household", "owner size members")
househould_array = []
# Create some households and put them in the array
household_array.append(household("Family #1", 3, []))
household_array.append(household("Family #2", 1, []))
household_array.append(household("Family #3", 7, []))
# Loop over every household in the household_array
for family in household_array:
# Each `household` namedtulpe's values can be accessed by
# attribute as well as by index number
# family[1] == family.size == 3
# (for Family #1)
for member_number in range(family.size):
# family[2] == family.members == []
# (before we put anything in it)
family.members.append(generate_family_member())

You are mixing program data with variable names. It is okay to call a variable something generic; you do this all the time: e.g. in your for-loop, you use currentFam rather than the name of the family. Asking to uniquely name the array makes (no offense) as much sense as either asking what to name currentFam (it doesn't matter what you name it), or alternatively trying to do:
Andersons[0] = genSex()
Andersons[1] = genFirstName()
Andersons[2] = genAge()
Longs[0] = genSex()
Longs[1] = genFirstName()
Longs[2] = genAge()
Smiths[0] = genSex()
Smiths[1] = genFirstName()
Smiths[2] = genAge()
...
Variables are separate from program data.
You should just name your array person, and store it with other arrays. Even better would be to define a class Person(object): ..., so you could do things like x.name and x.age, but you don't need to do that. For example:
class Person(object):
def __init__(self, **kw):
self.data = kw
self.__dict__.update(kw)
def __repr__(self):
return str('Person(**{})'.format(self.data))
__str__ = __repr__
M = Person.M = 'm'
F = Person.F = 'f'
ALL_PEOPLE = set()
for ...:
person = Person(name=..., age=..., sex=...)
people.add(person)
Then to find people:
def findPeople(name=None, age=None, custom=set()):
matchers = custom
if name!=None:
matchers.add(lambda x:name.lower() in x.name.lower())
if age!=None:
matchers.add(lambda x:age==x.age)
return set(p for p in ALL_PEOPLE if all(m(p) for m in matchers))
Demo:
ALL_PEOPLE = set([
Person(name='Alex', age=5, sex=M),
Person(name='Alexander', age=33, sex=M),
Person(name='Alexa', age=21, sex=F)
])
>>> pprint.pprint( findPeople(name='alex', custom={lambda p: p.age>10}) )
{Person(**{'age': 33, 'name': 'Alexander', 'sex': 'm'}),
Person(**{'age': 21, 'name': 'Alexa', 'sex': 'f'})}

Wow, I really enjoyed reading all of the other answers.
So many great suggestions including, but not limited to:
#Sean Vieira suggests named-tuples -- an excellent, light-weight choice;
#ninjagecko uses a neat trick to dynamically assign instance attributes;
#Li-aung Yip mentions using the built-in sqlite3 module.
Much if not all of what's here has already been suggested.
If nothing else I hope this answer is an introduction to what classes may provide beyond what is provided by other data-structures.
Caveat: If performance is a huge concern, modeling each entity as a class might be overkill.
from __future__ import division, print_function
class Town(object):
def __init__(self, name=None, country=None, area_km2=0, population=0):
self.name = name
self.area_km2 = area_km2
self.area_mi2 = self.area_km2 * 0.38610217499077215
self.population = population
self.households = []
#property
def total_households(self):
return len(self.households)
#property
def population_density_per_km2(self):
try:
return self.population / self.area_km2
except ZeroDivisionError:
return 0
#property
def population_density_per_mi2(self):
try:
return self.population / self.area_mi2
except ZeroDivisionError:
return 0
class Household(object):
def __init__(self, primary_lang='Esperanto'):
self.primary_lang = primary_lang
self.members = []
#property
def total_members(self):
return len(self.members)
class Person(object):
def __init__(self, age=0, gender=None, first_name=None):
self.age = age
self.gender = gender
self.first_name = first_name
if __name__ == '__main__':
londontown = Town(name='London',
country='UK',
area_km2=1572,
population=7753600)
print(londontown.population_density_per_km2)
print(londontown.population_density_per_mi2)
a_household = Household()
a_household.members.append(
Person(age=10, gender='m', first_name='john'),
)
a_household.members.append(
Person(age=10, gender='f', first_name='jane')
)
londontown.households.append(a_household)
print(londontown.total_households)
print(a_household.total_members)

question about managing values with multiple subvalues in python 3k

I have a question reguarding how I would perform the following task in python.
(I use python 3k)
what I have are several variables which can yield further variables on top of those
and each of those have even more variables
for example:
a generic name would be
item_version_type =
where each part (item, version, and type) refer to different variables(here there are 3 for each)
item = item_a, item_b, item_c
version = range(1,3)
itemtype = itemtype_a, itemtype_b, itemtype_c
simply listing each name and defining it is annoying:
itema_ver1_typea =
itemb_ver1_typea =
itemc_ver1_typea =
itema_ver2_typea =
etc.
etc.
etc.
especially when I have something where one variable is dependent on something else
for example:
if value == True:
version = ver + 1
and to top it off this whole example is rather simply compared to what I'm actually
working with.
one thing I am curious about is using multiple "." type of classes such as:
item.version.type
I know that this can be done
I just can't figure out how to get a class with more than one dot
either that or if anyone can point me to a better method
Thanks for help.

Grouping of data like this can be done in three ways in Python.
First way is tuples:
myvariable = ('Sammalamma', 1, 'Text')
The second way is a dictionary:
myvariable = {'value': 'Sammalamma', 'version': 1, 'type': 'Text'}
And the third way is a class:
class MyClass(object):
def __init__(self, value, version, type):
self.value = value
self.version = version
self.type = type
>>> myvariable = MyClass('Sammalamma', 1, 'Text')
>>> myvariable.value
'Sammalamma'
>>> myvariable.version
1
>>> myvariable.type
'Text'
Which one to use in each case is up to you, although in this case I would claim that the tuple doesn't seem to be the best choice, I would go for a dictionary or a class.
None of this is unique to Python 3, it works in any version of Python.

In addition to #Lennart Regebro's answer if items are immutable:
import collections
Item = collections.namedtuple('Item', 'value version type')
items = [Item(val, 'ver'+ver, t)
for val in 'abc' for ver in '12' for t in ['typea']]
print(items[0])
# -> Item(value='a', version='ver1', type='typea')
item = items[1]
print(item.value, item.type)
# -> b typea

sorry for posting this here instead of the comments but I have no clue how to work the site here.
for clarification
what I need is basically to have be able to get an output of said such as where
I could take a broad area (item) narrow it further (version) and even further (type as in type of item like lets say types are spoon, knife, fork)
or a better description is like arm.left.lower = lower left arm
where I could also have like leg.left.lower
so I could have arm.both.upper to get both left and right upper arms
where a value would be assigned to both.
what I need is to be able to do truth tests etc. and have it return the allowable values
such as
if leg == True
output is --> leg.both.lower, leg.both.upper, leg.left.upper leg.right.upper, etc., etc., etc.
if upper == True
output is --> leg.both.upper, leg.left.upper, etc., etc., etc.
hopefully that helps
Basically I get how to get something like item.version but how do I get something
like item.version.type
I need to have it to be more specific than just item.version
I need to be able to tell if item is this and version is that then type will be x
like
item.version.type
if version == 3:
item.version = spoon.3.bent
#which is different from
if version == 2:
item.version.type = spoon.2.bent