We Keep Coding

Finding and replacing values in specific columns in a CSV file using dictionaries

My Goal here is to clean up address data from individual CSV files using dictionaries for each individual column. Sort of like automating the find and replace feature from excel. The addresses are divided into columns. Housenumbers, streetnames, directions and streettype all in their own column. I used the following code to do the whole document.
missad = {
'Typo goes here': 'Corrected typo goes here'}
def replace_all(text, dic):
for i, j in missad.items():
text = text.replace(i, j)
return text
with open('original.csv','r') as csvfile:
text=csvfile.read()
text=replace_all(text,missad)
with open('cleanfile.csv','w') as cleancsv:
cleancsv.write(text)
While the code works, I need to have separate dictionaries as some columns need specific typo fixes.For example for the Housenumbers column housenum , stdir for the street direction and so on each with their column specific typos:
housenum = {
'One': '1',
'Two': '2
}
stdir = {
'NULL': ''}
I have no idea how to proceed, I feel it's something simple or that I would need pandas but am unsure how to continue. Would appreciate any help! Also is there anyway to group the typos together with one corrected typo? I tried the following but got an unhashable type error.
missad = {
['Typo goes here',Typo 2 goes here',Typo 3 goes here']: 'Corrected typo goes here'}

is something like this what you are looking for?
import pandas as pd
df = pd.read_csv(filename, index_col=False) #using pandas to read in the CSV file
#let's say in this dataframe you want to do corrections on the 'column for correction' column
correctiondict= {
'one': 1,
'two': 2
}
df['columnforcorrection']=df['columnforcorrection'].replace(correctiondict)
and use this idea for other columns of interest.

Python Pandas: export to excel format a row 'Bold' without condition

I have what I feel could be a silly question but it seems I can't find the answer:
How to, while exporting to Excel, set a row/range to bold?
I know how to conditional format e.g.
format1 = workbook.add_format({'bg_color': '#FFC7CE',
'font_color': '#9C0006'})
...
worksheet.conditional_format('C2:AF41', {'type': 'cell', 'criteria': '<', 'value': 0, 'format': format1})
or even to format columns:
border_format = workbook.add_format({
'border': 1,
'align': 'center',
'font_size': 10
})
...
worksheet.set_column(0, 0, 30, border_format)
But let's say I want for row A1:A40 to be written in bold, whitout any particular criteria? I just want to set to bold or a color or any formatting a given range whatever data it may contain.
In this case, for instance, apply 'format1' to range A1:A40 whatever the values.
Does anyone know the answer?

It looks like there isn't a helper function specifically designed for this, according to github.
But Gabriel's answer to a similar question proposes a workaround, where you use two (or more) conditional formats to cover all values:
worksheet.conditional_format('A1:A40', {'type': 'cell',
'criteria': '>=', 'value': 0, 'format': format1})
worksheet.conditional_format('A1:A40', {'type': 'cell',
'criteria': '<', 'value': 0, 'format': format1})

Conditional Formatting +1 in Excel

I want to apply conditional formatting to all cells that contain +1. To avoid Excel from converting +1 to 1 I used '+1. However, the below code does not work. Any ideas why?
my_formats = {-1: '#FF0000',
"'+1": '#00FF00',}
for val, color in my_formats.items():
fmt = workbook.add_format({'font_color': color})
worksheet.conditional_format('B3:EK500', {'type': 'cell',
'criteria': '=',
'value': val,
'format': fmt})

string Validation with xlsxwriter python

I am trying to put validations on excel file. I field should only accept characters or character with '_' symbol. The following code is only redistricting user to enter values more than length 10.
import xlsxwriter
wb = xlsxwriter.Workbook('staff.xlsx')
ws = workbook.add_worksheet()
ws.data_validation(1, 1, 10, 0,
{'validate': 'length',
'input_title': 'Enter value',
'criteria': '<',
'value': firstname_max_length,
'error_message': 'Max Length is{0}'.format(10)})
Help me to validate values that should only except characters.
Thanks.

The first step is to work out how to do the data validation in Excel and then transfer it to XlsxWriter.
This will probably require a "Custom" data validation like this:
import xlsxwriter
wb = xlsxwriter.Workbook('staff.xlsx')
ws = wb.add_worksheet()
ws.data_validation('A1:C3',
{'validate': 'custom',
'value': '=ISTEXT(A1)',
'input_title': 'Enter a non numeric value',
'error_message': 'Enter a string not a number'})
wb.close()
Output:
However, this doesn't do exactly what you are looking for. It is still possible to add non character data or even strings like 123h as shown in the screenshot. So you will need to figure out a formula that works in Excel and then apply it to XlsxWriter. I googled for an example but didn't find anything that worked.

xlsxwriter: how "inf" value can be uncolored?

using python package "xlsxwriter", I want to highlight cells in the following conditional range;
value > 1 or value <-1
However, some cells have -inf/inf values and it fill colors them too (to yellow). Is thare any way to unhighlight them?
I tried "conditional_format" function to uncolor them, but it doesn't work.
output example
format1 = workbook.add_format({'bg_color':'#FFBF00'}) #yellow
format2 = workbook.add_format({'bg_color':'#2E64FE'}) #blue
format3 = workbook.add_format({'bg_color':'#FFFFFF'}) #white
c_fold=[data.columns.get_loc(col) for col in data.columns if col.startswith("fold")]
c_fold.sort()
l=len(data)+1
worksheet.conditional_format(1,c_fold[0],l,c_fold[-1], {'type':'cell',
'criteria' : '>',
'value':1,
'format':format1,
})
worksheet.conditional_format(1,c_fold[0],l,c_fold[-1], {'type':'cell',
'criteria' : '<',
'value':-1,
'format':format2,
})
worksheet.conditional_format(1,c_fold[0],l,c_fold[-1], {'type':'text',
'criteria' : 'begins with',
'value':"-inf",
'format':format3,
})
Thanks in advance

In a lot of cases the answer to the question "how do I get this to work with XlsxWriter" is the same as the answer to the question "how do I get this to work with Excel".
If you try your example in Excel you will see that you get the same results as the XlsxWriter example. The > criteria is applies to -inf in Excel, thus it is highlighted as light orange. The fact that the following containing criteria also matches doesn't override the first matching criteria since Excel applies them in the order that the user supplies them.
The solution in Excel, and XlsxWriter, is to change the order that the rules are applied, like this:
import xlsxwriter
workbook = xlsxwriter.Workbook('conditional_format.xlsx')
worksheet1 = workbook.add_worksheet()
# Add some formats to use in the conditional formats.
format1 = workbook.add_format({'bg_color': '#FFBF00'})
format2 = workbook.add_format({'bg_color': '#2E64FE'})
format3 = workbook.add_format({'bg_color': '#FFFFFF'})
# Write some sample data.
worksheet1.write('A1', 2)
worksheet1.write('A2', '-inf')
worksheet1.write('A3', -2)
# Write a conditional formats over the same range.
worksheet1.conditional_format('A1:A3', {'type': 'text',
'criteria': 'begins with',
'value': "-inf",
'format': format3})
worksheet1.conditional_format('A1:A3', {'type': 'cell',
'criteria': '>',
'value': 1,
'format': format1})
worksheet1.conditional_format('A1:A3', {'type': 'cell',
'criteria': '<',
'value': 1,
'format': format2})
workbook.close()
Output:

This would solve
worksheet.conditional_format(1,c_fold[0],l,c_fold[-1], {'type':'text',
'criteria' : 'containing',
'value':"-inf",
'format':format3,
})