I'm trying to retrieve 2 output Arrays from an XRecord in AutoCAD 2016 using python 2.7, with comtypes imported, the first array is an array of integers (DXF Group Codes) and the second array is an array of variants (the values of XRecord).
The opposite way of what this question seeks to
The method of interest is GetXRecordData, which (according to AutoCAD's documentation) if successful returns None, and only accepts 2 output arguments.
when I try to retrieve it with code like
DxfGrCd = []
vals = []
an_XRecord.GetXRecordData(DxfGrCd, vals)
and see the values of DxfGrCd and vals I found no change happened to them, both of them still equal to [], the same is also with
DxfGrCd = {}
vals = {}
anXRecord.GetXRecordData(DxfGrCd, vals)
also no change is applied on them, both of them still equal to {}, even though dictionaries and lists are mutable.
Is there any way to deal with that kind of methods in python?
Well, I haven't figured out any way to do so from python, however, since the data stored in XRecords are just numbers and strings (in my application), stored in the XRecord as variants, I've used MS Excel as a middle man to pass me data.
Note: All numbers I've got were retrieved but as floats.
And all strings were retrieved but their type is unicode. (you can convert them to string easily with the built-in function str())
Here's how I've done that.
First: Creation of The Facilitator Workbook (Our Middle Man)
1-Normally as a regular windows user, open Excel, then open Visual Basic Editor, one way to do that is to go to Developer tab and click on Visual Basic Editor.
2-From the Editor, insert a module (one way is from the menu bar: insert>Module), then left-double click on its default name and type "mod_facilitate", then hit Enter.
3-Left-double click on its icon at the project viewer.
4- A window will appear, copy the following code to it.
Sub getxrecord()
'get running AutoCAD object
Dim mycad As AcadApplication, mydoc As AcadDocument, filepath As String
Set mycad = GetObject(, "AutoCAD.Application.20")
'get the selected drawing, provided from python code
With Sheet1
filepath = .Range(.Cells(1, 1), .Cells(1, 1)).Value
End With
Dim iCount As Integer, i As Integer, j As Integer, CompName As String
iCount = mycad.Documents.Count
For i = 0 To iCount - 1
CompName = mycad.Documents.Item(i).FullName
If CompName Like filepath Then
j = i
Exit For
End If
Next i
Set mydoc = mycad.Documents.Item(j)
Dim name2 As String
'get the object from its provided handle
With Sheet1
handler = .Range(.Cells(2, 1), .Cells(2, 1)).Value
End With
Dim myXRecord As AcadXRecord
Set myXRecord = mydoc.HandleToObject(handler)
Dim DxfGrcd As Variant, Val As Variant
DxfGrcd = Array()
Val = Array()
myXRecord.GetXRecordData DxfGrcd, Val
Dim UB As Integer
UB = UBound(DxfGrcd)
For i = 0 To UB
With Sheet1
.Range(.Cells((i + 1), 2), .Cells((i + 1), 2)).Value = DxfGrcd(i)
.Range(.Cells((i + 1), 3), .Cells((i + 1), 3)).Value = Val(i)
End With
Next i
End Sub
5- From Tools>References Select these reference names, leaving the others at their previous states
AcSmComponents20 1.0 Type Library
AutoCAD 2016 Type Library
CAO 1.0 Type Library
Then click on OK, then hit Ctrl+s to save.
6- Save the file and name it "facilitator", save it within the same directory of your python file. Save it of type Excel Macro-Enabled Workbook (has the extension .xlsm)
7- At your python file, define the function to retrieve XRecord's data as following, I'll tell what are its arguments for:
def XRecord_return(namefile,handle,size):
xl.Range["A1"].Value[xlRangeValueDefault] = namefile
xl.Range["A2"].Value[xlRangeValueDefault] = handle
xl.Application.Run("facilitator.xlsm!mod_facilitate.getxrecord")
dxfgrcd = []
vals = []
for i in range(0,size):
CellB = 'B' + str(i+1)
CellC = 'C' + str(i+1)
dxfgrcd.append(xl.Range[CellB].Value[xlRangeValueDefault])
vals.append(xl.Range[CellC].Value[xlRangeValueDefault])
return dxfgrcd,vals
Second: What to Insure
Note: All the following steps must be written before the definition of XRecord_return
1- AutoCAD must be instantiated from python using a line like autocad = CreateObject("AutoCAD.Application.20",dynamic=True) or autocad = comtypes.client.CreateObject("AutoCAD.Application.20",dynamic=True) depending on the scope of importing and importing form [ import comtypes.client or from comtypes.client import CreateObject ], here, importing scope is the python file's module scope.
2-instantiate Excel using xl = CreateObject("Excel.Application") and open the facilitator file with
xlpath = os.getcwd()
xlpath += '\\facilitator.xlsm'
xl = CreateObject("Excel.Application")
from comtypes.gen.Excel import xlRangeValueDefault
xlwb = xl.Workbooks.Open(Filename=xlpath,ReadOnly=0)
3- You have to know how many elements are stored in the XRecord (excluding the number of associated DXF group codes), this number of elements is what you'll supply to XRecord_return as its size argument.
e.g. An XRecord that stores 3.0 "abc" 5 and have correspondent DXF group codes 1 2 3 is of size 3, not 6.
Third: Supplying Data to The Facilitator Workbook
We need only its first worksheet, you must provide the following data:-
1- The drawing's full path/directory to cell "A1".
To get the drawing's full path if you have its Document object you can get it from the property FullName. This value is what you'll supply to XRecord_return as its namefile argument.
To assign, for instance: xl.Range["A1"].Values[xlRangeValueDefault] = filepath
2-The XRecord's handle value to cell "A2", you can get it from the property Handle of the XRecord. This value is what you'll supply to XRecord_return as its 'handle' argument.
To assign, for instance: xl.Range["A1"].Values[xlRangeValueDefault] = handlevalue
3- After that, wherever you need to get the XRecords data, call the XRecord_return function, like
DxfGrCd,vals = XRecord_return(filepath,handlevalue,size_of_XRecord)
The outputs are lists that contain the correspondent data.
Last, But not Least
When you finish using Excel for retrieving data from as many XRecords as you need, close the facilitator workbook using xlwb.Close(SaveChanges=0)
Related
How do I pass the following commands into the latex environment?
\centering (I need landscape tables to be centered)
and
\caption* (I need to skip for a panel the table numbering)
In addition, I would need to add parentheses and asterisks to the t-statistics, meaning row-specific formatting on the dataframes.
For example:
Current
variable
value
const
2.439628
t stat
13.921319
FamFirm
0.114914
t stat
0.351283
founder
0.154914
t stat
2.351283
Adjusted R Square
0.291328
I want this
variable
value
const
2.439628
t stat
(13.921319)***
FamFirm
0.114914
t stat
(0.351283)
founder
0.154914
t stat
(1.651283)**
Adjusted R Square
0.291328
I'm doing my research papers in DataSpell. All empirical work is in Python, and then I use Latex (TexiFy) to create the pdf within DataSpell. Due to this workflow, I can't edit tables in latex code while they get overwritten every time I run the jupyter notebook.
In case it helps, here's an example of how I pass a table to the latex environment:
# drop index to column
panel_a.reset_index(inplace=True)
# write Latex index and cut names to appropriate length
ind_list = [
"ageFirm",
"meanAgeF",
"lnAssets",
"bsVol",
"roa",
"fndrCeo",
"lnQ",
"sic",
"hightech",
"nonFndrFam"
]
# assign the list of values to the column
panel_a["index"] = ind_list
# format column names
header = ["", "count","mean", "std", "min", "25%", "50%", "75%", "max"]
panel_a.columns = header
with open(
os.path.join(r"/.../tables/panel_a.tex"),"w"
) as tf:
tf.write(
panel_a
.style
.format(precision=3)
.format_index(escape="latex", axis=1)
.hide(level=0, axis=0)
.to_latex(
caption = "Panel A: Summary Statistics for the Full Sample",
label = "tab:table_label",
hrules=True,
))
You're asking three questions in one. I think I can do you two out of three (I hear that "ain't bad").
How to pass \centering to the LaTeX env using Styler.to_latex?
Use the position_float parameter. Simplified:
df.style.to_latex(position_float='centering')
How to pass \caption*?
This one I don't know. Perhaps useful: Why is caption not working.
How to apply row-specific formatting?
This one's a little tricky. Let me give an example of how I would normally do this:
df = pd.DataFrame({'a':['some_var','t stat'],'b':[1.01235,2.01235]})
df.style.format({'a': str, 'b': lambda x: "{:.3f}".format(x)
if x < 2 else '({:.3f})***'.format(x)})
Result:
You can see from this example that style.format accepts a callable (here nested inside a dict, but you could also do: .format(func, subset='value')). So, this is great if each value itself is evaluated (x < 2).
The problem in your case is that the evaluation is over some other value, namely a (not supplied) P value combined with panel_a['variable'] == 't stat'. Now, assuming you have those P values in a different column, I suggest you create a for loop to populate a list that becomes like this:
fmt_list = ['{:.3f}','({:.3f})***','{:.3f}','({:.3f})','{:.3f}','({:.3f})***','{:.3f}']
Now, we can apply a function to df.style.format, and pop/select from the list like so:
fmt_list = ['{:.3f}','({:.3f})***','{:.3f}','({:.3f})','{:.3f}','({:.3f})***','{:.3f}']
def func(v):
fmt = fmt_list.pop(0)
return fmt.format(v)
panel_a.style.format({'variable': str, 'value': func})
Result:
This solution is admittedly a bit "hacky", since modifying a globally declared list inside a function is far from good practice; e.g. if you modify the list again before calling func, its functionality is unlikely to result in the expected behaviour or worse, it may throw an error that is difficult to track down. I'm not sure how to remedy this other than simply turning all the floats into strings in panel_a.value inplace. In that case, of course, you don't need .format anymore, but it will alter your df and that's also not ideal. I guess you could make a copy first (df2 = df.copy()), but that will affect memory.
Anyway, hope this helps. So, in full you add this as follows to your code:
fmt_list = ['{:.3f}','({:.3f})***','{:.3f}','({:.3f})','{:.3f}','({:.3f})***','{:.3f}']
def func(v):
fmt = fmt_list.pop(0)
return fmt.format(v)
with open(fname, "w") as tf:
tf.write(
panel_a
.style
.format({'variable': str, 'value': func})
...
.to_latex(
...
position_float='centering'
))
I am trying to obtain the actual A1 values using the Sheetfu library's get_data_range().
When I use the code below, it works perfectly, and I get what I would expect.
invoice_sheet = spreadsheet.get_sheet_by_name('Invoice')
invoice_data_range = invoice_sheet.get_data_range()
invoice_values = invoice_data_range.get_values()
print(invoice_data_range)
print(invoice_values)
From the print() statements I get:
<Range object Invoice!A1:Q42>
[['2019-001', '01/01/2019', 'Services']...] #cut for brevity
What is the best way to get that "A1:Q42" value? I really only want the end of the range (Q42), because I need to build the get_range_from_a1() argument "A4:Q14". My sheet has known headers (rows 1-3), and the get_values() includes 3 rows that I don't want in the get_values() list.
I guess I could do some string manipulation to pull out the text between the ":" and ">" in
<Range object Invoice!A1:Q42>
...but that seems a bit sloppy.
As a quick aside, it would be fantastic to be able to call get_data_range() like so:
invoice_sheet = spreadsheet.get_sheet_by_name('Invoice')
invoice_data_range = invoice_sheet.get_data_range(start="A4", end="")
invoice_values = invoice_data_range.get_values()
...but that's more like a feature request. (Which I'm happy to do BTW).
Author here. Alan answers it well.
I added some methods at Range level to the library, that are simply shortcuts to the coordinates properties.
from sheetfu import SpreadsheetApp
spreadsheet = SpreadsheetApp("....access_file.json").open_by_id('long_string_id')
sheet = spreadsheet.get_sheet_by_name('test')
data_range = sheet.get_data_range()
starting_row = data_range.get_row()
starting_column = data_range.get_column()
max_row = data_range.get_max_row()
max_column = data_range.get_max_column()
This will effectively tell you the max row and max column that contains data in your sheet.
If you use the get_data_range method, the first row and first column typically is 1.
I received a response from the owner of Sheetfu, and the following code provides the information that I'm looking for.
Example code:
from sheetfu import SpreadsheetApp
spreadsheet = SpreadsheetApp("....access_file.json").open_by_id('long_string_id')
sheet = spreadsheet.get_sheet_by_name('test')
data_range = sheet.get_data_range()
range_max_row = data_range.coordinates.row + data_range.coordinates.number_of_rows - 1
range_max_column = data_range.coordinates.column + data_range.coordinates.number_of_columns - 1
As of this writing, the .coordinates properties are not currently documented, but they are usable, and should be officially documented within the next couple of weeks.
I have an excel file with a table A6:E233. I had to concatenate columns A and B so that values from B are displayed in a new line. I have achieved that with the CONCATENATE function (and CHAR(10) for new line) that is built into Excel.
After concatenation the spreadsheets looks like this:
EXAMPLE1
Now i would also need different formatting for each line inside the cell, namely size 12, bold for the first line and size 8 for second line:
EXAMPLE2
How do achieve this? If it would be a short table, I would do it manually, but since I have a few files, totally well over 5000 rows, maybe an automated way would be better.
I have found answers that touch upon this problem, but since I dont know how to use VBA, I am more or less lost. I am also using a lot of python and have looked through openpyexl and csv, but have not found a way how to achieve this.
Thank you for your help!
With Excel VBA, you need to use the Characters property of the Range object. For example:
Sub Test()
Dim rngCell As Range
Dim lngPos As Long
'get cell
Set rngCell = Sheet1.Range("A1")
'find linebreak
lngPos = InStr(1, rngCell.Value, vbLf, vbBinaryCompare)
'format either side
rngCell.Characters(1, lngPos).Font.Bold = True
rngCell.Characters(lngPos + 1, Len(rngCell.Value) - lngPos).Font.Color = 1234
End Sub
Which will format like this:
Here, try this code. I built this according to your screenshot.
Sub partialFormatting()
Dim tws As Worksheet
Dim fr, lr As Integer
Dim pos As Integer
Set tws = ThisWorkbook.Worksheets("Sheet1")
fr = 7
lr = tws.Range("A1000000").End(xlUp).Row
For r = fr To lr
With tws.Range("A" & r)
pos = InStr(.Value, vbLf)
With .Characters(Start:=1, Length:=pos - 1).Font
.FontStyle = "Bold"
.Size = 12
End With
With .Characters(Start:=pos + 1, Length:=Len(.Value) - pos).Font
.FontStyle = "Normal"
.Size = 8
End With
End With
Next r
End Sub
Please let me know if you have any questions on how the code works!
I imported an excel spreadsheet and I am trying to clean up empty values with default values in all rows in my spreadsheet. I don't need to update the spreadsheet, I just need to set default values because I am using this information to insert into a local database. Whenever I try to do so, it never gets processed correctly. Here is my original iteration of the code:
for root,dirs,files in os.walk(path):
xlsfiles=['1128CNLOAD.xlsx']
#xlsfiles=[ _ for _ in files if _.endswith('CNLOAD.xlsx') ]
print (xlsfiles)
for xlsfile in xlsfiles:
book=xlrd.open_workbook(os.path.join(root,xlsfile))
sheet=book.sheet_by_index(0)
cell=sheet.cell(1,1)
print (sheet)
sheet0 = book.sheet_by_index(0)
#sheet1 = book.sheet_by_index(1)
for rownum in range(sheet0.nrows):
print sheet0.row_values(rownum)
values=()
print sheet0.nrows
for row_index in range(1, sheet.nrows):
if sheet.cell(row_index,4).value == '':
sheet.cell(row_index,4).value = 0.0
print sheet.row(row_index)
The code spits returns no errors but nothing gets updated and the cells I am trying to update are still empty.
I also tried to change the loop to just do a value replace for the list which is seen below:
for row_index in range(1, sheet.nrows):
if sheet.row(1)[4] == "empty:''":
sheet.row(1)[4] = "number:0.0"
When I print after running this update, the list has not changed.
print(sheet.row(1))
[text:u'FRFHF', text:u' ', number:0.15, number:0.15, empty:'', empty:'', number:2.5, number:2.5, empty:'', empty:'']
Thank you for any help and let me know if you have any questions.
xlrd isn't really set up to edit the spreadsheet once you have it in memory. You can do it, but you have to use the undocumented internal implementation.
On my version (0.7.1), cells are stored internally to the sheet in a couple of different two-dimensional arrays - sheet._cell_types and sheet._cell_values are the main two. The types are defined by a set of constants in biffh.py, which the xlrd module imports. When you call cell, it constructs a new Cell instance using the value and type looked up for the given row/column pair. You could update those directly, or you could use the put_cell method.
So it looks like this would work:
if sheet.cell_type(1, 4) == xlrd.XL_CELL_EMPTY:
sheet._cell_types[1][4] = xlrd.XL_CELL_NUMBER
sheet._cell_values[1][4] = 0.0
Alternately:
if sheet.cell_type(1, 4) == xlrd.XL_CELL_EMPTY:
sheet.put_cell(1, 4, xlrd.XL_CELL_NUMBER, 0.0, sheet.cell_xf_index(1, 4))
You may need to review the code to make sure this didn't change if you're on a different version.
I'm looking for the equivalent to the vlookup function in excel. I have a script where I read in a csv file. I would like to be able to query an associated value from another column in the .csv. Script so far:
import matplotlib
import matplotlib.mlab as mlab
import glob
for files in glob.glob("*.csv"):
print files
r = mlab.csv2rec(files)
r.cols = r.dtype.names
depVar = r[r.cols[0]]
indVar = r[r.cols[1]]
print indVar
This will read in from .csv files in the same folder the script is in. In the above example depVar is the first column in the .csv, and indVar is the second column. In my case, I know a value for indVar, and I want to return the associated value for depVar. I'd like to add a command like:
depVar = r[r.cols[0]]
indVar = r[r.cols[1]]
print indVar
depVarAt5 = lookup value in depVar where indVar = 5 (I could sub in things for the 5 later)
In my case, all values in all fields are numbers and all of the values of indVar are unique. I want to be able to define a new variable (depVarAt5 in last example) equal to the associated value.
Here's example .csv contents, name the file anything and place it in same folder as script. In this example, depVarAt5 should be set equal to 16.1309.
Temp,Depth
16.1309,5
16.1476,94.4007
16.2488,100.552
16.4232,106.573
16.4637,112.796
16.478,118.696
16.4961,124.925
16.5105,131.101
16.5462,137.325
16.7016,143.186
16.8575,149.101
16.9369,155.148
17.0462,161.187
I think this solves your problem quite directly:
import numpy
import glob
for f in glob.glob("*.csv"):
print f
r = numpy.recfromcsv(f)
print numpy.interp(5, r.depth, r.temp)
I'm pretty sure numpy is a prerequisite for matplotlib.
Not sure what that r object is, but since it has a member called cols, I'm going to assume it also has a member called rows which contains the row data.
>>> r.rows
[[16.1309, 5], [16.1476, 94.4007], ...]
In that case, your pseudocode very nearly contains a valid generator expression/list comprehension.
depVarAt5 = lookup value in depVar where indVar = 5 (I could sub in things for the 5 later)
becomes
depVarAt5 = [row[0] for row in r.rows if row[1] == 5]
Or, more generally
depVarValue = [row[depVarColIndex] for row in r.rows if row[indVarColIndex] == searchValue]
so
def vlookup(rows, searchColumn, dataColumn, searchValue):
return [row[dataColumn] for row in rows if row[searchColumn] == searchValue]
Throw a [0] on the end of that if you can guarantee there will be exactly one output per input.
There's also a csv module in the Python standard libary which you might prefer to work with. =)
For arbitrary orderings and exact matches you can use indVar.index() and index depVar with the returned index.
If indVar is ordered and (well, "or", sort of) you need closest match then you should look at using bisect on indVar.