We Keep Coding

QTableView dynamic row heigh for large QAbstractTableModel

I know there have been a lot of times question was answered on stackoverflow about how to set row height for QTableView. I'm asking one more time but my question is not exactly about "how", at least not so simple. I'm setting row height successfully with help of Qt.SizeHintRole in data method of my custom model derived from QAbstractTableModel - see code below. (Also tried very similar example but with help of sizeHint() method of QStyledItemDelegate - the result is exactly the same.)
It works pretty good when I have MODEL_ROW_COUNT about 100 as in example below. But my dataset has ~30-40 thousands of rows. As result this simple application starts about 30 seconds with MODEL_ROW_COUNT=35000 for example.
The reason of this big delay is this line of code:
self.table_view.verticalHeader().setSectionResizeMode(QHeaderView.ResizeToContents)
Everything works really fast with MODEL_ROW_COUNT=35000 if I would comment this line. But in this case data() method is not called with Qt.SizeHintRole and I can't manipulate row height.
So, my question is - how to set row height on a per row basis for dataset with thousands of rows? Below example works but takes 30 seconds to start with 35 000 rows (after window is shown everything is fluent)...
At the same time if I use QSqlTableModel it doesn't have this problem and I may use sizeHint() of QStyledItemDelegate without big problems. But it's a mess to have too many delegates... May I subclass QStyledItemDelegate instead of QAbstractTableModel to implement my custom model? (I'm not sure that it will work as every source recomment to subclass QAbstractTableModel for custom models...)
Or I did something wrong and there is a better way than usage of QHeaderView.ResizeToContents?
P.S. I really need different heights. Some rows in database have less data and I may show them in a couple of cells. But others have more data and I need extra space to display it. The same height for all rows will mean either waste of space (a lot of white space on a screen) or lack of essential details for some data rows. I'm using contant CUSTOM_ROW_HEIGHT only too keep example as much simple as possible and reproducible with ease - you may use any DB with any large table (I think I may re-create it even without DB... will try soon)
from PySide2.QtWidgets import QApplication, QWidget, QVBoxLayout, QTableView, QHeaderView
from PySide2.QtSql import QSqlDatabase, QSqlQuery
from PySide2.QtCore import Qt, QAbstractTableModel, QSize
class MyWindow(QWidget):
def __init__(self):
QWidget.__init__(self)
self.db = QSqlDatabase.addDatabase("QSQLITE")
self.db.setDatabaseName("/home/db.sqlite")
self.db.open()
self.table_model = MyModel(parent=self, db=self.db)
self.table_view = QTableView()
self.table_view.setModel(self.table_model)
# SizeHint is not triggered without this line but it causes delay
self.table_view.verticalHeader().setSectionResizeMode(QHeaderView.ResizeToContents)
layout = QVBoxLayout(self)
layout.addWidget(self.table_view)
self.setLayout(layout)
class MyModel(QAbstractTableModel):
CUSTOM_ROW_HEIGHT = 300
MODEL_ROW_COUNT = 100
MODEL_COL_COUNT = 5
def __init__(self, parent, db):
QAbstractTableModel.__init__(self, parent)
self.query = QSqlQuery(db)
self.query.prepare("SELECT * FROM big_table")
self.query.exec_()
def rowCount(self, parent=None):
return self.MODEL_ROW_COUNT
def columnCount(self, parent=None):
return self.MODEL_COL_COUNT
def data(self, index, role=Qt.DisplayRole):
if not index.isValid():
return None
if role == Qt.DisplayRole:
if self.query.seek(index.row()):
return str(self.query.value(index.column()))
if role == Qt.SizeHintRole:
return QSize(0, self.CUSTOM_ROW_HEIGHT)
return None
def main():
app = QApplication([])
win = MyWindow()
win.show()
app.exec_()
if __name__ == "__main__":
main()

Ok, Thanks to #musicamante I realized that I missed canFetchMore() and fetchMore() methods. So, I implemented dynamic size property and these methods in MyModel class. It was not hard at all and now I have better performance than QSqlTableModel and identical visual behavior with direct conrol of visible buffer size. Below is new code of MyModel class:
class MyModel(QAbstractTableModel):
CUSTOM_ROW_HEIGHT = 300
MODEL_ROW_COUNT = 37000
MODEL_COL_COUNT = 5
PAGE_SIZE = 500
def __init__(self, parent, db):
QAbstractTableModel.__init__(self, parent)
self.query = QSqlQuery(db)
self.query.prepare("SELECT * FROM big_table")
self.query.exec_()
self._current_size = self.PAGE_SIZE
def rowCount(self, parent=None):
return self._current_size
def columnCount(self, parent=None):
return self.MODEL_COL_COUNT
def data(self, index, role=Qt.DisplayRole):
if not index.isValid():
return None
if role == Qt.DisplayRole:
if self.query.seek(index.row()):
return str(self.query.value(index.column()))
if role == Qt.SizeHintRole:
return QSize(0, self.CUSTOM_ROW_HEIGHT)
return None
def canFetchMore(self, index):
return self._current_size < self.MODEL_ROW_COUNT
def fetchMore(self, index):
self.beginInsertRows(index, self._current_size, self._current_size + self.PAGE_SIZE - 1)
self._current_size += self.PAGE_SIZE
self.endInsertRows()

Tab over non-editable cells in QtableView

I am using PYQT5. I have a QtableView widget which I use for quick data entry similar to a spreadsheet. Some columns are editable while others (labels etc) are not. I have achieved this quite simply by subclassing QItemDelegate.
What I would like to do is when a user tabs from an editable cell, it will skip any non-editable cell and go to the next editable cell. I think I need to examine a keypress event after editing somewhere and determine which column is next. Alternatively, when I land in a non-editable cell, I should move immediately to the next editable cell. My code is:
class Rates_sheet(QDialog, Ui_rates_sheet):
"""Displays rates for the next x days for quick changes"""
#----------------------------------------------------------------------
def __init__(self):
"""Constructor"""
QDialog.__init__(self)
self.ui = Ui_rates_sheet()
self.ui.setupUi(self)
self.ui.num_days.setValue(45)
self.ui.get_avail.clicked.connect(self.make_rate_sheet)
self.ui.publish_avail.clicked.connect(self.publish_rates)
self.populate()
def populate(self):
self.rates_model = QSqlTableModel()
self.rates_model.setTable("rates_sheet")
self.rates_model.select()
self.rates_view = self.ui.rates_grid
self.rates_view.setModel(self.rates_model)
self.rates_view.resizeColumnsToContents()
self.rates_view.setSortingEnabled(True)
self.rates_view.setColumnHidden(0, True)
for x in range(7,12):
self.rates_view.setColumnHidden(x, True)
self.rates_view.horizontalHeader().moveSection(3,10)
self.rates_view.horizontalHeader().moveSection(3,12)
self.rates_view.horizontalHeader().moveSection(13,2)
self.rates_view.setItemDelegate(Tmodel(self))
self.rates_model.setHeaderData(1, Qt.Horizontal, "Date")
self.rates_model.sort(1, Qt.AscendingOrder)
def make_rate_sheet(self):
pass
def publish_rates(self):
pass
class Tmodel(QItemDelegate):
"""Remplement rates_sheet table"""
def __init__(self, parent=None):
QItemDelegate.__init__(self)
def createEditor(self, parent, option, index):
if index.column() == 4:
spinbox = QSpinBox(parent)
spinbox.setRange(1,4)
return spinbox
elif index.column() in [5,6,11,12]:
spinbox = QSpinBox(parent)
spinbox.setRange(49,499)
return spinbox
EDIT:
I have tried to reimplement QSqlTableModel to change the value in flags but end up with a runtime error:
class MySqlTableModel(QSqlTableModel):
def __init__(self):
QSqlTableModel.__init__(self)
def flags(self, index):
if index.isValid():
if index.column() == 4:
flags ^= 1
return flags
The error I get is:
File "f:\Dropbox\pms\main-5.py", line 2658, in <module>
sys.exit(app.exec_())
builtins.TypeError: invalid result from MySqlTableModel.flags(), NoneType cannot be converted to PyQt5.QtCore.ItemFlags in this context
Now I am even more confused.

After sweating over this for days, the answer is that you can successfully reimplement QsqlTableModel. Here is how I arrived at the answer.
First I saw recommendations on the net mostly in the QT forums for C++ that you should reimplement QsqlQueryModel instead of QsqlTableModel. But then you have to create the methods for setdata, data, rowcount, columncount, insertrows, deleterows not to mention flags yourself. This seemed like a lot of work and prone to error. I could not get it to work.
Thinking that all of the above was pointless and a lot of work, I found a code snippet from stackoverflow where someone was trying to use the dataChanged signal from QsqlQueryModel and they were recommended to overide QsqlTableModel. Finally I saw an example of how to do precisely what I was attempting. Therefore I updated my code as follows:
class Rates_sheet(QDialog, Ui_rates_sheet):
"""Displays rates for the next x days for quick changes"""
def __init__(self):
"""Constructor"""
QDialog.__init__(self)
self.ui = Ui_rates_sheet()
self.ui.setupUi(self)
self.ui.num_days.setValue(45)
self.ui.get_avail.clicked.connect(self.make_rate_sheet)
self.ui.publish_avail.clicked.connect(self.publish_rates)
self.populate()
def populate(self):
self.rates_model = MySqlTableModel()
self.rates_model.setTable("rates_sheet")
self.rates_model.select()
self.rates_view = self.ui.rates_grid
self.rates_view.setModel(self.rates_model)
self.rates_view.resizeColumnsToContents()
self.rates_view.setSortingEnabled(True)
self.rates_view.setColumnHidden(0, True)
self.rates_view.setItemDelegate(Tmodel(self))
self.rates_model.setHeaderData(1, Qt.Horizontal, "Date")
self.rates_model.sort(1, Qt.AscendingOrder)
def make_rate_sheet(self):
pass
def publish_rates(self):
pass
class MySqlTableModel(QSqlTableModel):
"""Overides QSqlTableModel to make columns not selectable"""
def __init__(self):
QSqlTableModel.__init__(self)
def setData(self, index, value, role=Qt.EditRole):
if role == Qt.EditRole:
value = value.strip() if type(value) == str else value
return super(MySqlTableModel, self).setData(index, value, role)
def flags(self, index):
flags = super(MySqlTableModel, self).flags(index)
if index.column() in (4, 5, 6, 11):
flags |= Qt.ItemIsEditable
else:
flags &= Qt.ItemIsSelectable
return flags
This was the solution. I am still testing but I haven't found any problems yet.

spanning multiple columns, using QTreeView and QAbstractItemModel

I want to make a TreeView where the top level entries span all columns (e.g. they have one row), while children span multiple columns (e.g. they have multiple rows.)
I was trying to accomplish this with QTreeView.setFirstColumnSpanned. However, I can't really figure out where/when to call this (and maybe I'm going about things completely wrong to begin with.)
I thought maybe I could make a function in my view that would be called after the view is populated, to check for any items that need to have their spans updated.
However, I couldn't find any signals to connect to that might help me do this (I tried several, but none of them ever seemed to fire.)
I also tried reimplementing insertRows in my model and/or rowsInserted in my view, but they never seem to get called.
Here's an extremely simplified version of what I'm doing:
class MyModel(QtCore.QAbstractItemModel):
def __init__(self, top_level_nodes):
QtCore.QAbstractItemModel.__init__(self)
self.top_level_nodes = top_level_nodes
self.columns = 5
def columnCount(self, parent):
return self.columns
def rowCount(self, parent):
total = len(self.top_level_nodes)
for node in self.top_level_nodes:
total += len(node.subnodes)
return total
def data(self, index, role):
if not index.isValid():
return QtCore.QVariant()
if role == QtCore.Qt.DisplayRole:
obj = index.internalPointer()
return obj.name
return QtCore.QVariant()
def index(self, row, column, parent):
if not parent.isValid():
if row > (len(self.top_level_nodes) - 1):
return QtCore.QModelIndex()
return self.createIndex(row, column, self.top_level_nodes[row])
return QtCore.QModelIndex()
def parent(self, index):
if not index.isValid():
return QtCore.QModelIndex()
node = index.internalPointer()
if node.parent is None:
return QtCore.QModelIndex()
else:
return self.createIndex(node.parent.row, 0, node.parent)
class FakeEntry(object):
def __init__(self, name, row, children=[]):
self.parent = None
self.row = row
self.name = 'foo'
self.subnodes = children
class MainWindow(QtGui.QWidget):
def __init__(self):
QtGui.QWidget.__init__(self)
self.resize(800, 600)
self.layout = QtGui.QVBoxLayout(self)
self.tree_view = QtGui.QTreeView(self)
self.layout.addWidget(self.tree_view)
entry = FakeEntry('foo', 0, children=[])
self.model = MyModel([entry])
self.tree_view.setModel(self.model)
def main():
app = QtGui.QApplication(sys.argv)
frame = MainWindow()
frame.show()
sys.exit(app.exec_())
if __name__ == '__main__':
main()
This code runs fine. What I need to know is:
How can I make the entry created by FakeEntry span all 5 columns instead of creating 5 identical cells?
Other questions:
Can I reimplement QAbstractItemModel.insertRows or QTreeView.rowsInserted? Is there a "trick" to doing this that I'm missing, which is causing my code to pick up the default implementation?
What actually handles creating the rows/columns in the view, once the model is populated with data and attached to the view?

This works for me if I add a call to setFirstColumnSpanned after the setModel() call in MainWindow.__init__().
class MainWindow(QtGui.QWidget):
def __init__(self):
QtGui.QWidget.__init__(self)
self.resize(800, 600)
self.layout = QtGui.QVBoxLayout(self)
self.tree_view = QtGui.QTreeView(self)
self.layout.addWidget(self.tree_view)
entry = FakeEntry('foo', 0, children=[])
self.model = MyModel([entry])
self.tree_view.setModel(self.model)
self.tree_view.setFirstColumnSpanned(0, QtCore.QModelIndex(), True)
I believe the view keeps track of the spanned columns internally, separate from the model. But it will reset the span information whenever setModel() is called. So, you should call setFirstColumnSpanned() after calling setModel() on the view. If your model is doing lazy loading with fetchMore(), you'll have to work out some way to inform the view that it should update the spans after the model has loaded new data.
With regard to what actually handles creating the rows/columns in the view, I think about that as a partnership between the model and the view. Once the view has a model set, it begins asking various questions to the model. For the top-most parent item (an invalid index), how many columns are there? Cool, there's 5. How many rows are there? Ok, 1. Just from that information the view's header can start to configure itself. At a certain point, the view iterates over the rows, columns, and children in the model, and asks each index for it's data in turn, for each data role. It can then start to create the delegates to display that data in the appropriate spot in the view.

PyQt: Adding rows to QTableView using QAbstractTableModel

I am super new to Qt programming. I am trying to make a simple table that can have rows added by clicking a button. I can implement the table fine but can't seem to get the updated data to show on the table. I believe my problem stems from the fact that I can't seem to properly call any sort of "change data" method using the button. I've tried several different solutions online all of which have lead to 4 year old, dead-end posts. What I have so far is the basic structure, I just can't figure out how to make the table update with new data.
This is the basic view
I have set up with some test data.
In the final implementation, the table will start empty and I would like to append rows and have them displayed in the table view.
import sys
from PyQt4.QtCore import *
from PyQt4.QtGui import *
class MyWindow(QWidget):
def __init__(self):
QWidget.__init__(self)
# create table
self.get_table_data()
self.table = self.createTable()
# layout
self.layout = QVBoxLayout()
self.testButton = QPushButton("test")
self.connect(self.testButton, SIGNAL("released()"), self.test)
self.layout.addWidget(self.testButton)
self.layout.addWidget(self.table)
self.setLayout(self.layout)
def get_table_data(self):
self.tabledata = [[1234567890,2,3,4,5],
[6,7,8,9,10],
[11,12,13,14,15],
[16,17,18,19,20]]
def createTable(self):
# create the view
tv = QTableView()
# set the table model
header = ['col_0', 'col_1', 'col_2', 'col_3', 'col_4']
tablemodel = MyTableModel(self.tabledata, header, self)
tv.setModel(tablemodel)
# set the minimum size
tv.setMinimumSize(400, 300)
# hide grid
tv.setShowGrid(False)
# hide vertical header
vh = tv.verticalHeader()
vh.setVisible(False)
# set horizontal header properties
hh = tv.horizontalHeader()
hh.setStretchLastSection(True)
# set column width to fit contents
tv.resizeColumnsToContents()
# set row height
tv.resizeRowsToContents()
# enable sorting
tv.setSortingEnabled(False)
return tv
def test(self):
self.tabledata.append([1,1,1,1,1])
self.emit(SIGNAL('dataChanged()'))
print 'success'
class MyTableModel(QAbstractTableModel):
def __init__(self, datain, headerdata, parent=None):
"""
Args:
datain: a list of lists\n
headerdata: a list of strings
"""
QAbstractTableModel.__init__(self, parent)
self.arraydata = datain
self.headerdata = headerdata
def rowCount(self, parent):
return len(self.arraydata)
def columnCount(self, parent):
if len(self.arraydata) > 0:
return len(self.arraydata[0])
return 0
def data(self, index, role):
if not index.isValid():
return QVariant()
elif role != Qt.DisplayRole:
return QVariant()
return QVariant(self.arraydata[index.row()][index.column()])
def setData(self, index, value, role):
pass # not sure what to put here
def headerData(self, col, orientation, role):
if orientation == Qt.Horizontal and role == Qt.DisplayRole:
return QVariant(self.headerdata[col])
return QVariant()
def sort(self, Ncol, order):
"""
Sort table by given column number.
"""
self.emit(SIGNAL("layoutAboutToBeChanged()"))
self.arraydata = sorted(self.arraydata, key=operator.itemgetter(Ncol))
if order == Qt.DescendingOrder:
self.arraydata.reverse()
self.emit(SIGNAL("layoutChanged()"))
if __name__ == "__main__":
app = QApplication(sys.argv)
w = MyWindow()
w.show()
sys.exit(app.exec_())

When the underlying data of the model changes, the model should emit either layoutChanged or layoutAboutToBeChanged, so that view updates properly (there's also dataChanged, if you want to update a specific range of cells).
So you just need something like this:
def test(self):
self.tabledata.append([1,1,1,1,1])
self.table.model().layoutChanged.emit()
print 'success'

QAbstractTableModel have two special methods for that ( beginInsertRows() and endInsertRows()).
You can add api-point in your custom model. For example:
def insertGuest(self, guest):
self.beginInsertRows(QtCore.QModelIndex(), self.rowCount(), self.rowCount())
self.guestsTableData.append(guest)
self.endInsertRows()

I've made your table reference a class variable instead of an instance variable, so you could edit the data for the table from virtually anywhere in your code.
# First access the data of the table
self.tv_model = self.tv.model()
Secondly, I use the sort of pandas-dataframe-editing type approach.
Lets say your data that you want to add is stored in a variable on its own:
# These can be whatever, but for consistency,
# I used the data in the OP's example
new_values = [1, 1, 1, 1, 1]
There are different ways the next step can be approached, depending on whether the data is being added to the table, or updating existing values. Adding the data as a new row would be as follows.
# The headers should also be a class variable,
# but I left it as the OP had it
header = ['col_0', 'col_1', 'col_2', 'col_3', 'col_4']
# There are multiple ways of establishing what the row reference should be,
# this is just one example how to add a new row
new_row = len(self.tv_model.dataFrame.index)
for i, col in enumerate(header):
self.tv_model.dataFrame.loc[new_row, col] = new_values[i]
Since self.tv_model is a reference to the actual data of the table,
emitting the following signal will update the data, or 'commit' it to the model,
so to speak.
self.tv_model.layoutChanged.emit()

PyQt - QTableView search by hiding rows

Here's my problem. I have a QTableView displaying some data (set up in a model) and a QLineEdit Widget that i want to use for searching text in all displayed rows. The expected behavior should be: i type some text in the QLineEdit and the QTableView update itself hiding all rows that not contain that data.
The question is, how should i implement that? I have found a QTableView member function named hideRows() that seems the right choice, but i can't figure out how should i iterate through all the data and where to put that method. It should be contained inside the model or the dialog? (this is actually my first time using models, so i just grasped how they work)
Plus i need to implement an export function (csv, html or whatever) but just with the currently displayed rows (those who are not hiding). Is this possible?
Thank you for any advice.
Here is my code till now:
from PyQt4.QtCore import *
from PyQt4.QtGui import *
import sys
import Android_extractor
import ui_android_dialog
class recordsTableModel(QAbstractTableModel):
def __init__(self, records, parent = None):
QAbstractTableModel.__init__(self, parent)
self.__records = records
def rowCount(self, parent):
return len(self.__records)
def columnCount(self, parent):
return len(self.__records[0])
def flags(self, index):
return Qt.ItemIsEnabled | Qt.ItemIsSelectable
def data(self, index, role):
if role == Qt.EditRole:
row = index.row()
column = index.column()
return self.__colors[row][column].name()
if role == Qt.DisplayRole:
row = index.row()
column = index.column()
value = self.__records[row][column]
return value
def headerData(self, section, orientation, role):
if role == Qt.DisplayRole:
if orientation == Qt.Horizontal:
return self.__records[0]._fields[section]
class AndroidDialog(QDialog, ui_android_dialog.Ui_androidDialog):
def __init__(self, parent=None):
super(AndroidDialog, self).__init__(parent)
self.setupUi(self)
self.buttonMapper = QSignalMapper(self)
self.buttonMapper.setMapping(self.contactsToolButton, 0)
self.buttonMapper.setMapping(self.groupsToolButton, 1)
self.buttonMapper.setMapping(self.chatsessionToolButton, 2)
self.buttonMapper.setMapping(self.messageToolButton, 3)
self.contactsToolButton.clicked.connect(self.buttonMapper.map)
self.groupsToolButton.clicked.connect(self.buttonMapper.map)
self.chatsessionToolButton.clicked.connect(self.buttonMapper.map)
self.messageToolButton.clicked.connect(self.buttonMapper.map)
self.buttonMapper.mapped.connect(self.dataStackedWidget.setCurrentIndex)
self.newQuery = Android_extractor.AndroidQuery(sys.argv[1])
self.contacts = self.newQuery.getContacts()
self.groups = self.newQuery.getGroups()
self.chats = self.newQuery.getChats()
self.contactsTableView.setModel(recordsTableModel(self.contacts))
self.contactsTableView.resizeColumnsToContents()
self.contactsTableView.resizeRowsToContents()
self.groupsTableView.setModel(recordsTableModel(self.groups))
self.groupsTableView.resizeColumnsToContents()
self.groupsTableView.resizeRowsToContents()
self.chatsessionTableView.setModel(recordsTableModel(self.chats))
self.chatsessionTableView.resizeColumnsToContents()
self.chatsessionTableView.resizeRowsToContents()
app = QApplication(sys.argv)
form = AndroidDialog()
form.show()
app.exec_()

You should have a look at QSortFilterProxyModel
Instead of setting your custom model directly on your tableview, set in as Source model on the proxy, then set the proxy model on your view.
self.proxyModelContact = QSortFilterProxyModel(self)
self.proxyModelContact.setSourceModel(recordsTableModel(self.contacts))
self.contactsTableView.setModel(self.proxyModelContact)
QSortFilterProxyModel provides two methods:
setFilterRegExp(pattern) allows you to set a regex filter on your view (i.e only the items that match the pattern will be displayed)
setFilterKeyColumn(index) allows you to define which column will be used to do the filtering (if index = -1, all columns will be looked at).
You just have to link the textChanged signal of your linedit to a slot which will update the filter eg:
def onTextChanged(self, text):
self.proxyModelContact.setFilterRegExp(str(text))