Messagelib

model.cpp
1 /******************************************************************************
2  *
3  * SPDX-FileCopyrightText: 2008 Szymon Tomasz Stefanek <[email protected]>
4  *
5  * SPDX-License-Identifier: GPL-2.0-or-later
6  *
7  *******************************************************************************/
8 
9 //
10 // This class is a rather huge monster. It's something that resembles a QAbstractItemModel
11 // (because it has to provide the interface for a QTreeView) but isn't entirely one
12 // (for optimization reasons). It basically manages a tree of items of two types:
13 // GroupHeaderItem and MessageItem. Be sure to read the docs for ViewItemJob.
14 //
15 // A huge credit here goes to Till Adam which seems to have written most
16 // (if not all) of the original KMail threading code. The KMHeaders implementation,
17 // the documentation and his clever ideas were my starting points and essential tools.
18 // This is why I'm adding his copyright entry (copied from headeritem.cpp) here even if
19 // he didn't write a byte in this file until now :)
20 //
21 // Szymon Tomasz Stefanek, 03 Aug 2008 04:50 (am)
22 //
23 // This class contains ideas from:
24 //
25 // kmheaders.cpp / kmheaders.h, headeritem.cpp / headeritem.h
26 // Copyright: (c) 2004 Till Adam < adam at kde dot org >
27 //
28 #include "core/model.h"
29 #include "core/filter.h"
30 #include "core/groupheaderitem.h"
31 #include "core/item_p.h"
32 #include "core/messageitem.h"
33 #include "core/messageitemsetmanager.h"
34 #include "core/model_p.h"
35 #include "core/modelinvariantrowmapper.h"
36 #include "core/storagemodelbase.h"
37 #include "core/theme.h"
38 #include "core/view.h"
39 #include "messagelist_debug.h"
40 #include <config-messagelist.h>
41 
42 #include "MessageCore/StringUtil"
43 #include <Akonadi/Item>
44 #include <Akonadi/MessageStatus>
45 
46 #include <KLocalizedString>
47 
48 #include <QApplication>
49 #include <QDateTime>
50 #include <QElapsedTimer>
51 #include <QIcon>
52 #include <QLocale>
53 #include <QScrollBar>
54 #include <QTimer>
55 
56 #include <algorithm>
57 #include <chrono>
58 
59 using namespace std::chrono_literals;
60 
61 namespace MessageList
62 {
63 namespace Core
64 {
65 Q_GLOBAL_STATIC(QTimer, _k_heartBeatTimer)
66 
67 /**
68  * A job in a "View Fill" or "View Cleanup" or "View Update" task.
69  *
70  * For a "View Fill" task a job is a set of messages
71  * that are contiguous in the storage. The set is expressed as a range
72  * of row indexes. The task "sweeps" the storage in the specified
73  * range, creates the appropriate Item instances and places them
74  * in the right position in the tree.
75  *
76  * The idea is that in a single instance and for the same StorageModel
77  * the jobs should never "cover" the same message twice. This assertion
78  * is enforced all around this source file.
79  *
80  * For a "View Cleanup" task the job is a list of ModelInvariantIndex
81  * objects (that are in fact MessageItem objects) that need to be removed
82  * from the view.
83  *
84  * For a "View Update" task the job is a list of ModelInvariantIndex
85  * objects (that are in fact MessageItem objects) that need to be updated.
86  *
87  * The interesting fact is that all the tasks need
88  * very similar operations to be performed on the message tree.
89  *
90  * For a "View Fill" we have 5 passes.
91  *
92  * Pass 1 scans the underlying storage, creates the MessageItem objects
93  * (which are subclasses of ModelInvariantIndex) and retrieves invariant
94  * storage indexes for them. It also builds threading caches and
95  * attempts to do some "easy" threading. If it succeeds in threading
96  * and some conditions apply then it also attaches the items to the view.
97  * Any unattached message is placed in a list.
98  *
99  * Pass 2 scans the list of messages that haven't been attached in
100  * the first pass and performs perfect and reference based threading.
101  * Since grouping of messages may depend on the "shape" of the thread
102  * then certain threads aren't attached to the view yet.
103  * Unassigned messages get stuffed into a list waiting for Pass3
104  * or directly to a list waiting for Pass4 (that is, Pass3 may be skipped
105  * if there is no hope to find an imperfect parent by subject based threading).
106  *
107  * Pass 3 scans the list of messages that haven't been attached in
108  * the first and second passes and performs subject based threading.
109  * Since grouping of messages may depend on the "shape" of the thread
110  * then certain threads aren't attached to the view yet.
111  * Anything unattached gets stuffed into the list waiting for Pass4.
112  *
113  * Pass 4 scans the unattached threads and puts them in the appropriate
114  * groups. After this pass nothing is unattached.
115  *
116  * Pass 5 eventually re-sorts the groups and removes the empty ones.
117  *
118  * For a "View Cleanup" we still have 5 passes.
119  *
120  * Pass 1 scans the list of invalidated ModelInvariantIndex-es, casts
121  * them to MessageItem objects and detaches them from the view.
122  * The orphan children of the destroyed items get stuffed in the list
123  * of unassigned messages that has been used also in the "View Fill" task above.
124  *
125  * Pass 2, 3, 4 and 5: same as "View Fill", just operating on the "orphaned"
126  * messages that need to be reattached to the view.
127  *
128  * For a "View Update" we still have 5 passes.
129  *
130  * Pass 1 scans the list of ModelInvariantIndex-es that need an update, casts
131  * them to MessageItem objects and handles the updates from storage.
132  * The updates may cause a regrouping so items might be stuffed in one
133  * of the lists for pass 4 or 5.
134  *
135  * Pass 2, 3 and 4 are simply empty.
136  *
137  * Pass 5: same as "View Fill", just operating on groups that require updates
138  * after the messages have been moved in pass 1.
139  *
140  * That's why we in fact have Pass1Fill, Pass1Cleanup, Pass1Update, Pass2, Pass3, Pass4 and Pass5 below.
141  * Pass1Fill, Pass1Cleanup and Pass1Update are exclusive and all of them proceed with Pass2 when finished.
142  */
143 class ViewItemJob
144 {
145 public:
146  enum Pass {
147  Pass1Fill = 0, ///< Build threading caches, *TRY* to do some threading, try to attach something to the view
148  Pass1Cleanup = 1, ///< Kill messages, build list of orphans
149  Pass1Update = 2, ///< Update messages
150  Pass2 = 3, ///< Thread everything by using caches, try to attach more to the view
151  Pass3 = 4, ///< Do more threading (this time try to guess), try to attach more to the view
152  Pass4 = 5, ///< Attach anything is still unattached
153  Pass5 = 6, ///< Eventually Re-sort group headers and remove the empty ones
154  LastIndex = 7 ///< Keep this at the end, needed to get the size of the enum
155  };
156 
157 private:
158  // Data for "View Fill" jobs
159  int mStartIndex; ///< The first index (in the underlying storage) of this job
160  int mCurrentIndex; ///< The current index (in the underlying storage) of this job
161  int mEndIndex; ///< The last index (in the underlying storage) of this job
162 
163  // Data for "View Cleanup" jobs
164  QList<ModelInvariantIndex *> *mInvariantIndexList; ///< Owned list of shallow pointers
165 
166  // Common data
167 
168  // The maximum time that we can spend "at once" inside viewItemJobStep() (milliseconds)
169  // The bigger this value, the larger chunks of work we do at once and less the time
170  // we loose in "breaking and resuming" the job. On the other side large values tend
171  // to make the view less responsive up to a "freeze" perception if this value is larger
172  // than 2000.
173  int mChunkTimeout;
174 
175  // The interval between two fillView steps. The larger the interval, the more interactivity
176  // we have. The shorter the interval the more work we get done per second.
177  int mIdleInterval;
178 
179  // The minimum number of messages we process in every viewItemJobStep() call
180  // The larger this value the less time we loose in checking the timeout every N messages.
181  // On the other side, making this very large may make the view less responsive
182  // if we're processing very few messages at a time and very high values (say > 10000) may
183  // eventually make our job unbreakable until the end.
184  int mMessageCheckCount;
185  Pass mCurrentPass;
186 
187  // If this parameter is true then this job uses a "disconnected" UI.
188  // It's FAR faster since we don't need to call beginInsertRows()/endInsertRows()
189  // and we simply Q_EMIT a layoutChanged() at the end. It can be done only as the first
190  // job though: subsequent jobs can't use layoutChanged() as it looses the expanded
191  // state of items.
192  bool mDisconnectUI;
193 
194 public:
195  /**
196  * Creates a "View Fill" operation job
197  */
198  ViewItemJob(int startIndex, int endIndex, int chunkTimeout, int idleInterval, int messageCheckCount, bool disconnectUI = false)
199  : mStartIndex(startIndex)
200  , mCurrentIndex(startIndex)
201  , mEndIndex(endIndex)
202  , mInvariantIndexList(nullptr)
203  , mChunkTimeout(chunkTimeout)
204  , mIdleInterval(idleInterval)
205  , mMessageCheckCount(messageCheckCount)
206  , mCurrentPass(Pass1Fill)
207  , mDisconnectUI(disconnectUI)
208  {
209  }
210 
211  /**
212  * Creates a "View Cleanup" or "View Update" operation job
213  */
214  ViewItemJob(Pass pass, QList<ModelInvariantIndex *> *invariantIndexList, int chunkTimeout, int idleInterval, int messageCheckCount)
215  : mStartIndex(0)
216  , mCurrentIndex(0)
217  , mEndIndex(invariantIndexList->count() - 1)
218  , mInvariantIndexList(invariantIndexList)
219  , mChunkTimeout(chunkTimeout)
220  , mIdleInterval(idleInterval)
221  , mMessageCheckCount(messageCheckCount)
222  , mCurrentPass(pass)
223  , mDisconnectUI(false)
224  {
225  }
226 
227  ~ViewItemJob()
228  {
229  delete mInvariantIndexList;
230  }
231 
232 public:
233  Q_REQUIRED_RESULT int startIndex() const
234  {
235  return mStartIndex;
236  }
237 
238  void setStartIndex(int startIndex)
239  {
240  mStartIndex = startIndex;
241  mCurrentIndex = startIndex;
242  }
243 
244  Q_REQUIRED_RESULT int currentIndex() const
245  {
246  return mCurrentIndex;
247  }
248 
249  void setCurrentIndex(int currentIndex)
250  {
251  mCurrentIndex = currentIndex;
252  }
253 
254  Q_REQUIRED_RESULT int endIndex() const
255  {
256  return mEndIndex;
257  }
258 
259  void setEndIndex(int endIndex)
260  {
261  mEndIndex = endIndex;
262  }
263 
264  Q_REQUIRED_RESULT Pass currentPass() const
265  {
266  return mCurrentPass;
267  }
268 
269  void setCurrentPass(Pass pass)
270  {
271  mCurrentPass = pass;
272  }
273 
274  Q_REQUIRED_RESULT int idleInterval() const
275  {
276  return mIdleInterval;
277  }
278 
279  Q_REQUIRED_RESULT int chunkTimeout() const
280  {
281  return mChunkTimeout;
282  }
283 
284  Q_REQUIRED_RESULT int messageCheckCount() const
285  {
286  return mMessageCheckCount;
287  }
288 
289  Q_REQUIRED_RESULT QList<ModelInvariantIndex *> *invariantIndexList() const
290  {
291  return mInvariantIndexList;
292  }
293 
294  Q_REQUIRED_RESULT bool disconnectUI() const
295  {
296  return mDisconnectUI;
297  }
298 };
299 } // namespace Core
300 } // namespace MessageList
301 
302 using namespace MessageList::Core;
303 
304 Model::Model(View *pParent)
305  : QAbstractItemModel(pParent)
306  , d(new ModelPrivate(this))
307 {
308  d->mRecursionCounterForReset = 0;
309  d->mStorageModel = nullptr;
310  d->mView = pParent;
311  d->mAggregation = nullptr;
312  d->mTheme = nullptr;
313  d->mSortOrder = nullptr;
314  d->mFilter = nullptr;
315  d->mPersistentSetManager = nullptr;
316  d->mInLengthyJobBatch = false;
317  d->mLastSelectedMessageInFolder = nullptr;
318  d->mLoading = false;
319 
320  d->mRootItem = new Item(Item::InvisibleRoot);
321  d->mRootItem->setViewable(nullptr, true);
322 
323  d->mFillStepTimer.setSingleShot(true);
324  d->mInvariantRowMapper = new ModelInvariantRowMapper();
325  d->mModelForItemFunctions = this;
326  connect(&d->mFillStepTimer, &QTimer::timeout, this, [this]() {
327  d->viewItemJobStep();
328  });
329 
330  d->mCachedTodayLabel = i18n("Today");
331  d->mCachedYesterdayLabel = i18n("Yesterday");
332  d->mCachedUnknownLabel = i18nc("Unknown date", "Unknown");
333  d->mCachedLastWeekLabel = i18n("Last Week");
334  d->mCachedTwoWeeksAgoLabel = i18n("Two Weeks Ago");
335  d->mCachedThreeWeeksAgoLabel = i18n("Three Weeks Ago");
336  d->mCachedFourWeeksAgoLabel = i18n("Four Weeks Ago");
337  d->mCachedFiveWeeksAgoLabel = i18n("Five Weeks Ago");
338 
339  d->mCachedWatchedOrIgnoredStatusBits = Akonadi::MessageStatus::statusIgnored().toQInt32() | Akonadi::MessageStatus::statusWatched().toQInt32();
340 
341  connect(_k_heartBeatTimer(), &QTimer::timeout, this, [this]() {
342  d->checkIfDateChanged();
343  });
344 
345  if (!_k_heartBeatTimer->isActive()) { // First model starts it
346  _k_heartBeatTimer->start(1min); // 1 minute
347  }
348 }
349 
351 {
352  setStorageModel(nullptr);
353 
354  d->clearJobList();
355  d->mOldestItem = nullptr;
356  d->mNewestItem = nullptr;
357  d->clearUnassignedMessageLists();
358  d->clearOrphanChildrenHash();
359  d->clearThreadingCacheReferencesIdMD5ToMessageItem();
360  d->clearThreadingCacheMessageSubjectMD5ToMessageItem();
361  delete d->mPersistentSetManager;
362  // Delete the invariant row mapper before removing the items.
363  // It's faster since the items will not need to call the invariant
364  delete d->mInvariantRowMapper;
365  delete d->mRootItem;
366 }
367 
368 void Model::setAggregation(const Aggregation *aggregation)
369 {
370  d->mAggregation = aggregation;
371  d->mView->setRootIsDecorated((d->mAggregation->grouping() == Aggregation::NoGrouping) && (d->mAggregation->threading() != Aggregation::NoThreading));
372 }
373 
374 void Model::setTheme(const Theme *theme)
375 {
376  d->mTheme = theme;
377 }
378 
379 void Model::setSortOrder(const SortOrder *sortOrder)
380 {
381  d->mSortOrder = sortOrder;
382 }
383 
385 {
386  return d->mSortOrder;
387 }
388 
389 void Model::setFilter(const Filter *filter)
390 {
391  d->mFilter = filter;
392 
393  if (d->mFilter) {
394  connect(d->mFilter, &Filter::finished, this, [this]() {
395  d->slotApplyFilter();
396  });
397  }
398 
399  d->slotApplyFilter();
400 }
401 
402 void ModelPrivate::slotApplyFilter()
403 {
404  auto childList = mRootItem->childItems();
405  if (!childList) {
406  return;
407  }
408 
409  QModelIndex idx; // invalid
410 
412  for (const auto child : std::as_const(*childList)) {
413  applyFilterToSubtree(child, idx);
414  }
415 
417 }
418 
419 bool ModelPrivate::applyFilterToSubtree(Item *item, const QModelIndex &parentIndex)
420 {
421  // This function applies the current filter (eventually empty)
422  // to a message tree starting at "item".
423 
424  if (!mModelForItemFunctions) {
425  qCWarning(MESSAGELIST_LOG) << "Cannot apply filter, the UI must be not disconnected.";
426  return true;
427  }
428  Q_ASSERT(item); // the item must obviously be valid
429  Q_ASSERT(item->isViewable()); // the item must be viewable
430 
431  // Apply to children first
432 
433  auto childList = item->childItems();
434 
435  bool childrenMatch = false;
436 
437  QModelIndex thisIndex = q->index(item, 0);
438 
439  if (childList) {
440  for (const auto child : std::as_const(*childList)) {
441  if (applyFilterToSubtree(child, thisIndex)) {
442  childrenMatch = true;
443  }
444  }
445  }
446 
447  if (!mFilter) { // empty filter always matches (but does not expand items)
448  mView->setRowHidden(thisIndex.row(), parentIndex, false);
449  return true;
450  }
451 
452  if (childrenMatch) {
453  mView->setRowHidden(thisIndex.row(), parentIndex, false);
454 
455  if (!mView->isExpanded(thisIndex)) {
456  mView->expand(thisIndex);
457  }
458  return true;
459  }
460 
461  if (item->type() == Item::Message) {
462  if (mFilter->match((MessageItem *)item)) {
463  mView->setRowHidden(thisIndex.row(), parentIndex, false);
464  return true;
465  }
466  } // else this is a group header and it never explicitly matches
467 
468  // filter doesn't match, hide the item
469  mView->setRowHidden(thisIndex.row(), parentIndex, true);
470 
471  return false;
472 }
473 
474 int Model::columnCount(const QModelIndex &parent) const
475 {
476  if (!d->mTheme) {
477  return 0;
478  }
479  if (parent.column() > 0) {
480  return 0;
481  }
482  return d->mTheme->columns().count();
483 }
484 
485 QVariant Model::data(const QModelIndex &index, int role) const
486 {
487  /// this is called only when Akonadi is using the selectionmodel
488  /// for item actions. since akonadi uses the ETM ItemRoles, and the
489  /// messagelist uses its own internal roles, here we respond
490  /// to the ETM ones.
491 
492  auto item = static_cast<Item *>(index.internalPointer());
493 
494  switch (role) {
495  /// taken from entitytreemodel.h
496  case Qt::UserRole + 1: // EntityTreeModel::ItemIdRole
497  if (item->type() == MessageList::Core::Item::Message) {
498  auto mItem = static_cast<MessageItem *>(item);
499  return QVariant::fromValue(mItem->akonadiItem().id());
500  } else {
501  return {};
502  }
503  break;
504  case Qt::UserRole + 2: // EntityTreeModel::ItemRole
505  if (item->type() == MessageList::Core::Item::Message) {
506  auto mItem = static_cast<MessageItem *>(item);
507  return QVariant::fromValue(mItem->akonadiItem());
508  } else {
509  return {};
510  }
511  break;
512  case Qt::UserRole + 3: // EntityTreeModel::MimeTypeRole
513  if (item->type() == MessageList::Core::Item::Message) {
514  return QStringLiteral("message/rfc822");
515  } else {
516  return {};
517  }
518  break;
520  if (item->type() == MessageList::Core::Item::Message) {
521  auto mItem = static_cast<MessageItem *>(item);
522  return mItem->accessibleText(d->mTheme, index.column());
523  } else if (item->type() == MessageList::Core::Item::GroupHeader) {
524  if (index.column() > 0) {
525  return QString();
526  }
527  auto hItem = static_cast<GroupHeaderItem *>(item);
528  return hItem->label();
529  }
530  return QString();
531  break;
532  default:
533  return {};
534  }
535 }
536 
537 QVariant Model::headerData(int section, Qt::Orientation, int role) const
538 {
539  if (!d->mTheme) {
540  return {};
541  }
542 
543  auto column = d->mTheme->column(section);
544  if (!column) {
545  return {};
546  }
547 
548  if (d->mStorageModel && column->isSenderOrReceiver() && (role == Qt::DisplayRole)) {
549  if (d->mStorageModelContainsOutboundMessages) {
550  return QVariant(i18n("Receiver"));
551  }
552  return QVariant(i18n("Sender"));
553  }
554 
555  const bool columnPixmapEmpty(column->pixmapName().isEmpty());
556  if ((role == Qt::DisplayRole) && columnPixmapEmpty) {
557  return QVariant(column->label());
558  } else if ((role == Qt::ToolTipRole) && !columnPixmapEmpty) {
559  return QVariant(column->label());
560  } else if ((role == Qt::DecorationRole) && !columnPixmapEmpty) {
561  return QVariant(QIcon::fromTheme(column->pixmapName()));
562  }
563 
564  return {};
565 }
566 
567 QModelIndex Model::index(Item *item, int column) const
568 {
569  if (!d->mModelForItemFunctions) {
570  return {}; // called with disconnected UI: the item isn't known on the Qt side, yet
571  }
572 
573  if (!item) {
574  return {};
575  }
576  // FIXME: This function is a bottleneck (the caching in indexOfChildItem only works 30% of the time)
577  auto par = item->parent();
578  if (!par) {
579  if (item != d->mRootItem) {
580  item->dump(QString());
581  }
582  return {};
583  }
584 
585  const int index = par->indexOfChildItem(item);
586  if (index < 0) {
587  return {}; // BUG
588  }
589  return createIndex(index, column, item);
590 }
591 
592 QModelIndex Model::index(int row, int column, const QModelIndex &parent) const
593 {
594  if (!d->mModelForItemFunctions) {
595  return {}; // called with disconnected UI: the item isn't known on the Qt side, yet
596  }
597 
598 #ifdef READD_THIS_IF_YOU_WANT_TO_PASS_MODEL_TEST
599  if (column < 0) {
600  return QModelIndex(); // senseless column (we could optimize by skipping this check but ModelTest from trolltech is pedantic)
601  }
602 #endif
603 
604  const Item *item;
605  if (parent.isValid()) {
606  item = static_cast<const Item *>(parent.internalPointer());
607  if (!item) {
608  return {}; // should never happen
609  }
610  } else {
611  item = d->mRootItem;
612  }
613 
614  if (parent.column() > 0) {
615  return {}; // parent column is not 0: shouldn't have children (as per Qt documentation)
616  }
617 
618  Item *child = item->childItem(row);
619  if (!child) {
620  return {}; // no such row in parent
621  }
622  return createIndex(row, column, child);
623 }
624 
625 QModelIndex Model::parent(const QModelIndex &modelIndex) const
626 {
627  Q_ASSERT(d->mModelForItemFunctions); // should be never called with disconnected UI
628 
629  if (!modelIndex.isValid()) {
630  return {}; // should never happen
631  }
632  auto item = static_cast<Item *>(modelIndex.internalPointer());
633  if (!item) {
634  return {};
635  }
636  auto par = item->parent();
637  if (!par) {
638  return {}; // should never happen
639  }
640  // return index( par, modelIndex.column() );
641  return index(par, 0); // parents are always in column 0 (as per Qt documentation)
642 }
643 
644 int Model::rowCount(const QModelIndex &parent) const
645 {
646  if (!d->mModelForItemFunctions) {
647  return 0; // called with disconnected UI
648  }
649 
650  const Item *item;
651  if (parent.isValid()) {
652  item = static_cast<const Item *>(parent.internalPointer());
653  if (!item) {
654  return 0; // should never happen
655  }
656  } else {
657  item = d->mRootItem;
658  }
659 
660  if (!item->isViewable()) {
661  return 0;
662  }
663 
664  return item->childItemCount();
665 }
666 
667 class RecursionPreventer
668 {
669 public:
670  RecursionPreventer(int &counter)
671  : mCounter(counter)
672  {
673  mCounter++;
674  }
675 
676  ~RecursionPreventer()
677  {
678  mCounter--;
679  }
680 
681  Q_REQUIRED_RESULT bool isRecursive() const
682  {
683  return mCounter > 1;
684  }
685 
686 private:
687  int &mCounter;
688 };
689 
691 {
692  return d->mStorageModel;
693 }
694 
695 void ModelPrivate::clear()
696 {
697  q->beginResetModel();
698  if (mFillStepTimer.isActive()) {
699  mFillStepTimer.stop();
700  }
701 
702  // Kill pre-selection at this stage
703  mPreSelectionMode = PreSelectNone;
704  mLastSelectedMessageInFolder = nullptr;
705  mOldestItem = nullptr;
706  mNewestItem = nullptr;
707 
708  // Reset the row mapper before removing items
709  // This is faster since the items don't need to access the mapper.
710  mInvariantRowMapper->modelReset();
711 
712  clearJobList();
713  clearUnassignedMessageLists();
714  clearOrphanChildrenHash();
715  mGroupHeaderItemHash.clear();
716  mGroupHeadersThatNeedUpdate.clear();
717  mThreadingCacheMessageIdMD5ToMessageItem.clear();
718  mThreadingCacheMessageInReplyToIdMD5ToMessageItem.clear();
719  clearThreadingCacheReferencesIdMD5ToMessageItem();
720  clearThreadingCacheMessageSubjectMD5ToMessageItem();
721  mViewItemJobStepChunkTimeout = 100;
722  mViewItemJobStepIdleInterval = 10;
723  mViewItemJobStepMessageCheckCount = 10;
724  delete mPersistentSetManager;
725  mPersistentSetManager = nullptr;
726  mCurrentItemToRestoreAfterViewItemJobStep = nullptr;
727 
728  mTodayDate = QDate::currentDate();
729 
730  // FIXME: CLEAR THE FILTER HERE AS WE CAN'T APPLY IT WITH UI DISCONNECTED!
731 
732  mRootItem->killAllChildItems();
733 
734  q->endResetModel();
735  // Q_EMIT headerDataChanged();
736 
737  mView->selectionModel()->clearSelection();
738 }
739 
740 void Model::setStorageModel(StorageModel *storageModel, PreSelectionMode preSelectionMode)
741 {
742  // Prevent a case of recursion when opening a folder that has a message and the folder was
743  // never opened before.
744  RecursionPreventer preventer(d->mRecursionCounterForReset);
745  if (preventer.isRecursive()) {
746  return;
747  }
748 
749  d->clear();
750 
751  if (d->mStorageModel) {
752  // Disconnect all signals from old storageModel
753  std::for_each(d->mStorageModelConnections.cbegin(), d->mStorageModelConnections.cend(), [](const QMetaObject::Connection &c) -> bool {
754  return QObject::disconnect(c);
755  });
756  d->mStorageModelConnections.clear();
757  }
758 
759  const bool isReload = (d->mStorageModel == storageModel);
760  d->mStorageModel = storageModel;
761 
762  if (!d->mStorageModel) {
763  return; // no folder: nothing to fill
764  }
765 
766  // Save threading cache of the previous folder, but only if the cache was
767  // enabled and a different folder is being loaded - reload of the same folder
768  // means change in aggregation in which case we will have to re-build the
769  // cache so there's no point saving the current threading cache.
770  if (d->mThreadingCache.isEnabled() && !isReload) {
771  d->mThreadingCache.save();
772  } else {
773  if (isReload) {
774  qCDebug(MESSAGELIST_LOG) << "Identical folder reloaded, not saving old threading cache";
775  } else {
776  qCDebug(MESSAGELIST_LOG) << "Threading disabled in previous folder, not saving threading cache";
777  }
778  }
779  // Load threading cache for the new folder, but only if threading is enabled,
780  // otherwise we would just be caching a flat list.
781  if (d->mAggregation->threading() != Aggregation::NoThreading) {
782  d->mThreadingCache.setEnabled(true);
783  d->mThreadingCache.load(d->mStorageModel->id(), d->mAggregation);
784  } else {
785  // No threading, no cache - don't even bother inserting entries into the
786  // cache or trying to look them up there
787  d->mThreadingCache.setEnabled(false);
788  qCDebug(MESSAGELIST_LOG) << "Threading disabled in folder" << d->mStorageModel->id() << ", not using threading cache";
789  }
790 
791  d->mPreSelectionMode = preSelectionMode;
792  d->mStorageModelContainsOutboundMessages = d->mStorageModel->containsOutboundMessages();
793 
794  d->mStorageModelConnections = {connect(d->mStorageModel,
796  this,
797  [this](const QModelIndex &parent, int first, int last) {
798  d->slotStorageModelRowsInserted(parent, first, last);
799  }),
800  connect(d->mStorageModel,
802  this,
803  [this](const QModelIndex &parent, int first, int last) {
804  d->slotStorageModelRowsRemoved(parent, first, last);
805  }),
806  connect(d->mStorageModel,
808  this,
809  [this]() {
810  d->slotStorageModelLayoutChanged();
811  }),
812  connect(d->mStorageModel,
814  this,
815  [this]() {
816  d->slotStorageModelLayoutChanged();
817  }),
818  connect(d->mStorageModel,
820  this,
821  [this](const QModelIndex &topLeft, const QModelIndex &bottomRight) {
822  d->slotStorageModelDataChanged(topLeft, bottomRight);
823  }),
824  connect(d->mStorageModel, &StorageModel::headerDataChanged, this, [this](Qt::Orientation orientation, int first, int last) {
825  d->slotStorageModelHeaderDataChanged(orientation, first, last);
826  })};
827 
828  if (d->mStorageModel->rowCount() == 0) {
829  return; // folder empty: nothing to fill
830  }
831 
832  // Here we use different strategies based on user preference and the folder size.
833  // The knobs we can tune are:
834  //
835  // - The number of jobs used to scan the whole folder and their order
836  //
837  // There are basically two approaches to this. One is the "single big job"
838  // approach. It scans the folder from the beginning to the end in a single job
839  // entry. The job passes are done only once. It's advantage is that it's simpler
840  // and it's less likely to generate imperfect parent threadings. The bad
841  // side is that since the folders are "sort of" date ordered then the most interesting
842  // messages show up at the end of the work. Not nice for large folders.
843  // The other approach uses two jobs. This is a bit slower but smarter strategy.
844  // First we scan the latest 1000 messages and *then* take care of the older ones.
845  // This will show up the most interesting messages almost immediately. (Well...
846  // All this assuming that the underlying storage always appends the newly arrived messages)
847  // The strategy is slower since it generates some imperfect parent threadings which must be
848  // adjusted by the second job. For instance, in my kernel mailing list folder this "smart" approach
849  // generates about 150 additional imperfectly threaded children... but the "today"
850  // messages show up almost immediately. The two-chunk job also makes computing
851  // the percentage user feedback a little harder and might break some optimization
852  // in the insertions (we're able to optimize appends and prepends but a chunked
853  // job is likely to split our work at a boundary where messages are always inserted
854  // in the middle of the list).
855  //
856  // - The maximum time to spend inside a single job step
857  //
858  // The larger this time, the greater the number of messages per second that this
859  // engine can process but also greater time with frozen UI -> less interactivity.
860  // Reasonable values start at 50 msecs. Values larger than 300 msecs are very likely
861  // to be perceived by the user as UI non-reactivity.
862  //
863  // - The number of messages processed in each job step subchunk.
864  //
865  // A job subchunk is processed without checking the maximum time above. This means
866  // that each job step will process at least the number of messages specified by this value.
867  // Very low values mean that we respect the maximum time very carefully but we also
868  // waste time to check if we ran out of time :)
869  // Very high values are likely to cause the engine to not respect the maximum step time.
870  // Reasonable values go from 5 to 100.
871  //
872  // - The "idle" time between two steps
873  //
874  // The lower this time, the greater the number of messages per second that this
875  // engine can process but also lower time for the UI to process events -> less interactivity.
876  // A value of 0 here means that Qt will trigger the timer as soon as it has some
877  // idle time to spend. UI events will be still processed but slowdowns are possible.
878  // 0 is reasonable though. Values larger than 200 will tend to make the total job
879  // completion times high.
880  //
881 
882  // If we have no filter it seems that we can apply a huge optimization.
883  // We disconnect the UI for the first huge filling job. This allows us
884  // to save the extremely expensive beginInsertRows()/endInsertRows() calls
885  // and call a single layoutChanged() at the end. This slows down a lot item
886  // expansion. But on the other side if only few items need to be expanded
887  // then this strategy is better. If filtering is enabled then this strategy
888  // isn't applicable (because filtering requires interaction with the UI
889  // while the data is loading).
890 
891  // So...
892 
893  // For the very first small chunk it's ok to work with disconnected UI as long
894  // as we have no filter. The first small chunk is always 1000 messages, so
895  // even if all of them are expanded, it's still somewhat acceptable.
896  bool canDoFirstSmallChunkWithDisconnectedUI = !d->mFilter;
897 
898  // Larger works need a bigger condition: few messages must be expanded in the end.
899  bool canDoJobWithDisconnectedUI = // we have no filter
900  !d->mFilter
901  && (
902  // we do no threading at all
903  (d->mAggregation->threading() == Aggregation::NoThreading) || // or we never expand threads
904  (d->mAggregation->threadExpandPolicy() == Aggregation::NeverExpandThreads) || // or we expand threads but we'll be going to expand really only a few
905  (
906  // so we don't expand them all
907  (d->mAggregation->threadExpandPolicy() != Aggregation::AlwaysExpandThreads) && // and we'd expand only a few in fact
908  (d->mStorageModel->initialUnreadRowCountGuess() < 1000)));
909 
910  switch (d->mAggregation->fillViewStrategy()) {
912  // favor interactivity
913  if ((!canDoJobWithDisconnectedUI) && (d->mStorageModel->rowCount() > 3000)) { // empiric value
914  // First a small job with the most recent messages. Large chunk, small (but non zero) idle interval
915  // and a larger number of messages to process at once.
916  auto job1 =
917  new ViewItemJob(d->mStorageModel->rowCount() - 1000, d->mStorageModel->rowCount() - 1, 200, 20, 100, canDoFirstSmallChunkWithDisconnectedUI);
918  d->mViewItemJobs.append(job1);
919  // Then a larger job with older messages. Small chunk, bigger idle interval, small number of messages to
920  // process at once.
921  auto job2 = new ViewItemJob(0, d->mStorageModel->rowCount() - 1001, 100, 50, 10, false);
922  d->mViewItemJobs.append(job2);
923 
924  // We could even extremize this by splitting the folder in several
925  // chunks and scanning them from the newest to the oldest... but the overhead
926  // due to imperfectly threaded children would be probably too big.
927  } else {
928  // small folder or can be done with disconnected UI: single chunk work.
929  // Lag the CPU a bit more but not too much to destroy even the earliest interactivity.
930  auto job = new ViewItemJob(0, d->mStorageModel->rowCount() - 1, 150, 30, 30, canDoJobWithDisconnectedUI);
931  d->mViewItemJobs.append(job);
932  }
933  break;
935  // More batchy jobs, still interactive to a certain degree
936  if ((!canDoJobWithDisconnectedUI) && (d->mStorageModel->rowCount() > 3000)) { // empiric value
937  // large folder, but favor speed
938  auto job1 =
939  new ViewItemJob(d->mStorageModel->rowCount() - 1000, d->mStorageModel->rowCount() - 1, 250, 0, 100, canDoFirstSmallChunkWithDisconnectedUI);
940  d->mViewItemJobs.append(job1);
941  auto job2 = new ViewItemJob(0, d->mStorageModel->rowCount() - 1001, 200, 0, 10, false);
942  d->mViewItemJobs.append(job2);
943  } else {
944  // small folder or can be done with disconnected UI and favor speed: single chunk work.
945  // Lag the CPU more, get more work done
946  auto job = new ViewItemJob(0, d->mStorageModel->rowCount() - 1, 250, 0, 100, canDoJobWithDisconnectedUI);
947  d->mViewItemJobs.append(job);
948  }
949  break;
951  // one large job, never interrupt, block UI
952  auto job = new ViewItemJob(0, d->mStorageModel->rowCount() - 1, 60000, 0, 100000, canDoJobWithDisconnectedUI);
953  d->mViewItemJobs.append(job);
954  break;
955  }
956  default:
957  qCWarning(MESSAGELIST_LOG) << "Unrecognized fill view strategy";
958  Q_ASSERT(false);
959  break;
960  }
961 
962  d->mLoading = true;
963 
964  d->viewItemJobStep();
965 }
966 
967 void ModelPrivate::checkIfDateChanged()
968 {
969  // This function is called by MessageList::Core::Manager once in a while (every 1 minute or sth).
970  // It is used to check if the current date has changed (with respect to mTodayDate).
971  //
972  // Our message items cache the formatted dates (as formatting them
973  // on the fly would be too expensive). We also cache the labels of the groups which often display dates.
974  // When the date changes we would need to fix all these strings.
975  //
976  // A dedicated algorithm to refresh the labels of the items would be either too complex
977  // or would block on large trees. Fixing the labels of the groups is also quite hard...
978  //
979  // So to keep the things simple we just reload the view.
980 
981  if (!mStorageModel) {
982  return; // nothing to do
983  }
984 
985  if (mLoading) {
986  return; // not now
987  }
988 
989  if (!mViewItemJobs.isEmpty()) {
990  return; // not now
991  }
992 
993  if (mTodayDate == QDate::currentDate()) {
994  return; // date not changed
995  }
996 
997  // date changed, reload the view (and try to preserve the current selection)
998  q->setStorageModel(mStorageModel, PreSelectLastSelected);
999 }
1000 
1002 {
1003  d->mPreSelectionMode = preSelect;
1004  d->mLastSelectedMessageInFolder = nullptr;
1005 }
1006 
1007 //
1008 // The "view fill" algorithm implemented in the functions below is quite smart but also quite complex.
1009 // It's governed by the following goals:
1010 //
1011 // - Be flexible: allow different configurations from "unsorted flat list" to a "grouped and threaded
1012 // list with different sorting algorithms applied to each aggregation level"
1013 // - Be reasonably fast
1014 // - Be non blocking: UI shouldn't freeze while the algorithm is running
1015 // - Be interruptible: user must be able to abort the execution and just switch to another folder in the middle
1016 //
1017 
1018 void ModelPrivate::clearUnassignedMessageLists()
1019 {
1020  // This is a bit tricky...
1021  // The three unassigned message lists contain messages that have been created
1022  // but not yet attached to the view. There may be two major cases for a message:
1023  // - it has no parent -> it must be deleted and it will delete its children too
1024  // - it has a parent -> it must NOT be deleted since it will be deleted by its parent.
1025 
1026  // Sometimes the things get a little complicated since in Pass2 and Pass3
1027  // we have transitional states in that the MessageItem object can be in two of these lists.
1028 
1029  // WARNING: This function does NOT fixup mNewestItem and mOldestItem. If one of these
1030  // two messages is in the lists below, it's deleted and the member becomes a dangling pointer.
1031  // The caller must ensure that both mNewestItem and mOldestItem are set to 0
1032  // and this is enforced in the assert below to avoid errors. This basically means
1033  // that this function should be called only when the storage model changes or
1034  // when the model is destroyed.
1035  Q_ASSERT((mOldestItem == nullptr) && (mNewestItem == nullptr));
1036 
1037  if (!mUnassignedMessageListForPass2.isEmpty()) {
1038  // We're actually in Pass1* or Pass2: everything is mUnassignedMessageListForPass2
1039  // Something may *also* be in mUnassignedMessageListForPass3 and mUnassignedMessageListForPass4
1040  // but that are duplicates for sure.
1041 
1042  // We can't just sweep the list and delete parentless items since each delete
1043  // could kill children which are somewhere AFTER in the list: accessing the children
1044  // would then lead to a SIGSEGV. We first sweep the list gathering parentless
1045  // items and *then* delete them without accessing the parented ones.
1046 
1047  QList<MessageItem *> parentless;
1048  for (const auto mi : std::as_const(mUnassignedMessageListForPass2)) {
1049  if (!mi->parent()) {
1050  parentless.append(mi);
1051  }
1052  }
1053 
1054  for (const auto mi : std::as_const(parentless)) {
1055  delete mi;
1056  }
1057 
1058  mUnassignedMessageListForPass2.clear();
1059  // Any message these list contain was also in mUnassignedMessageListForPass2
1060  mUnassignedMessageListForPass3.clear();
1061  mUnassignedMessageListForPass4.clear();
1062  return;
1063  }
1064 
1065  // mUnassignedMessageListForPass2 is empty
1066 
1067  if (!mUnassignedMessageListForPass3.isEmpty()) {
1068  // We're actually at the very end of Pass2 or inside Pass3
1069  // Pass2 pushes stuff in mUnassignedMessageListForPass3 *or* mUnassignedMessageListForPass4
1070  // Pass3 pushes stuff from mUnassignedMessageListForPass3 to mUnassignedMessageListForPass4
1071  // So if we're in Pass2 then the two lists contain distinct messages but if we're in Pass3
1072  // then the two lists may contain the same messages.
1073 
1074  if (!mUnassignedMessageListForPass4.isEmpty()) {
1075  // We're actually in Pass3: the messiest one.
1076 
1077  QSet<MessageItem *> itemsToDelete;
1078  for (const auto mi : std::as_const(mUnassignedMessageListForPass3)) {
1079  if (!mi->parent()) {
1080  itemsToDelete.insert(mi);
1081  }
1082  }
1083  for (const auto mi : std::as_const(mUnassignedMessageListForPass4)) {
1084  if (!mi->parent()) {
1085  itemsToDelete.insert(mi);
1086  }
1087  }
1088  for (const auto mi : std::as_const(itemsToDelete)) {
1089  delete mi;
1090  }
1091 
1092  mUnassignedMessageListForPass3.clear();
1093  mUnassignedMessageListForPass4.clear();
1094  return;
1095  }
1096 
1097  // mUnassignedMessageListForPass4 is empty so we must be at the end of a very special kind of Pass2
1098  // We have the same problem as in mUnassignedMessageListForPass2.
1099  QList<MessageItem *> parentless;
1100  for (const auto mi : std::as_const(mUnassignedMessageListForPass3)) {
1101  if (!mi->parent()) {
1102  parentless.append(mi);
1103  }
1104  }
1105  for (const auto mi : std::as_const(parentless)) {
1106  delete mi;
1107  }
1108 
1109  mUnassignedMessageListForPass3.clear();
1110  return;
1111  }
1112 
1113  // mUnassignedMessageListForPass3 is empty
1114  if (!mUnassignedMessageListForPass4.isEmpty()) {
1115  // we're in Pass4.. this is easy.
1116 
1117  // We have the same problem as in mUnassignedMessageListForPass2.
1118  QList<MessageItem *> parentless;
1119  for (const auto mi : std::as_const(mUnassignedMessageListForPass4)) {
1120  if (!mi->parent()) {
1121  parentless.append(mi);
1122  }
1123  }
1124  for (const auto mi : std::as_const(parentless)) {
1125  delete mi;
1126  }
1127 
1128  mUnassignedMessageListForPass4.clear();
1129  return;
1130  }
1131 }
1132 
1133 void ModelPrivate::clearThreadingCacheReferencesIdMD5ToMessageItem()
1134 {
1135  qDeleteAll(mThreadingCacheMessageReferencesIdMD5ToMessageItem);
1136  mThreadingCacheMessageReferencesIdMD5ToMessageItem.clear();
1137 }
1138 
1139 void ModelPrivate::clearThreadingCacheMessageSubjectMD5ToMessageItem()
1140 {
1141  qDeleteAll(mThreadingCacheMessageSubjectMD5ToMessageItem);
1142  mThreadingCacheMessageSubjectMD5ToMessageItem.clear();
1143 }
1144 
1145 void ModelPrivate::clearOrphanChildrenHash()
1146 {
1147  qDeleteAll(mOrphanChildrenHash);
1148  mOrphanChildrenHash.clear();
1149 }
1150 
1151 void ModelPrivate::clearJobList()
1152 {
1153  if (mViewItemJobs.isEmpty()) {
1154  return;
1155  }
1156 
1157  if (mInLengthyJobBatch) {
1158  mInLengthyJobBatch = false;
1159  }
1160 
1161  qDeleteAll(mViewItemJobs);
1162  mViewItemJobs.clear();
1163 
1164  mModelForItemFunctions = q; // make sure it's true, as there remains no job with disconnected UI
1165 }
1166 
1167 void ModelPrivate::attachGroup(GroupHeaderItem *ghi)
1168 {
1169  if (ghi->parent()) {
1170  if (((ghi)->childItemCount() > 0) // has children
1171  && (ghi)->isViewable() // is actually attached to the viewable root
1172  && mModelForItemFunctions // the UI is not disconnected
1173  && mView->isExpanded(q->index(ghi, 0)) // is actually expanded
1174  ) {
1175  saveExpandedStateOfSubtree(ghi);
1176  }
1177 
1178  // FIXME: This *WILL* break selection and current index... :/
1179 
1180  ghi->parent()->takeChildItem(mModelForItemFunctions, ghi);
1181  }
1182 
1183  ghi->setParent(mRootItem);
1184 
1185  // I'm using a macro since it does really improve readability.
1186  // I'm NOT using a helper function since gcc will refuse to inline some of
1187  // the calls because they make this function grow too much.
1188 #define INSERT_GROUP_WITH_COMPARATOR(_ItemComparator) \
1189  switch (mSortOrder->groupSortDirection()) { \
1190  case SortOrder::Ascending: \
1191  mRootItem->d_ptr->insertChildItem<_ItemComparator, true>(mModelForItemFunctions, ghi); \
1192  break; \
1193  case SortOrder::Descending: \
1194  mRootItem->d_ptr->insertChildItem<_ItemComparator, false>(mModelForItemFunctions, ghi); \
1195  break; \
1196  default: /* should never happen... */ \
1197  mRootItem->appendChildItem(mModelForItemFunctions, ghi); \
1198  break; \
1199  }
1200 
1201  switch (mSortOrder->groupSorting()) {
1203  INSERT_GROUP_WITH_COMPARATOR(ItemDateComparator)
1204  break;
1206  INSERT_GROUP_WITH_COMPARATOR(ItemMaxDateComparator)
1207  break;
1209  INSERT_GROUP_WITH_COMPARATOR(ItemSenderOrReceiverComparator)
1210  break;
1212  INSERT_GROUP_WITH_COMPARATOR(ItemSenderComparator)
1213  break;
1215  INSERT_GROUP_WITH_COMPARATOR(ItemReceiverComparator)
1216  break;
1218  mRootItem->appendChildItem(mModelForItemFunctions, ghi);
1219  break;
1220  default: // should never happen
1221  mRootItem->appendChildItem(mModelForItemFunctions, ghi);
1222  break;
1223  }
1224 
1225  if (ghi->initialExpandStatus() == Item::ExpandNeeded) { // this actually is a "non viewable expanded state"
1226  if (ghi->childItemCount() > 0) {
1227  if (mModelForItemFunctions) { // the UI is not disconnected
1228  syncExpandedStateOfSubtree(ghi);
1229  }
1230  }
1231  }
1232 
1233  // A group header is always viewable, when attached: apply the filter, if we have it.
1234  if (mFilter) {
1235  Q_ASSERT(mModelForItemFunctions); // UI must be NOT disconnected
1236  // apply the filter to subtree
1237  applyFilterToSubtree(ghi, QModelIndex());
1238  }
1239 }
1240 
1241 void ModelPrivate::saveExpandedStateOfSubtree(Item *root)
1242 {
1243  Q_ASSERT(mModelForItemFunctions); // UI must be NOT disconnected here
1244  Q_ASSERT(root);
1245 
1247 
1248  auto children = root->childItems();
1249  if (!children) {
1250  return;
1251  }
1252  for (const auto mi : std::as_const(*children)) {
1253  if (mi->childItemCount() > 0 // has children
1254  && mi->isViewable() // is actually attached to the viewable root
1255  && mView->isExpanded(q->index(mi, 0))) { // is actually expanded
1256  saveExpandedStateOfSubtree(mi);
1257  }
1258  }
1259 }
1260 
1261 void ModelPrivate::syncExpandedStateOfSubtree(Item *root)
1262 {
1263  Q_ASSERT(mModelForItemFunctions); // UI must be NOT disconnected here
1264 
1265  // WE ASSUME that:
1266  // - the item is viewable
1267  // - its initialExpandStatus() is Item::ExpandNeeded
1268  // - it has at least one children (well.. this is not a strict requirement, but it's a waste of resources to expand items that don't have children)
1269 
1270  QModelIndex idx = q->index(root, 0);
1271 
1272  // if ( !mView->isExpanded( idx ) ) // this is O(logN!) in Qt.... very ugly... but it should never happen here
1273  mView->expand(idx); // sync the real state in the view
1275 
1276  auto children = root->childItems();
1277  if (!children) {
1278  return;
1279  }
1280 
1281  for (const auto mi : std::as_const(*children)) {
1282  if (mi->initialExpandStatus() == Item::ExpandNeeded) {
1283  if (mi->childItemCount() > 0) {
1284  syncExpandedStateOfSubtree(mi);
1285  }
1286  }
1287  }
1288 }
1289 
1290 void ModelPrivate::attachMessageToGroupHeader(MessageItem *mi)
1291 {
1292  QString groupLabel;
1293  time_t date;
1294 
1295  // compute the group header label and the date
1296  switch (mAggregation->grouping()) {
1299  if (mAggregation->threadLeader() == Aggregation::MostRecentMessage) {
1300  date = mi->maxDate();
1301  } else {
1302  date = mi->date();
1303  }
1304 
1305  QDateTime dt;
1306  dt.setSecsSinceEpoch(date);
1307  QDate dDate = dt.date();
1308  int daysAgo = -1;
1309  const int daysInWeek = 7;
1310  if (dDate.isValid() && mTodayDate.isValid()) {
1311  daysAgo = dDate.daysTo(mTodayDate);
1312  }
1313 
1314  if ((daysAgo < 0) // In the future
1315  || (static_cast<uint>(date) == static_cast<uint>(-1))) { // Invalid
1316  groupLabel = mCachedUnknownLabel;
1317  } else if (daysAgo == 0) { // Today
1318  groupLabel = mCachedTodayLabel;
1319  } else if (daysAgo == 1) { // Yesterday
1320  groupLabel = mCachedYesterdayLabel;
1321  } else if (daysAgo > 1 && daysAgo < daysInWeek) { // Within last seven days
1322  auto dayName = mCachedDayNameLabel.find(dDate.dayOfWeek()); // non-const call, but non-shared container
1323  if (dayName == mCachedDayNameLabel.end()) {
1324  dayName = mCachedDayNameLabel.insert(dDate.dayOfWeek(), QLocale::system().standaloneDayName(dDate.dayOfWeek()));
1325  }
1326  groupLabel = *dayName;
1327  } else if (mAggregation->grouping() == Aggregation::GroupByDate) { // GroupByDate seven days or more ago
1328  groupLabel = QLocale::system().toString(dDate, QLocale::ShortFormat);
1329  } else if (dDate.month() == mTodayDate.month() // GroupByDateRange within this month
1330  && dDate.year() == mTodayDate.year()) {
1331  int startOfWeekDaysAgo = (daysInWeek + mTodayDate.dayOfWeek() - QLocale().firstDayOfWeek()) % daysInWeek;
1332  int weeksAgo = ((daysAgo - startOfWeekDaysAgo) / daysInWeek) + 1;
1333  switch (weeksAgo) {
1334  case 0: // This week
1335  groupLabel = QLocale::system().standaloneDayName(dDate.dayOfWeek());
1336  break;
1337  case 1: // 1 week ago
1338  groupLabel = mCachedLastWeekLabel;
1339  break;
1340  case 2:
1341  groupLabel = mCachedTwoWeeksAgoLabel;
1342  break;
1343  case 3:
1344  groupLabel = mCachedThreeWeeksAgoLabel;
1345  break;
1346  case 4:
1347  groupLabel = mCachedFourWeeksAgoLabel;
1348  break;
1349  case 5:
1350  groupLabel = mCachedFiveWeeksAgoLabel;
1351  break;
1352  default: // should never happen
1353  groupLabel = mCachedUnknownLabel;
1354  }
1355  } else if (dDate.year() == mTodayDate.year()) { // GroupByDateRange within this year
1356  auto monthName = mCachedMonthNameLabel.find(dDate.month()); // non-const call, but non-shared container
1357  if (monthName == mCachedMonthNameLabel.end()) {
1358  monthName = mCachedMonthNameLabel.insert(dDate.month(), QLocale::system().standaloneMonthName(dDate.month()));
1359  }
1360  groupLabel = *monthName;
1361  } else { // GroupByDateRange in previous years
1362  auto monthName = mCachedMonthNameLabel.find(dDate.month()); // non-const call, but non-shared container
1363  if (monthName == mCachedMonthNameLabel.end()) {
1364  monthName = mCachedMonthNameLabel.insert(dDate.month(), QLocale::system().standaloneMonthName(dDate.month()));
1365  }
1366  groupLabel = i18nc("Message Aggregation Group Header: Month name and Year number",
1367  "%1 %2",
1368  *monthName,
1369  QLocale::system().toString(dDate, QLatin1String("yyyy")));
1370  }
1371  break;
1372  }
1373 
1375  date = mi->date();
1376  groupLabel = mi->displaySenderOrReceiver();
1377  break;
1378 
1380  date = mi->date();
1381  groupLabel = mi->displaySender();
1382  break;
1383 
1385  date = mi->date();
1386  groupLabel = mi->displayReceiver();
1387  break;
1388 
1390  // append directly to root
1391  attachMessageToParent(mRootItem, mi);
1392  return;
1393 
1394  default:
1395  // should never happen
1396  attachMessageToParent(mRootItem, mi);
1397  return;
1398  }
1399 
1400  GroupHeaderItem *ghi;
1401 
1402  ghi = mGroupHeaderItemHash.value(groupLabel, nullptr);
1403  if (!ghi) {
1404  // not found
1405 
1406  ghi = new GroupHeaderItem(groupLabel);
1407  ghi->initialSetup(date, mi->size(), mi->sender(), mi->receiver(), mi->useReceiver());
1408 
1409  switch (mAggregation->groupExpandPolicy()) {
1411  // nothing to do
1412  break;
1414  // expand always
1415  ghi->setInitialExpandStatus(Item::ExpandNeeded);
1416  break;
1418  // expand only if "close" to today
1419  if (mViewItemJobStepStartTime > ghi->date()) {
1420  if ((mViewItemJobStepStartTime - ghi->date()) < (3600 * 72)) {
1421  ghi->setInitialExpandStatus(Item::ExpandNeeded);
1422  }
1423  } else {
1424  if ((ghi->date() - mViewItemJobStepStartTime) < (3600 * 72)) {
1425  ghi->setInitialExpandStatus(Item::ExpandNeeded);
1426  }
1427  }
1428  break;
1429  default:
1430  // b0rken
1431  break;
1432  }
1433 
1434  attachMessageToParent(ghi, mi);
1435 
1436  attachGroup(ghi); // this will expand the group if required
1437 
1438  mGroupHeaderItemHash.insert(groupLabel, ghi);
1439  } else {
1440  // the group was already there (certainly viewable)
1441 
1442  // This function may be also called to re-group a message.
1443  // That is, to eventually find a new group for a message that has changed
1444  // its properties (but was already attached to a group).
1445  // So it may happen that we find out that in fact re-grouping wasn't really
1446  // needed because the message is already in the correct group.
1447  if (mi->parent() == ghi) {
1448  return; // nothing to be done
1449  }
1450 
1451  attachMessageToParent(ghi, mi);
1452  }
1453 
1454  // Remember this message as a thread leader
1455  mThreadingCache.updateParent(mi, nullptr);
1456 }
1457 
1458 MessageItem *ModelPrivate::findMessageParent(MessageItem *mi)
1459 {
1460  Q_ASSERT(mAggregation->threading() != Aggregation::NoThreading); // caller must take care of this
1461 
1462  // This function attempts to find a thread parent for the item "mi"
1463  // which actually may already have a children subtree.
1464 
1465  // Forged or plain broken message trees are dangerous here.
1466  // For example, a message tree with circular references like
1467  //
1468  // Message mi, Id=1, In-Reply-To=2
1469  // Message childOfMi, Id=2, In-Reply-To=1
1470  //
1471  // is perfectly possible and will cause us to find childOfMi
1472  // as parent of mi. This will then create a loop in the message tree
1473  // (which will then no longer be a tree in fact) and cause us to freeze
1474  // once we attempt to climb the parents. We need to take care of that.
1475 
1476  bool bMessageWasThreadable = false;
1477  MessageItem *pParent;
1478 
1479  // First of all try to find a "perfect parent", that is the message for that
1480  // we have the ID in the "In-Reply-To" field. This is actually done by using
1481  // MD5 caches of the message ids because of speed. Collisions are very unlikely.
1482 
1483  QByteArray md5 = mi->inReplyToIdMD5();
1484  if (!md5.isEmpty()) {
1485  // have an In-Reply-To field MD5
1486  pParent = mThreadingCacheMessageIdMD5ToMessageItem.value(md5, nullptr);
1487  if (pParent) {
1488  // Take care of circular references
1489  if ((mi == pParent) // self referencing message
1490  || ((mi->childItemCount() > 0) // mi already has children, this is fast to determine
1491  && pParent->hasAncestor(mi) // pParent is in the mi's children tree
1492  )) {
1493  qCWarning(MESSAGELIST_LOG) << "Circular In-Reply-To reference loop detected in the message tree";
1494  mi->setThreadingStatus(MessageItem::NonThreadable);
1495  return nullptr; // broken message: throw it away
1496  }
1497  mi->setThreadingStatus(MessageItem::PerfectParentFound);
1498  return pParent; // got a perfect parent for this message
1499  }
1500 
1501  // got no perfect parent
1502  bMessageWasThreadable = true; // but the message was threadable
1503  }
1504 
1505  if (mAggregation->threading() == Aggregation::PerfectOnly) {
1506  mi->setThreadingStatus(bMessageWasThreadable ? MessageItem::ParentMissing : MessageItem::NonThreadable);
1507  return nullptr; // we're doing only perfect parent matches
1508  }
1509 
1510  // Try to use the "References" field. In fact we have the MD5 of the
1511  // (n-1)th entry in References.
1512  //
1513  // Original rationale from KMHeaders:
1514  //
1515  // If we don't have a replyToId, or if we have one and the
1516  // corresponding message is not in this folder, as happens
1517  // if you keep your outgoing messages in an OUTBOX, for
1518  // example, try the list of references, because the second
1519  // to last will likely be in this folder. replyToAuxIdMD5
1520  // contains the second to last one.
1521 
1522  md5 = mi->referencesIdMD5();
1523  if (!md5.isEmpty()) {
1524  pParent = mThreadingCacheMessageIdMD5ToMessageItem.value(md5, nullptr);
1525  if (pParent) {
1526  // Take care of circular references
1527  if ((mi == pParent) // self referencing message
1528  || ((mi->childItemCount() > 0) // mi already has children, this is fast to determine
1529  && pParent->hasAncestor(mi) // pParent is in the mi's children tree
1530  )) {
1531  qCWarning(MESSAGELIST_LOG) << "Circular reference loop detected in the message tree";
1532  mi->setThreadingStatus(MessageItem::NonThreadable);
1533  return nullptr; // broken message: throw it away
1534  }
1535  mi->setThreadingStatus(MessageItem::ImperfectParentFound);
1536  return pParent; // got an imperfect parent for this message
1537  }
1538 
1539  auto messagesWithTheSameReferences = mThreadingCacheMessageReferencesIdMD5ToMessageItem.value(md5, nullptr);
1540  if (messagesWithTheSameReferences) {
1541  Q_ASSERT(!messagesWithTheSameReferences->isEmpty());
1542 
1543  pParent = messagesWithTheSameReferences->first();
1544  if (mi != pParent && (mi->childItemCount() == 0 || !pParent->hasAncestor(mi))) {
1545  mi->setThreadingStatus(MessageItem::ImperfectParentFound);
1546  return pParent;
1547  }
1548  }
1549 
1550  // got no imperfect parent
1551  bMessageWasThreadable = true; // but the message was threadable
1552  }
1553 
1554  if (mAggregation->threading() == Aggregation::PerfectAndReferences) {
1555  mi->setThreadingStatus(bMessageWasThreadable ? MessageItem::ParentMissing : MessageItem::NonThreadable);
1556  return nullptr; // we're doing only perfect parent matches
1557  }
1558 
1559  Q_ASSERT(mAggregation->threading() == Aggregation::PerfectReferencesAndSubject);
1560 
1561  // We are supposed to do subject based threading but we can't do it now.
1562  // This is because the subject based threading *may* be wrong and waste
1563  // time by creating circular references (that we'd need to detect and fix).
1564  // We first try the perfect and references based threading on all the messages
1565  // and then run subject based threading only on the remaining ones.
1566 
1567  mi->setThreadingStatus((bMessageWasThreadable || mi->subjectIsPrefixed()) ? MessageItem::ParentMissing : MessageItem::NonThreadable);
1568  return nullptr;
1569 }
1570 
1571 // Subject threading cache stuff
1572 
1573 #if 0
1574 // Debug helpers
1575 void dump_iterator_and_list(QList< MessageItem * >::Iterator &iter, QList< MessageItem * > *list)
1576 {
1577  qCDebug(MESSAGELIST_LOG) << "Threading cache part dump";
1578  if (iter == list->end()) {
1579  qCDebug(MESSAGELIST_LOG) << "Iterator pointing to end of the list";
1580  } else {
1581  qCDebug(MESSAGELIST_LOG) << "Iterator pointing to " << *iter << " subject [" << (*iter)->subject() << "] date [" << (*iter)->date() << "]";
1582  }
1583 
1584  for (QList< MessageItem * >::Iterator it = list->begin(); it != list->end(); ++it) {
1585  qCDebug(MESSAGELIST_LOG) << "List element " << *it << " subject [" << (*it)->subject() << "] date [" << (*it)->date() << "]";
1586  }
1587 
1588  qCDebug(MESSAGELIST_LOG) << "End of threading cache part dump";
1589 }
1590 
1591 void dump_list(QList< MessageItem * > *list)
1592 {
1593  qCDebug(MESSAGELIST_LOG) << "Threading cache part dump";
1594 
1595  for (QList< MessageItem * >::Iterator it = list->begin(); it != list->end(); ++it) {
1596  qCDebug(MESSAGELIST_LOG) << "List element " << *it << " subject [" << (*it)->subject() << "] date [" << (*it)->date() << "]";
1597  }
1598 
1599  qCDebug(MESSAGELIST_LOG) << "End of threading cache part dump";
1600 }
1601 
1602 #endif // debug helpers
1603 
1604 // a helper class used in a qLowerBound() call below
1605 class MessageLessThanByDate
1606 {
1607 public:
1608  inline bool operator()(const MessageItem *mi1, const MessageItem *mi2) const
1609  {
1610  if (mi1->date() < mi2->date()) { // likely
1611  return true;
1612  }
1613  if (mi1->date() > mi2->date()) { // likely
1614  return false;
1615  }
1616  // dates are equal, compare by pointer
1617  return mi1 < mi2;
1618  }
1619 };
1620 
1621 void ModelPrivate::addMessageToReferencesBasedThreadingCache(MessageItem *mi)
1622 {
1623  // Messages in this cache are sorted by date, and if dates are equal then they are sorted by pointer value.
1624  // Sorting by date is used to optimize the parent lookup in guessMessageParent() below.
1625 
1626  // WARNING: If the message date changes for some reason (like in the "update" step)
1627  // then the cache may become unsorted. For this reason the message about to
1628  // be changed must be first removed from the cache and then reinserted.
1629 
1630  auto messagesWithTheSameReference = mThreadingCacheMessageReferencesIdMD5ToMessageItem.value(mi->referencesIdMD5(), nullptr);
1631 
1632  if (!messagesWithTheSameReference) {
1633  messagesWithTheSameReference = new QList<MessageItem *>();
1634  mThreadingCacheMessageReferencesIdMD5ToMessageItem.insert(mi->referencesIdMD5(), messagesWithTheSameReference);
1635  messagesWithTheSameReference->append(mi);
1636  return;
1637  }
1638 
1639  // Found: assert that we have no duplicates in the cache.
1640  Q_ASSERT(!messagesWithTheSameReference->contains(mi));
1641 
1642  // Ordered insert: first by date then by pointer value.
1643  auto it = std::lower_bound(messagesWithTheSameReference->begin(), messagesWithTheSameReference->end(), mi, MessageLessThanByDate());
1644  messagesWithTheSameReference->insert(it, mi);
1645 }
1646 
1647 void ModelPrivate::removeMessageFromReferencesBasedThreadingCache(MessageItem *mi)
1648 {
1649  // We assume that the caller knows what he is doing and the message is actually in the cache.
1650  // If the message isn't in the cache then we should not be called at all.
1651 
1652  auto messagesWithTheSameReference = mThreadingCacheMessageReferencesIdMD5ToMessageItem.value(mi->referencesIdMD5(), nullptr);
1653 
1654  // We assume that the message is there so the list must be non null.
1655  Q_ASSERT(messagesWithTheSameReference);
1656 
1657  // The cache *MUST* be ordered first by date then by pointer value
1658  auto it = std::lower_bound(messagesWithTheSameReference->begin(), messagesWithTheSameReference->end(), mi, MessageLessThanByDate());
1659 
1660  // The binary based search must have found a message
1661  Q_ASSERT(it != messagesWithTheSameReference->end());
1662 
1663  // and it must have found exactly the message requested
1664  Q_ASSERT(*it == mi);
1665 
1666  // Kill it
1667  messagesWithTheSameReference->erase(it);
1668 
1669  // And kill the list if it was the last one
1670  if (messagesWithTheSameReference->isEmpty()) {
1671  mThreadingCacheMessageReferencesIdMD5ToMessageItem.remove(mi->referencesIdMD5());
1672  delete messagesWithTheSameReference;
1673  }
1674 }
1675 
1676 void ModelPrivate::addMessageToSubjectBasedThreadingCache(MessageItem *mi)
1677 {
1678  // Messages in this cache are sorted by date, and if dates are equal then they are sorted by pointer value.
1679  // Sorting by date is used to optimize the parent lookup in guessMessageParent() below.
1680 
1681  // WARNING: If the message date changes for some reason (like in the "update" step)
1682  // then the cache may become unsorted. For this reason the message about to
1683  // be changed must be first removed from the cache and then reinserted.
1684 
1685  // Lookup the list of messages with the same stripped subject
1686  auto messagesWithTheSameStrippedSubject = mThreadingCacheMessageSubjectMD5ToMessageItem.value(mi->strippedSubjectMD5(), nullptr);
1687 
1688  if (!messagesWithTheSameStrippedSubject) {
1689  // Not there yet: create it and append.
1690  messagesWithTheSameStrippedSubject = new QList<MessageItem *>();
1691  mThreadingCacheMessageSubjectMD5ToMessageItem.insert(mi->strippedSubjectMD5(), messagesWithTheSameStrippedSubject);
1692  messagesWithTheSameStrippedSubject->append(mi);
1693  return;
1694  }
1695 
1696  // Found: assert that we have no duplicates in the cache.
1697  Q_ASSERT(!messagesWithTheSameStrippedSubject->contains(mi));
1698 
1699  // Ordered insert: first by date then by pointer value.
1700  auto it = std::lower_bound(messagesWithTheSameStrippedSubject->begin(), messagesWithTheSameStrippedSubject->end(), mi, MessageLessThanByDate());
1701  messagesWithTheSameStrippedSubject->insert(it, mi);
1702 }
1703 
1704 void ModelPrivate::removeMessageFromSubjectBasedThreadingCache(MessageItem *mi)
1705 {
1706  // We assume that the caller knows what he is doing and the message is actually in the cache.
1707  // If the message isn't in the cache then we should not be called at all.
1708  //
1709  // The game is called "performance"
1710 
1711  // Grab the list of all the messages with the same stripped subject (all potential parents)
1712  auto messagesWithTheSameStrippedSubject = mThreadingCacheMessageSubjectMD5ToMessageItem.value(mi->strippedSubjectMD5(), nullptr);
1713 
1714  // We assume that the message is there so the list must be non null.
1715  Q_ASSERT(messagesWithTheSameStrippedSubject);
1716 
1717  // The cache *MUST* be ordered first by date then by pointer value
1718  auto it = std::lower_bound(messagesWithTheSameStrippedSubject->begin(), messagesWithTheSameStrippedSubject->end(), mi, MessageLessThanByDate());
1719 
1720  // The binary based search must have found a message
1721  Q_ASSERT(it != messagesWithTheSameStrippedSubject->end());
1722 
1723  // and it must have found exactly the message requested
1724  Q_ASSERT(*it == mi);
1725 
1726  // Kill it
1727  messagesWithTheSameStrippedSubject->erase(it);
1728 
1729  // And kill the list if it was the last one
1730  if (messagesWithTheSameStrippedSubject->isEmpty()) {
1731  mThreadingCacheMessageSubjectMD5ToMessageItem.remove(mi->strippedSubjectMD5());
1732  delete messagesWithTheSameStrippedSubject;
1733  }
1734 }
1735 
1736 MessageItem *ModelPrivate::guessMessageParent(MessageItem *mi)
1737 {
1738  // This function implements subject based threading
1739  // It attempts to guess a thread parent for the item "mi"
1740  // which actually may already have a children subtree.
1741 
1742  // We have all the problems of findMessageParent() plus the fact that
1743  // we're actually guessing (and often we may be *wrong*).
1744 
1745  Q_ASSERT(mAggregation->threading() == Aggregation::PerfectReferencesAndSubject); // caller must take care of this
1746  Q_ASSERT(mi->subjectIsPrefixed()); // caller must take care of this
1747  Q_ASSERT(mi->threadingStatus() == MessageItem::ParentMissing);
1748 
1749  // Do subject based threading
1750  const QByteArray md5 = mi->strippedSubjectMD5();
1751  if (!md5.isEmpty()) {
1752  auto messagesWithTheSameStrippedSubject = mThreadingCacheMessageSubjectMD5ToMessageItem.value(md5, nullptr);
1753 
1754  if (messagesWithTheSameStrippedSubject) {
1755  Q_ASSERT(!messagesWithTheSameStrippedSubject->isEmpty());
1756 
1757  // Need to find the message with the maximum date lower than the one of this message
1758 
1759  auto maxTime = (time_t)0;
1760  MessageItem *pParent = nullptr;
1761 
1762  // Here'we re really guessing so circular references are possible
1763  // even on perfectly valid trees. This is why we don't consider it
1764  // an error but just continue searching.
1765 
1766  // FIXME: This might be speed up with an initial binary search (?)
1767  // ANSWER: No. We can't rely on date order (as it can be updated on the fly...)
1768  for (const auto it : std::as_const(*messagesWithTheSameStrippedSubject)) {
1769  int delta = mi->date() - it->date();
1770 
1771  // We don't take into account messages with a delta smaller than 120.
1772  // Assuming that our date() values are correct (that is, they take into
1773  // account timezones etc..) then one usually needs more than 120 seconds
1774  // to answer to a message. Better safe than sorry.
1775 
1776  // This check also includes negative deltas so messages later than mi aren't considered
1777 
1778  if (delta < 120) {
1779  break; // The list is ordered by date (ascending) so we can stop searching here
1780  }
1781 
1782  // About the "magic" 3628899 value here comes a Till's comment from the original KMHeaders:
1783  //
1784  // "Parents more than six weeks older than the message are not accepted. The reasoning being
1785  // that if a new message with the same subject turns up after such a long time, the chances
1786  // that it is still part of the same thread are slim. The value of six weeks is chosen as a
1787  // result of a poll conducted on kde-devel, so it's probably bogus. :)"
1788 
1789  if (delta < 3628899) {
1790  // Compute the closest.
1791  if ((maxTime < it->date())) {
1792  // This algorithm *can* be (and often is) wrong.
1793  // Take care of circular threading which is really possible at this level.
1794  // If mi contains "it" inside its children subtree then we have
1795  // found such a circular threading problem.
1796 
1797  // Note that here we can't have it == mi because of the delta >= 120 check above.
1798 
1799  if ((mi->childItemCount() == 0) || !it->hasAncestor(mi)) {
1800  maxTime = it->date();
1801  pParent = it;
1802  }
1803  }
1804  }
1805  }
1806 
1807  if (pParent) {
1808  mi->setThreadingStatus(MessageItem::ImperfectParentFound);
1809  return pParent; // got an imperfect parent for this message
1810  }
1811  }
1812  }
1813 
1814  return nullptr;
1815 }
1816 
1817 //
1818 // A little template helper, hopefully inlineable.
1819 //
1820 // Return true if the specified message item is in the wrong position
1821 // inside the specified parent and needs re-sorting. Return false otherwise.
1822 // Both parent and messageItem must not be null.
1823 //
1824 // Checking if a message needs re-sorting instead of just re-sorting it
1825 // is very useful since re-sorting is an expensive operation.
1826 //
1827 template<class ItemComparator>
1828 static bool messageItemNeedsReSorting(SortOrder::SortDirection messageSortDirection, ItemPrivate *parent, MessageItem *messageItem)
1829 {
1830  if ((messageSortDirection == SortOrder::Ascending) || (parent->mType == Item::Message)) {
1831  return parent->childItemNeedsReSorting<ItemComparator, true>(messageItem);
1832  }
1833  return parent->childItemNeedsReSorting<ItemComparator, false>(messageItem);
1834 }
1835 
1836 bool ModelPrivate::handleItemPropertyChanges(int propertyChangeMask, Item *parent, Item *item)
1837 {
1838  // The facts:
1839  //
1840  // - If dates changed:
1841  // - If we're sorting messages by min/max date then at each level the messages might need resorting.
1842  // - If the thread leader is the most recent message of a thread then the uppermost
1843  // message of the thread might need re-grouping.
1844  // - If the groups are sorted by min/max date then the group might need re-sorting too.
1845  //
1846  // This function explicitly doesn't re-apply the filter when ActionItemStatus changes.
1847  // This is because filters must be re-applied due to a broader range of status variations:
1848  // this is done in viewItemJobStepInternalForJobPass1Update() instead (which is the only
1849  // place in that ActionItemStatus may be set).
1850 
1851  if (parent->type() == Item::InvisibleRoot) {
1852  // item is either a message or a group attached to the root.
1853  // It might need resorting.
1854  if (item->type() == Item::GroupHeader) {
1855  // item is a group header attached to the root.
1856  if ((
1857  // max date changed
1858  (propertyChangeMask & MaxDateChanged) && // groups sorted by max date
1859  (mSortOrder->groupSorting() == SortOrder::SortGroupsByDateTimeOfMostRecent))
1860  || (
1861  // date changed
1862  (propertyChangeMask & DateChanged) && // groups sorted by date
1863  (mSortOrder->groupSorting() == SortOrder::SortGroupsByDateTime))) {
1864  // This group might need re-sorting.
1865 
1866  // Groups are large container of messages so it's likely that
1867  // another message inserted will cause this group to be marked again.
1868  // So we wait until the end to do the grand final re-sorting: it will be done in Pass4.
1869  mGroupHeadersThatNeedUpdate.insert(static_cast<GroupHeaderItem *>(item), static_cast<GroupHeaderItem *>(item));
1870  }
1871  } else {
1872  // item is a message. It might need re-sorting.
1873 
1874  // Since sorting is an expensive operation, we first check if it's *really* needed.
1875  // Re-sorting will actually not change min/max dates at all and
1876  // will not climb up the parent's ancestor tree.
1877 
1878  switch (mSortOrder->messageSorting()) {
1880  if (propertyChangeMask & DateChanged) { // date changed
1881  if (messageItemNeedsReSorting<ItemDateComparator>(mSortOrder->messageSortDirection(), parent->d_ptr, static_cast<MessageItem *>(item))) {
1882  attachMessageToParent(parent, static_cast<MessageItem *>(item));
1883  }
1884  } // else date changed, but it doesn't match sorting order: no need to re-sort
1885  break;
1887  if (propertyChangeMask & MaxDateChanged) { // max date changed
1888  if (messageItemNeedsReSorting<ItemMaxDateComparator>(mSortOrder->messageSortDirection(), parent->d_ptr, static_cast<MessageItem *>(item))) {
1889  attachMessageToParent(parent, static_cast<MessageItem *>(item));
1890  }
1891  } // else max date changed, but it doesn't match sorting order: no need to re-sort
1892  break;
1894  if (propertyChangeMask & ActionItemStatusChanged) { // todo status changed
1895  if (messageItemNeedsReSorting<ItemActionItemStatusComparator>(mSortOrder->messageSortDirection(),
1896  parent->d_ptr,
1897  static_cast<MessageItem *>(item))) {
1898  attachMessageToParent(parent, static_cast<MessageItem *>(item));
1899  }
1900  } // else to do status changed, but it doesn't match sorting order: no need to re-sort
1901  break;
1903  if (propertyChangeMask & UnreadStatusChanged) { // new / unread status changed
1904  if (messageItemNeedsReSorting<ItemUnreadStatusComparator>(mSortOrder->messageSortDirection(),
1905  parent->d_ptr,
1906  static_cast<MessageItem *>(item))) {
1907  attachMessageToParent(parent, static_cast<MessageItem *>(item));
1908  }
1909  } // else new/unread status changed, but it doesn't match sorting order: no need to re-sort
1910  break;
1911  case SortOrder::SortMessagesByImportantStatus:
1912  if (propertyChangeMask & ImportantStatusChanged) { // important status changed
1913  if (messageItemNeedsReSorting<ItemImportantStatusComparator>(mSortOrder->messageSortDirection(),
1914  parent->d_ptr,
1915  static_cast<MessageItem *>(item))) {
1916  attachMessageToParent(parent, static_cast<MessageItem *>(item));
1917  }
1918  } // else new/unread status changed, but it doesn't match sorting order: no need to re-sort
1919  break;
1921  if (propertyChangeMask & AttachmentStatusChanged) { // attachment status changed
1922  if (messageItemNeedsReSorting<ItemAttachmentStatusComparator>(mSortOrder->messageSortDirection(),
1923  parent->d_ptr,
1924  static_cast<MessageItem *>(item))) {
1925  attachMessageToParent(parent, static_cast<MessageItem *>(item));
1926  }
1927  } // else new/unread status changed, but it doesn't match sorting order: no need to re-sort
1928  break;
1929  default:
1930  // this kind of message sorting isn't affected by the property changes: nothing to do.
1931  break;
1932  }
1933  }
1934 
1935  return false; // the invisible root isn't affected by any change.
1936  }
1937 
1938  if (parent->type() == Item::GroupHeader) {
1939  // item is a message attached to a GroupHeader.
1940  // It might need re-grouping or re-sorting (within the same group)
1941 
1942  // Check re-grouping here.
1943  if ((
1944  // max date changed
1945  (propertyChangeMask & MaxDateChanged) && // thread leader is most recent message
1946  (mAggregation->threadLeader() == Aggregation::MostRecentMessage))
1947  || (
1948  // date changed
1949  (propertyChangeMask & DateChanged) && // thread leader the topmost message
1950  (mAggregation->threadLeader() == Aggregation::TopmostMessage))) {
1951  // Might really need re-grouping.
1952  // attachMessageToGroupHeader() will find the right group for this message
1953  // and if it's different than the current it will move it.
1954  attachMessageToGroupHeader(static_cast<MessageItem *>(item));
1955  // Re-grouping fixes the properties of the involved group headers
1956  // so at exit of attachMessageToGroupHeader() the parent can't be affected
1957  // by the change anymore.
1958  return false;
1959  }
1960 
1961  // Re-grouping wasn't needed. Re-sorting might be.
1962  } // else item is a message attached to another message and might need re-sorting only.
1963 
1964  // Check if message needs re-sorting.
1965 
1966  switch (mSortOrder->messageSorting()) {
1968  if (propertyChangeMask & DateChanged) { // date changed
1969  if (messageItemNeedsReSorting<ItemDateComparator>(mSortOrder->messageSortDirection(), parent->d_ptr, static_cast<MessageItem *>(item))) {
1970  attachMessageToParent(parent, static_cast<MessageItem *>(item));
1971  }
1972  } // else date changed, but it doesn't match sorting order: no need to re-sort
1973  break;
1975  if (propertyChangeMask & MaxDateChanged) { // max date changed
1976  if (messageItemNeedsReSorting<ItemMaxDateComparator>(mSortOrder->messageSortDirection(), parent->d_ptr, static_cast<MessageItem *>(item))) {
1977  attachMessageToParent(parent, static_cast<MessageItem *>(item));
1978  }
1979  } // else max date changed, but it doesn't match sorting order: no need to re-sort
1980  break;
1982  if (propertyChangeMask & ActionItemStatusChanged) { // todo status changed
1983  if (messageItemNeedsReSorting<ItemActionItemStatusComparator>(mSortOrder->messageSortDirection(),
1984  parent->d_ptr,
1985  static_cast<MessageItem *>(item))) {
1986  attachMessageToParent(parent, static_cast<MessageItem *>(item));
1987  }
1988  } // else to do status changed, but it doesn't match sorting order: no need to re-sort
1989  break;
1991  if (propertyChangeMask & UnreadStatusChanged) { // new / unread status changed
1992  if (messageItemNeedsReSorting<ItemUnreadStatusComparator>(mSortOrder->messageSortDirection(), parent->d_ptr, static_cast<MessageItem *>(item))) {
1993  attachMessageToParent(parent, static_cast<MessageItem *>(item));
1994  }
1995  } // else new/unread status changed, but it doesn't match sorting order: no need to re-sort
1996  break;
1997  case SortOrder::SortMessagesByImportantStatus:
1998  if (propertyChangeMask & ImportantStatusChanged) { // important status changed
1999  if (messageItemNeedsReSorting<ItemImportantStatusComparator>(mSortOrder->messageSortDirection(), parent->d_ptr, static_cast<MessageItem *>(item))) {
2000  attachMessageToParent(parent, static_cast<MessageItem *>(item));
2001  }
2002  } // else important status changed, but it doesn't match sorting order: no need to re-sort
2003  break;
2005  if (propertyChangeMask & AttachmentStatusChanged) { // attachment status changed
2006  if (messageItemNeedsReSorting<ItemAttachmentStatusComparator>(mSortOrder->messageSortDirection(),
2007  parent->d_ptr,
2008  static_cast<MessageItem *>(item))) {
2009  attachMessageToParent(parent, static_cast<MessageItem *>(item));
2010  }
2011  } // else important status changed, but it doesn't match sorting order: no need to re-sort
2012  break;
2013  default:
2014  // this kind of message sorting isn't affected by property changes: nothing to do.
2015  break;
2016  }
2017 
2018  return true; // parent might be affected too.
2019 }
2020 
2021 void ModelPrivate::messageDetachedUpdateParentProperties(Item *oldParent, MessageItem *mi)
2022 {
2023  Q_ASSERT(oldParent);
2024  Q_ASSERT(mi);
2025  Q_ASSERT(oldParent != mRootItem);
2026 
2027  // oldParent might have its properties changed because of the child removal.
2028  // propagate the changes up.
2029  for (;;) {
2030  // pParent is not the root item now. This is assured by how we enter this loop
2031  // and by the fact that handleItemPropertyChanges returns false when grandParent
2032  // is Item::InvisibleRoot. We could actually assert it here...
2033 
2034  // Check if its dates need an update.
2035  int propertyChangeMask;
2036 
2037  if ((mi->maxDate() == oldParent->maxDate()) && oldParent->recomputeMaxDate()) {
2038  propertyChangeMask = MaxDateChanged;
2039  } else {
2040  break; // from the for(;;) loop
2041  }
2042 
2043  // One of the oldParent properties has changed for sure
2044 
2045  Item *grandParent = oldParent->parent();
2046 
2047  // If there is no grandParent then oldParent isn't attached to the view.
2048  // Re-sorting / re-grouping isn't needed for sure.
2049  if (!grandParent) {
2050  break; // from the for(;;) loop
2051  }
2052 
2053  // The following function will return true if grandParent may be affected by the change.
2054  // If the grandParent isn't affected, we stop climbing.
2055  if (!handleItemPropertyChanges(propertyChangeMask, grandParent, oldParent)) {
2056  break; // from the for(;;) loop
2057  }
2058 
2059  // Now we need to climb up one level and check again.
2060  oldParent = grandParent;
2061  } // for(;;) loop
2062 
2063  // If the last message was removed from a group header then this group will need an update
2064  // for sure. We will need to remove it (unless a message is attached back to it)
2065  if (oldParent->type() == Item::GroupHeader) {
2066  if (oldParent->childItemCount() == 0) {
2067  mGroupHeadersThatNeedUpdate.insert(static_cast<GroupHeaderItem *>(oldParent), static_cast<GroupHeaderItem *>(oldParent));
2068  }
2069  }
2070 }
2071 
2072 void ModelPrivate::propagateItemPropertiesToParent(Item *item)
2073 {
2074  Item *pParent = item->parent();
2075  Q_ASSERT(pParent);
2076  Q_ASSERT(pParent != mRootItem);
2077 
2078  for (;;) {
2079  // pParent is not the root item now. This is assured by how we enter this loop
2080  // and by the fact that handleItemPropertyChanges returns false when grandParent
2081  // is Item::InvisibleRoot. We could actually assert it here...
2082 
2083  // Check if its dates need an update.
2084  int propertyChangeMask;
2085 
2086  if (item->maxDate() > pParent->maxDate()) {
2087  pParent->setMaxDate(item->maxDate());
2088  propertyChangeMask = MaxDateChanged;
2089  } else {
2090  // No parent dates have changed: no further work is needed. Stop climbing here.
2091  break; // from the for(;;) loop
2092  }
2093 
2094  // One of the pParent properties has changed.
2095 
2096  Item *grandParent = pParent->parent();
2097 
2098  // If there is no grandParent then pParent isn't attached to the view.
2099  // Re-sorting / re-grouping isn't needed for sure.
2100  if (!grandParent) {
2101  break; // from the for(;;) loop
2102  }
2103 
2104  // The following function will return true if grandParent may be affected by the change.
2105  // If the grandParent isn't affected, we stop climbing.
2106  if (!handleItemPropertyChanges(propertyChangeMask, grandParent, pParent)) {
2107  break; // from the for(;;) loop
2108  }
2109 
2110  // Now we need to climb up one level and check again.
2111  pParent = grandParent;
2112  } // for(;;)
2113 }
2114 
2115 void ModelPrivate::attachMessageToParent(Item *pParent, MessageItem *mi, AttachOptions attachOptions)
2116 {
2117  Q_ASSERT(pParent);
2118  Q_ASSERT(mi);
2119 
2120  // This function may be called to do a simple "re-sort" of the item inside the parent.
2121  // In that case mi->parent() is equal to pParent.
2122  bool oldParentWasTheSame;
2123 
2124  if (mi->parent()) {
2125  Item *oldParent = mi->parent();
2126 
2127  // The item already had a parent and this means that we're moving it.
2128  oldParentWasTheSame = oldParent == pParent; // just re-sorting ?
2129 
2130  if (mi->isViewable()) { // is actually
2131  // The message is actually attached to the viewable root
2132 
2133  // Unfortunately we need to hack the model/view architecture
2134  // since it's somewhat flawed in this. At the moment of writing
2135  // there is simply no way to atomically move a subtree.
2136  // We must detach, call beginRemoveRows()/endRemoveRows(),
2137  // save the expanded state, save the selection, save the current item,
2138  // save the view position (YES! As we are removing items the view
2139  // will hopelessly jump around so we're just FORCED to break
2140  // the isolation from the view)...
2141  // ...*then* reattach, restore the expanded state, restore the selection,
2142  // restore the current item, restore the view position and pray
2143  // that nothing will fail in the (rather complicated) process....
2144 
2145  // Yet more unfortunately, while saving the expanded state might stop
2146  // at a certain (unexpanded) point in the tree, saving the selection
2147  // is hopelessly recursive down to the bare leafs.
2148 
2149  // Furthermore the expansion of items is a common case while selection
2150  // in the subtree is rare, so saving it would be a huge cost with
2151  // a low revenue.
2152 
2153  // This is why we just let the selection screw up. I hereby refuse to call
2154  // yet another expensive recursive function here :D
2155 
2156  // The current item saving can be somewhat optimized doing it once for
2157  // a single job step...
2158 
2159  if (((mi)->childItemCount() > 0) // has children
2160  && mModelForItemFunctions // the UI is not actually disconnected
2161  && mView->isExpanded(q->index(mi, 0)) // is actually expanded
2162  ) {
2163  saveExpandedStateOfSubtree(mi);
2164  }
2165  }
2166 
2167  // If the parent is viewable (so mi was viewable too) then the beginRemoveRows()
2168  // and endRemoveRows() functions of this model will be called too.
2169  oldParent->takeChildItem(mModelForItemFunctions, mi);
2170 
2171  if ((!oldParentWasTheSame) && (oldParent != mRootItem)) {
2172  messageDetachedUpdateParentProperties(oldParent, mi);
2173  }
2174  } else {
2175  // The item had no parent yet.
2176  oldParentWasTheSame = false;
2177  }
2178 
2179  // Take care of perfect / imperfect threading.
2180  // Items that are now perfectly threaded, but already have a different parent
2181  // might have been imperfectly threaded before. Remove them from the caches.
2182  // Items that are now imperfectly threaded must be added to the caches.
2183  //
2184  // If we're just re-sorting the item inside the same parent then the threading
2185  // caches don't need to be updated (since they actually depend on the parent).
2186 
2187  if (!oldParentWasTheSame) {
2188  switch (mi->threadingStatus()) {
2190  if (!mi->inReplyToIdMD5().isEmpty()) {
2191  mThreadingCacheMessageInReplyToIdMD5ToMessageItem.remove(mi->inReplyToIdMD5(), mi);
2192  }
2193  if (attachOptions == StoreInCache && pParent->type() == Item::Message) {
2194  mThreadingCache.updateParent(mi, static_cast<MessageItem *>(pParent));
2195  }
2196  break;
2198  case MessageItem::ParentMissing: // may be: temporary or just fallback assignment
2199  if (!mi->inReplyToIdMD5().isEmpty()) {
2200  if (!mThreadingCacheMessageInReplyToIdMD5ToMessageItem.contains(mi->inReplyToIdMD5(), mi)) {
2201  mThreadingCacheMessageInReplyToIdMD5ToMessageItem.insert(mi->inReplyToIdMD5(), mi);
2202  }
2203  }
2204  break;
2205  case MessageItem::NonThreadable: // this also happens when we do no threading at all
2206  // make gcc happy
2207  Q_ASSERT(!mThreadingCacheMessageInReplyToIdMD5ToMessageItem.contains(mi->inReplyToIdMD5(), mi));
2208  break;
2209  }
2210  }
2211 
2212  // Set the new parent
2213  mi->setParent(pParent);
2214 
2215  // Propagate watched and ignored status
2216  if ((pParent->status().toQInt32() & mCachedWatchedOrIgnoredStatusBits) // unlikely
2217  && (pParent->type() == Item::Message) // likely
2218  ) {
2219  // the parent is either watched or ignored: propagate to the child
2220  if (pParent->status().isWatched()) {
2221  int row = mInvariantRowMapper->modelInvariantIndexToModelIndexRow(mi);
2223  mStorageModel->setMessageItemStatus(mi, row, Akonadi::MessageStatus::statusWatched());
2224  } else if (pParent->status().isIgnored()) {
2225  int row = mInvariantRowMapper->modelInvariantIndexToModelIndexRow(mi);
2227  mStorageModel->setMessageItemStatus(mi, row, Akonadi::MessageStatus::statusIgnored());
2228  }
2229  }
2230 
2231  // And insert into its child list
2232 
2233  // If pParent is viewable then the insert/append functions will call this model's
2234  // beginInsertRows() and endInsertRows() functions. This is EXTREMELY
2235  // expensive and ugly but it's the only way with the Qt4 imposed Model/View method.
2236  // Dude... (citation from Lost, if it wasn't clear).
2237 
2238  // I'm using a macro since it does really improve readability.
2239  // I'm NOT using a helper function since gcc will refuse to inline some of
2240  // the calls because they make this function grow too much.
2241 #define INSERT_MESSAGE_WITH_COMPARATOR(_ItemComparator) \
2242  if ((mSortOrder->messageSortDirection() == SortOrder::Ascending) || (pParent->type() == Item::Message)) { \
2243  pParent->d_ptr->insertChildItem<_ItemComparator, true>(mModelForItemFunctions, mi); \
2244  } else { \
2245  pParent->d_ptr->insertChildItem<_ItemComparator, false>(mModelForItemFunctions, mi); \
2246  }
2247 
2248  // If pParent is viewable then the insertion call will also set the child state to viewable.
2249  // Since mi MAY have children, then this call may make them viewable.
2250  switch (mSortOrder->messageSorting()) {
2252  INSERT_MESSAGE_WITH_COMPARATOR(ItemDateComparator)
2253  break;
2255  INSERT_MESSAGE_WITH_COMPARATOR(ItemMaxDateComparator)
2256  break;
2258  INSERT_MESSAGE_WITH_COMPARATOR(ItemSizeComparator)
2259  break;
2261  INSERT_MESSAGE_WITH_COMPARATOR(ItemSenderOrReceiverComparator)
2262  break;
2264  INSERT_MESSAGE_WITH_COMPARATOR(ItemSenderComparator)
2265  break;
2267  INSERT_MESSAGE_WITH_COMPARATOR(ItemReceiverComparator)
2268  break;
2270  INSERT_MESSAGE_WITH_COMPARATOR(ItemSubjectComparator)
2271  break;
2273  INSERT_MESSAGE_WITH_COMPARATOR(ItemActionItemStatusComparator)
2274  break;
2276  INSERT_MESSAGE_WITH_COMPARATOR(ItemUnreadStatusComparator)
2277  break;
2278  case SortOrder::SortMessagesByImportantStatus:
2279  INSERT_MESSAGE_WITH_COMPARATOR(ItemImportantStatusComparator)
2280  break;
2282  INSERT_MESSAGE_WITH_COMPARATOR(ItemAttachmentStatusComparator)
2283  break;
2285  pParent->appendChildItem(mModelForItemFunctions, mi);
2286  break;
2287  default: // should never happen
2288  pParent->appendChildItem(mModelForItemFunctions, mi);
2289  break;
2290  }
2291 
2292  // Decide if we need to expand parents
2293  bool childNeedsExpanding = (mi->initialExpandStatus() == Item::ExpandNeeded);
2294 
2295  if (pParent->initialExpandStatus() == Item::NoExpandNeeded) {
2296  switch (mAggregation->threadExpandPolicy()) {
2298  // just do nothing unless this child has children and is already marked for expansion
2299  if (childNeedsExpanding) {
2301  }
2302  break;
2303  case Aggregation::ExpandThreadsWithNewMessages: // No more new status. fall through to unread if it exists in config
2305  // expand only if unread (or it has children marked for expansion)
2306  if (childNeedsExpanding || !mi->status().isRead()) {
2308  }
2309  break;
2311  // expand only if unread, important or todo (or it has children marked for expansion)
2312  // FIXME: Wouldn't it be nice to be able to test for bitmasks in MessageStatus ?
2313  if (childNeedsExpanding || !mi->status().isRead() || mi->status().isImportant() || mi->status().isToAct()) {
2315  }
2316  break;
2318  // expand everything
2320  break;
2321  default:
2322  // BUG
2323  break;
2324  }
2325  } // else it's already marked for expansion or expansion has been already executed
2326 
2327  // expand parent first, if possible
2328  if (pParent->initialExpandStatus() == Item::ExpandNeeded) {
2329  // If UI is not disconnected and parent is viewable, go up and expand
2330  if (mModelForItemFunctions && pParent->isViewable()) {
2331  // Now expand parents as needed
2332  Item *parentToExpand = pParent;
2333  while (parentToExpand) {
2334  if (parentToExpand == mRootItem) {
2335  break; // no need to set it expanded
2336  }
2337  // parentToExpand is surely viewable (because this item is)
2338  if (parentToExpand->initialExpandStatus() == Item::ExpandExecuted) {
2339  break;
2340  }
2341 
2342  mView->expand(q->index(parentToExpand, 0));
2343 
2345  parentToExpand = parentToExpand->parent();
2346  }
2347  } else {
2348  // It isn't viewable or UI is disconnected: climb up marking only
2349  Item *parentToExpand = pParent->parent();
2350  while (parentToExpand) {
2351  if (parentToExpand == mRootItem) {
2352  break; // no need to set it expanded
2353  }
2354  parentToExpand->setInitialExpandStatus(Item::ExpandNeeded);
2355  parentToExpand = parentToExpand->parent();
2356  }
2357  }
2358  }
2359 
2360  if (mi->isViewable()) {
2361  // mi is now viewable
2362 
2363  // sync subtree expanded status
2364  if (childNeedsExpanding) {
2365  if (mi->childItemCount() > 0) {
2366  if (mModelForItemFunctions) { // the UI is not disconnected
2367  syncExpandedStateOfSubtree(mi); // sync the real state in the view
2368  }
2369  }
2370  }
2371 
2372  // apply the filter, if needed
2373  if (mFilter) {
2374  Q_ASSERT(mModelForItemFunctions); // the UI must be NOT disconnected here
2375 
2376  // apply the filter to subtree
2377  if (applyFilterToSubtree(mi, q->index(pParent, 0))) {
2378  // mi matched, expand parents (unconditionally)
2379  mView->ensureDisplayedWithParentsExpanded(mi);
2380  }
2381  }
2382  }
2383 
2384  // Now we need to propagate the property changes the upper levels.
2385 
2386  // If we have just inserted a message inside the root then no work needs to be done:
2387  // no grouping is in effect and the message is already in the right place.
2388  if (pParent == mRootItem) {
2389  return;
2390  }
2391 
2392  // If we have just removed the item from this parent and re-inserted it
2393  // then this operation was a simple re-sort. The code above didn't update
2394  // the properties when removing the item so we don't actually need
2395  // to make the updates back.
2396  if (oldParentWasTheSame) {
2397  return;
2398  }
2399 
2400  // FIXME: OPTIMIZE THIS: First propagate changes THEN syncExpandedStateOfSubtree()
2401  // and applyFilterToSubtree... (needs some thinking though).
2402 
2403  // Time to propagate up.
2404  propagateItemPropertiesToParent(mi);
2405 
2406  // Aaah.. we're done. Time for a thea ? :)
2407 }
2408 
2409 // FIXME: ThreadItem ?
2410 //
2411 // Foo Bar, Joe Thommason, Martin Rox ... Eddie Maiden <date of the thread>
2412 // Title <number of messages>, Last by xxx <inner status>
2413 //
2414 // When messages are added, mark it as dirty only (?)
2415 
2416 ModelPrivate::ViewItemJobResult ModelPrivate::viewItemJobStepInternalForJobPass5(ViewItemJob *job, QElapsedTimer elapsedTimer)
2417 {
2418  // In this pass we scan the group headers that are in mGroupHeadersThatNeedUpdate.
2419  // Empty groups get deleted while the other ones are re-sorted.
2420 
2421  int curIndex = job->currentIndex();
2422 
2423  auto it = mGroupHeadersThatNeedUpdate.begin();
2424  auto end = mGroupHeadersThatNeedUpdate.end();
2425 
2426  while (it != end) {
2427  if ((*it)->childItemCount() == 0) {
2428  // group with no children, kill it
2429  (*it)->parent()->takeChildItem(mModelForItemFunctions, *it);
2430  mGroupHeaderItemHash.remove((*it)->label());
2431 
2432  // If we were going to restore its position after the job step, well.. we can't do it anymore.
2433  if (mCurrentItemToRestoreAfterViewItemJobStep == (*it)) {
2434  mCurrentItemToRestoreAfterViewItemJobStep = nullptr;
2435  }
2436 
2437  // bye bye
2438  delete *it;
2439  } else {
2440  // Group with children: probably needs re-sorting.
2441 
2442  // Re-sorting here is an expensive operation.
2443  // In fact groups have been put in the QHash above on the assumption
2444  // that re-sorting *might* be needed but no real (expensive) check
2445  // has been done yet. Also by sorting a single group we might actually
2446  // put the others in the right place.
2447  // So finally check if re-sorting is *really* needed.
2448  bool needsReSorting;
2449 
2450  // A macro really improves readability here.
2451 #define CHECK_IF_GROUP_NEEDS_RESORTING(_ItemDateComparator) \
2452  switch (mSortOrder->groupSortDirection()) { \
2453  case SortOrder::Ascending: \
2454  needsReSorting = (*it)->parent()->d_ptr->childItemNeedsReSorting<_ItemDateComparator, true>(*it); \
2455  break; \
2456  case SortOrder::Descending: \
2457  needsReSorting = (*it)->parent()->d_ptr->childItemNeedsReSorting<_ItemDateComparator, false>(*it); \
2458  break; \
2459  default: /* should never happen */ \
2460  needsReSorting = false; \
2461  break; \
2462  }
2463 
2464  switch (mSortOrder->groupSorting()) {
2466  CHECK_IF_GROUP_NEEDS_RESORTING(ItemDateComparator)
2467  break;
2469  CHECK_IF_GROUP_NEEDS_RESORTING(ItemMaxDateComparator)
2470  break;
2472  CHECK_IF_GROUP_NEEDS_RESORTING(ItemSenderOrReceiverComparator)
2473  break;
2475  CHECK_IF_GROUP_NEEDS_RESORTING(ItemSenderComparator)
2476  break;
2478  CHECK_IF_GROUP_NEEDS_RESORTING(ItemReceiverComparator)
2479  break;
2481  needsReSorting = false;
2482  break;
2483  default:
2484  // Should never happen... just assume re-sorting is not needed
2485  needsReSorting = false;
2486  break;
2487  }
2488 
2489  if (needsReSorting) {
2490  attachGroup(*it); // it will first detach and then re-attach in the proper place
2491  }
2492  }
2493 
2494  it = mGroupHeadersThatNeedUpdate.erase(it);
2495 
2496  curIndex++;
2497 
2498  // FIXME: In fact a single update is likely to manipulate
2499  // a subtree with a LOT of messages inside. If interactivity is favored
2500  // we should check the time really more often.
2501  if ((curIndex % mViewItemJobStepMessageCheckCount) == 0) {
2502  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
2503  if (it != mGroupHeadersThatNeedUpdate.end()) {
2504  job->setCurrentIndex(curIndex);
2505  return ViewItemJobInterrupted;
2506  }
2507  }
2508  }
2509  }
2510 
2511  return ViewItemJobCompleted;
2512 }
2513 
2514 ModelPrivate::ViewItemJobResult ModelPrivate::viewItemJobStepInternalForJobPass4(ViewItemJob *job, QElapsedTimer elapsedTimer)
2515 {
2516  // In this pass we scan mUnassignedMessageListForPass4 which now
2517  // contains both items with parents and items without parents.
2518  // We scan mUnassignedMessageList for messages without parent (the ones that haven't been
2519  // attached to the viewable tree yet) and find a suitable group for them. Then we simply
2520  // clear mUnassignedMessageList.
2521 
2522  // We call this pass "Grouping"
2523 
2524  int curIndex = job->currentIndex();
2525  int endIndex = job->endIndex();
2526 
2527  while (curIndex <= endIndex) {
2528  MessageItem *mi = mUnassignedMessageListForPass4[curIndex];
2529  if (!mi->parent()) {
2530  // Unassigned item: thread leader, insert into the proper group.
2531  // Locate the group (or root if no grouping requested)
2532  attachMessageToGroupHeader(mi);
2533  } else {
2534  // A parent was already assigned in Pass3: we have nothing to do here
2535  }
2536  curIndex++;
2537 
2538  // FIXME: In fact a single call to attachMessageToGroupHeader() is likely to manipulate
2539  // a subtree with a LOT of messages inside. If interactivity is favored
2540  // we should check the time really more often.
2541  if ((curIndex % mViewItemJobStepMessageCheckCount) == 0) {
2542  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
2543  if (curIndex <= endIndex) {
2544  job->setCurrentIndex(curIndex);
2545  return ViewItemJobInterrupted;
2546  }
2547  }
2548  }
2549  }
2550 
2551  mUnassignedMessageListForPass4.clear();
2552  return ViewItemJobCompleted;
2553 }
2554 
2555 ModelPrivate::ViewItemJobResult ModelPrivate::viewItemJobStepInternalForJobPass3(ViewItemJob *job, QElapsedTimer elapsedTimer)
2556 {
2557  // In this pass we scan the mUnassignedMessageListForPass3 and try to do construct the threads
2558  // by using subject based threading. If subject based threading is not in effect then
2559  // this pass turns to a nearly-no-op: at the end of Pass2 we have swapped the lists
2560  // and mUnassignedMessageListForPass3 is actually empty.
2561 
2562  // We don't shrink the mUnassignedMessageListForPass3 for two reasons:
2563  // - It would mess up this chunked algorithm by shifting indexes
2564  // - mUnassignedMessageList is a QList which is basically an array. It's faster
2565  // to traverse an array of N entries than to remove K>0 entries one by one and
2566  // to traverse the remaining N-K entries.
2567 
2568  int curIndex = job->currentIndex();
2569  int endIndex = job->endIndex();
2570 
2571  while (curIndex <= endIndex) {
2572  // If we're here, then threading is requested for sure.
2573  auto mi = mUnassignedMessageListForPass3[curIndex];
2574  if ((!mi->parent()) || (mi->threadingStatus() == MessageItem::ParentMissing)) {
2575  // Parent is missing (either "physically" with the item being not attached or "logically"
2576  // with the item being attached to a group or directly to the root.
2577  if (mi->subjectIsPrefixed()) {
2578  // We can try to guess it
2579  auto mparent = guessMessageParent(mi);
2580 
2581  if (mparent) {
2582  // imperfect parent found
2583  if (mi->isViewable()) {
2584  // mi was already viewable, we're just trying to re-parent it better...
2585  attachMessageToParent(mparent, mi);
2586  if (!mparent->isViewable()) {
2587  // re-attach it immediately (so current item is not lost)
2588  auto topmost = mparent->topmostMessage();
2589  Q_ASSERT(!topmost->parent()); // groups are always viewable!
2590  topmost->setThreadingStatus(MessageItem::ParentMissing);
2591  attachMessageToGroupHeader(topmost);
2592  }
2593  } else {
2594  // mi wasn't viewable yet.. no need to attach parent
2595  attachMessageToParent(mparent, mi);
2596  }
2597  // and we're done for now
2598  } else {
2599  // so parent not found, (threadingStatus() is either MessageItem::ParentMissing or MessageItem::NonThreadable)
2600  Q_ASSERT((mi->threadingStatus() == MessageItem::ParentMissing) || (mi->threadingStatus() == MessageItem::NonThreadable));
2601  mUnassignedMessageListForPass4.append(mi); // this is ~O(1)
2602  // and wait for Pass4
2603  }
2604  } else {
2605  // can't guess the parent as the subject isn't prefixed
2606  Q_ASSERT((mi->threadingStatus() == MessageItem::ParentMissing) || (mi->threadingStatus() == MessageItem::NonThreadable));
2607  mUnassignedMessageListForPass4.append(mi); // this is ~O(1)
2608  // and wait for Pass4
2609  }
2610  } else {
2611  // Has a parent: either perfect parent already found or non threadable.
2612  // Since we don't end here if mi has status of parent missing then mi must not have imperfect parent.
2613  Q_ASSERT(mi->threadingStatus() != MessageItem::ImperfectParentFound);
2614  Q_ASSERT(mi->isViewable());
2615  }
2616 
2617  curIndex++;
2618 
2619  // FIXME: In fact a single call to attachMessageToGroupHeader() is likely to manipulate
2620  // a subtree with a LOT of messages inside. If interactivity is favored
2621  // we should check the time really more often.
2622  if ((curIndex % mViewItemJobStepMessageCheckCount) == 0) {
2623  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
2624  if (curIndex <= endIndex) {
2625  job->setCurrentIndex(curIndex);
2626  return ViewItemJobInterrupted;
2627  }
2628  }
2629  }
2630  }
2631 
2632  mUnassignedMessageListForPass3.clear();
2633  return ViewItemJobCompleted;
2634 }
2635 
2636 ModelPrivate::ViewItemJobResult ModelPrivate::viewItemJobStepInternalForJobPass2(ViewItemJob *job, QElapsedTimer elapsedTimer)
2637 {
2638  // In this pass we scan the mUnassignedMessageList and try to do construct the threads.
2639  // If some thread leader message got attached to the viewable tree in Pass1Fill then
2640  // we'll also attach all of its children too. The thread leaders we were unable
2641  // to attach in Pass1Fill and their children (which we find here) will make it to the small Pass3
2642 
2643  // We don't shrink the mUnassignedMessageList for two reasons:
2644  // - It would mess up this chunked algorithm by shifting indexes
2645  // - mUnassignedMessageList is a QList which is basically an array. It's faster
2646  // to traverse an array of N entries than to remove K>0 entries one by one and
2647  // to traverse the remaining N-K entries.
2648 
2649  // We call this pass "Threading"
2650 
2651  int curIndex = job->currentIndex();
2652  int endIndex = job->endIndex();
2653 
2654  while (curIndex <= endIndex) {
2655  // If we're here, then threading is requested for sure.
2656  auto mi = mUnassignedMessageListForPass2[curIndex];
2657  // The item may or may not have a parent.
2658  // If it has no parent or it has a temporary one (mi->parent() && mi->threadingStatus() == MessageItem::ParentMissing)
2659  // then we attempt to (re-)thread it. Otherwise we just do nothing (the job has already been done by the previous steps).
2660  if ((!mi->parent()) || (mi->threadingStatus() == MessageItem::ParentMissing)) {
2661  qint64 parentId;
2662  auto mparent = mThreadingCache.parentForItem(mi, parentId);
2663  if (mparent && !mparent->hasAncestor(mi)) {
2664  mi->setThreadingStatus(MessageItem::PerfectParentFound);
2665  attachMessageToParent(mparent, mi, SkipCacheUpdate);
2666  } else {
2667  if (parentId > 0) {
2668  // In second pass we have all available Items in mThreadingCache already. If
2669  // mThreadingCache.parentForItem() returns null, but returns valid parentId then
2670  // the Item was removed from Akonadi and our threading cache is out-of-date.
2671  mThreadingCache.expireParent(mi);
2672  mparent = findMessageParent(mi);
2673  } else if (parentId < 0) {
2674  mparent = findMessageParent(mi);
2675  } else {
2676  // parentId = 0: this message is a thread leader so don't
2677  // bother resolving parent, it will be moved directly to
2678  // Pass4 in the code below
2679  }
2680 
2681  if (mparent) {
2682  // parent found, either perfect or imperfect
2683  if (mi->isViewable()) {
2684  // mi was already viewable, we're just trying to re-parent it better...
2685  attachMessageToParent(mparent, mi);
2686  if (!mparent->isViewable()) {
2687  // re-attach it immediately (so current item is not lost)
2688  auto topmost = mparent->topmostMessage();
2689  Q_ASSERT(!topmost->parent()); // groups are always viewable!
2690  topmost->setThreadingStatus(MessageItem::ParentMissing);
2691  attachMessageToGroupHeader(topmost);
2692  }
2693  } else {
2694  // mi wasn't viewable yet.. no need to attach parent
2695  attachMessageToParent(mparent, mi);
2696  }
2697  // and we're done for now
2698  } else {
2699  // so parent not found, (threadingStatus() is either MessageItem::ParentMissing or MessageItem::NonThreadable)
2700  switch (mi->threadingStatus()) {
2702  if (mAggregation->threading() == Aggregation::PerfectReferencesAndSubject) {
2703  // parent missing but still can be found in Pass3
2704  mUnassignedMessageListForPass3.append(mi); // this is ~O(1)
2705  } else {
2706  // We're not doing subject based threading: will never be threaded, go straight to Pass4
2707  mUnassignedMessageListForPass4.append(mi); // this is ~O(1)
2708  }
2709  break;
2711  // will never be threaded, go straight to Pass4
2712  mUnassignedMessageListForPass4.append(mi); // this is ~O(1)
2713  break;
2714  default:
2715  // a bug for sure
2716  qCWarning(MESSAGELIST_LOG) << "ERROR: Invalid message threading status returned by findMessageParent()!";
2717  Q_ASSERT(false);
2718  break;
2719  }
2720  }
2721  }
2722  } else {
2723  // Has a parent: either perfect parent already found or non threadable.
2724  // Since we don't end here if mi has status of parent missing then mi must not have imperfect parent.
2725  Q_ASSERT(mi->threadingStatus() != MessageItem::ImperfectParentFound);
2726  if (!mi->isViewable()) {
2727  qCWarning(MESSAGELIST_LOG) << "Non viewable message " << mi << " subject " << mi->subject().toUtf8().data();
2728  Q_ASSERT(mi->isViewable());
2729  }
2730  }
2731 
2732  curIndex++;
2733 
2734  // FIXME: In fact a single call to attachMessageToGroupHeader() is likely to manipulate
2735  // a subtree with a LOT of messages inside. If interactivity is favored
2736  // we should check the time really more often.
2737  if ((curIndex % mViewItemJobStepMessageCheckCount) == 0) {
2738  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
2739  if (curIndex <= endIndex) {
2740  job->setCurrentIndex(curIndex);
2741  return ViewItemJobInterrupted;
2742  }
2743  }
2744  }
2745  }
2746 
2747  mUnassignedMessageListForPass2.clear();
2748  return ViewItemJobCompleted;
2749 }
2750 
2751 ModelPrivate::ViewItemJobResult ModelPrivate::viewItemJobStepInternalForJobPass1Fill(ViewItemJob *job, QElapsedTimer elapsedTimer)
2752 {
2753  // In this pass we scan the a contiguous region of the underlying storage (that is
2754  // assumed to be FLAT) and create the corresponding MessageItem objects.
2755  // The deal is to show items to the user as soon as possible so in this pass we
2756  // *TRY* to attach them to the viewable tree (which is rooted on mRootItem).
2757  // Messages we're unable to attach for some reason (mainly due to threading) get appended
2758  // to mUnassignedMessageList and wait for Pass2.
2759 
2760  // We call this pass "Processing"
2761 
2762  // Should we use the receiver or the sender field for sorting ?
2763  bool bUseReceiver = mStorageModelContainsOutboundMessages;
2764 
2765  // The begin storage index of our work
2766  int curIndex = job->currentIndex();
2767  // The end storage index of our work.
2768  int endIndex = job->endIndex();
2769 
2770  unsigned long msgToSelect = mPreSelectionMode == PreSelectLastSelected ? mStorageModel->preSelectedMessage() : 0;
2771 
2772  MessageItem *mi = nullptr;
2773 
2774  while (curIndex <= endIndex) {
2775  // Create the message item with no parent: we'll set it later
2776  if (!mi) {
2777  mi = new MessageItem();
2778  } else {
2779  // a MessageItem discarded by a previous iteration: reuse it.
2780  Q_ASSERT(mi->parent() == nullptr);
2781  }
2782 
2783  if (!mStorageModel->initializeMessageItem(mi, curIndex, bUseReceiver)) {
2784  // ugh
2785  qCWarning(MESSAGELIST_LOG) << "Fill of the MessageItem at storage row index " << curIndex << " failed";
2786  curIndex++;
2787  continue;
2788  }
2789 
2790  // If we're supposed to pre-select a specific message, check if it's this one.
2791  if (msgToSelect != 0 && msgToSelect == mi->uniqueId()) {
2792  // Found, it's this one.
2793  // But actually it's not viewable (so not selectable). We must wait
2794  // until the end of the job to be 100% sure. So here we just translate
2795  // the unique id to a MessageItem pointer and wait.
2796  mLastSelectedMessageInFolder = mi;
2797  msgToSelect = 0; // already found, don't bother checking anymore
2798  }
2799 
2800  // Update the newest/oldest message, since we might be supposed to select those later
2801  if (mi->date() != static_cast<uint>(-1)) {
2802  if (!mOldestItem || mOldestItem->date() > mi->date()) {
2803  mOldestItem = mi;
2804  }
2805  if (!mNewestItem || mNewestItem->date() < mi->date()) {
2806  mNewestItem = mi;
2807  }
2808  }
2809 
2810  // Ok.. it passed the initial checks: we will not be discarding it.
2811  // Make this message item an invariant index to the underlying model storage.
2812  mInvariantRowMapper->createModelInvariantIndex(curIndex, mi);
2813 
2814  // Attempt to do threading as soon as possible (to display items to the user)
2815  if (mAggregation->threading() != Aggregation::NoThreading) {
2816  // Threading is requested
2817 
2818  // Fetch the data needed for proper threading
2819  // Add the item to the threading caches
2820 
2821  switch (mAggregation->threading()) {
2823  mStorageModel->fillMessageItemThreadingData(mi, curIndex, StorageModel::PerfectThreadingReferencesAndSubject);
2824 
2825  // We also need to build the subject/reference-based threading cache
2826  addMessageToReferencesBasedThreadingCache(mi);
2827  addMessageToSubjectBasedThreadingCache(mi);
2828  break;
2830  mStorageModel->fillMessageItemThreadingData(mi, curIndex, StorageModel::PerfectThreadingPlusReferences);
2831  addMessageToReferencesBasedThreadingCache(mi);
2832  break;
2833  default:
2834  mStorageModel->fillMessageItemThreadingData(mi, curIndex, StorageModel::PerfectThreadingOnly);
2835  break;
2836  }
2837 
2838  // Perfect/References threading cache
2839  mThreadingCacheMessageIdMD5ToMessageItem.insert(mi->messageIdMD5(), mi);
2840 
2841  // Register the current item into the threading cache
2842  mThreadingCache.addItemToCache(mi);
2843 
2844  // First of all look into the persistent cache
2845  qint64 parentId;
2846  Item *pParent = mThreadingCache.parentForItem(mi, parentId);
2847  if (pParent) {
2848  // We already have the parent MessageItem. Attach current message
2849  // to it and mark it as perfect
2850  mi->setThreadingStatus(MessageItem::PerfectParentFound);
2851  attachMessageToParent(pParent, mi);
2852  } else if (parentId > 0) {
2853  // We don't have the parent MessageItem yet, but we do know the
2854  // parent: delay for pass 2 when we will have the parent MessageItem
2855  // for sure.
2856  mi->setThreadingStatus(MessageItem::ParentMissing);
2857  mUnassignedMessageListForPass2.append(mi);
2858  } else if (parentId == 0) {
2859  // Message is a thread leader, skip straight to Pass4
2860  mi->setThreadingStatus(MessageItem::NonThreadable);
2861  mUnassignedMessageListForPass4.append(mi);
2862  } else {
2863  // Check if this item is a perfect parent for some imperfectly threaded
2864  // message (that is actually attached to it, but not necessarily to the
2865  // viewable root). If it is, then remove the imperfect child from its
2866  // current parent rebuild the hierarchy on the fly.
2867  bool needsImmediateReAttach = false;
2868 
2869  if (!mThreadingCacheMessageInReplyToIdMD5ToMessageItem.isEmpty()) { // unlikely
2870  const auto lImperfectlyThreaded = mThreadingCacheMessageInReplyToIdMD5ToMessageItem.values(mi->messageIdMD5());
2871  for (const auto it : lImperfectlyThreaded) {
2872  Q_ASSERT(it->parent());
2873  Q_ASSERT(it->parent() != mi);
2874 
2875  if (!((it->threadingStatus() == MessageItem::ImperfectParentFound) || (it->threadingStatus() == MessageItem::ParentMissing))) {
2876  qCritical() << "Got message " << it << " with threading status" << it->threadingStatus();
2877  Q_ASSERT_X(false, "ModelPrivate::viewItemJobStepInternalForJobPass1Fill", "Wrong threading status");
2878  }
2879 
2880  // If the item was already attached to the view then
2881  // re-attach it immediately. This will avoid a message
2882  // being displayed for a short while in the view and then
2883  // disappear until a perfect parent isn't found.
2884  if (it->isViewable()) {
2885  needsImmediateReAttach = true;
2886  }
2887 
2888  it->setThreadingStatus(MessageItem::PerfectParentFound);
2889  attachMessageToParent(mi, it);
2890  }
2891  }
2892 
2893  // FIXME: Might look by "References" too, here... (?)
2894 
2895  // Attempt to do threading with anything we already have in caches until now
2896  // Note that this is likely to work since thread-parent messages tend
2897  // to come before thread-children messages in the folders (simply because of
2898  // date of arrival).
2899 
2900  // First of all try to find a "perfect parent", that is the message for that
2901  // we have the ID in the "In-Reply-To" field. This is actually done by using
2902  // MD5 caches of the message ids because of speed. Collisions are very unlikely.
2903 
2904  const QByteArray md5 = mi->inReplyToIdMD5();
2905  if (!md5.isEmpty()) {
2906  // Have an In-Reply-To field MD5.
2907  // In well behaved mailing lists 70% of the threadable messages get a parent here :)
2908  pParent = mThreadingCacheMessageIdMD5ToMessageItem.value(md5, nullptr);
2909 
2910  if (pParent) { // very likely
2911  // Take care of self-referencing (which is always possible)
2912  // and circular In-Reply-To reference loops which are possible
2913  // in case this item was found to be a perfect parent for some
2914  // imperfectly threaded message just above.
2915  if ((mi == pParent) // self referencing message
2916  || ((mi->childItemCount() > 0) // mi already has children, this is fast to determine
2917  && pParent->hasAncestor(mi) // pParent is in the mi's children tree
2918  )) {
2919  // Bad, bad message.. it has In-Reply-To equal to Message-Id
2920  // or it's in a circular In-Reply-To reference loop.
2921  // Will wait for Pass2 with References-Id only
2922  qCWarning(MESSAGELIST_LOG) << "Circular In-Reply-To reference loop detected in the message tree";
2923  mUnassignedMessageListForPass2.append(mi);
2924  } else {
2925  // wow, got a perfect parent for this message!
2926  mi->setThreadingStatus(MessageItem::PerfectParentFound);
2927  attachMessageToParent(pParent, mi);
2928  // we're done with this message (also for Pass2)
2929  }
2930  } else {
2931  // got no parent
2932  // will have to wait Pass2
2933  mUnassignedMessageListForPass2.append(mi);
2934  }
2935  } else {
2936  // No In-Reply-To header.
2937 
2938  bool mightHaveOtherMeansForThreading;
2939 
2940  switch (mAggregation->threading()) {
2942  mightHaveOtherMeansForThreading = mi->subjectIsPrefixed() || !mi->referencesIdMD5().isEmpty();
2943  break;
2945  mightHaveOtherMeansForThreading = !mi->referencesIdMD5().isEmpty();
2946  break;
2948  mightHaveOtherMeansForThreading = false;
2949  break;
2950  default:
2951  // BUG: there shouldn't be other values (NoThreading is excluded in an upper branch)
2952  Q_ASSERT(false);
2953  mightHaveOtherMeansForThreading = false; // make gcc happy
2954  break;
2955  }
2956 
2957  if (mightHaveOtherMeansForThreading) {
2958  // We might have other means for threading this message, wait until Pass2
2959  mUnassignedMessageListForPass2.append(mi);
2960  } else {
2961  // No other means for threading this message. This is either
2962  // a standalone message or a thread leader.
2963  // If there is no grouping in effect or thread leaders are just the "topmost"
2964  // messages then we might be done with this one.
2965  if ((mAggregation->grouping() == Aggregation::NoGrouping) || (mAggregation->threadLeader() == Aggregation::TopmostMessage)) {
2966  // We're done with this message: it will be surely either toplevel (no grouping in effect)
2967  // or a thread leader with a well defined group. Do it :)
2968  // qCDebug(MESSAGELIST_LOG) << "Setting message status from " << mi->threadingStatus() << " to non threadable (1) " << mi;
2969  mi->setThreadingStatus(MessageItem::NonThreadable);
2970  // Locate the parent group for this item
2971  attachMessageToGroupHeader(mi);
2972  // we're done with this message (also for Pass2)
2973  } else {
2974  // Threads belong to the most recent message in the thread. This means
2975  // that we have to wait until Pass2 or Pass3 to assign a group.
2976  mUnassignedMessageListForPass2.append(mi);
2977  }
2978  }
2979  }
2980 
2981  if (needsImmediateReAttach && !mi->isViewable()) {
2982  // The item gathered previously viewable children. They must be immediately
2983  // re-shown. So this item must currently be attached to the view.
2984  // This is a temporary measure: it will be probably still moved.
2985  MessageItem *topmost = mi->topmostMessage();
2986  Q_ASSERT(topmost->threadingStatus() == MessageItem::ParentMissing);
2987  attachMessageToGroupHeader(topmost);
2988  }
2989  }
2990  } else {
2991  // else no threading requested: we don't even need Pass2
2992  // set not threadable status (even if it might be not true, but in this mode we don't care)
2993  // qCDebug(MESSAGELIST_LOG) << "Setting message status from " << mi->threadingStatus() << " to non threadable (2) " << mi;
2994  mi->setThreadingStatus(MessageItem::NonThreadable);
2995  // locate the parent group for this item
2996  if (mAggregation->grouping() == Aggregation::NoGrouping) {
2997  attachMessageToParent(mRootItem, mi); // no groups requested, attach directly to root
2998  } else {
2999  attachMessageToGroupHeader(mi);
3000  }
3001  // we're done with this message (also for Pass2)
3002  }
3003 
3004  mi = nullptr; // this item was pushed somewhere, create a new one at next iteration
3005  curIndex++;
3006 
3007  if ((curIndex % mViewItemJobStepMessageCheckCount) == 0) {
3008  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
3009  if (curIndex <= endIndex) {
3010  job->setCurrentIndex(curIndex);
3011  return ViewItemJobInterrupted;
3012  }
3013  }
3014  }
3015  }
3016 
3017  if (mi) {
3018  delete mi;
3019  }
3020  return ViewItemJobCompleted;
3021 }
3022 
3023 ModelPrivate::ViewItemJobResult ModelPrivate::viewItemJobStepInternalForJobPass1Cleanup(ViewItemJob *job, QElapsedTimer elapsedTimer)
3024 {
3025  Q_ASSERT(mModelForItemFunctions); // UI must be not disconnected here
3026  // In this pass we remove the MessageItem objects that are present in the job
3027  // and put their children in the unassigned message list.
3028 
3029  // Note that this list in fact contains MessageItem objects (we need dynamic_cast<>).
3030  QList<ModelInvariantIndex *> *invalidatedMessages = job->invariantIndexList();
3031 
3032  // We don't shrink the invalidatedMessages because it's basically an array.
3033  // It's faster to traverse an array of N entries than to remove K>0 entries
3034  // one by one and to traverse the remaining N-K entries.
3035 
3036  // The begin index of our work
3037  int curIndex = job->currentIndex();
3038  // The end index of our work.
3039  int endIndex = job->endIndex();
3040 
3041  if (curIndex == job->startIndex()) {
3042  Q_ASSERT(mOrphanChildrenHash.isEmpty());
3043  }
3044 
3045  while (curIndex <= endIndex) {
3046  // Get the underlying storage message data...
3047  auto dyingMessage = dynamic_cast<MessageItem *>(invalidatedMessages->at(curIndex));
3048  // This MUST NOT be null (otherwise we have a bug somewhere in this file).
3049  Q_ASSERT(dyingMessage);
3050 
3051  // If we were going to pre-select this message but we were interrupted
3052  // *before* it was actually made viewable, we just clear the pre-selection pointer
3053  // and unique id (abort pre-selection).
3054  if (dyingMessage == mLastSelectedMessageInFolder) {
3055  mLastSelectedMessageInFolder = nullptr;
3056  mPreSelectionMode = PreSelectNone;
3057  }
3058 
3059  // remove the message from any pending user job
3060  if (mPersistentSetManager) {
3061  mPersistentSetManager->removeMessageItemFromAllSets(dyingMessage);
3062  if (mPersistentSetManager->setCount() < 1) {
3063  delete mPersistentSetManager;
3064  mPersistentSetManager = nullptr;
3065  }
3066  }
3067 
3068  // Remove the message from threading cache before we start moving up the
3069  // children, so that they don't get mislead by the cache
3070  mThreadingCache.expireParent(dyingMessage);
3071 
3072  if (dyingMessage->parent()) {
3073  // Handle saving the current selection: if this item was the current before the step
3074  // then zero it out. We have killed it and it's OK for the current item to change.
3075 
3076  if (dyingMessage == mCurrentItemToRestoreAfterViewItemJobStep) {
3077  Q_ASSERT(dyingMessage->isViewable());
3078  // Try to select the item below the removed one as it helps in doing a "readon" of emails:
3079  // you read a message, decide to delete it and then go to the next.
3080  // Qt tends to select the message above the removed one instead (this is a hardcoded logic in
3081  // QItemSelectionModelPrivate::_q_rowsAboutToBeRemoved()).
3082  mCurrentItemToRestoreAfterViewItemJobStep = mView->messageItemAfter(dyingMessage, MessageTypeAny, false);
3083 
3084  if (!mCurrentItemToRestoreAfterViewItemJobStep) {
3085  // There is no item below. Try the item above.
3086  // We still do it better than qt which tends to find the *thread* above
3087  // instead of the item above.
3088  mCurrentItemToRestoreAfterViewItemJobStep = mView->messageItemBefore(dyingMessage, MessageTypeAny, false);
3089  }
3090 
3091  Q_ASSERT((!mCurrentItemToRestoreAfterViewItemJobStep) || mCurrentItemToRestoreAfterViewItemJobStep->isViewable());
3092  }
3093 
3094  if (dyingMessage->isViewable() && ((dyingMessage)->childItemCount() > 0) // has children
3095  && mView->isExpanded(q->index(dyingMessage, 0)) // is actually expanded
3096  ) {
3097  saveExpandedStateOfSubtree(dyingMessage);
3098  }
3099 
3100  auto oldParent = dyingMessage->parent();
3101  oldParent->takeChildItem(q, dyingMessage);
3102 
3103  // FIXME: This can generate many message movements.. it would be nicer
3104  // to start from messages that are higher in the hierarchy so
3105  // we would need to move less stuff above.
3106 
3107  if (oldParent != mRootItem) {
3108  messageDetachedUpdateParentProperties(oldParent, dyingMessage);
3109  }
3110 
3111  // We might have already removed its parent from the view, so it
3112  // might already be in the orphan child hash...
3113  if (dyingMessage->threadingStatus() == MessageItem::ParentMissing) {
3114  mOrphanChildrenHash.remove(dyingMessage); // this can turn to a no-op (dyingMessage not present in fact)
3115  }
3116  } else {
3117  // The dying message had no parent: this should happen only if it's already an orphan
3118 
3119  Q_ASSERT(dyingMessage->threadingStatus() == MessageItem::ParentMissing);
3120  Q_ASSERT(mOrphanChildrenHash.contains(dyingMessage));
3121  Q_ASSERT(dyingMessage != mCurrentItemToRestoreAfterViewItemJobStep);
3122 
3123  mOrphanChildrenHash.remove(dyingMessage);
3124  }
3125 
3126  if (mAggregation->threading() != Aggregation::NoThreading) {
3127  // Threading is requested: remove the message from threading caches.
3128 
3129  // Remove from the cache of potential parent items
3130  mThreadingCacheMessageIdMD5ToMessageItem.remove(dyingMessage->messageIdMD5());
3131 
3132  // If we also have a cache for subject/reference-based threading then remove the message from there too
3133  if (mAggregation->threading() == Aggregation::PerfectReferencesAndSubject) {
3134  removeMessageFromReferencesBasedThreadingCache(dyingMessage);
3135  removeMessageFromSubjectBasedThreadingCache(dyingMessage);
3136  } else if (mAggregation->threading() == Aggregation::PerfectAndReferences) {
3137  removeMessageFromReferencesBasedThreadingCache(dyingMessage);
3138  }
3139 
3140  // If this message wasn't perfectly parented then it might still be in another cache.
3141  switch (dyingMessage->threadingStatus()) {
3144  if (!dyingMessage->inReplyToIdMD5().isEmpty()) {
3145  mThreadingCacheMessageInReplyToIdMD5ToMessageItem.remove(dyingMessage->inReplyToIdMD5());
3146  }
3147  break;
3148  default:
3149  Q_ASSERT(!mThreadingCacheMessageInReplyToIdMD5ToMessageItem.contains(dyingMessage->inReplyToIdMD5(), dyingMessage));
3150  // make gcc happy
3151  break;
3152  }
3153  }
3154 
3155  while (auto childItem = dyingMessage->firstChildItem()) {
3156  auto childMessage = dynamic_cast<MessageItem *>(childItem);
3157  Q_ASSERT(childMessage);
3158 
3159  dyingMessage->takeChildItem(q, childMessage);
3160 
3161  if (mAggregation->threading() != Aggregation::NoThreading) {
3162  if (childMessage->threadingStatus() == MessageItem::PerfectParentFound) {
3163  // If the child message was perfectly parented then now it had
3164  // lost its perfect parent. Add to the cache of imperfectly parented.
3165  if (!childMessage->inReplyToIdMD5().isEmpty()) {
3166  Q_ASSERT(!mThreadingCacheMessageInReplyToIdMD5ToMessageItem.contains(childMessage->inReplyToIdMD5(), childMessage));
3167  mThreadingCacheMessageInReplyToIdMD5ToMessageItem.insert(childMessage->inReplyToIdMD5(), childMessage);
3168  }
3169  }
3170  }
3171 
3172  // Parent is gone
3173  childMessage->setThreadingStatus(MessageItem::ParentMissing);
3174 
3175  // If the child (or any message in its subtree) is going to be selected,
3176  // then we must immediately reattach it to a temporary group in order for the
3177  // selection to be preserved across multiple steps. Otherwise we could end
3178  // with the child-to-be-selected being non viewable at the end
3179  // of the view job step. Attach to a temporary group.
3180  if (
3181  // child is going to be re-selected
3182  (childMessage == mCurrentItemToRestoreAfterViewItemJobStep)
3183  || (
3184  // there is a message that is going to be re-selected
3185  mCurrentItemToRestoreAfterViewItemJobStep && // that message is in the childMessage subtree
3186  mCurrentItemToRestoreAfterViewItemJobStep->hasAncestor(childMessage))) {
3187  attachMessageToGroupHeader(childMessage);
3188 
3189  Q_ASSERT(childMessage->isViewable());
3190  }
3191 
3192  mOrphanChildrenHash.insert(childMessage, childMessage);
3193  }
3194 
3195  if (mNewestItem == dyingMessage) {
3196  mNewestItem = nullptr;
3197  }
3198  if (mOldestItem == dyingMessage) {
3199  mOldestItem = nullptr;
3200  }
3201 
3202  delete dyingMessage;
3203 
3204  curIndex++;
3205 
3206  // FIXME: Maybe we should check smaller steps here since the
3207  // code above can generate large message tree movements
3208  // for each single item we sweep in the invalidatedMessages list.
3209  if ((curIndex % mViewItemJobStepMessageCheckCount) == 0) {
3210  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
3211  if (curIndex <= endIndex) {
3212  job->setCurrentIndex(curIndex);
3213  return ViewItemJobInterrupted;
3214  }
3215  }
3216  }
3217  }
3218 
3219  // We looped over the entire deleted message list.
3220 
3221  job->setCurrentIndex(endIndex + 1);
3222 
3223  // A quick last cleaning pass: this is usually very fast so we don't have a real
3224  // Pass enumeration for it. We just include it as trailer of Pass1Cleanup to be executed
3225  // when job->currentIndex() > job->endIndex();
3226 
3227  // We move all the messages from the orphan child hash to the unassigned message
3228  // list and get them ready for the standard Pass2.
3229 
3230  auto it = mOrphanChildrenHash.begin();
3231  auto end = mOrphanChildrenHash.end();
3232 
3233  curIndex = 0;
3234 
3235  while (it != end) {
3236  mUnassignedMessageListForPass2.append(*it);
3237 
3238  it = mOrphanChildrenHash.erase(it);
3239 
3240  // This is still interruptible
3241 
3242  curIndex++;
3243 
3244  // FIXME: We could take "larger" steps here
3245  if ((curIndex % mViewItemJobStepMessageCheckCount) == 0) {
3246  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
3247  if (it != mOrphanChildrenHash.end()) {
3248  return ViewItemJobInterrupted;
3249  }
3250  }
3251  }
3252  }
3253 
3254  return ViewItemJobCompleted;
3255 }
3256 
3257 ModelPrivate::ViewItemJobResult ModelPrivate::viewItemJobStepInternalForJobPass1Update(ViewItemJob *job, QElapsedTimer elapsedTimer)
3258 {
3259  Q_ASSERT(mModelForItemFunctions); // UI must be not disconnected here
3260 
3261  // In this pass we simply update the MessageItem objects that are present in the job.
3262 
3263  // Note that this list in fact contains MessageItem objects (we need dynamic_cast<>).
3264  auto messagesThatNeedUpdate = job->invariantIndexList();
3265 
3266  // We don't shrink the messagesThatNeedUpdate because it's basically an array.
3267  // It's faster to traverse an array of N entries than to remove K>0 entries
3268  // one by one and to traverse the remaining N-K entries.
3269 
3270  // The begin index of our work
3271  int curIndex = job->currentIndex();
3272  // The end index of our work.
3273  int endIndex = job->endIndex();
3274 
3275  while (curIndex <= endIndex) {
3276  // Get the underlying storage message data...
3277  auto message = dynamic_cast<MessageItem *>(messagesThatNeedUpdate->at(curIndex));
3278  // This MUST NOT be null (otherwise we have a bug somewhere in this file).
3279  Q_ASSERT(message);
3280 
3281  int row = mInvariantRowMapper->modelInvariantIndexToModelIndexRow(message);
3282 
3283  if (row < 0) {
3284  // Must have been invalidated (so it's basically about to be deleted)
3285  Q_ASSERT(!message->isValid());
3286  // Skip it here.
3287  curIndex++;
3288  continue;
3289  }
3290 
3291  time_t prevDate = message->date();
3292  time_t prevMaxDate = message->maxDate();
3293  bool toDoStatus = message->status().isToAct();
3294  bool prevUnreadStatus = !message->status().isRead();
3295  bool prevImportantStatus = message->status().isImportant();
3296 
3297  // The subject/reference based threading cache is sorted by date: we must remove
3298  // the item and re-insert it since updateMessageItemData() may change the date too.
3299  if (mAggregation->threading() == Aggregation::PerfectReferencesAndSubject) {
3300  removeMessageFromReferencesBasedThreadingCache(message);
3301  removeMessageFromSubjectBasedThreadingCache(message);
3302  } else if (mAggregation->threading() == Aggregation::PerfectAndReferences) {
3303  removeMessageFromReferencesBasedThreadingCache(message);
3304  }
3305 
3306  // Do update
3307  mStorageModel->updateMessageItemData(message, row);
3308  QModelIndex idx = q->index(message, 0);
3309  Q_EMIT q->dataChanged(idx, idx);
3310 
3311  // Reinsert the item to the cache, if needed
3312  if (mAggregation->threading() == Aggregation::PerfectReferencesAndSubject) {
3313  addMessageToReferencesBasedThreadingCache(message);
3314  addMessageToSubjectBasedThreadingCache(message);
3315  } else if (mAggregation->threading() == Aggregation::PerfectAndReferences) {
3316  addMessageToReferencesBasedThreadingCache(message);
3317  }
3318 
3319  int propertyChangeMask = 0;
3320 
3321  if (prevDate != message->date()) {
3322  propertyChangeMask |= DateChanged;
3323  }
3324  if (prevMaxDate != message->maxDate()) {
3325  propertyChangeMask |= MaxDateChanged;
3326  }
3327  if (toDoStatus != message->status().isToAct()) {
3328  propertyChangeMask |= ActionItemStatusChanged;
3329  }
3330  if (prevUnreadStatus != (!message->status().isRead())) {
3331  propertyChangeMask |= UnreadStatusChanged;
3332  }
3333  if (prevImportantStatus != (!message->status().isImportant())) {
3334  propertyChangeMask |= ImportantStatusChanged;
3335  }
3336 
3337  if (propertyChangeMask) {
3338  // Some message data has changed
3339  // now we need to handle the changes that might cause re-grouping/re-sorting
3340  // and propagate them to the parents.
3341 
3342  Item *pParent = message->parent();
3343 
3344  if (pParent && (pParent != mRootItem)) {
3345  // The following function will return true if itemParent may be affected by the change.
3346  // If the itemParent isn't affected, we stop climbing.
3347  if (handleItemPropertyChanges(propertyChangeMask, pParent, message)) {
3348  Q_ASSERT(message->parent()); // handleItemPropertyChanges() must never leave an item detached
3349 
3350  // Note that actually message->parent() may be different than pParent since
3351  // handleItemPropertyChanges() may have re-grouped it.
3352 
3353  // Time to propagate up.
3354  propagateItemPropertiesToParent(message);
3355  }
3356  } // else there is no parent so the item isn't attached to the view: re-grouping/re-sorting not needed.
3357  } // else message data didn't change an there is nothing interesting to do
3358 
3359  // (re-)apply the filter, if needed
3360  if (mFilter && message->isViewable()) {
3361  // In all the other cases we (re-)apply the filter to the topmost subtree that this message is in.
3362  Item *pTopMostNonRoot = message->topmostNonRoot();
3363 
3364  Q_ASSERT(pTopMostNonRoot);
3365  Q_ASSERT(pTopMostNonRoot != mRootItem);
3366  Q_ASSERT(pTopMostNonRoot->parent() == mRootItem);
3367 
3368  // FIXME: The call below works, but it's expensive when we are updating
3369  // a lot of items with filtering enabled. This is because the updated
3370  // items are likely to be in the same subtree which we then filter multiple times.
3371  // A point for us is that when filtering there shouldn't be really many
3372  // items in the view so the user isn't going to update a lot of them at once...
3373  // Well... anyway, the alternative would be to write yet another
3374  // specialized routine that would update only the "message" item
3375  // above and climb up eventually hiding parents (without descending the sibling subtrees again).
3376  // If people complain about performance in this particular case I'll consider that solution.
3377 
3378  applyFilterToSubtree(pTopMostNonRoot, QModelIndex());
3379  } // otherwise there is no filter or the item isn't viewable: very likely
3380  // left detached while propagating property changes. Will filter it
3381  // on reattach.
3382 
3383  // Done updating this message
3384 
3385  curIndex++;
3386 
3387  // FIXME: Maybe we should check smaller steps here since the
3388  // code above can generate large message tree movements
3389  // for each single item we sweep in the messagesThatNeedUpdate list.
3390  if ((curIndex % mViewItemJobStepMessageCheckCount) == 0) {
3391  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
3392  if (curIndex <= endIndex) {
3393  job->setCurrentIndex(curIndex);
3394  return ViewItemJobInterrupted;
3395  }
3396  }
3397  }
3398  }
3399 
3400  return ViewItemJobCompleted;
3401 }
3402 
3403 ModelPrivate::ViewItemJobResult ModelPrivate::viewItemJobStepInternalForJob(ViewItemJob *job, QElapsedTimer elapsedTimer)
3404 {
3405  // This function does a timed chunk of work for a single Fill View job.
3406  // It attempts to process messages until a timeout forces it to return to the caller.
3407 
3408  // A macro would improve readability here but since this is a good point
3409  // to place debugger breakpoints then we need it explicitly.
3410  // A (template) helper would need to pass many parameters and would not be inlined...
3411 
3412  if (job->currentPass() == ViewItemJob::Pass1Fill) {
3413  // We're in Pass1Fill of the job.
3414  switch (viewItemJobStepInternalForJobPass1Fill(job, elapsedTimer)) {
3415  case ViewItemJobInterrupted:
3416  // current job interrupted by timeout: propagate status to caller
3417  return ViewItemJobInterrupted;
3418  break;
3419  case ViewItemJobCompleted:
3420  // pass 1 has been completed
3421  // # TODO: Refactor this, make it virtual or whatever, but switch == bad, code duplication etc
3422  job->setCurrentPass(ViewItemJob::Pass2);
3423  job->setStartIndex(0);
3424  job->setEndIndex(mUnassignedMessageListForPass2.count() - 1);
3425  // take care of small jobs which never timeout by themselves because
3426  // of a small number of messages. At the end of each job check
3427  // the time used and if we're timeoutting and there is another job
3428  // then interrupt.
3429  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
3430  return ViewItemJobInterrupted;
3431  } // else proceed with the next pass
3432  break;
3433  default:
3434  // This is *really* a BUG
3435  qCWarning(MESSAGELIST_LOG) << "ERROR: returned an invalid result";
3436  Q_ASSERT(false);
3437  break;
3438  }
3439  } else if (job->currentPass() == ViewItemJob::Pass1Cleanup) {
3440  // We're in Pass1Cleanup of the job.
3441  switch (viewItemJobStepInternalForJobPass1Cleanup(job, elapsedTimer)) {
3442  case ViewItemJobInterrupted:
3443  // current job interrupted by timeout: propagate status to caller
3444  return ViewItemJobInterrupted;
3445  break;
3446  case ViewItemJobCompleted:
3447  // pass 1 has been completed
3448  job->setCurrentPass(ViewItemJob::Pass2);
3449  job->setStartIndex(0);
3450  job->setEndIndex(mUnassignedMessageListForPass2.count() - 1);
3451  // take care of small jobs which never timeout by themselves because
3452  // of a small number of messages. At the end of each job check
3453  // the time used and if we're timeoutting and there is another job
3454  // then interrupt.
3455  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
3456  return ViewItemJobInterrupted;
3457  } // else proceed with the next pass
3458  break;
3459  default:
3460  // This is *really* a BUG
3461  qCWarning(MESSAGELIST_LOG) << "ERROR: returned an invalid result";
3462  Q_ASSERT(false);
3463  break;
3464  }
3465  } else if (job->currentPass() == ViewItemJob::Pass1Update) {
3466  // We're in Pass1Update of the job.
3467  switch (viewItemJobStepInternalForJobPass1Update(job, elapsedTimer)) {
3468  case ViewItemJobInterrupted:
3469  // current job interrupted by timeout: propagate status to caller
3470  return ViewItemJobInterrupted;
3471  break;
3472  case ViewItemJobCompleted:
3473  // pass 1 has been completed
3474  // Since Pass2, Pass3 and Pass4 are empty for an Update operation
3475  // we simply skip them. (TODO: Triple-verify this assertion...).
3476  job->setCurrentPass(ViewItemJob::Pass5);
3477  job->setStartIndex(0);
3478  job->setEndIndex(mGroupHeadersThatNeedUpdate.count() - 1);
3479  // take care of small jobs which never timeout by themselves because
3480  // of a small number of messages. At the end of each job check
3481  // the time used and if we're timeoutting and there is another job
3482  // then interrupt.
3483  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
3484  return ViewItemJobInterrupted;
3485  } // else proceed with the next pass
3486  break;
3487  default:
3488  // This is *really* a BUG
3489  qCWarning(MESSAGELIST_LOG) << "ERROR: returned an invalid result";
3490  Q_ASSERT(false);
3491  break;
3492  }
3493  }
3494 
3495  // Pass1Fill/Pass1Cleanup/Pass1Update has been already completed.
3496 
3497  if (job->currentPass() == ViewItemJob::Pass2) {
3498  // We're in Pass2 of the job.
3499  switch (viewItemJobStepInternalForJobPass2(job, elapsedTimer)) {
3500  case ViewItemJobInterrupted:
3501  // current job interrupted by timeout: propagate status to caller
3502  return ViewItemJobInterrupted;
3503  break;
3504  case ViewItemJobCompleted:
3505  // pass 2 has been completed
3506  job->setCurrentPass(ViewItemJob::Pass3);
3507  job->setStartIndex(0);
3508  job->setEndIndex(mUnassignedMessageListForPass3.count() - 1);
3509  // take care of small jobs which never timeout by themselves because
3510  // of a small number of messages. At the end of each job check
3511  // the time used and if we're timeoutting and there is another job
3512  // then interrupt.
3513  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
3514  return ViewItemJobInterrupted;
3515  }
3516  // else proceed with the next pass
3517  break;
3518  default:
3519  // This is *really* a BUG
3520  qCWarning(MESSAGELIST_LOG) << "ERROR: returned an invalid result";
3521  Q_ASSERT(false);
3522  break;
3523  }
3524  }
3525 
3526  if (job->currentPass() == ViewItemJob::Pass3) {
3527  // We're in Pass3 of the job.
3528  switch (viewItemJobStepInternalForJobPass3(job, elapsedTimer)) {
3529  case ViewItemJobInterrupted:
3530  // current job interrupted by timeout: propagate status to caller
3531  return ViewItemJobInterrupted;
3532  case ViewItemJobCompleted:
3533  // pass 3 has been completed
3534  job->setCurrentPass(ViewItemJob::Pass4);
3535  job->setStartIndex(0);
3536  job->setEndIndex(mUnassignedMessageListForPass4.count() - 1);
3537  // take care of small jobs which never timeout by themselves because
3538  // of a small number of messages. At the end of each job check
3539  // the time used and if we're timeoutting and there is another job
3540  // then interrupt.
3541  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
3542  return ViewItemJobInterrupted;
3543  }
3544  // else proceed with the next pass
3545  break;
3546  default:
3547  // This is *really* a BUG
3548  qCWarning(MESSAGELIST_LOG) << "ERROR: returned an invalid result";
3549  Q_ASSERT(false);
3550  break;
3551  }
3552  }
3553 
3554  if (job->currentPass() == ViewItemJob::Pass4) {
3555  // We're in Pass4 of the job.
3556  switch (viewItemJobStepInternalForJobPass4(job, elapsedTimer)) {
3557  case ViewItemJobInterrupted:
3558  // current job interrupted by timeout: propagate status to caller
3559  return ViewItemJobInterrupted;
3560  case ViewItemJobCompleted:
3561  // pass 4 has been completed
3562  job->setCurrentPass(ViewItemJob::Pass5);
3563  job->setStartIndex(0);
3564  job->setEndIndex(mGroupHeadersThatNeedUpdate.count() - 1);
3565  // take care of small jobs which never timeout by themselves because
3566  // of a small number of messages. At the end of each job check
3567  // the time used and if we're timeoutting and there is another job
3568  // then interrupt.
3569  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
3570  return ViewItemJobInterrupted;
3571  }
3572  // else proceed with the next pass
3573  break;
3574  default:
3575  // This is *really* a BUG
3576  qCWarning(MESSAGELIST_LOG) << "ERROR: returned an invalid result";
3577  Q_ASSERT(false);
3578  break;
3579  }
3580  }
3581 
3582  // Pass4 has been already completed. Proceed to Pass5.
3583  return viewItemJobStepInternalForJobPass5(job, elapsedTimer);
3584 }
3585 
3586 #ifdef KDEPIM_FOLDEROPEN_PROFILE
3587 
3588 // Namespace to collect all the vars and functions for KDEPIM_FOLDEROPEN_PROFILE
3589 namespace Stats
3590 {
3591 // Number of existing jobs/passes
3592 static const int numberOfPasses = ViewItemJob::LastIndex;
3593 
3594 // The pass in the last call of viewItemJobStepInternal(), used to detect when
3595 // a new pass starts
3596 static int lastPass = -1;
3597 
3598 // Total number of messages in the folder
3599 static int totalMessages;
3600 
3601 // Per-Job data
3602 static int numElements[numberOfPasses];
3603 static int totalTime[numberOfPasses];
3604 static int chunks[numberOfPasses];
3605 
3606 // Time, in msecs for some special operations
3607 static int expandingTreeTime;
3608 static int layoutChangeTime;
3609 
3610 // Descriptions of the job, for nicer debug output
3611 static const char *jobDescription[numberOfPasses] = {"Creating items from messages and simple threading",
3612  "Removing messages",
3613  "Updating messages",
3614  "Additional Threading",
3615  "Subject-Based threading",
3616  "Grouping",
3617  "Group resorting + cleanup"};
3618 
3619 // Timer to track time between start of first job and end of last job
3620 static QTime firstStartTime;
3621 
3622 // Timer to track time the current job takes
3623 static QTime currentJobStartTime;
3624 
3625 // Zeros the stats, to be called when the first job starts
3626 static void resetStats()
3627 {
3628  totalMessages = 0;
3629  layoutChangeTime = 0;
3630  expandingTreeTime = 0;
3631  lastPass = -1;
3632  for (int i = 0; i < numberOfPasses; ++i) {
3633  numElements[i] = 0;
3634  totalTime[i] = 0;
3635  chunks[i] = 0;
3636  }
3637 }
3638 } // namespace Stats
3639 
3640 void ModelPrivate::printStatistics()
3641 {
3642  using namespace Stats;
3643  int totalTotalTime = 0;
3644  int completeTime = firstStartTime.elapsed();
3645  for (int i = 0; i < numberOfPasses; ++i) {
3646  totalTotalTime += totalTime[i];
3647  }
3648 
3649  float msgPerSecond = totalMessages / (totalTotalTime / 1000.0f);
3650  float msgPerSecondComplete = totalMessages / (completeTime / 1000.0f);
3651 
3652  int messagesWithSameSubjectAvg = 0;
3653  int messagesWithSameSubjectMax = 0;
3654  for (const auto messages : std::as_const(mThreadingCacheMessageSubjectMD5ToMessageItem)) {
3655  if (messages->size() > messagesWithSameSubjectMax) {
3656  messagesWithSameSubjectMax = messages->size();
3657  }
3658  messagesWithSameSubjectAvg += messages->size();
3659  }
3660  messagesWithSameSubjectAvg = messagesWithSameSubjectAvg / (float)mThreadingCacheMessageSubjectMD5ToMessageItem.size();
3661 
3662  int totalThreads = 0;
3663  if (!mGroupHeaderItemHash.isEmpty()) {
3664  for (const GroupHeaderItem *groupHeader : std::as_const(mGroupHeaderItemHash)) {
3665  totalThreads += groupHeader->childItemCount();
3666  }
3667  } else {
3668  totalThreads = mRootItem->childItemCount();
3669  }
3670 
3671  qCDebug(MESSAGELIST_LOG) << "Finished filling the view with" << totalMessages << "messages";
3672  qCDebug(MESSAGELIST_LOG) << "That took" << totalTotalTime << "msecs inside the model and" << completeTime << "in total.";
3673  qCDebug(MESSAGELIST_LOG) << (totalTotalTime / (float)completeTime) * 100.0f << "percent of the time was spent in the model.";
3674  qCDebug(MESSAGELIST_LOG) << "Time for layoutChanged(), in msecs:" << layoutChangeTime << "(" << (layoutChangeTime / (float)totalTotalTime) * 100.0f
3675  << "percent )";
3676  qCDebug(MESSAGELIST_LOG) << "Time to expand tree, in msecs:" << expandingTreeTime << "(" << (expandingTreeTime / (float)totalTotalTime) * 100.0f
3677  << "percent )";
3678  qCDebug(MESSAGELIST_LOG) << "Number of messages per second in the model:" << msgPerSecond;
3679  qCDebug(MESSAGELIST_LOG) << "Number of messages per second in total:" << msgPerSecondComplete;
3680  qCDebug(MESSAGELIST_LOG) << "Number of threads:" << totalThreads;
3681  qCDebug(MESSAGELIST_LOG) << "Number of groups:" << mGroupHeaderItemHash.size();
3682  qCDebug(MESSAGELIST_LOG) << "Messages per thread:" << totalMessages / (float)totalThreads;
3683  qCDebug(MESSAGELIST_LOG) << "Threads per group:" << totalThreads / (float)mGroupHeaderItemHash.size();
3684  qCDebug(MESSAGELIST_LOG) << "Messages with the same subject:"
3685  << "Max:" << messagesWithSameSubjectMax << "Avg:" << messagesWithSameSubjectAvg;
3686  qCDebug(MESSAGELIST_LOG);
3687  qCDebug(MESSAGELIST_LOG) << "Now follows a breakdown of the jobs.";
3688  qCDebug(MESSAGELIST_LOG);
3689  for (int i = 0; i < numberOfPasses; ++i) {
3690  if (totalTime[i] == 0) {
3691  continue;
3692  }
3693  float elementsPerSecond = numElements[i] / (totalTime[i] / 1000.0f);
3694  float percent = totalTime[i] / (float)totalTotalTime * 100.0f;
3695  qCDebug(MESSAGELIST_LOG) << "----------------------------------------------";
3696  qCDebug(MESSAGELIST_LOG) << "Job" << i + 1 << "(" << jobDescription[i] << ")";
3697  qCDebug(MESSAGELIST_LOG) << "Share of complete time:" << percent << "percent";
3698  qCDebug(MESSAGELIST_LOG) << "Time in msecs:" << totalTime[i];
3699  qCDebug(MESSAGELIST_LOG) << "Number of elements:" << numElements[i]; // TODO: map of element string
3700  qCDebug(MESSAGELIST_LOG) << "Elements per second:" << elementsPerSecond;
3701  qCDebug(MESSAGELIST_LOG) << "Number of chunks:" << chunks[i];
3702  qCDebug(MESSAGELIST_LOG);
3703  }
3704 
3705  qCDebug(MESSAGELIST_LOG) << "==========================================================";
3706  resetStats();
3707 }
3708 
3709 #endif
3710 
3711 ModelPrivate::ViewItemJobResult ModelPrivate::viewItemJobStepInternal()
3712 {
3713  // This function does a timed chunk of work in our View Fill operation.
3714  // It attempts to do processing until it either runs out of jobs
3715  // to be done or a timeout forces it to interrupt and jump back to the caller.
3716 
3717  QElapsedTimer elapsedTimer;
3718  elapsedTimer.start();
3719 
3720  while (!mViewItemJobs.isEmpty()) {
3721  // Have a job to do.
3722  ViewItemJob *job = mViewItemJobs.constFirst();
3723 
3724 #ifdef KDEPIM_FOLDEROPEN_PROFILE
3725 
3726  // Here we check if an old job has just completed or if we are at the start of the
3727  // first job. We then initialize job data stuff and timers based on this.
3728 
3729  const int currentPass = job->currentPass();
3730  const bool firstChunk = currentPass != Stats::lastPass;
3731  if (currentPass != Stats::lastPass && Stats::lastPass != -1) {
3732  Stats::totalTime[Stats::lastPass] = Stats::currentJobStartTime.elapsed();
3733  }
3734  const bool firstJob = job->currentPass() == ViewItemJob::Pass1Fill && firstChunk;
3735  const int elements = job->endIndex() - job->startIndex();
3736  if (firstJob) {
3737  Stats::resetStats();
3738  Stats::totalMessages = elements;
3739  Stats::firstStartTime.restart();
3740  }
3741  if (firstChunk) {
3742  Stats::numElements[currentPass] = elements;
3743  Stats::currentJobStartTime.restart();
3744  }
3745  Stats::chunks[currentPass]++;
3746  Stats::lastPass = currentPass;
3747 
3748 #endif
3749 
3750  mViewItemJobStepIdleInterval = job->idleInterval();
3751  mViewItemJobStepChunkTimeout = job->chunkTimeout();
3752  mViewItemJobStepMessageCheckCount = job->messageCheckCount();
3753 
3754  if (job->disconnectUI()) {
3755  mModelForItemFunctions = nullptr; // disconnect the UI for this job
3756  Q_ASSERT(mLoading); // this must be true in the first job
3757  // FIXME: Should assert yet more that this is the very first job for this StorageModel
3758  // Asserting only mLoading is not enough as we could be using a two-jobs loading strategy
3759  // or this could be a job enqueued before the first job has completed.
3760  } else {
3761  // With a connected UI we need to avoid the view to update the scrollbars at EVERY insertion or expansion.
3762  // QTreeViewPrivate::updateScrollBars() is very expensive as it loops through ALL the items in the view every time.
3763  // We can't disable the function directly as it's hidden in the private data object of QTreeView
3764  // but we can disable the parent QTreeView::updateGeometries() instead.
3765  // We will trigger it "manually" at the end of the step.
3766  mView->ignoreUpdateGeometries(true);
3767 
3768  // Ok.. I know that this seems unbelieveable but disabling updates actually
3769  // causes a (significant) performance loss in most cases. This is probably because QTreeView
3770  // uses delayed layouts when updates are disabled which should be delayed but in
3771  // fact are "forced" by next item insertions. The delayed layout algorithm, then
3772  // is probably slower than the non-delayed one.
3773  // Disabling the paintEvent() doesn't seem to work either.
3774  // mView->setUpdatesEnabled( false );
3775  }
3776 
3777  switch (viewItemJobStepInternalForJob(job, elapsedTimer)) {
3778  case ViewItemJobInterrupted:
3779  // current job interrupted by timeout: will propagate status to caller
3780  // but before this, give some feedback to the user
3781 
3782  // FIXME: This is now inaccurate, think of something else
3783  switch (job->currentPass()) {
3784  case ViewItemJob::Pass1Fill:
3785  case ViewItemJob::Pass1Cleanup:
3786  case ViewItemJob::Pass1Update:
3787  Q_EMIT q->statusMessage(i18np("Processed 1 Message of %2",
3788  "Processed %1 Messages of %2",
3789  job->currentIndex() - job->startIndex(),
3790  job->endIndex() - job->startIndex() + 1));
3791  break;
3792  case ViewItemJob::Pass2:
3793  Q_EMIT q->statusMessage(i18np("Threaded 1 Message of %2",
3794  "Threaded %1 Messages of %2",
3795  job->currentIndex() - job->startIndex(),
3796  job->endIndex() - job->startIndex() + 1));
3797  break;
3798  case ViewItemJob::Pass3:
3799  Q_EMIT q->statusMessage(i18np("Threaded 1 Message of %2",
3800  "Threaded %1 Messages of %2",
3801  job->currentIndex() - job->startIndex(),
3802  job->endIndex() - job->startIndex() + 1));
3803  break;
3804  case ViewItemJob::Pass4:
3805  Q_EMIT q->statusMessage(i18np("Grouped 1 Thread of %2",
3806  "Grouped %1 Threads of %2",
3807  job->currentIndex() - job->startIndex(),
3808  job->endIndex() - job->startIndex() + 1));
3809  break;
3810  case ViewItemJob::Pass5:
3811  Q_EMIT q->statusMessage(i18np("Updated 1 Group of %2",
3812  "Updated %1 Groups of %2",
3813  job->currentIndex() - job->startIndex(),
3814  job->endIndex() - job->startIndex() + 1));
3815  break;
3816  default:
3817  break;
3818  }
3819 
3820  if (!job->disconnectUI()) {
3821  mView->ignoreUpdateGeometries(false);
3822  // explicit call to updateGeometries() here
3823  mView->updateGeometries();
3824  }
3825 
3826  return ViewItemJobInterrupted;
3827  break;
3828  case ViewItemJobCompleted:
3829 
3830  // If this job worked with a disconnected UI, Q_EMIT layoutChanged()
3831  // to reconnect it. We go back to normal operation now.
3832  if (job->disconnectUI()) {
3833  mModelForItemFunctions = q;
3834  // This call would destroy the expanded state of items.
3835  // This is why when mModelForItemFunctions was 0 we didn't actually expand them
3836  // but we just set a "ExpandNeeded" mark...
3837 #ifdef KDEPIM_FOLDEROPEN_PROFILE
3838  QTime layoutChangedTimer;
3839  layoutChangedTimer.start();
3840 #endif
3841  mView->modelAboutToEmitLayoutChanged();
3842  Q_EMIT q->layoutChanged();
3843  mView->modelEmittedLayoutChanged();
3844 
3845 #ifdef KDEPIM_FOLDEROPEN_PROFILE
3846  Stats::layoutChangeTime = layoutChangedTimer.elapsed();
3847  QTime expandingTime;
3848  expandingTime.start();
3849 #endif
3850 
3851  // expand all the items that need it in a single sweep
3852 
3853  // FIXME: This takes quite a lot of time, it could be made an interruptible job
3854 
3855  auto rootChildItems = mRootItem->childItems();
3856  if (rootChildItems) {
3857  for (const auto it : std::as_const(*rootChildItems)) {
3858  if (it->initialExpandStatus() == Item::ExpandNeeded) {
3859  syncExpandedStateOfSubtree(it);
3860  }
3861  }
3862  }
3863 #ifdef KDEPIM_FOLDEROPEN_PROFILE
3864  Stats::expandingTreeTime = expandingTime.elapsed();
3865 #endif
3866  } else {
3867  mView->ignoreUpdateGeometries(false);
3868  // explicit call to updateGeometries() here
3869  mView->updateGeometries();
3870  }
3871 
3872  // this job has been completed
3873  delete mViewItemJobs.takeFirst();
3874 
3875 #ifdef KDEPIM_FOLDEROPEN_PROFILE
3876  // Last job finished!
3877  Stats::totalTime[currentPass] = Stats::currentJobStartTime.elapsed();
3878  printStatistics();
3879 #endif
3880 
3881  // take care of small jobs which never timeout by themselves because
3882  // of a small number of messages. At the end of each job check
3883  // the time used and if we're timeoutting and there is another job
3884  // then interrupt.
3885  if ((elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) || (elapsedTimer.elapsed() < 0)) {
3886  if (!mViewItemJobs.isEmpty()) {
3887  return ViewItemJobInterrupted;
3888  }
3889  // else it's completed in fact
3890  } // else proceed with the next job
3891 
3892  break;
3893  default:
3894  // This is *really* a BUG
3895  qCWarning(MESSAGELIST_LOG) << "ERROR: returned an invalid result";
3896  Q_ASSERT(false);
3897  break;
3898  }
3899  }
3900 
3901  // no more jobs
3902 
3903  Q_EMIT q->statusMessage(i18nc("@info:status Finished view fill", "Ready"));
3904 
3905  return ViewItemJobCompleted;
3906 }
3907 
3908 void ModelPrivate::viewItemJobStep()
3909 {
3910  // A single step in the View Fill operation.
3911  // This function wraps viewItemJobStepInternal() which does the step job
3912  // and either completes it or stops because of a timeout.
3913  // If the job is stopped then we start a zero-msecs timer to call us
3914  // back and resume the job. Otherwise we're just done.
3915 
3916  mViewItemJobStepStartTime = ::time(nullptr);
3917 
3918  if (mFillStepTimer.isActive()) {
3919  mFillStepTimer.stop();
3920  }
3921 
3922  if (!mStorageModel) {
3923  return; // nothing more to do
3924  }
3925 
3926  // Save the current item in the view as our process may
3927  // cause items to be reparented (and QTreeView will forget the current item in the meantime).
3928  // This machinery is also needed when we're about to remove items from the view in
3929  // a cleanup job: we'll be trying to set as current the item after the one removed.
3930 
3931  QModelIndex currentIndexBeforeStep = mView->currentIndex();
3932  Item *currentItemBeforeStep = currentIndexBeforeStep.isValid() ? static_cast<Item *>(currentIndexBeforeStep.internalPointer()) : nullptr;
3933 
3934  // mCurrentItemToRestoreAfterViewItemJobStep will be zeroed out if it's killed
3935  mCurrentItemToRestoreAfterViewItemJobStep = currentItemBeforeStep;
3936 
3937  // Save the current item position in the viewport as QTreeView fails to keep
3938  // the current item in the sample place when items are added or removed...
3939  QRect rectBeforeViewItemJobStep;
3940 
3941  const bool lockView = mView->isScrollingLocked();
3942 
3943  // This is generally SLOW AS HELL... (so we avoid it if we lock the view and thus don't need it)
3944  if (mCurrentItemToRestoreAfterViewItemJobStep && (!lockView)) {
3945  rectBeforeViewItemJobStep = mView->visualRect(currentIndexBeforeStep);
3946  }
3947 
3948  // FIXME: If the current item is NOT in the view, preserve the position
3949  // of the top visible item. This will make the view move yet less.
3950 
3951  // Insulate the View from (very likely spurious) "currentChanged()" signals.
3952  mView->ignoreCurrentChanges(true);
3953 
3954  // And go to real work.
3955  switch (viewItemJobStepInternal()) {
3956  case ViewItemJobInterrupted:
3957  // Operation timed out, need to resume in a while
3958  if (!mInLengthyJobBatch) {
3959  mInLengthyJobBatch = true;
3960  }
3961  mFillStepTimer.start(mViewItemJobStepIdleInterval); // this is a single shot timer connected to viewItemJobStep()
3962  // and go dealing with current/selection out of the switch.
3963  break;
3964  case ViewItemJobCompleted:
3965  // done :)
3966 
3967  Q_ASSERT(mModelForItemFunctions); // UI must be no (longer) disconnected in this state
3968 
3969  // Ask the view to remove the eventual busy indications
3970  if (mInLengthyJobBatch) {
3971  mInLengthyJobBatch = false;
3972  }
3973 
3974  if (mLoading) {
3975  mLoading = false;
3976  mView->modelFinishedLoading();
3977  slotApplyFilter();
3978  }
3979 
3980  // Apply pre-selection, if any
3981  if (mPreSelectionMode != PreSelectNone) {
3982  mView->ignoreCurrentChanges(false);
3983 
3984  bool bSelectionDone = false;
3985 
3986  switch (mPreSelectionMode) {
3987  case PreSelectLastSelected:
3988  // fall down
3989  break;
3990  case PreSelectFirstUnreadCentered:
3991  bSelectionDone = mView->selectFirstMessageItem(MessageTypeUnreadOnly, true); // center
3992  break;
3993  case PreSelectOldestCentered:
3994  mView->setCurrentMessageItem(mOldestItem, true /* center */);
3995  bSelectionDone = true;
3996  break;
3997  case PreSelectNewestCentered:
3998  mView->setCurrentMessageItem(mNewestItem, true /* center */);
3999  bSelectionDone = true;
4000  break;
4001  case PreSelectNone:
4002  // deal with selection below
4003  break;
4004  default:
4005  qCWarning(MESSAGELIST_LOG) << "ERROR: Unrecognized pre-selection mode " << static_cast<int>(mPreSelectionMode);
4006  break;
4007  }
4008 
4009  if ((!bSelectionDone) && (mPreSelectionMode != PreSelectNone)) {
4010  // fallback to last selected, if possible
4011  if (mLastSelectedMessageInFolder) { // we found it in the loading process: select and jump out
4012  mView->setCurrentMessageItem(mLastSelectedMessageInFolder);
4013  bSelectionDone = true;
4014  }
4015  }
4016 
4017  if (bSelectionDone) {
4018  mLastSelectedMessageInFolder = nullptr;
4019  mPreSelectionMode = PreSelectNone;
4020  return; // already taken care of current / selection
4021  }
4022  }
4023  // deal with current/selection out of the switch
4024 
4025  break;
4026  default:
4027  // This is *really* a BUG
4028  qCWarning(MESSAGELIST_LOG) << "ERROR: returned an invalid result";
4029  Q_ASSERT(false);
4030  break;
4031  }
4032 
4033  // Everything else here deals with the selection
4034 
4035  // If UI is disconnected then we don't have anything else to do here
4036  if (!mModelForItemFunctions) {
4037  mView->ignoreCurrentChanges(false);
4038  return;
4039  }
4040 
4041  // Restore current/selection and/or scrollbar position
4042 
4043  if (mCurrentItemToRestoreAfterViewItemJobStep) {
4044  bool stillIgnoringCurrentChanges = true;
4045 
4046  // If the assert below fails then the previously current item got detached
4047  // and didn't get reattached in the step: this should never happen.
4048  Q_ASSERT(mCurrentItemToRestoreAfterViewItemJobStep->isViewable());
4049 
4050  // Check if the current item changed
4051  QModelIndex currentIndexAfterStep = mView->currentIndex();
4052  Item *currentAfterStep = currentIndexAfterStep.isValid() ? static_cast<Item *>(currentIndexAfterStep.internalPointer()) : nullptr;
4053 
4054  if (mCurrentItemToRestoreAfterViewItemJobStep != currentAfterStep) {
4055  // QTreeView lost the current item...
4056  if (mCurrentItemToRestoreAfterViewItemJobStep != currentItemBeforeStep) {
4057  // Some view job code expects us to actually *change* the current item.
4058  // This is done by the cleanup step which removes items and tries
4059  // to set as current the item *after* the removed one, if possible.
4060  // We need the view to handle the change though.
4061  stillIgnoringCurrentChanges = false;
4062  mView->ignoreCurrentChanges(false);
4063  } else {
4064  // we just have to restore the old current item. The code
4065  // outside shouldn't have noticed that we lost it (e.g. the message viewer
4066  // still should have the old message opened). So we don't need to
4067  // actually notify the view of the restored setting.
4068  }
4069  // Restore it
4070  qCDebug(MESSAGELIST_LOG) << "Gonna restore current here" << mCurrentItemToRestoreAfterViewItemJobStep->subject();
4071  mView->setCurrentIndex(q->index(mCurrentItemToRestoreAfterViewItemJobStep, 0));
4072  } else {
4073  // The item we're expected to set as current is already current
4074  if (mCurrentItemToRestoreAfterViewItemJobStep != currentItemBeforeStep) {
4075  // But we have changed it in the job step.
4076  // This means that: we have deleted the current item and chosen a
4077  // new candidate as current but Qt also has chosen it as candidate
4078  // and already made it current. The problem is that (as of Qt 4.4)
4079  // it probably didn't select it.
4080  if (!mView->selectionModel()->hasSelection()) {
4081  stillIgnoringCurrentChanges = false;
4082  mView->ignoreCurrentChanges(false);
4083 
4084  qCDebug(MESSAGELIST_LOG) << "Gonna restore selection here" << mCurrentItemToRestoreAfterViewItemJobStep->subject();
4085 
4086  QItemSelection selection;
4087  selection.append(QItemSelectionRange(q->index(mCurrentItemToRestoreAfterViewItemJobStep, 0)));
4088  mView->selectionModel()->select(selection, QItemSelectionModel::Select | QItemSelectionModel::Rows);
4089  }
4090  }
4091  }
4092 
4093  // FIXME: If it was selected before the change, then re-select it (it may happen that it's not)
4094  if (!lockView) {
4095  // we prefer to keep the currently selected item steady in the view
4096  QRect rectAfterViewItemJobStep = mView->visualRect(q->index(mCurrentItemToRestoreAfterViewItemJobStep, 0));
4097  if (rectBeforeViewItemJobStep.y() != rectAfterViewItemJobStep.y()) {
4098  // QTreeView lost its position...
4099  mView->verticalScrollBar()->setValue(mView->verticalScrollBar()->value() + rectAfterViewItemJobStep.y() - rectBeforeViewItemJobStep.y());
4100  }
4101  }
4102 
4103  // and kill the insulation, if not yet done
4104  if (stillIgnoringCurrentChanges) {
4105  mView->ignoreCurrentChanges(false);
4106  }
4107 
4108  return;
4109  }
4110 
4111  // Either there was no current item before, or it was lost in a cleanup step and another candidate for
4112  // current item couldn't be found (possibly empty view)
4113  mView->ignoreCurrentChanges(false);
4114 
4115  if (currentItemBeforeStep) {
4116  // lost in a cleanup..
4117  // tell the view that we have a new current, this time with no insulation
4118  mView->slotSelectionChanged(QItemSelection(), QItemSelection());
4119  }
4120 }
4121 
4122 void ModelPrivate::slotStorageModelRowsInserted(const QModelIndex &parent, int from, int to)
4123 {
4124  if (parent.isValid()) {
4125  return; // ugh... should never happen
4126  }
4127 
4128  Q_ASSERT(from <= to);
4129 
4130  int count = (to - from) + 1;
4131 
4132  mInvariantRowMapper->modelRowsInserted(from, count);
4133 
4134  // look if no current job is in the middle
4135 
4136  int jobCount = mViewItemJobs.count();
4137 
4138  for (int idx = 0; idx < jobCount; idx++) {
4139  ViewItemJob *job = mViewItemJobs.at(idx);
4140 
4141  if (job->currentPass() != ViewItemJob::Pass1Fill) {
4142  // The job is a cleanup or in a later pass: the storage has been already accessed
4143  // and the messages created... no need to care anymore: the invariant row mapper will do the job.
4144  continue;
4145  }
4146 
4147  if (job->currentIndex() > job->endIndex()) {
4148  // The job finished the Pass1Fill but still waits for the pass indicator to be
4149  // changed. This is unlikely but still may happen if the job has been interrupted
4150  // and then a call to slotStorageModelRowsRemoved() caused it to be forcibly completed.
4151  continue;
4152  }
4153 
4154  //
4155  // The following cases are possible:
4156  //
4157  // from to
4158  // | | -> shift up job
4159  // from to
4160  // | | -> shift up job
4161  // from to
4162  // | | -> shift up job
4163  // from to
4164  // | | -> split job
4165  // from to
4166  // | | -> split job
4167  // from to
4168  // | | -> job unaffected
4169  //
4170  //
4171  // FOLDER
4172  // |-------------------------|---------|--------------|
4173  // 0 currentIndex endIndex count
4174  // +-- job --+
4175  //
4176 
4177  if (from > job->endIndex()) {
4178  // The change is completely above the job, the job is not affected
4179  continue;
4180  }
4181 
4182  if (from > job->currentIndex()) { // and from <= job->endIndex()
4183  // The change starts in the middle of the job in a way that it must be split in two.
4184  // The first part is unaffected by the shift and ranges from job->currentIndex() to from - 1.
4185  // The second part ranges from "from" to job->endIndex() that are now shifted up by count steps.
4186 
4187  // First add a new job for the second part.
4188  auto newJob = new ViewItemJob(from + count, job->endIndex() + count, job->chunkTimeout(), job->idleInterval(), job->messageCheckCount());
4189 
4190  Q_ASSERT(newJob->currentIndex() <= newJob->endIndex());
4191 
4192  idx++; // we can skip this job in the loop, it's already ok
4193  jobCount++; // and our range increases by one.
4194  mViewItemJobs.insert(idx, newJob);
4195 
4196  // Then limit the original job to the first part
4197  job->setEndIndex(from - 1);
4198 
4199  Q_ASSERT(job->currentIndex() <= job->endIndex());
4200 
4201  continue;
4202  }
4203 
4204  // The change starts below (or exactly on the beginning of) the job.
4205  // The job must be shifted up.
4206  job->setCurrentIndex(job->currentIndex() + count);
4207  job->setEndIndex(job->endIndex() + count);
4208 
4209  Q_ASSERT(job->currentIndex() <= job->endIndex());
4210  }
4211 
4212  bool newJobNeeded = true;
4213 
4214  // Try to attach to an existing fill job, if any.
4215  // To enforce consistency we can attach only if the Fill job
4216  // is the last one in the list (might be eventually *also* the first,
4217  // and even being already processed but we must make sure that there
4218  // aren't jobs _after_ it).
4219  if (jobCount > 0) {
4220  ViewItemJob *job = mViewItemJobs.at(jobCount - 1);
4221  if (job->currentPass() == ViewItemJob::Pass1Fill) {
4222  if (
4223  // The job ends just before the added rows
4224  (from == (job->endIndex() + 1)) && // The job didn't reach the end of Pass1Fill yet
4225  (job->currentIndex() <= job->endIndex())) {
4226  // We can still attach this :)
4227  job->setEndIndex(to);
4228  Q_ASSERT(job->currentIndex() <= job->endIndex());
4229  newJobNeeded = false;
4230  }
4231  }
4232  }
4233 
4234  if (newJobNeeded) {
4235  // FIXME: Should take timing options from aggregation here ?
4236  auto job = new ViewItemJob(from, to, 100, 50, 10);
4237  mViewItemJobs.append(job);
4238  }
4239 
4240  if (!mFillStepTimer.isActive()) {
4241  mFillStepTimer.start(mViewItemJobStepIdleInterval);
4242  }
4243 }
4244 
4245 void ModelPrivate::slotStorageModelRowsRemoved(const QModelIndex &parent, int from, int to)
4246 {
4247  // This is called when the underlying StorageModel emits the rowsRemoved signal.
4248 
4249  if (parent.isValid()) {
4250  return; // ugh... should never happen
4251  }
4252 
4253  // look if no current job is in the middle
4254 
4255  Q_ASSERT(from <= to);
4256 
4257  const int count = (to - from) + 1;
4258 
4259  int jobCount = mViewItemJobs.count();
4260 
4261  if (mRootItem && from == 0 && count == mRootItem->childItemCount() && jobCount == 0) {
4262  clear();
4263  return;
4264  }
4265 
4266  for (int idx = 0; idx < jobCount; idx++) {
4267  ViewItemJob *job = mViewItemJobs.at(idx);
4268 
4269  if (job->currentPass() != ViewItemJob::Pass1Fill) {
4270  // The job is a cleanup or in a later pass: the storage has been already accessed
4271  // and the messages created... no need to care: we will invalidate the messages in a while.
4272  continue;
4273  }
4274 
4275  if (job->currentIndex() > job->endIndex()) {
4276  // The job finished the Pass1Fill but still waits for the pass indicator to be
4277  // changed. This is unlikely but still may happen if the job has been interrupted
4278  // and then a call to slotStorageModelRowsRemoved() caused it to be forcibly completed.
4279  continue;
4280  }
4281 
4282  //
4283  // The following cases are possible:
4284  //
4285  // from to
4286  // | | -> shift down job
4287  // from to
4288  // | | -> shift down and crop job
4289  // from to
4290  // | | -> kill job
4291  // from to
4292  // | | -> split job, crop and shift
4293  // from to
4294  // | | -> crop job
4295  // from to
4296  // | | -> job unaffected
4297  //
4298  //
4299  // FOLDER
4300  // |-------------------------|---------|--------------|
4301  // 0 currentIndex endIndex count
4302  // +-- job --+
4303  //
4304 
4305  if (from > job->endIndex()) {
4306  // The change is completely above the job, the job is not affected
4307  continue;
4308  }
4309 
4310  if (from > job->currentIndex()) { // and from <= job->endIndex()
4311  // The change starts in the middle of the job and ends in the middle or after the job.
4312  // The first part is unaffected by the shift and ranges from job->currentIndex() to from - 1
4313  // We use the existing job for this.
4314  job->setEndIndex(from - 1); // stop before the first removed row
4315 
4316  Q_ASSERT(job->currentIndex() <= job->endIndex());
4317 
4318  if (to < job->endIndex()) {
4319  // The change ends inside the job and a part of it can be completed.
4320 
4321  // We create a new job for the shifted remaining part. It would actually
4322  // range from to + 1 up to job->endIndex(), but we need to shift it down by count.
4323  // since count = ( to - from ) + 1 so from = to + 1 - count
4324 
4325  auto newJob = new ViewItemJob(from, job->endIndex() - count, job->chunkTimeout(), job->idleInterval(), job->messageCheckCount());
4326 
4327  Q_ASSERT(newJob->currentIndex() < newJob->endIndex());
4328 
4329  idx++; // we can skip this job in the loop, it's already ok
4330  jobCount++; // and our range increases by one.
4331  mViewItemJobs.insert(idx, newJob);
4332  } // else the change includes completely the end of the job and no other part of it can be completed.
4333 
4334  continue;
4335  }
4336 
4337  // The change starts below (or exactly on the beginning of) the job. ( from <= job->currentIndex() )
4338  if (to >= job->endIndex()) {
4339  // The change completely covers the job: kill it
4340 
4341  // We don't delete the job since we want the other passes to be completed
4342  // This is because the Pass1Fill may have already filled mUnassignedMessageListForPass2
4343  // and may have set mOldestItem and mNewestItem. We *COULD* clear the unassigned
4344  // message list with clearUnassignedMessageLists() but mOldestItem and mNewestItem
4345  // could be still dangling pointers. So we just move the current index of the job
4346  // after the end (so storage model scan terminates) and let it complete spontaneously.
4347  job->setCurrentIndex(job->endIndex() + 1);
4348 
4349  continue;
4350  }
4351 
4352  if (to >= job->currentIndex()) {
4353  // The change partially covers the job. Only a part of it can be completed
4354  // and it must be shifted down. It would actually
4355  // range from to + 1 up to job->endIndex(), but we need to shift it down by count.
4356  // since count = ( to - from ) + 1 so from = to + 1 - count
4357  job->setCurrentIndex(from);
4358  job->setEndIndex(job->endIndex() - count);
4359 
4360  Q_ASSERT(job->currentIndex() <= job->endIndex());
4361 
4362  continue;
4363  }
4364 
4365  // The change is completely below the job: it must be shifted down.
4366  job->setCurrentIndex(job->currentIndex() - count);
4367  job->setEndIndex(job->endIndex() - count);
4368  }
4369 
4370  // This will invalidate the ModelInvariantIndex-es that have been removed and return
4371  // them all in a nice list that we can feed to a view removal job.
4372  auto invalidatedIndexes = mInvariantRowMapper->modelRowsRemoved(from, count);
4373 
4374  if (invalidatedIndexes) {
4375  // Try to attach to an existing cleanup job, if any.
4376  // To enforce consistency we can attach only if the Cleanup job
4377  // is the last one in the list (might be eventually *also* the first,
4378  // and even being already processed but we must make sure that there
4379  // aren't jobs _after_ it).
4380  if (jobCount > 0) {
4381  ViewItemJob *job = mViewItemJobs.at(jobCount - 1);
4382  if (job->currentPass() == ViewItemJob::Pass1Cleanup) {
4383  if ((job->currentIndex() <= job->endIndex()) && job->invariantIndexList()) {
4384  // qCDebug(MESSAGELIST_LOG) << "Appending " << invalidatedIndexes->count() << " invalidated indexes to existing cleanup job";
4385  // We can still attach this :)
4386  *(job->invariantIndexList()) += *invalidatedIndexes;
4387  job->setEndIndex(job->endIndex() + invalidatedIndexes->count());
4388  delete invalidatedIndexes;
4389  invalidatedIndexes = nullptr;
4390  }
4391  }
4392  }
4393 
4394  if (invalidatedIndexes) {
4395  // Didn't append to any existing cleanup job.. create a new one
4396 
4397  // qCDebug(MESSAGELIST_LOG) << "Creating new cleanup job for " << invalidatedIndexes->count() << " invalidated indexes";
4398  // FIXME: Should take timing options from aggregation here ?
4399  auto job = new ViewItemJob(ViewItemJob::Pass1Cleanup, invalidatedIndexes, 100, 50, 10);
4400  mViewItemJobs.append(job);
4401  }
4402 
4403  if (!mFillStepTimer.isActive()) {
4404  mFillStepTimer.start(mViewItemJobStepIdleInterval);
4405  }
4406  }
4407 }
4408 
4409 void ModelPrivate::slotStorageModelLayoutChanged()
4410 {
4411  qCDebug(MESSAGELIST_LOG) << "Storage model layout changed";
4412  // need to reset everything...
4413  q->setStorageModel(mStorageModel);
4414  qCDebug(MESSAGELIST_LOG) << "Storage model layout changed done";
4415 }
4416 
4417 void ModelPrivate::slotStorageModelDataChanged(const QModelIndex &fromIndex, const QModelIndex &toIndex)
4418 {
4419  Q_ASSERT(mStorageModel); // must exist (and be the sender of the signal connected to this slot)
4420 
4421  int from = fromIndex.row();
4422  int to = toIndex.row();
4423 
4424  Q_ASSERT(from <= to);
4425 
4426  int count = (to - from) + 1;
4427 
4428  int jobCount = mViewItemJobs.count();
4429 
4430  // This will find out the ModelInvariantIndex-es that need an update and will return
4431  // them all in a nice list that we can feed to a view removal job.
4432  auto indexesThatNeedUpdate = mInvariantRowMapper->modelIndexRowRangeToModelInvariantIndexList(from, count);
4433 
4434  if (indexesThatNeedUpdate) {
4435  // Try to attach to an existing update job, if any.
4436  // To enforce consistency we can attach only if the Update job
4437  // is the last one in the list (might be eventually *also* the first,
4438  // and even being already processed but we must make sure that there
4439  // aren't jobs _after_ it).
4440  if (jobCount > 0) {
4441  ViewItemJob *job = mViewItemJobs.at(jobCount - 1);
4442  if (job->currentPass() == ViewItemJob::Pass1Update) {
4443  if ((job->currentIndex() <= job->endIndex()) && job->invariantIndexList()) {
4444  // We can still attach this :)
4445  *(job->invariantIndexList()) += *indexesThatNeedUpdate;
4446  job->setEndIndex(job->endIndex() + indexesThatNeedUpdate->count());
4447  delete indexesThatNeedUpdate;
4448  indexesThatNeedUpdate = nullptr;
4449  }
4450  }
4451  }
4452 
4453  if (indexesThatNeedUpdate) {
4454  // Didn't append to any existing update job.. create a new one
4455  // FIXME: Should take timing options from aggregation here ?
4456  auto job = new ViewItemJob(ViewItemJob::Pass1Update, indexesThatNeedUpdate, 100, 50, 10);
4457  mViewItemJobs.append(job);
4458  }
4459 
4460  if (!mFillStepTimer.isActive()) {
4461  mFillStepTimer.start(mViewItemJobStepIdleInterval);
4462  }
4463  }
4464 }
4465 
4466 void ModelPrivate::slotStorageModelHeaderDataChanged(Qt::Orientation, int, int)
4467 {
4468  if (mStorageModelContainsOutboundMessages != mStorageModel->containsOutboundMessages()) {
4469  mStorageModelContainsOutboundMessages = mStorageModel->containsOutboundMessages();
4470  Q_EMIT q->headerDataChanged(Qt::Horizontal, 0, q->columnCount());
4471  }
4472 }
4473 
4474 Qt::ItemFlags Model::flags(const QModelIndex &index) const
4475 {
4476  if (!index.isValid()) {
4477  return Qt::NoItemFlags;
4478  }
4479 
4480  Q_ASSERT(d->mModelForItemFunctions); // UI must be connected if a valid index was queried
4481 
4482  Item *it = static_cast<Item *>(index.internalPointer());
4483 
4484  Q_ASSERT(it);
4485 
4486  if (it->type() == Item::GroupHeader) {
4487  return Qt::ItemIsEnabled;
4488  }
4489 
4490  Q_ASSERT(it->type() == Item::Message);
4491 
4492  if (!static_cast<MessageItem *>(it)->isValid()) {
4493  return Qt::NoItemFlags; // not enabled, not selectable
4494  }
4495 
4496  if (static_cast<MessageItem *>(it)->aboutToBeRemoved()) {
4497  return Qt::NoItemFlags; // not enabled, not selectable
4498  }
4499 
4500  if (static_cast<MessageItem *>(it)->status().isDeleted()) {
4501  return Qt::NoItemFlags; // not enabled, not selectable
4502  }
4503 
4505 }
4506 
4507 QMimeData *MessageList::Core::Model::mimeData(const QModelIndexList &indexes) const
4508 {
4510  for (const QModelIndex &idx : indexes) {
4511  if (idx.isValid()) {
4512  Item *item = static_cast<Item *>(idx.internalPointer());
4513  if (item->type() == MessageList::Core::Item::Message) {
4514  msgs << static_cast<MessageItem *>(idx.internalPointer());
4515  }
4516  }
4517  }
4518  return storageModel()->mimeData(msgs);
4519 }
4520 
4522 {
4523  return d->mRootItem;
4524 }
4525 
4526 bool Model::isLoading() const
4527 {
4528  return d->mLoading;
4529 }
4530 
4532 {
4533  if (!d->mStorageModel) {
4534  return nullptr;
4535  }
4536  auto idx = d->mInvariantRowMapper->modelIndexRowToModelInvariantIndex(row);
4537  if (!idx) {
4538  return nullptr;
4539  }
4540 
4541  return static_cast<MessageItem *>(idx);
4542 }
4543 
4544 MessageItemSetReference Model::createPersistentSet(const QVector<MessageItem *> &items)
4545 {
4546  if (!d->mPersistentSetManager) {
4547  d->mPersistentSetManager = new MessageItemSetManager();
4548  }
4549 
4550  MessageItemSetReference ref = d->mPersistentSetManager->createSet();
4551  for (const auto mi : items) {
4552  d->mPersistentSetManager->addMessageItem(ref, mi);
4553  }
4554 
4555  return ref;
4556 }
4557 
4559 {
4560  if (d->mPersistentSetManager) {
4561  return d->mPersistentSetManager->messageItems(ref);
4562  }
4563  return {};
4564 }
4565 
4566 void Model::deletePersistentSet(MessageItemSetReference ref)
4567 {
4568  if (!d->mPersistentSetManager) {
4569  return;
4570  }
4571 
4572  d->mPersistentSetManager->removeSet(ref);
4573 
4574  if (d->mPersistentSetManager->setCount() < 1) {
4575  delete d->mPersistentSetManager;
4576  d->mPersistentSetManager = nullptr;
4577  }
4578 }
4579 
4580 #include "moc_model.cpp"
const QString & receiver() const
Returns the receiver associated to this item.
Definition: item.cpp:501
void append(const T &value)
@ SortMessagesByDateTime
Sort the messages by date and time.
Definition: sortorder.h:80
@ PerfectReferencesAndSubject
Thread by all of the above and try to match subjects too.
Definition: aggregation.h:69
@ GroupBySenderOrReceiver
Group by sender (incoming) or receiver (outgoing) field.
Definition: aggregation.h:40
size_t size() const
Returns the size of this item (size of the Message, mainly)
Definition: item.cpp:456
int month() const const
@ SortGroupsByReceiver
Sort groups by receiver (makes sense only with GroupByReceiver)
Definition: sortorder.h:65
const SortOrder * sortOrder() const
Returns the sort order.
Definition: model.cpp:384
The MessageItem class.
Definition: messageitem.h:34
@ GroupBySender
Group by sender, always.
Definition: aggregation.h:41
UserRole
AKONADICORE_EXPORT Collection fromIndex(const QModelIndex &index)
void setStorageModel(StorageModel *storageModel, PreSelectionMode preSelectionMode=PreSelectLastSelected)
Sets the storage model from that the messages to be displayed should be fetched.
Definition: model.cpp:740
QVariant fromValue(const T &value)
SortDirection
The "generic" sort direction: used for groups and for messages If you add values here please look at ...
Definition: sortorder.h:68
void * internalPointer() const const
The MessageList::View is the real display of the message list.
Definition: view.h:47
bool isViewable() const
Is this item attached to the viewable root ?
Definition: item.cpp:357
@ SortMessagesByReceiver
Sort the messages by receiver.
Definition: sortorder.h:84
@ SortMessagesByActionItemStatus
Sort the messages by the "Action Item" flag of status.
Definition: sortorder.h:87
int column() const const
void layoutChanged(const QList< QPersistentModelIndex > &parents, QAbstractItemModel::LayoutChangeHint hint)
@ NoGroupSorting
Don't sort the groups at all, add them as they come in.
Definition: sortorder.h:60
@ PerfectThreadingOnly
Only the data for messageIdMD5 and inReplyToMD5 is needed.
int year() const const
Qt::DayOfWeek firstDayOfWeek() const const
@ SortGroupsByDateTimeOfMostRecent
Sort groups by date/time of the most recent message.
Definition: sortorder.h:62
QIcon fromTheme(const QString &name)
static const MessageStatus statusWatched()
@ AlwaysExpandGroups
All groups are expanded as they are inserted.
Definition: aggregation.h:56
@ NoThreading
Perform no threading at all.
Definition: aggregation.h:66
Type type() const
Returns the type of this item.
Definition: item.cpp:342
Item * rootItem() const
Returns the hidden root item that all the messages are (or will be) attached to.
Definition: model.cpp:4521
This class is an optimizing helper for dealing with large flat QAbstractItemModel objects.
int y() const const
@ TopmostMessage
The thread grouping is computed from the topmost message (very similar to least recent,...
Definition: aggregation.h:79
@ ExpandRecentGroups
Makes sense only with GroupByDate or GroupByDateRange.
Definition: aggregation.h:55
void deletePersistentSet(MessageItemSetReference ref)
Deletes the persistent set pointed by the specified reference.
Definition: model.cpp:4566
void setStatus(Akonadi::MessageStatus status)
Sets the status associated to this Item.
Definition: item.cpp:451
void setSecsSinceEpoch(qint64 secs)
@ InvisibleRoot
This item is just Item and it's the only InvisibleRoot per Model.
Definition: item.h:59
QString standaloneMonthName(int month, QLocale::FormatType type) const const
KIOFILEWIDGETS_EXPORT QStringList list(const QString &fileClass)
Item * parent() const
Returns the parent Item in the tree, or 0 if this item isn't attached to the tree.
Definition: item.cpp:436
void setSortOrder(const SortOrder *sortOrder)
Sets the sort order.
Definition: model.cpp:379
@ PerfectOnly
Thread by "In-Reply-To" field only.
Definition: aggregation.h:67
QMetaObject::Connection connect(const QObject *sender, const char *signal, const QObject *receiver, const char *method, Qt::ConnectionType type)
@ GroupByDate
Group the messages by the date of the thread leader.
Definition: aggregation.h:38
@ NeverExpandGroups
Never expand groups during a view fill algorithm.
Definition: aggregation.h:54
@ ExpandThreadsWithUnreadMessages
Expand threads with unread messages (this includes new)
Definition: aggregation.h:93
QString displaySender() const
Display sender.
Definition: item.cpp:496
Q_GLOBAL_STATIC(Internal::StaticControl, s_instance) class ControlPrivate
@ SortMessagesBySender
Sort the messages by sender.
Definition: sortorder.h:83
@ NeverExpandThreads
Never expand any thread, this is fast.
Definition: aggregation.h:91
time_t maxDate() const
Returns the maximum date in the subtree originating from this item.
Definition: item.cpp:476
QLocale system()
void takeChildItem(Model *model, Item *child)
Removes a child from this item's child list without deleting it.
Definition: item.cpp:637
@ ImperfectParentFound
this message found an imperfect parent to attach to (might be fixed later)
Definition: messageitem.h:62
bool isWatched() const
QList< Item * > * childItems() const
Return the list of child items.
Definition: item.cpp:59
QModelIndex createIndex(int row, int column, void *ptr) const const
@ ExpandNeeded
Must expand when this item becomes viewable.
Definition: item.h:68
typedef ItemFlags
@ NoGrouping
Don't group messages at all.
Definition: aggregation.h:37
QString i18n(const char *text, const TYPE &arg...)
void dump(const QString &prefix)
Debug helper.
Definition: item.cpp:620
void dataChanged(const QModelIndex &topLeft, const QModelIndex &bottomRight, const QVector< int > &roles)
char * toString(const T &value)
bool isIgnored() const
@ AlwaysExpandThreads
Expand all threads (this might be very slow)
Definition: aggregation.h:94
Orientation
void timeout()
WaitCursor
QByteArray toUtf8() const const
Item * childItem(int idx) const
Returns the child item at position idx or 0 if idx is out of the allowable range.
Definition: item.cpp:64
@ NoMessageSorting
Don't sort the messages at all.
Definition: sortorder.h:79
const T & at(int i) const const
StorageModel * storageModel() const
Returns the StorageModel currently set.
Definition: model.cpp:690
@ PerfectThreadingReferencesAndSubject
All of the above plus subject stuff.
Q_SCRIPTABLE CaptureState status()
MessageItem * messageItemByStorageRow(int row) const
Returns the message item that is at the current storage row index or zero if no such storage item is ...
Definition: model.cpp:4531
QString toString(qlonglong i) const const
@ ExpandThreadsWithUnreadOrImportantMessages
Expand threads with "hot" messages (this includes new, unread, important, todo)
Definition: aggregation.h:95
bool isValid() const const
static const MessageStatus statusIgnored()
@ GroupByReceiver
Group by receiver, always.
Definition: aggregation.h:42
QString displaySenderOrReceiver() const
Display sender or receiver.
Definition: item.cpp:521
void headerDataChanged(Qt::Orientation orientation, int first, int last)
PreSelectionMode
Pre-selection is the action of automatically selecting a message just after the folder has finished l...
int childItemCount() const
Returns the number of children of this Item.
Definition: item.cpp:158
QDate currentDate()
A class which holds information about sorting, e.g.
Definition: sortorder.h:34
@ ParentMissing
this message might belong to a thread but its parent is actually missing
Definition: messageitem.h:63
QMimeData * mimeData(const QModelIndexList &indexes) const override
Called when user initiates a drag from the messagelist.
Definition: model.cpp:4507
@ SortMessagesBySize
Sort the messages by size.
Definition: sortorder.h:86
void setAggregation(const Aggregation *aggregation)
Sets the Aggregation mode.
Definition: model.cpp:368
bool useReceiver() const
Returns whether sender or receiver is supposed to be displayed.
Definition: item.cpp:526
qint64 elapsed() const const
bool isValid() const const
@ NoExpandNeeded
No expand needed at all.
Definition: item.h:69
@ SortGroupsBySenderOrReceiver
Sort groups by sender or receiver (makes sense only with GroupBySenderOrReceiver)
Definition: sortorder.h:63
@ SortMessagesByUnreadStatus
Sort the messages by the "Unread" flags of status.
Definition: sortorder.h:88
void setParent(Item *pParent)
Sets the parent for this item.
Definition: item.cpp:441
bool recomputeMaxDate()
Recompute the maximum date from the current children list.
Definition: item.cpp:323
@ MostRecentMessage
The thread grouping is computed from the most recent message.
Definition: aggregation.h:81
@ Message
This item is a MessageItem.
Definition: item.h:58
int row() const const
QString i18np(const char *singular, const char *plural, const TYPE &arg...)
QVariant data(const QModelIndex &index, int role=Qt::DisplayRole) const override
Definition: model.cpp:485
@ FavorSpeed
Do larger chunks of work, zero intervals between chunks.
Definition: aggregation.h:106
The Theme class defines the visual appearance of the MessageList.
Definition: theme.h:48
QString displayReceiver() const
Display receiver.
Definition: item.cpp:511
QAction * clear(const QObject *recvr, const char *slot, QObject *parent)
@ NonThreadable
this message does not look as being threadable
Definition: messageitem.h:64
bool isEmpty() const const
void start()
void rowsInserted(const QModelIndex &parent, int first, int last)
void setOverrideCursor(const QCursor &cursor)
QString & insert(int position, QChar ch)
int elapsed() const const
void rowsRemoved(const QModelIndex &parent, int first, int last)
bool isValid(QStringView ifopt)
@ SortMessagesByAttachmentStatus
Sort the messages By "Important" flags of status.
Definition: sortorder.h:90
QDate date() const const
void setPreSelectionMode(PreSelectionMode preSelect)
Sets the pre-selection mode.
Definition: model.cpp:1001
The QAbstractItemModel based interface that you need to provide for your storage to work with Message...
@ SortMessagesBySubject
Sort the messages by subject.
Definition: sortorder.h:85
@ PerfectParentFound
this message found a perfect parent to attach to
Definition: messageitem.h:61
QString i18nc(const char *context, const char *text, const TYPE &arg...)
QSet::iterator insert(const T &value)
@ BatchNoInteractivity
Do one large chunk, no interactivity at all.
Definition: aggregation.h:107
QList::iterator begin()
InitialExpandStatus initialExpandStatus() const
The initial expand status we have to honor when attaching to the viewable root.
Definition: item.cpp:347
const Akonadi::MessageStatus & status() const
Returns the status associated to this Item.
Definition: item.cpp:446
@ SortMessagesByDateTimeOfMostRecent
Sort the messages by date and time of the most recent message in subtree.
Definition: sortorder.h:81
@ FavorInteractivity
Do small chunks of work, small intervals between chunks to allow for UI event processing.
Definition: aggregation.h:105
@ ExpandExecuted
Item already expanded.
Definition: item.h:70
qint32 toQInt32() const
This class manages sets of messageitem references.
void setInitialExpandStatus(InitialExpandStatus initialExpandStatus)
Set the initial expand status we have to honor when attaching to the viewable root.
Definition: item.cpp:352
void restoreOverrideCursor()
This class is responsible of matching messages that should be displayed in the View.
Definition: filter.h:44
@ SortGroupsByDateTime
Sort groups by date/time of the group.
Definition: sortorder.h:61
time_t date() const
Returns the date of this item.
Definition: item.cpp:466
@ SortMessagesBySenderOrReceiver
Sort the messages by sender or receiver.
Definition: sortorder.h:82
int appendChildItem(Model *model, Item *child)
Appends an Item to this item's child list.
Definition: item.cpp:597
~Model() override
Destroys the mighty model along with the tree of items it manages.
Definition: model.cpp:350
A single item of the MessageList tree managed by MessageList::Model.
Definition: item.h:47
QList::iterator end()
int dayOfWeek() const const
@ SortGroupsBySender
Sort groups by sender (makes sense only with GroupBySender)
Definition: sortorder.h:64
QString standaloneDayName(int day, QLocale::FormatType type) const const
bool hasAncestor(const Item *it) const
Return true if Item pointed by it is an ancestor of this item (that is, if it is its parent,...
Definition: item.cpp:362
const QString & sender() const
Returns the sender associated to this item.
Definition: item.cpp:486
@ ExpandThreadsWithNewMessages
DEPRECATED. New message status no longer exists.
Definition: aggregation.h:92
@ PerfectAndReferences
Thread by "In-Reply-To" and "References" fields.
Definition: aggregation.h:68
void setFilter(const Filter *filter)
Sets the Filter to be applied on messages.
Definition: model.cpp:389
void setTheme(const Theme *theme)
Sets the Theme.
Definition: model.cpp:374
void setMaxDate(time_t date)
Sets the maximum date in the subtree originating from this item.
Definition: item.cpp:481
QList< MessageItem * > persistentSetCurrentMessageItemList(MessageItemSetReference ref)
Returns the list of MessageItems that are still existing in the set pointed by the specified referenc...
Definition: model.cpp:4558
A set of aggregation options that can be applied to the MessageList::Model in a single shot.
Definition: aggregation.h:28
bool isImportant() const
QObject * parent() const const
QString message
qint64 daysTo(const QDate &d) const const
@ GroupByDateRange
Use smart (thread leader) date ranges ("Today","Yesterday","Last Week"...)
Definition: aggregation.h:39
const QList< QKeySequence > & end()
const QString & subject() const
Returns the subject associated to this Item.
Definition: item.cpp:531
@ GroupHeader
This item is a GroupHeaderItem.
Definition: item.h:57
char * data()
@ PerfectThreadingPlusReferences
messageIdMD5, inReplyToMD5, referencesIdMD5
MessageItemSetReference createPersistentSet(const QVector< MessageItem * > &items)
Creates a persistent set for the specified MessageItems and returns its reference.
Definition: model.cpp:4544
bool isLoading() const
Returns true if the view is currently loading, that is it's in the first (possibly lengthy) job batch...
Definition: model.cpp:4526
This file is part of the KDE documentation.
Documentation copyright © 1996-2023 The KDE developers.
Generated on Mon Mar 27 2023 04:08:18 by doxygen 1.8.17 written by Dimitri van Heesch, © 1997-2006

KDE's Doxygen guidelines are available online.