Messagelib

model.cpp
1 /******************************************************************************
2  *
3  * SPDX-FileCopyrightText: 2008 Szymon Tomasz Stefanek <[email protected]>
4  *
5  * SPDX-License-Identifier: GPL-2.0-or-later
6  *
7  *******************************************************************************/
8 
9 //
10 // This class is a rather huge monster. It's something that resembles a QAbstractItemModel
11 // (because it has to provide the interface for a QTreeView) but isn't entirely one
12 // (for optimization reasons). It basically manages a tree of items of two types:
13 // GroupHeaderItem and MessageItem. Be sure to read the docs for ViewItemJob.
14 //
15 // A huge credit here goes to Till Adam which seems to have written most
16 // (if not all) of the original KMail threading code. The KMHeaders implementation,
17 // the documentation and his clever ideas were my starting points and essential tools.
18 // This is why I'm adding his copyright entry (copied from headeritem.cpp) here even if
19 // he didn't write a byte in this file until now :)
20 //
21 // Szymon Tomasz Stefanek, 03 Aug 2008 04:50 (am)
22 //
23 // This class contains ideas from:
24 //
25 // kmheaders.cpp / kmheaders.h, headeritem.cpp / headeritem.h
26 // Copyright: (c) 2004 Till Adam < adam at kde dot org >
27 //
28 #include "core/model.h"
29 #include "core/delegate.h"
30 #include "core/filter.h"
31 #include "core/groupheaderitem.h"
32 #include "core/item_p.h"
33 #include "core/manager.h"
34 #include "core/messageitem.h"
35 #include "core/messageitemsetmanager.h"
36 #include "core/model_p.h"
37 #include "core/modelinvariantrowmapper.h"
38 #include "core/storagemodelbase.h"
39 #include "core/theme.h"
40 #include "core/view.h"
41 #include "messagelist_debug.h"
42 #include <config-messagelist.h>
43 
44 #include "MessageCore/StringUtil"
45 #include <Akonadi/Item>
46 #include <Akonadi/MessageStatus>
47 
48 #include <KLocalizedString>
49 
50 #include <QApplication>
51 #include <QDateTime>
52 #include <QElapsedTimer>
53 #include <QIcon>
54 #include <QLocale>
55 #include <QScrollBar>
56 #include <QTimer>
57 
58 #include <algorithm>
59 #include <chrono>
60 
61 using namespace std::chrono_literals;
62 
63 namespace MessageList
64 {
65 namespace Core
66 {
67 Q_GLOBAL_STATIC(QTimer, _k_heartBeatTimer)
68 
69 /**
70  * A job in a "View Fill" or "View Cleanup" or "View Update" task.
71  *
72  * For a "View Fill" task a job is a set of messages
73  * that are contiguous in the storage. The set is expressed as a range
74  * of row indexes. The task "sweeps" the storage in the specified
75  * range, creates the appropriate Item instances and places them
76  * in the right position in the tree.
77  *
78  * The idea is that in a single instance and for the same StorageModel
79  * the jobs should never "cover" the same message twice. This assertion
80  * is enforced all around this source file.
81  *
82  * For a "View Cleanup" task the job is a list of ModelInvariantIndex
83  * objects (that are in fact MessageItem objects) that need to be removed
84  * from the view.
85  *
86  * For a "View Update" task the job is a list of ModelInvariantIndex
87  * objects (that are in fact MessageItem objects) that need to be updated.
88  *
89  * The interesting fact is that all the tasks need
90  * very similar operations to be performed on the message tree.
91  *
92  * For a "View Fill" we have 5 passes.
93  *
94  * Pass 1 scans the underlying storage, creates the MessageItem objects
95  * (which are subclasses of ModelInvariantIndex) and retrieves invariant
96  * storage indexes for them. It also builds threading caches and
97  * attempts to do some "easy" threading. If it succeeds in threading
98  * and some conditions apply then it also attaches the items to the view.
99  * Any unattached message is placed in a list.
100  *
101  * Pass 2 scans the list of messages that haven't been attached in
102  * the first pass and performs perfect and reference based threading.
103  * Since grouping of messages may depend on the "shape" of the thread
104  * then certain threads aren't attached to the view yet.
105  * Unassigned messages get stuffed into a list waiting for Pass3
106  * or directly to a list waiting for Pass4 (that is, Pass3 may be skipped
107  * if there is no hope to find an imperfect parent by subject based threading).
108  *
109  * Pass 3 scans the list of messages that haven't been attached in
110  * the first and second passes and performs subject based threading.
111  * Since grouping of messages may depend on the "shape" of the thread
112  * then certain threads aren't attached to the view yet.
113  * Anything unattached gets stuffed into the list waiting for Pass4.
114  *
115  * Pass 4 scans the unattached threads and puts them in the appropriate
116  * groups. After this pass nothing is unattached.
117  *
118  * Pass 5 eventually re-sorts the groups and removes the empty ones.
119  *
120  * For a "View Cleanup" we still have 5 passes.
121  *
122  * Pass 1 scans the list of invalidated ModelInvariantIndex-es, casts
123  * them to MessageItem objects and detaches them from the view.
124  * The orphan children of the destroyed items get stuffed in the list
125  * of unassigned messages that has been used also in the "View Fill" task above.
126  *
127  * Pass 2, 3, 4 and 5: same as "View Fill", just operating on the "orphaned"
128  * messages that need to be reattached to the view.
129  *
130  * For a "View Update" we still have 5 passes.
131  *
132  * Pass 1 scans the list of ModelInvariantIndex-es that need an update, casts
133  * them to MessageItem objects and handles the updates from storage.
134  * The updates may cause a regrouping so items might be stuffed in one
135  * of the lists for pass 4 or 5.
136  *
137  * Pass 2, 3 and 4 are simply empty.
138  *
139  * Pass 5: same as "View Fill", just operating on groups that require updates
140  * after the messages have been moved in pass 1.
141  *
142  * That's why we in fact have Pass1Fill, Pass1Cleanup, Pass1Update, Pass2, Pass3, Pass4 and Pass5 below.
143  * Pass1Fill, Pass1Cleanup and Pass1Update are exclusive and all of them proceed with Pass2 when finished.
144  */
145 class ViewItemJob
146 {
147 public:
148  enum Pass {
149  Pass1Fill = 0, ///< Build threading caches, *TRY* to do some threading, try to attach something to the view
150  Pass1Cleanup = 1, ///< Kill messages, build list of orphans
151  Pass1Update = 2, ///< Update messages
152  Pass2 = 3, ///< Thread everything by using caches, try to attach more to the view
153  Pass3 = 4, ///< Do more threading (this time try to guess), try to attach more to the view
154  Pass4 = 5, ///< Attach anything is still unattached
155  Pass5 = 6, ///< Eventually Re-sort group headers and remove the empty ones
156  LastIndex = 7 ///< Keep this at the end, needed to get the size of the enum
157  };
158 
159 private:
160  // Data for "View Fill" jobs
161  int mStartIndex; ///< The first index (in the underlying storage) of this job
162  int mCurrentIndex; ///< The current index (in the underlying storage) of this job
163  int mEndIndex; ///< The last index (in the underlying storage) of this job
164 
165  // Data for "View Cleanup" jobs
166  QList<ModelInvariantIndex *> *mInvariantIndexList; ///< Owned list of shallow pointers
167 
168  // Common data
169 
170  // The maximum time that we can spend "at once" inside viewItemJobStep() (milliseconds)
171  // The bigger this value, the larger chunks of work we do at once and less the time
172  // we loose in "breaking and resuming" the job. On the other side large values tend
173  // to make the view less responsive up to a "freeze" perception if this value is larger
174  // than 2000.
175  int mChunkTimeout;
176 
177  // The interval between two fillView steps. The larger the interval, the more interactivity
178  // we have. The shorter the interval the more work we get done per second.
179  int mIdleInterval;
180 
181  // The minimum number of messages we process in every viewItemJobStep() call
182  // The larger this value the less time we loose in checking the timeout every N messages.
183  // On the other side, making this very large may make the view less responsive
184  // if we're processing very few messages at a time and very high values (say > 10000) may
185  // eventually make our job unbreakable until the end.
186  int mMessageCheckCount;
187  Pass mCurrentPass;
188 
189  // If this parameter is true then this job uses a "disconnected" UI.
190  // It's FAR faster since we don't need to call beginInsertRows()/endInsertRows()
191  // and we simply Q_EMIT a layoutChanged() at the end. It can be done only as the first
192  // job though: subsequent jobs can't use layoutChanged() as it looses the expanded
193  // state of items.
194  bool mDisconnectUI;
195 
196 public:
197  /**
198  * Creates a "View Fill" operation job
199  */
200  ViewItemJob(int startIndex, int endIndex, int chunkTimeout, int idleInterval, int messageCheckCount, bool disconnectUI = false)
201  : mStartIndex(startIndex)
202  , mCurrentIndex(startIndex)
203  , mEndIndex(endIndex)
204  , mInvariantIndexList(nullptr)
205  , mChunkTimeout(chunkTimeout)
206  , mIdleInterval(idleInterval)
207  , mMessageCheckCount(messageCheckCount)
208  , mCurrentPass(Pass1Fill)
209  , mDisconnectUI(disconnectUI)
210  {
211  }
212 
213  /**
214  * Creates a "View Cleanup" or "View Update" operation job
215  */
216  ViewItemJob(Pass pass, QList<ModelInvariantIndex *> *invariantIndexList, int chunkTimeout, int idleInterval, int messageCheckCount)
217  : mStartIndex(0)
218  , mCurrentIndex(0)
219  , mEndIndex(invariantIndexList->count() - 1)
220  , mInvariantIndexList(invariantIndexList)
221  , mChunkTimeout(chunkTimeout)
222  , mIdleInterval(idleInterval)
223  , mMessageCheckCount(messageCheckCount)
224  , mCurrentPass(pass)
225  , mDisconnectUI(false)
226  {
227  }
228 
229  ~ViewItemJob()
230  {
231  delete mInvariantIndexList;
232  }
233 
234 public:
235  Q_REQUIRED_RESULT int startIndex() const
236  {
237  return mStartIndex;
238  }
239 
240  void setStartIndex(int startIndex)
241  {
242  mStartIndex = startIndex;
243  mCurrentIndex = startIndex;
244  }
245 
246  Q_REQUIRED_RESULT int currentIndex() const
247  {
248  return mCurrentIndex;
249  }
250 
251  void setCurrentIndex(int currentIndex)
252  {
253  mCurrentIndex = currentIndex;
254  }
255 
256  Q_REQUIRED_RESULT int endIndex() const
257  {
258  return mEndIndex;
259  }
260 
261  void setEndIndex(int endIndex)
262  {
263  mEndIndex = endIndex;
264  }
265 
266  Q_REQUIRED_RESULT Pass currentPass() const
267  {
268  return mCurrentPass;
269  }
270 
271  void setCurrentPass(Pass pass)
272  {
273  mCurrentPass = pass;
274  }
275 
276  Q_REQUIRED_RESULT int idleInterval() const
277  {
278  return mIdleInterval;
279  }
280 
281  Q_REQUIRED_RESULT int chunkTimeout() const
282  {
283  return mChunkTimeout;
284  }
285 
286  Q_REQUIRED_RESULT int messageCheckCount() const
287  {
288  return mMessageCheckCount;
289  }
290 
291  Q_REQUIRED_RESULT QList<ModelInvariantIndex *> *invariantIndexList() const
292  {
293  return mInvariantIndexList;
294  }
295 
296  Q_REQUIRED_RESULT bool disconnectUI() const
297  {
298  return mDisconnectUI;
299  }
300 };
301 } // namespace Core
302 } // namespace MessageList
303 
304 using namespace MessageList::Core;
305 
306 Model::Model(View *pParent)
307  : QAbstractItemModel(pParent)
308  , d(new ModelPrivate(this))
309 {
310  d->mRecursionCounterForReset = 0;
311  d->mStorageModel = nullptr;
312  d->mView = pParent;
313  d->mAggregation = nullptr;
314  d->mTheme = nullptr;
315  d->mSortOrder = nullptr;
316  d->mFilter = nullptr;
317  d->mPersistentSetManager = nullptr;
318  d->mInLengthyJobBatch = false;
319  d->mLastSelectedMessageInFolder = nullptr;
320  d->mLoading = false;
321 
322  d->mRootItem = new Item(Item::InvisibleRoot);
323  d->mRootItem->setViewable(nullptr, true);
324 
325  d->mFillStepTimer.setSingleShot(true);
326  d->mInvariantRowMapper = new ModelInvariantRowMapper();
327  d->mModelForItemFunctions = this;
328  connect(&d->mFillStepTimer, &QTimer::timeout, this, [this]() {
329  d->viewItemJobStep();
330  });
331 
332  d->mCachedTodayLabel = i18n("Today");
333  d->mCachedYesterdayLabel = i18n("Yesterday");
334  d->mCachedUnknownLabel = i18nc("Unknown date", "Unknown");
335  d->mCachedLastWeekLabel = i18n("Last Week");
336  d->mCachedTwoWeeksAgoLabel = i18n("Two Weeks Ago");
337  d->mCachedThreeWeeksAgoLabel = i18n("Three Weeks Ago");
338  d->mCachedFourWeeksAgoLabel = i18n("Four Weeks Ago");
339  d->mCachedFiveWeeksAgoLabel = i18n("Five Weeks Ago");
340 
341  d->mCachedWatchedOrIgnoredStatusBits = Akonadi::MessageStatus::statusIgnored().toQInt32() | Akonadi::MessageStatus::statusWatched().toQInt32();
342 
343  connect(_k_heartBeatTimer(), &QTimer::timeout, this, [this]() {
344  d->checkIfDateChanged();
345  });
346 
347  if (!_k_heartBeatTimer->isActive()) { // First model starts it
348  _k_heartBeatTimer->start(1min); // 1 minute
349  }
350 }
351 
353 {
354  setStorageModel(nullptr);
355 
356  d->clearJobList();
357  d->mOldestItem = nullptr;
358  d->mNewestItem = nullptr;
359  d->clearUnassignedMessageLists();
360  d->clearOrphanChildrenHash();
361  d->clearThreadingCacheReferencesIdMD5ToMessageItem();
362  d->clearThreadingCacheMessageSubjectMD5ToMessageItem();
363  delete d->mPersistentSetManager;
364  // Delete the invariant row mapper before removing the items.
365  // It's faster since the items will not need to call the invariant
366  delete d->mInvariantRowMapper;
367  delete d->mRootItem;
368 }
369 
370 void Model::setAggregation(const Aggregation *aggregation)
371 {
372  d->mAggregation = aggregation;
373  d->mView->setRootIsDecorated((d->mAggregation->grouping() == Aggregation::NoGrouping) && (d->mAggregation->threading() != Aggregation::NoThreading));
374 }
375 
376 void Model::setTheme(const Theme *theme)
377 {
378  d->mTheme = theme;
379 }
380 
381 void Model::setSortOrder(const SortOrder *sortOrder)
382 {
383  d->mSortOrder = sortOrder;
384 }
385 
387 {
388  return d->mSortOrder;
389 }
390 
391 void Model::setFilter(const Filter *filter)
392 {
393  d->mFilter = filter;
394 
395  if (d->mFilter) {
396  connect(d->mFilter, &Filter::finished, this, [this]() {
397  d->slotApplyFilter();
398  });
399  }
400 
401  d->slotApplyFilter();
402 }
403 
404 void ModelPrivate::slotApplyFilter()
405 {
406  auto childList = mRootItem->childItems();
407  if (!childList) {
408  return;
409  }
410 
411  QModelIndex idx; // invalid
412 
414  for (const auto child : std::as_const(*childList)) {
415  applyFilterToSubtree(child, idx);
416  }
417 
419 }
420 
421 bool ModelPrivate::applyFilterToSubtree(Item *item, const QModelIndex &parentIndex)
422 {
423  // This function applies the current filter (eventually empty)
424  // to a message tree starting at "item".
425 
426  if (!mModelForItemFunctions) {
427  qCWarning(MESSAGELIST_LOG) << "Cannot apply filter, the UI must be not disconnected.";
428  return true;
429  }
430  Q_ASSERT(item); // the item must obviously be valid
431  Q_ASSERT(item->isViewable()); // the item must be viewable
432 
433  // Apply to children first
434 
435  auto childList = item->childItems();
436 
437  bool childrenMatch = false;
438 
439  QModelIndex thisIndex = q->index(item, 0);
440 
441  if (childList) {
442  for (const auto child : std::as_const(*childList)) {
443  if (applyFilterToSubtree(child, thisIndex)) {
444  childrenMatch = true;
445  }
446  }
447  }
448 
449  if (!mFilter) { // empty filter always matches (but does not expand items)
450  mView->setRowHidden(thisIndex.row(), parentIndex, false);
451  return true;
452  }
453 
454  if (childrenMatch) {
455  mView->setRowHidden(thisIndex.row(), parentIndex, false);
456 
457  if (!mView->isExpanded(thisIndex)) {
458  mView->expand(thisIndex);
459  }
460  return true;
461  }
462 
463  if (item->type() == Item::Message) {
464  if (mFilter->match((MessageItem *)item)) {
465  mView->setRowHidden(thisIndex.row(), parentIndex, false);
466  return true;
467  }
468  } // else this is a group header and it never explicitly matches
469 
470  // filter doesn't match, hide the item
471  mView->setRowHidden(thisIndex.row(), parentIndex, true);
472 
473  return false;
474 }
475 
476 int Model::columnCount(const QModelIndex &parent) const
477 {
478  if (!d->mTheme) {
479  return 0;
480  }
481  if (parent.column() > 0) {
482  return 0;
483  }
484  return d->mTheme->columns().count();
485 }
486 
487 QVariant Model::data(const QModelIndex &index, int role) const
488 {
489  /// this is called only when Akonadi is using the selectionmodel
490  /// for item actions. since akonadi uses the ETM ItemRoles, and the
491  /// messagelist uses its own internal roles, here we respond
492  /// to the ETM ones.
493 
494  auto item = static_cast<Item *>(index.internalPointer());
495 
496  switch (role) {
497  /// taken from entitytreemodel.h
498  case Qt::UserRole + 1: // EntityTreeModel::ItemIdRole
499  if (item->type() == MessageList::Core::Item::Message) {
500  auto mItem = static_cast<MessageItem *>(item);
501  return QVariant::fromValue(mItem->akonadiItem().id());
502  } else {
503  return {};
504  }
505  break;
506  case Qt::UserRole + 2: // EntityTreeModel::ItemRole
507  if (item->type() == MessageList::Core::Item::Message) {
508  auto mItem = static_cast<MessageItem *>(item);
509  return QVariant::fromValue(mItem->akonadiItem());
510  } else {
511  return {};
512  }
513  break;
514  case Qt::UserRole + 3: // EntityTreeModel::MimeTypeRole
515  if (item->type() == MessageList::Core::Item::Message) {
516  return QStringLiteral("message/rfc822");
517  } else {
518  return {};
519  }
520  break;
522  if (item->type() == MessageList::Core::Item::Message) {
523  auto mItem = static_cast<MessageItem *>(item);
524  return mItem->accessibleText(d->mTheme, index.column());
525  } else if (item->type() == MessageList::Core::Item::GroupHeader) {
526  if (index.column() > 0) {
527  return QString();
528  }
529  auto hItem = static_cast<GroupHeaderItem *>(item);
530  return hItem->label();
531  }
532  return QString();
533  break;
534  default:
535  return {};
536  }
537 }
538 
539 QVariant Model::headerData(int section, Qt::Orientation, int role) const
540 {
541  if (!d->mTheme) {
542  return {};
543  }
544 
545  auto column = d->mTheme->column(section);
546  if (!column) {
547  return {};
548  }
549 
550  if (d->mStorageModel && column->isSenderOrReceiver() && (role == Qt::DisplayRole)) {
551  if (d->mStorageModelContainsOutboundMessages) {
552  return QVariant(i18n("Receiver"));
553  }
554  return QVariant(i18n("Sender"));
555  }
556 
557  const bool columnPixmapEmpty(column->pixmapName().isEmpty());
558  if ((role == Qt::DisplayRole) && columnPixmapEmpty) {
559  return QVariant(column->label());
560  } else if ((role == Qt::ToolTipRole) && !columnPixmapEmpty) {
561  return QVariant(column->label());
562  } else if ((role == Qt::DecorationRole) && !columnPixmapEmpty) {
563  return QVariant(QIcon::fromTheme(column->pixmapName()));
564  }
565 
566  return {};
567 }
568 
569 QModelIndex Model::index(Item *item, int column) const
570 {
571  if (!d->mModelForItemFunctions) {
572  return {}; // called with disconnected UI: the item isn't known on the Qt side, yet
573  }
574 
575  if (!item) {
576  return {};
577  }
578  // FIXME: This function is a bottleneck (the caching in indexOfChildItem only works 30% of the time)
579  auto par = item->parent();
580  if (!par) {
581  if (item != d->mRootItem) {
582  item->dump(QString());
583  }
584  return {};
585  }
586 
587  const int index = par->indexOfChildItem(item);
588  if (index < 0) {
589  return {}; // BUG
590  }
591  return createIndex(index, column, item);
592 }
593 
594 QModelIndex Model::index(int row, int column, const QModelIndex &parent) const
595 {
596  if (!d->mModelForItemFunctions) {
597  return {}; // called with disconnected UI: the item isn't known on the Qt side, yet
598  }
599 
600 #ifdef READD_THIS_IF_YOU_WANT_TO_PASS_MODEL_TEST
601  if (column < 0) {
602  return QModelIndex(); // senseless column (we could optimize by skipping this check but ModelTest from trolltech is pedantic)
603  }
604 #endif
605 
606  const Item *item;
607  if (parent.isValid()) {
608  item = static_cast<const Item *>(parent.internalPointer());
609  if (!item) {
610  return {}; // should never happen
611  }
612  } else {
613  item = d->mRootItem;
614  }
615 
616  if (parent.column() > 0) {
617  return {}; // parent column is not 0: shouldn't have children (as per Qt documentation)
618  }
619 
620  Item *child = item->childItem(row);
621  if (!child) {
622  return {}; // no such row in parent
623  }
624  return createIndex(row, column, child);
625 }
626 
627 QModelIndex Model::parent(const QModelIndex &modelIndex) const
628 {
629  Q_ASSERT(d->mModelForItemFunctions); // should be never called with disconnected UI
630 
631  if (!modelIndex.isValid()) {
632  return {}; // should never happen
633  }
634  auto item = static_cast<Item *>(modelIndex.internalPointer());
635  if (!item) {
636  return {};
637  }
638  auto par = item->parent();
639  if (!par) {
640  return {}; // should never happen
641  }
642  // return index( par, modelIndex.column() );
643  return index(par, 0); // parents are always in column 0 (as per Qt documentation)
644 }
645 
646 int Model::rowCount(const QModelIndex &parent) const
647 {
648  if (!d->mModelForItemFunctions) {
649  return 0; // called with disconnected UI
650  }
651 
652  const Item *item;
653  if (parent.isValid()) {
654  item = static_cast<const Item *>(parent.internalPointer());
655  if (!item) {
656  return 0; // should never happen
657  }
658  } else {
659  item = d->mRootItem;
660  }
661 
662  if (!item->isViewable()) {
663  return 0;
664  }
665 
666  return item->childItemCount();
667 }
668 
669 class RecursionPreventer
670 {
671 public:
672  RecursionPreventer(int &counter)
673  : mCounter(counter)
674  {
675  mCounter++;
676  }
677 
678  ~RecursionPreventer()
679  {
680  mCounter--;
681  }
682 
683  Q_REQUIRED_RESULT bool isRecursive() const
684  {
685  return mCounter > 1;
686  }
687 
688 private:
689  int &mCounter;
690 };
691 
693 {
694  return d->mStorageModel;
695 }
696 
697 void ModelPrivate::clear()
698 {
699  q->beginResetModel();
700  if (mFillStepTimer.isActive()) {
701  mFillStepTimer.stop();
702  }
703 
704  // Kill pre-selection at this stage
705  mPreSelectionMode = PreSelectNone;
706  mLastSelectedMessageInFolder = nullptr;
707  mOldestItem = nullptr;
708  mNewestItem = nullptr;
709 
710  // Reset the row mapper before removing items
711  // This is faster since the items don't need to access the mapper.
712  mInvariantRowMapper->modelReset();
713 
714  clearJobList();
715  clearUnassignedMessageLists();
716  clearOrphanChildrenHash();
717  mGroupHeaderItemHash.clear();
718  mGroupHeadersThatNeedUpdate.clear();
719  mThreadingCacheMessageIdMD5ToMessageItem.clear();
720  mThreadingCacheMessageInReplyToIdMD5ToMessageItem.clear();
721  clearThreadingCacheReferencesIdMD5ToMessageItem();
722  clearThreadingCacheMessageSubjectMD5ToMessageItem();
723  mViewItemJobStepChunkTimeout = 100;
724  mViewItemJobStepIdleInterval = 10;
725  mViewItemJobStepMessageCheckCount = 10;
726  delete mPersistentSetManager;
727  mPersistentSetManager = nullptr;
728  mCurrentItemToRestoreAfterViewItemJobStep = nullptr;
729 
730  mTodayDate = QDate::currentDate();
731 
732  // FIXME: CLEAR THE FILTER HERE AS WE CAN'T APPLY IT WITH UI DISCONNECTED!
733 
734  mRootItem->killAllChildItems();
735 
736  q->endResetModel();
737  // Q_EMIT headerDataChanged();
738 
739  mView->selectionModel()->clearSelection();
740 }
741 
742 void Model::setStorageModel(StorageModel *storageModel, PreSelectionMode preSelectionMode)
743 {
744  // Prevent a case of recursion when opening a folder that has a message and the folder was
745  // never opened before.
746  RecursionPreventer preventer(d->mRecursionCounterForReset);
747  if (preventer.isRecursive()) {
748  return;
749  }
750 
751  d->clear();
752 
753  if (d->mStorageModel) {
754  // Disconnect all signals from old storageModel
755  std::for_each(d->mStorageModelConnections.cbegin(), d->mStorageModelConnections.cend(), [](const QMetaObject::Connection &c) -> bool {
756  return QObject::disconnect(c);
757  });
758  d->mStorageModelConnections.clear();
759  }
760 
761  const bool isReload = (d->mStorageModel == storageModel);
762  d->mStorageModel = storageModel;
763 
764  if (!d->mStorageModel) {
765  return; // no folder: nothing to fill
766  }
767 
768  // Save threading cache of the previous folder, but only if the cache was
769  // enabled and a different folder is being loaded - reload of the same folder
770  // means change in aggregation in which case we will have to re-build the
771  // cache so there's no point saving the current threading cache.
772  if (d->mThreadingCache.isEnabled() && !isReload) {
773  d->mThreadingCache.save();
774  } else {
775  if (isReload) {
776  qCDebug(MESSAGELIST_LOG) << "Identical folder reloaded, not saving old threading cache";
777  } else {
778  qCDebug(MESSAGELIST_LOG) << "Threading disabled in previous folder, not saving threading cache";
779  }
780  }
781  // Load threading cache for the new folder, but only if threading is enabled,
782  // otherwise we would just be caching a flat list.
783  if (d->mAggregation->threading() != Aggregation::NoThreading) {
784  d->mThreadingCache.setEnabled(true);
785  d->mThreadingCache.load(d->mStorageModel->id(), d->mAggregation);
786  } else {
787  // No threading, no cache - don't even bother inserting entries into the
788  // cache or trying to look them up there
789  d->mThreadingCache.setEnabled(false);
790  qCDebug(MESSAGELIST_LOG) << "Threading disabled in folder" << d->mStorageModel->id() << ", not using threading cache";
791  }
792 
793  d->mPreSelectionMode = preSelectionMode;
794  d->mStorageModelContainsOutboundMessages = d->mStorageModel->containsOutboundMessages();
795 
796  d->mStorageModelConnections = {connect(d->mStorageModel,
798  this,
799  [this](const QModelIndex &parent, int first, int last) {
800  d->slotStorageModelRowsInserted(parent, first, last);
801  }),
802  connect(d->mStorageModel,
804  this,
805  [this](const QModelIndex &parent, int first, int last) {
806  d->slotStorageModelRowsRemoved(parent, first, last);
807  }),
808  connect(d->mStorageModel,
810  this,
811  [this]() {
812  d->slotStorageModelLayoutChanged();
813  }),
814  connect(d->mStorageModel,
816  this,
817  [this]() {
818  d->slotStorageModelLayoutChanged();
819  }),
820  connect(d->mStorageModel,
822  this,
823  [this](const QModelIndex &topLeft, const QModelIndex &bottomRight) {
824  d->slotStorageModelDataChanged(topLeft, bottomRight);
825  }),
826  connect(d->mStorageModel, &StorageModel::headerDataChanged, this, [this](Qt::Orientation orientation, int first, int last) {
827  d->slotStorageModelHeaderDataChanged(orientation, first, last);
828  })};
829 
830  if (d->mStorageModel->rowCount() == 0) {
831  return; // folder empty: nothing to fill
832  }
833 
834  // Here we use different strategies based on user preference and the folder size.
835  // The knobs we can tune are:
836  //
837  // - The number of jobs used to scan the whole folder and their order
838  //
839  // There are basically two approaches to this. One is the "single big job"
840  // approach. It scans the folder from the beginning to the end in a single job
841  // entry. The job passes are done only once. It's advantage is that it's simpler
842  // and it's less likely to generate imperfect parent threadings. The bad
843  // side is that since the folders are "sort of" date ordered then the most interesting
844  // messages show up at the end of the work. Not nice for large folders.
845  // The other approach uses two jobs. This is a bit slower but smarter strategy.
846  // First we scan the latest 1000 messages and *then* take care of the older ones.
847  // This will show up the most interesting messages almost immediately. (Well...
848  // All this assuming that the underlying storage always appends the newly arrived messages)
849  // The strategy is slower since it generates some imperfect parent threadings which must be
850  // adjusted by the second job. For instance, in my kernel mailing list folder this "smart" approach
851  // generates about 150 additional imperfectly threaded children... but the "today"
852  // messages show up almost immediately. The two-chunk job also makes computing
853  // the percentage user feedback a little harder and might break some optimization
854  // in the insertions (we're able to optimize appends and prepends but a chunked
855  // job is likely to split our work at a boundary where messages are always inserted
856  // in the middle of the list).
857  //
858  // - The maximum time to spend inside a single job step
859  //
860  // The larger this time, the greater the number of messages per second that this
861  // engine can process but also greater time with frozen UI -> less interactivity.
862  // Reasonable values start at 50 msecs. Values larger than 300 msecs are very likely
863  // to be perceived by the user as UI non-reactivity.
864  //
865  // - The number of messages processed in each job step subchunk.
866  //
867  // A job subchunk is processed without checking the maximum time above. This means
868  // that each job step will process at least the number of messages specified by this value.
869  // Very low values mean that we respect the maximum time very carefully but we also
870  // waste time to check if we ran out of time :)
871  // Very high values are likely to cause the engine to not respect the maximum step time.
872  // Reasonable values go from 5 to 100.
873  //
874  // - The "idle" time between two steps
875  //
876  // The lower this time, the greater the number of messages per second that this
877  // engine can process but also lower time for the UI to process events -> less interactivity.
878  // A value of 0 here means that Qt will trigger the timer as soon as it has some
879  // idle time to spend. UI events will be still processed but slowdowns are possible.
880  // 0 is reasonable though. Values larger than 200 will tend to make the total job
881  // completion times high.
882  //
883 
884  // If we have no filter it seems that we can apply a huge optimization.
885  // We disconnect the UI for the first huge filling job. This allows us
886  // to save the extremely expensive beginInsertRows()/endInsertRows() calls
887  // and call a single layoutChanged() at the end. This slows down a lot item
888  // expansion. But on the other side if only few items need to be expanded
889  // then this strategy is better. If filtering is enabled then this strategy
890  // isn't applicable (because filtering requires interaction with the UI
891  // while the data is loading).
892 
893  // So...
894 
895  // For the very first small chunk it's ok to work with disconnected UI as long
896  // as we have no filter. The first small chunk is always 1000 messages, so
897  // even if all of them are expanded, it's still somewhat acceptable.
898  bool canDoFirstSmallChunkWithDisconnectedUI = !d->mFilter;
899 
900  // Larger works need a bigger condition: few messages must be expanded in the end.
901  bool canDoJobWithDisconnectedUI = // we have no filter
902  !d->mFilter
903  && (
904  // we do no threading at all
905  (d->mAggregation->threading() == Aggregation::NoThreading) || // or we never expand threads
906  (d->mAggregation->threadExpandPolicy() == Aggregation::NeverExpandThreads) || // or we expand threads but we'll be going to expand really only a few
907  (
908  // so we don't expand them all
909  (d->mAggregation->threadExpandPolicy() != Aggregation::AlwaysExpandThreads) && // and we'd expand only a few in fact
910  (d->mStorageModel->initialUnreadRowCountGuess() < 1000)));
911 
912  switch (d->mAggregation->fillViewStrategy()) {
914  // favor interactivity
915  if ((!canDoJobWithDisconnectedUI) && (d->mStorageModel->rowCount() > 3000)) { // empiric value
916  // First a small job with the most recent messages. Large chunk, small (but non zero) idle interval
917  // and a larger number of messages to process at once.
918  auto job1 =
919  new ViewItemJob(d->mStorageModel->rowCount() - 1000, d->mStorageModel->rowCount() - 1, 200, 20, 100, canDoFirstSmallChunkWithDisconnectedUI);
920  d->mViewItemJobs.append(job1);
921  // Then a larger job with older messages. Small chunk, bigger idle interval, small number of messages to
922  // process at once.
923  auto job2 = new ViewItemJob(0, d->mStorageModel->rowCount() - 1001, 100, 50, 10, false);
924  d->mViewItemJobs.append(job2);
925 
926  // We could even extremize this by splitting the folder in several
927  // chunks and scanning them from the newest to the oldest... but the overhead
928  // due to imperfectly threaded children would be probably too big.
929  } else {
930  // small folder or can be done with disconnected UI: single chunk work.
931  // Lag the CPU a bit more but not too much to destroy even the earliest interactivity.
932  auto job = new ViewItemJob(0, d->mStorageModel->rowCount() - 1, 150, 30, 30, canDoJobWithDisconnectedUI);
933  d->mViewItemJobs.append(job);
934  }
935  break;
937  // More batchy jobs, still interactive to a certain degree
938  if ((!canDoJobWithDisconnectedUI) && (d->mStorageModel->rowCount() > 3000)) { // empiric value
939  // large folder, but favor speed
940  auto job1 =
941  new ViewItemJob(d->mStorageModel->rowCount() - 1000, d->mStorageModel->rowCount() - 1, 250, 0, 100, canDoFirstSmallChunkWithDisconnectedUI);
942  d->mViewItemJobs.append(job1);
943  auto job2 = new ViewItemJob(0, d->mStorageModel->rowCount() - 1001, 200, 0, 10, false);
944  d->mViewItemJobs.append(job2);
945  } else {
946  // small folder or can be done with disconnected UI and favor speed: single chunk work.
947  // Lag the CPU more, get more work done
948  auto job = new ViewItemJob(0, d->mStorageModel->rowCount() - 1, 250, 0, 100, canDoJobWithDisconnectedUI);
949  d->mViewItemJobs.append(job);
950  }
951  break;
953  // one large job, never interrupt, block UI
954  auto job = new ViewItemJob(0, d->mStorageModel->rowCount() - 1, 60000, 0, 100000, canDoJobWithDisconnectedUI);
955  d->mViewItemJobs.append(job);
956  break;
957  }
958  default:
959  qCWarning(MESSAGELIST_LOG) << "Unrecognized fill view strategy";
960  Q_ASSERT(false);
961  break;
962  }
963 
964  d->mLoading = true;
965 
966  d->viewItemJobStep();
967 }
968 
969 void ModelPrivate::checkIfDateChanged()
970 {
971  // This function is called by MessageList::Core::Manager once in a while (every 1 minute or sth).
972  // It is used to check if the current date has changed (with respect to mTodayDate).
973  //
974  // Our message items cache the formatted dates (as formatting them
975  // on the fly would be too expensive). We also cache the labels of the groups which often display dates.
976  // When the date changes we would need to fix all these strings.
977  //
978  // A dedicated algorithm to refresh the labels of the items would be either too complex
979  // or would block on large trees. Fixing the labels of the groups is also quite hard...
980  //
981  // So to keep the things simple we just reload the view.
982 
983  if (!mStorageModel) {
984  return; // nothing to do
985  }
986 
987  if (mLoading) {
988  return; // not now
989  }
990 
991  if (!mViewItemJobs.isEmpty()) {
992  return; // not now
993  }
994 
995  if (mTodayDate == QDate::currentDate()) {
996  return; // date not changed
997  }
998 
999  // date changed, reload the view (and try to preserve the current selection)
1000  q->setStorageModel(mStorageModel, PreSelectLastSelected);
1001 }
1002 
1004 {
1005  d->mPreSelectionMode = preSelect;
1006  d->mLastSelectedMessageInFolder = nullptr;
1007 }
1008 
1009 //
1010 // The "view fill" algorithm implemented in the functions below is quite smart but also quite complex.
1011 // It's governed by the following goals:
1012 //
1013 // - Be flexible: allow different configurations from "unsorted flat list" to a "grouped and threaded
1014 // list with different sorting algorithms applied to each aggregation level"
1015 // - Be reasonably fast
1016 // - Be non blocking: UI shouldn't freeze while the algorithm is running
1017 // - Be interruptible: user must be able to abort the execution and just switch to another folder in the middle
1018 //
1019 
1020 void ModelPrivate::clearUnassignedMessageLists()
1021 {
1022  // This is a bit tricky...
1023  // The three unassigned message lists contain messages that have been created
1024  // but not yet attached to the view. There may be two major cases for a message:
1025  // - it has no parent -> it must be deleted and it will delete its children too
1026  // - it has a parent -> it must NOT be deleted since it will be deleted by its parent.
1027 
1028  // Sometimes the things get a little complicated since in Pass2 and Pass3
1029  // we have transitional states in that the MessageItem object can be in two of these lists.
1030 
1031  // WARNING: This function does NOT fixup mNewestItem and mOldestItem. If one of these
1032  // two messages is in the lists below, it's deleted and the member becomes a dangling pointer.
1033  // The caller must ensure that both mNewestItem and mOldestItem are set to 0
1034  // and this is enforced in the assert below to avoid errors. This basically means
1035  // that this function should be called only when the storage model changes or
1036  // when the model is destroyed.
1037  Q_ASSERT((mOldestItem == nullptr) && (mNewestItem == nullptr));
1038 
1039  if (!mUnassignedMessageListForPass2.isEmpty()) {
1040  // We're actually in Pass1* or Pass2: everything is mUnassignedMessageListForPass2
1041  // Something may *also* be in mUnassignedMessageListForPass3 and mUnassignedMessageListForPass4
1042  // but that are duplicates for sure.
1043 
1044  // We can't just sweep the list and delete parentless items since each delete
1045  // could kill children which are somewhere AFTER in the list: accessing the children
1046  // would then lead to a SIGSEGV. We first sweep the list gathering parentless
1047  // items and *then* delete them without accessing the parented ones.
1048 
1049  QList<MessageItem *> parentless;
1050  for (const auto mi : std::as_const(mUnassignedMessageListForPass2)) {
1051  if (!mi->parent()) {
1052  parentless.append(mi);
1053  }
1054  }
1055 
1056  for (const auto mi : std::as_const(parentless)) {
1057  delete mi;
1058  }
1059 
1060  mUnassignedMessageListForPass2.clear();
1061  // Any message these list contain was also in mUnassignedMessageListForPass2
1062  mUnassignedMessageListForPass3.clear();
1063  mUnassignedMessageListForPass4.clear();
1064  return;
1065  }
1066 
1067  // mUnassignedMessageListForPass2 is empty
1068 
1069  if (!mUnassignedMessageListForPass3.isEmpty()) {
1070  // We're actually at the very end of Pass2 or inside Pass3
1071  // Pass2 pushes stuff in mUnassignedMessageListForPass3 *or* mUnassignedMessageListForPass4
1072  // Pass3 pushes stuff from mUnassignedMessageListForPass3 to mUnassignedMessageListForPass4
1073  // So if we're in Pass2 then the two lists contain distinct messages but if we're in Pass3
1074  // then the two lists may contain the same messages.
1075 
1076  if (!mUnassignedMessageListForPass4.isEmpty()) {
1077  // We're actually in Pass3: the messiest one.
1078 
1079  QSet<MessageItem *> itemsToDelete;
1080  for (const auto mi : std::as_const(mUnassignedMessageListForPass3)) {
1081  if (!mi->parent()) {
1082  itemsToDelete.insert(mi);
1083  }
1084  }
1085  for (const auto mi : std::as_const(mUnassignedMessageListForPass4)) {
1086  if (!mi->parent()) {
1087  itemsToDelete.insert(mi);
1088  }
1089  }
1090  for (const auto mi : std::as_const(itemsToDelete)) {
1091  delete mi;
1092  }
1093 
1094  mUnassignedMessageListForPass3.clear();
1095  mUnassignedMessageListForPass4.clear();
1096  return;
1097  }
1098 
1099  // mUnassignedMessageListForPass4 is empty so we must be at the end of a very special kind of Pass2
1100  // We have the same problem as in mUnassignedMessageListForPass2.
1101  QList<MessageItem *> parentless;
1102  for (const auto mi : std::as_const(mUnassignedMessageListForPass3)) {
1103  if (!mi->parent()) {
1104  parentless.append(mi);
1105  }
1106  }
1107  for (const auto mi : std::as_const(parentless)) {
1108  delete mi;
1109  }
1110 
1111  mUnassignedMessageListForPass3.clear();
1112  return;
1113  }
1114 
1115  // mUnassignedMessageListForPass3 is empty
1116  if (!mUnassignedMessageListForPass4.isEmpty()) {
1117  // we're in Pass4.. this is easy.
1118 
1119  // We have the same problem as in mUnassignedMessageListForPass2.
1120  QList<MessageItem *> parentless;
1121  for (const auto mi : std::as_const(mUnassignedMessageListForPass4)) {
1122  if (!mi->parent()) {
1123  parentless.append(mi);
1124  }
1125  }
1126  for (const auto mi : std::as_const(parentless)) {
1127  delete mi;
1128  }
1129 
1130  mUnassignedMessageListForPass4.clear();
1131  return;
1132  }
1133 }
1134 
1135 void ModelPrivate::clearThreadingCacheReferencesIdMD5ToMessageItem()
1136 {
1137  qDeleteAll(mThreadingCacheMessageReferencesIdMD5ToMessageItem);
1138  mThreadingCacheMessageReferencesIdMD5ToMessageItem.clear();
1139 }
1140 
1141 void ModelPrivate::clearThreadingCacheMessageSubjectMD5ToMessageItem()
1142 {
1143  qDeleteAll(mThreadingCacheMessageSubjectMD5ToMessageItem);
1144  mThreadingCacheMessageSubjectMD5ToMessageItem.clear();
1145 }
1146 
1147 void ModelPrivate::clearOrphanChildrenHash()
1148 {
1149  qDeleteAll(mOrphanChildrenHash);
1150  mOrphanChildrenHash.clear();
1151 }
1152 
1153 void ModelPrivate::clearJobList()
1154 {
1155  if (mViewItemJobs.isEmpty()) {
1156  return;
1157  }
1158 
1159  if (mInLengthyJobBatch) {
1160  mInLengthyJobBatch = false;
1161  }
1162 
1163  qDeleteAll(mViewItemJobs);
1164  mViewItemJobs.clear();
1165 
1166  mModelForItemFunctions = q; // make sure it's true, as there remains no job with disconnected UI
1167 }
1168 
1169 void ModelPrivate::attachGroup(GroupHeaderItem *ghi)
1170 {
1171  if (ghi->parent()) {
1172  if (((ghi)->childItemCount() > 0) // has children
1173  && (ghi)->isViewable() // is actually attached to the viewable root
1174  && mModelForItemFunctions // the UI is not disconnected
1175  && mView->isExpanded(q->index(ghi, 0)) // is actually expanded
1176  ) {
1177  saveExpandedStateOfSubtree(ghi);
1178  }
1179 
1180  // FIXME: This *WILL* break selection and current index... :/
1181 
1182  ghi->parent()->takeChildItem(mModelForItemFunctions, ghi);
1183  }
1184 
1185  ghi->setParent(mRootItem);
1186 
1187  // I'm using a macro since it does really improve readability.
1188  // I'm NOT using a helper function since gcc will refuse to inline some of
1189  // the calls because they make this function grow too much.
1190 #define INSERT_GROUP_WITH_COMPARATOR(_ItemComparator) \
1191  switch (mSortOrder->groupSortDirection()) { \
1192  case SortOrder::Ascending: \
1193  mRootItem->d_ptr->insertChildItem<_ItemComparator, true>(mModelForItemFunctions, ghi); \
1194  break; \
1195  case SortOrder::Descending: \
1196  mRootItem->d_ptr->insertChildItem<_ItemComparator, false>(mModelForItemFunctions, ghi); \
1197  break; \
1198  default: /* should never happen... */ \
1199  mRootItem->appendChildItem(mModelForItemFunctions, ghi); \
1200  break; \
1201  }
1202 
1203  switch (mSortOrder->groupSorting()) {
1205  INSERT_GROUP_WITH_COMPARATOR(ItemDateComparator)
1206  break;
1208  INSERT_GROUP_WITH_COMPARATOR(ItemMaxDateComparator)
1209  break;
1211  INSERT_GROUP_WITH_COMPARATOR(ItemSenderOrReceiverComparator)
1212  break;
1214  INSERT_GROUP_WITH_COMPARATOR(ItemSenderComparator)
1215  break;
1217  INSERT_GROUP_WITH_COMPARATOR(ItemReceiverComparator)
1218  break;
1220  mRootItem->appendChildItem(mModelForItemFunctions, ghi);
1221  break;
1222  default: // should never happen
1223  mRootItem->appendChildItem(mModelForItemFunctions, ghi);
1224  break;
1225  }
1226 
1227  if (ghi->initialExpandStatus() == Item::ExpandNeeded) { // this actually is a "non viewable expanded state"
1228  if (ghi->childItemCount() > 0) {
1229  if (mModelForItemFunctions) { // the UI is not disconnected
1230  syncExpandedStateOfSubtree(ghi);
1231  }
1232  }
1233  }
1234 
1235  // A group header is always viewable, when attached: apply the filter, if we have it.
1236  if (mFilter) {
1237  Q_ASSERT(mModelForItemFunctions); // UI must be NOT disconnected
1238  // apply the filter to subtree
1239  applyFilterToSubtree(ghi, QModelIndex());
1240  }
1241 }
1242 
1243 void ModelPrivate::saveExpandedStateOfSubtree(Item *root)
1244 {
1245  Q_ASSERT(mModelForItemFunctions); // UI must be NOT disconnected here
1246  Q_ASSERT(root);
1247 
1249 
1250  auto children = root->childItems();
1251  if (!children) {
1252  return;
1253  }
1254  for (const auto mi : std::as_const(*children)) {
1255  if (mi->childItemCount() > 0 // has children
1256  && mi->isViewable() // is actually attached to the viewable root
1257  && mView->isExpanded(q->index(mi, 0))) { // is actually expanded
1258  saveExpandedStateOfSubtree(mi);
1259  }
1260  }
1261 }
1262 
1263 void ModelPrivate::syncExpandedStateOfSubtree(Item *root)
1264 {
1265  Q_ASSERT(mModelForItemFunctions); // UI must be NOT disconnected here
1266 
1267  // WE ASSUME that:
1268  // - the item is viewable
1269  // - its initialExpandStatus() is Item::ExpandNeeded
1270  // - it has at least one children (well.. this is not a strict requirement, but it's a waste of resources to expand items that don't have children)
1271 
1272  QModelIndex idx = q->index(root, 0);
1273 
1274  // if ( !mView->isExpanded( idx ) ) // this is O(logN!) in Qt.... very ugly... but it should never happen here
1275  mView->expand(idx); // sync the real state in the view
1277 
1278  auto children = root->childItems();
1279  if (!children) {
1280  return;
1281  }
1282 
1283  for (const auto mi : std::as_const(*children)) {
1284  if (mi->initialExpandStatus() == Item::ExpandNeeded) {
1285  if (mi->childItemCount() > 0) {
1286  syncExpandedStateOfSubtree(mi);
1287  }
1288  }
1289  }
1290 }
1291 
1292 void ModelPrivate::attachMessageToGroupHeader(MessageItem *mi)
1293 {
1294  QString groupLabel;
1295  time_t date;
1296 
1297  // compute the group header label and the date
1298  switch (mAggregation->grouping()) {
1301  if (mAggregation->threadLeader() == Aggregation::MostRecentMessage) {
1302  date = mi->maxDate();
1303  } else {
1304  date = mi->date();
1305  }
1306 
1307  QDateTime dt;
1308  dt.setSecsSinceEpoch(date);
1309  QDate dDate = dt.date();
1310  int daysAgo = -1;
1311  const int daysInWeek = 7;
1312  if (dDate.isValid() && mTodayDate.isValid()) {
1313  daysAgo = dDate.daysTo(mTodayDate);
1314  }
1315 
1316  if ((daysAgo < 0) // In the future
1317  || (static_cast<uint>(date) == static_cast<uint>(-1))) { // Invalid
1318  groupLabel = mCachedUnknownLabel;
1319  } else if (daysAgo == 0) { // Today
1320  groupLabel = mCachedTodayLabel;
1321  } else if (daysAgo == 1) { // Yesterday
1322  groupLabel = mCachedYesterdayLabel;
1323  } else if (daysAgo > 1 && daysAgo < daysInWeek) { // Within last seven days
1324  auto dayName = mCachedDayNameLabel.find(dDate.dayOfWeek()); // non-const call, but non-shared container
1325  if (dayName == mCachedDayNameLabel.end()) {
1326  dayName = mCachedDayNameLabel.insert(dDate.dayOfWeek(), QLocale::system().standaloneDayName(dDate.dayOfWeek()));
1327  }
1328  groupLabel = *dayName;
1329  } else if (mAggregation->grouping() == Aggregation::GroupByDate) { // GroupByDate seven days or more ago
1330  groupLabel = QLocale::system().toString(dDate, QLocale::ShortFormat);
1331  } else if (dDate.month() == mTodayDate.month() // GroupByDateRange within this month
1332  && dDate.year() == mTodayDate.year()) {
1333  int startOfWeekDaysAgo = (daysInWeek + mTodayDate.dayOfWeek() - QLocale().firstDayOfWeek()) % daysInWeek;
1334  int weeksAgo = ((daysAgo - startOfWeekDaysAgo) / daysInWeek) + 1;
1335  switch (weeksAgo) {
1336  case 0: // This week
1337  groupLabel = QLocale::system().standaloneDayName(dDate.dayOfWeek());
1338  break;
1339  case 1: // 1 week ago
1340  groupLabel = mCachedLastWeekLabel;
1341  break;
1342  case 2:
1343  groupLabel = mCachedTwoWeeksAgoLabel;
1344  break;
1345  case 3:
1346  groupLabel = mCachedThreeWeeksAgoLabel;
1347  break;
1348  case 4:
1349  groupLabel = mCachedFourWeeksAgoLabel;
1350  break;
1351  case 5:
1352  groupLabel = mCachedFiveWeeksAgoLabel;
1353  break;
1354  default: // should never happen
1355  groupLabel = mCachedUnknownLabel;
1356  }
1357  } else if (dDate.year() == mTodayDate.year()) { // GroupByDateRange within this year
1358  auto monthName = mCachedMonthNameLabel.find(dDate.month()); // non-const call, but non-shared container
1359  if (monthName == mCachedMonthNameLabel.end()) {
1360  monthName = mCachedMonthNameLabel.insert(dDate.month(), QLocale::system().standaloneMonthName(dDate.month()));
1361  }
1362  groupLabel = *monthName;
1363  } else { // GroupByDateRange in previous years
1364  auto monthName = mCachedMonthNameLabel.find(dDate.month()); // non-const call, but non-shared container
1365  if (monthName == mCachedMonthNameLabel.end()) {
1366  monthName = mCachedMonthNameLabel.insert(dDate.month(), QLocale::system().standaloneMonthName(dDate.month()));
1367  }
1368  groupLabel = i18nc("Message Aggregation Group Header: Month name and Year number",
1369  "%1 %2",
1370  *monthName,
1371  QLocale::system().toString(dDate, QLatin1String("yyyy")));
1372  }
1373  break;
1374  }
1375 
1377  date = mi->date();
1378  groupLabel = mi->displaySenderOrReceiver();
1379  break;
1380 
1382  date = mi->date();
1383  groupLabel = mi->displaySender();
1384  break;
1385 
1387  date = mi->date();
1388  groupLabel = mi->displayReceiver();
1389  break;
1390 
1392  // append directly to root
1393  attachMessageToParent(mRootItem, mi);
1394  return;
1395 
1396  default:
1397  // should never happen
1398  attachMessageToParent(mRootItem, mi);
1399  return;
1400  }
1401 
1402  GroupHeaderItem *ghi;
1403 
1404  ghi = mGroupHeaderItemHash.value(groupLabel, nullptr);
1405  if (!ghi) {
1406  // not found
1407 
1408  ghi = new GroupHeaderItem(groupLabel);
1409  ghi->initialSetup(date, mi->size(), mi->sender(), mi->receiver(), mi->useReceiver());
1410 
1411  switch (mAggregation->groupExpandPolicy()) {
1413  // nothing to do
1414  break;
1416  // expand always
1417  ghi->setInitialExpandStatus(Item::ExpandNeeded);
1418  break;
1420  // expand only if "close" to today
1421  if (mViewItemJobStepStartTime > ghi->date()) {
1422  if ((mViewItemJobStepStartTime - ghi->date()) < (3600 * 72)) {
1423  ghi->setInitialExpandStatus(Item::ExpandNeeded);
1424  }
1425  } else {
1426  if ((ghi->date() - mViewItemJobStepStartTime) < (3600 * 72)) {
1427  ghi->setInitialExpandStatus(Item::ExpandNeeded);
1428  }
1429  }
1430  break;
1431  default:
1432  // b0rken
1433  break;
1434  }
1435 
1436  attachMessageToParent(ghi, mi);
1437 
1438  attachGroup(ghi); // this will expand the group if required
1439 
1440  mGroupHeaderItemHash.insert(groupLabel, ghi);
1441  } else {
1442  // the group was already there (certainly viewable)
1443 
1444  // This function may be also called to re-group a message.
1445  // That is, to eventually find a new group for a message that has changed
1446  // its properties (but was already attached to a group).
1447  // So it may happen that we find out that in fact re-grouping wasn't really
1448  // needed because the message is already in the correct group.
1449  if (mi->parent() == ghi) {
1450  return; // nothing to be done
1451  }
1452 
1453  attachMessageToParent(ghi, mi);
1454  }
1455 
1456  // Remember this message as a thread leader
1457  mThreadingCache.updateParent(mi, nullptr);
1458 }
1459 
1460 MessageItem *ModelPrivate::findMessageParent(MessageItem *mi)
1461 {
1462  Q_ASSERT(mAggregation->threading() != Aggregation::NoThreading); // caller must take care of this
1463 
1464  // This function attempts to find a thread parent for the item "mi"
1465  // which actually may already have a children subtree.
1466 
1467  // Forged or plain broken message trees are dangerous here.
1468  // For example, a message tree with circular references like
1469  //
1470  // Message mi, Id=1, In-Reply-To=2
1471  // Message childOfMi, Id=2, In-Reply-To=1
1472  //
1473  // is perfectly possible and will cause us to find childOfMi
1474  // as parent of mi. This will then create a loop in the message tree
1475  // (which will then no longer be a tree in fact) and cause us to freeze
1476  // once we attempt to climb the parents. We need to take care of that.
1477 
1478  bool bMessageWasThreadable = false;
1479  MessageItem *pParent;
1480 
1481  // First of all try to find a "perfect parent", that is the message for that
1482  // we have the ID in the "In-Reply-To" field. This is actually done by using
1483  // MD5 caches of the message ids because of speed. Collisions are very unlikely.
1484 
1485  QByteArray md5 = mi->inReplyToIdMD5();
1486  if (!md5.isEmpty()) {
1487  // have an In-Reply-To field MD5
1488  pParent = mThreadingCacheMessageIdMD5ToMessageItem.value(md5, nullptr);
1489  if (pParent) {
1490  // Take care of circular references
1491  if ((mi == pParent) // self referencing message
1492  || ((mi->childItemCount() > 0) // mi already has children, this is fast to determine
1493  && pParent->hasAncestor(mi) // pParent is in the mi's children tree
1494  )) {
1495  qCWarning(MESSAGELIST_LOG) << "Circular In-Reply-To reference loop detected in the message tree";
1496  mi->setThreadingStatus(MessageItem::NonThreadable);
1497  return nullptr; // broken message: throw it away
1498  }
1499  mi->setThreadingStatus(MessageItem::PerfectParentFound);
1500  return pParent; // got a perfect parent for this message
1501  }
1502 
1503  // got no perfect parent
1504  bMessageWasThreadable = true; // but the message was threadable
1505  }
1506 
1507  if (mAggregation->threading() == Aggregation::PerfectOnly) {
1508  mi->setThreadingStatus(bMessageWasThreadable ? MessageItem::ParentMissing : MessageItem::NonThreadable);
1509  return nullptr; // we're doing only perfect parent matches
1510  }
1511 
1512  // Try to use the "References" field. In fact we have the MD5 of the
1513  // (n-1)th entry in References.
1514  //
1515  // Original rationale from KMHeaders:
1516  //
1517  // If we don't have a replyToId, or if we have one and the
1518  // corresponding message is not in this folder, as happens
1519  // if you keep your outgoing messages in an OUTBOX, for
1520  // example, try the list of references, because the second
1521  // to last will likely be in this folder. replyToAuxIdMD5
1522  // contains the second to last one.
1523 
1524  md5 = mi->referencesIdMD5();
1525  if (!md5.isEmpty()) {
1526  pParent = mThreadingCacheMessageIdMD5ToMessageItem.value(md5, nullptr);
1527  if (pParent) {
1528  // Take care of circular references
1529  if ((mi == pParent) // self referencing message
1530  || ((mi->childItemCount() > 0) // mi already has children, this is fast to determine
1531  && pParent->hasAncestor(mi) // pParent is in the mi's children tree
1532  )) {
1533  qCWarning(MESSAGELIST_LOG) << "Circular reference loop detected in the message tree";
1534  mi->setThreadingStatus(MessageItem::NonThreadable);
1535  return nullptr; // broken message: throw it away
1536  }
1537  mi->setThreadingStatus(MessageItem::ImperfectParentFound);
1538  return pParent; // got an imperfect parent for this message
1539  }
1540 
1541  auto messagesWithTheSameReferences = mThreadingCacheMessageReferencesIdMD5ToMessageItem.value(md5, nullptr);
1542  if (messagesWithTheSameReferences) {
1543  Q_ASSERT(!messagesWithTheSameReferences->isEmpty());
1544 
1545  pParent = messagesWithTheSameReferences->first();
1546  if (mi != pParent && (mi->childItemCount() == 0 || !pParent->hasAncestor(mi))) {
1547  mi->setThreadingStatus(MessageItem::ImperfectParentFound);
1548  return pParent;
1549  }
1550  }
1551 
1552  // got no imperfect parent
1553  bMessageWasThreadable = true; // but the message was threadable
1554  }
1555 
1556  if (mAggregation->threading() == Aggregation::PerfectAndReferences) {
1557  mi->setThreadingStatus(bMessageWasThreadable ? MessageItem::ParentMissing : MessageItem::NonThreadable);
1558  return nullptr; // we're doing only perfect parent matches
1559  }
1560 
1561  Q_ASSERT(mAggregation->threading() == Aggregation::PerfectReferencesAndSubject);
1562 
1563  // We are supposed to do subject based threading but we can't do it now.
1564  // This is because the subject based threading *may* be wrong and waste
1565  // time by creating circular references (that we'd need to detect and fix).
1566  // We first try the perfect and references based threading on all the messages
1567  // and then run subject based threading only on the remaining ones.
1568 
1569  mi->setThreadingStatus((bMessageWasThreadable || mi->subjectIsPrefixed()) ? MessageItem::ParentMissing : MessageItem::NonThreadable);
1570  return nullptr;
1571 }
1572 
1573 // Subject threading cache stuff
1574 
1575 #if 0
1576 // Debug helpers
1577 void dump_iterator_and_list(QList< MessageItem * >::Iterator &iter, QList< MessageItem * > *list)
1578 {
1579  qCDebug(MESSAGELIST_LOG) << "Threading cache part dump";
1580  if (iter == list->end()) {
1581  qCDebug(MESSAGELIST_LOG) << "Iterator pointing to end of the list";
1582  } else {
1583  qCDebug(MESSAGELIST_LOG) << "Iterator pointing to " << *iter << " subject [" << (*iter)->subject() << "] date [" << (*iter)->date() << "]";
1584  }
1585 
1586  for (QList< MessageItem * >::Iterator it = list->begin(); it != list->end(); ++it) {
1587  qCDebug(MESSAGELIST_LOG) << "List element " << *it << " subject [" << (*it)->subject() << "] date [" << (*it)->date() << "]";
1588  }
1589 
1590  qCDebug(MESSAGELIST_LOG) << "End of threading cache part dump";
1591 }
1592 
1593 void dump_list(QList< MessageItem * > *list)
1594 {
1595  qCDebug(MESSAGELIST_LOG) << "Threading cache part dump";
1596 
1597  for (QList< MessageItem * >::Iterator it = list->begin(); it != list->end(); ++it) {
1598  qCDebug(MESSAGELIST_LOG) << "List element " << *it << " subject [" << (*it)->subject() << "] date [" << (*it)->date() << "]";
1599  }
1600 
1601  qCDebug(MESSAGELIST_LOG) << "End of threading cache part dump";
1602 }
1603 
1604 #endif // debug helpers
1605 
1606 // a helper class used in a qLowerBound() call below
1607 class MessageLessThanByDate
1608 {
1609 public:
1610  inline bool operator()(const MessageItem *mi1, const MessageItem *mi2) const
1611  {
1612  if (mi1->date() < mi2->date()) { // likely
1613  return true;
1614  }
1615  if (mi1->date() > mi2->date()) { // likely
1616  return false;
1617  }
1618  // dates are equal, compare by pointer
1619  return mi1 < mi2;
1620  }
1621 };
1622 
1623 void ModelPrivate::addMessageToReferencesBasedThreadingCache(MessageItem *mi)
1624 {
1625  // Messages in this cache are sorted by date, and if dates are equal then they are sorted by pointer value.
1626  // Sorting by date is used to optimize the parent lookup in guessMessageParent() below.
1627 
1628  // WARNING: If the message date changes for some reason (like in the "update" step)
1629  // then the cache may become unsorted. For this reason the message about to
1630  // be changed must be first removed from the cache and then reinserted.
1631 
1632  auto messagesWithTheSameReference = mThreadingCacheMessageReferencesIdMD5ToMessageItem.value(mi->referencesIdMD5(), nullptr);
1633 
1634  if (!messagesWithTheSameReference) {
1635  messagesWithTheSameReference = new QList<MessageItem *>();
1636  mThreadingCacheMessageReferencesIdMD5ToMessageItem.insert(mi->referencesIdMD5(), messagesWithTheSameReference);
1637  messagesWithTheSameReference->append(mi);
1638  return;
1639  }
1640 
1641  // Found: assert that we have no duplicates in the cache.
1642  Q_ASSERT(!messagesWithTheSameReference->contains(mi));
1643 
1644  // Ordered insert: first by date then by pointer value.
1645  auto it = std::lower_bound(messagesWithTheSameReference->begin(), messagesWithTheSameReference->end(), mi, MessageLessThanByDate());
1646  messagesWithTheSameReference->insert(it, mi);
1647 }
1648 
1649 void ModelPrivate::removeMessageFromReferencesBasedThreadingCache(MessageItem *mi)
1650 {
1651  // We assume that the caller knows what he is doing and the message is actually in the cache.
1652  // If the message isn't in the cache then we should not be called at all.
1653 
1654  auto messagesWithTheSameReference = mThreadingCacheMessageReferencesIdMD5ToMessageItem.value(mi->referencesIdMD5(), nullptr);
1655 
1656  // We assume that the message is there so the list must be non null.
1657  Q_ASSERT(messagesWithTheSameReference);
1658 
1659  // The cache *MUST* be ordered first by date then by pointer value
1660  auto it = std::lower_bound(messagesWithTheSameReference->begin(), messagesWithTheSameReference->end(), mi, MessageLessThanByDate());
1661 
1662  // The binary based search must have found a message
1663  Q_ASSERT(it != messagesWithTheSameReference->end());
1664 
1665  // and it must have found exactly the message requested
1666  Q_ASSERT(*it == mi);
1667 
1668  // Kill it
1669  messagesWithTheSameReference->erase(it);
1670 
1671  // And kill the list if it was the last one
1672  if (messagesWithTheSameReference->isEmpty()) {
1673  mThreadingCacheMessageReferencesIdMD5ToMessageItem.remove(mi->referencesIdMD5());
1674  delete messagesWithTheSameReference;
1675  }
1676 }
1677 
1678 void ModelPrivate::addMessageToSubjectBasedThreadingCache(MessageItem *mi)
1679 {
1680  // Messages in this cache are sorted by date, and if dates are equal then they are sorted by pointer value.
1681  // Sorting by date is used to optimize the parent lookup in guessMessageParent() below.
1682 
1683  // WARNING: If the message date changes for some reason (like in the "update" step)
1684  // then the cache may become unsorted. For this reason the message about to
1685  // be changed must be first removed from the cache and then reinserted.
1686 
1687  // Lookup the list of messages with the same stripped subject
1688  auto messagesWithTheSameStrippedSubject = mThreadingCacheMessageSubjectMD5ToMessageItem.value(mi->strippedSubjectMD5(), nullptr);
1689 
1690  if (!messagesWithTheSameStrippedSubject) {
1691  // Not there yet: create it and append.
1692  messagesWithTheSameStrippedSubject = new QList<MessageItem *>();
1693  mThreadingCacheMessageSubjectMD5ToMessageItem.insert(mi->strippedSubjectMD5(), messagesWithTheSameStrippedSubject);
1694  messagesWithTheSameStrippedSubject->append(mi);
1695  return;
1696  }
1697 
1698  // Found: assert that we have no duplicates in the cache.
1699  Q_ASSERT(!messagesWithTheSameStrippedSubject->contains(mi));
1700 
1701  // Ordered insert: first by date then by pointer value.
1702  auto it = std::lower_bound(messagesWithTheSameStrippedSubject->begin(), messagesWithTheSameStrippedSubject->end(), mi, MessageLessThanByDate());
1703  messagesWithTheSameStrippedSubject->insert(it, mi);
1704 }
1705 
1706 void ModelPrivate::removeMessageFromSubjectBasedThreadingCache(MessageItem *mi)
1707 {
1708  // We assume that the caller knows what he is doing and the message is actually in the cache.
1709  // If the message isn't in the cache then we should not be called at all.
1710  //
1711  // The game is called "performance"
1712 
1713  // Grab the list of all the messages with the same stripped subject (all potential parents)
1714  auto messagesWithTheSameStrippedSubject = mThreadingCacheMessageSubjectMD5ToMessageItem.value(mi->strippedSubjectMD5(), nullptr);
1715 
1716  // We assume that the message is there so the list must be non null.
1717  Q_ASSERT(messagesWithTheSameStrippedSubject);
1718 
1719  // The cache *MUST* be ordered first by date then by pointer value
1720  auto it = std::lower_bound(messagesWithTheSameStrippedSubject->begin(), messagesWithTheSameStrippedSubject->end(), mi, MessageLessThanByDate());
1721 
1722  // The binary based search must have found a message
1723  Q_ASSERT(it != messagesWithTheSameStrippedSubject->end());
1724 
1725  // and it must have found exactly the message requested
1726  Q_ASSERT(*it == mi);
1727 
1728  // Kill it
1729  messagesWithTheSameStrippedSubject->erase(it);
1730 
1731  // And kill the list if it was the last one
1732  if (messagesWithTheSameStrippedSubject->isEmpty()) {
1733  mThreadingCacheMessageSubjectMD5ToMessageItem.remove(mi->strippedSubjectMD5());
1734  delete messagesWithTheSameStrippedSubject;
1735  }
1736 }
1737 
1738 MessageItem *ModelPrivate::guessMessageParent(MessageItem *mi)
1739 {
1740  // This function implements subject based threading
1741  // It attempts to guess a thread parent for the item "mi"
1742  // which actually may already have a children subtree.
1743 
1744  // We have all the problems of findMessageParent() plus the fact that
1745  // we're actually guessing (and often we may be *wrong*).
1746 
1747  Q_ASSERT(mAggregation->threading() == Aggregation::PerfectReferencesAndSubject); // caller must take care of this
1748  Q_ASSERT(mi->subjectIsPrefixed()); // caller must take care of this
1749  Q_ASSERT(mi->threadingStatus() == MessageItem::ParentMissing);
1750 
1751  // Do subject based threading
1752  const QByteArray md5 = mi->strippedSubjectMD5();
1753  if (!md5.isEmpty()) {
1754  auto messagesWithTheSameStrippedSubject = mThreadingCacheMessageSubjectMD5ToMessageItem.value(md5, nullptr);
1755 
1756  if (messagesWithTheSameStrippedSubject) {
1757  Q_ASSERT(!messagesWithTheSameStrippedSubject->isEmpty());
1758 
1759  // Need to find the message with the maximum date lower than the one of this message
1760 
1761  auto maxTime = (time_t)0;
1762  MessageItem *pParent = nullptr;
1763 
1764  // Here'we re really guessing so circular references are possible
1765  // even on perfectly valid trees. This is why we don't consider it
1766  // an error but just continue searching.
1767 
1768  // FIXME: This might be speed up with an initial binary search (?)
1769  // ANSWER: No. We can't rely on date order (as it can be updated on the fly...)
1770  for (const auto it : std::as_const(*messagesWithTheSameStrippedSubject)) {
1771  int delta = mi->date() - it->date();
1772 
1773  // We don't take into account messages with a delta smaller than 120.
1774  // Assuming that our date() values are correct (that is, they take into
1775  // account timezones etc..) then one usually needs more than 120 seconds
1776  // to answer to a message. Better safe than sorry.
1777 
1778  // This check also includes negative deltas so messages later than mi aren't considered
1779 
1780  if (delta < 120) {
1781  break; // The list is ordered by date (ascending) so we can stop searching here
1782  }
1783 
1784  // About the "magic" 3628899 value here comes a Till's comment from the original KMHeaders:
1785  //
1786  // "Parents more than six weeks older than the message are not accepted. The reasoning being
1787  // that if a new message with the same subject turns up after such a long time, the chances
1788  // that it is still part of the same thread are slim. The value of six weeks is chosen as a
1789  // result of a poll conducted on kde-devel, so it's probably bogus. :)"
1790 
1791  if (delta < 3628899) {
1792  // Compute the closest.
1793  if ((maxTime < it->date())) {
1794  // This algorithm *can* be (and often is) wrong.
1795  // Take care of circular threading which is really possible at this level.
1796  // If mi contains "it" inside its children subtree then we have
1797  // found such a circular threading problem.
1798 
1799  // Note that here we can't have it == mi because of the delta >= 120 check above.
1800 
1801  if ((mi->childItemCount() == 0) || !it->hasAncestor(mi)) {
1802  maxTime = it->date();
1803  pParent = it;
1804  }
1805  }
1806  }
1807  }
1808 
1809  if (pParent) {
1810  mi->setThreadingStatus(MessageItem::ImperfectParentFound);
1811  return pParent; // got an imperfect parent for this message
1812  }
1813  }
1814  }
1815 
1816  return nullptr;
1817 }
1818 
1819 //
1820 // A little template helper, hopefully inlineable.
1821 //
1822 // Return true if the specified message item is in the wrong position
1823 // inside the specified parent and needs re-sorting. Return false otherwise.
1824 // Both parent and messageItem must not be null.
1825 //
1826 // Checking if a message needs re-sorting instead of just re-sorting it
1827 // is very useful since re-sorting is an expensive operation.
1828 //
1829 template<class ItemComparator>
1830 static bool messageItemNeedsReSorting(SortOrder::SortDirection messageSortDirection, ItemPrivate *parent, MessageItem *messageItem)
1831 {
1832  if ((messageSortDirection == SortOrder::Ascending) || (parent->mType == Item::Message)) {
1833  return parent->childItemNeedsReSorting<ItemComparator, true>(messageItem);
1834  }
1835  return parent->childItemNeedsReSorting<ItemComparator, false>(messageItem);
1836 }
1837 
1838 bool ModelPrivate::handleItemPropertyChanges(int propertyChangeMask, Item *parent, Item *item)
1839 {
1840  // The facts:
1841  //
1842  // - If dates changed:
1843  // - If we're sorting messages by min/max date then at each level the messages might need resorting.
1844  // - If the thread leader is the most recent message of a thread then the uppermost
1845  // message of the thread might need re-grouping.
1846  // - If the groups are sorted by min/max date then the group might need re-sorting too.
1847  //
1848  // This function explicitly doesn't re-apply the filter when ActionItemStatus changes.
1849  // This is because filters must be re-applied due to a broader range of status variations:
1850  // this is done in viewItemJobStepInternalForJobPass1Update() instead (which is the only
1851  // place in that ActionItemStatus may be set).
1852 
1853  if (parent->type() == Item::InvisibleRoot) {
1854  // item is either a message or a group attached to the root.
1855  // It might need resorting.
1856  if (item->type() == Item::GroupHeader) {
1857  // item is a group header attached to the root.
1858  if ((
1859  // max date changed
1860  (propertyChangeMask & MaxDateChanged) && // groups sorted by max date
1861  (mSortOrder->groupSorting() == SortOrder::SortGroupsByDateTimeOfMostRecent))
1862  || (
1863  // date changed
1864  (propertyChangeMask & DateChanged) && // groups sorted by date
1865  (mSortOrder->groupSorting() == SortOrder::SortGroupsByDateTime))) {
1866  // This group might need re-sorting.
1867 
1868  // Groups are large container of messages so it's likely that
1869  // another message inserted will cause this group to be marked again.
1870  // So we wait until the end to do the grand final re-sorting: it will be done in Pass4.
1871  mGroupHeadersThatNeedUpdate.insert(static_cast<GroupHeaderItem *>(item), static_cast<GroupHeaderItem *>(item));
1872  }
1873  } else {
1874  // item is a message. It might need re-sorting.
1875 
1876  // Since sorting is an expensive operation, we first check if it's *really* needed.
1877  // Re-sorting will actually not change min/max dates at all and
1878  // will not climb up the parent's ancestor tree.
1879 
1880  switch (mSortOrder->messageSorting()) {
1882  if (propertyChangeMask & DateChanged) { // date changed
1883  if (messageItemNeedsReSorting<ItemDateComparator>(mSortOrder->messageSortDirection(), parent->d_ptr, static_cast<MessageItem *>(item))) {
1884  attachMessageToParent(parent, static_cast<MessageItem *>(item));
1885  }
1886  } // else date changed, but it doesn't match sorting order: no need to re-sort
1887  break;
1889  if (propertyChangeMask & MaxDateChanged) { // max date changed
1890  if (messageItemNeedsReSorting<ItemMaxDateComparator>(mSortOrder->messageSortDirection(), parent->d_ptr, static_cast<MessageItem *>(item))) {
1891  attachMessageToParent(parent, static_cast<MessageItem *>(item));
1892  }
1893  } // else max date changed, but it doesn't match sorting order: no need to re-sort
1894  break;
1896  if (propertyChangeMask & ActionItemStatusChanged) { // todo status changed
1897  if (messageItemNeedsReSorting<ItemActionItemStatusComparator>(mSortOrder->messageSortDirection(),
1898  parent->d_ptr,
1899  static_cast<MessageItem *>(item))) {
1900  attachMessageToParent(parent, static_cast<MessageItem *>(item));
1901  }
1902  } // else to do status changed, but it doesn't match sorting order: no need to re-sort
1903  break;
1905  if (propertyChangeMask & UnreadStatusChanged) { // new / unread status changed
1906  if (messageItemNeedsReSorting<ItemUnreadStatusComparator>(mSortOrder->messageSortDirection(),
1907  parent->d_ptr,
1908  static_cast<MessageItem *>(item))) {
1909  attachMessageToParent(parent, static_cast<MessageItem *>(item));
1910  }
1911  } // else new/unread status changed, but it doesn't match sorting order: no need to re-sort
1912  break;
1913  case SortOrder::SortMessagesByImportantStatus:
1914  if (propertyChangeMask & ImportantStatusChanged) { // important status changed
1915  if (messageItemNeedsReSorting<ItemImportantStatusComparator>(mSortOrder->messageSortDirection(),
1916  parent->d_ptr,
1917  static_cast<MessageItem *>(item))) {
1918  attachMessageToParent(parent, static_cast<MessageItem *>(item));
1919  }
1920  } // else new/unread status changed, but it doesn't match sorting order: no need to re-sort
1921  break;
1923  if (propertyChangeMask & AttachmentStatusChanged) { // attachment status changed
1924  if (messageItemNeedsReSorting<ItemAttachmentStatusComparator>(mSortOrder->messageSortDirection(),
1925  parent->d_ptr,
1926  static_cast<MessageItem *>(item))) {
1927  attachMessageToParent(parent, static_cast<MessageItem *>(item));
1928  }
1929  } // else new/unread status changed, but it doesn't match sorting order: no need to re-sort
1930  break;
1931  default:
1932  // this kind of message sorting isn't affected by the property changes: nothing to do.
1933  break;
1934  }
1935  }
1936 
1937  return false; // the invisible root isn't affected by any change.
1938  }
1939 
1940  if (parent->type() == Item::GroupHeader) {
1941  // item is a message attached to a GroupHeader.
1942  // It might need re-grouping or re-sorting (within the same group)
1943 
1944  // Check re-grouping here.
1945  if ((
1946  // max date changed
1947  (propertyChangeMask & MaxDateChanged) && // thread leader is most recent message
1948  (mAggregation->threadLeader() == Aggregation::MostRecentMessage))
1949  || (
1950  // date changed
1951  (propertyChangeMask & DateChanged) && // thread leader the topmost message
1952  (mAggregation->threadLeader() == Aggregation::TopmostMessage))) {
1953  // Might really need re-grouping.
1954  // attachMessageToGroupHeader() will find the right group for this message
1955  // and if it's different than the current it will move it.
1956  attachMessageToGroupHeader(static_cast<MessageItem *>(item));
1957  // Re-grouping fixes the properties of the involved group headers
1958  // so at exit of attachMessageToGroupHeader() the parent can't be affected
1959  // by the change anymore.
1960  return false;
1961  }
1962 
1963  // Re-grouping wasn't needed. Re-sorting might be.
1964  } // else item is a message attached to another message and might need re-sorting only.
1965 
1966  // Check if message needs re-sorting.
1967 
1968  switch (mSortOrder->messageSorting()) {
1970  if (propertyChangeMask & DateChanged) { // date changed
1971  if (messageItemNeedsReSorting<ItemDateComparator>(mSortOrder->messageSortDirection(), parent->d_ptr, static_cast<MessageItem *>(item))) {
1972  attachMessageToParent(parent, static_cast<MessageItem *>(item));
1973  }
1974  } // else date changed, but it doesn't match sorting order: no need to re-sort
1975  break;
1977  if (propertyChangeMask & MaxDateChanged) { // max date changed
1978  if (messageItemNeedsReSorting<ItemMaxDateComparator>(mSortOrder->messageSortDirection(), parent->d_ptr, static_cast<MessageItem *>(item))) {
1979  attachMessageToParent(parent, static_cast<MessageItem *>(item));
1980  }
1981  } // else max date changed, but it doesn't match sorting order: no need to re-sort
1982  break;
1984  if (propertyChangeMask & ActionItemStatusChanged) { // todo status changed
1985  if (messageItemNeedsReSorting<ItemActionItemStatusComparator>(mSortOrder->messageSortDirection(),
1986  parent->d_ptr,
1987  static_cast<MessageItem *>(item))) {
1988  attachMessageToParent(parent, static_cast<MessageItem *>(item));
1989  }
1990  } // else to do status changed, but it doesn't match sorting order: no need to re-sort
1991  break;
1993  if (propertyChangeMask & UnreadStatusChanged) { // new / unread status changed
1994  if (messageItemNeedsReSorting<ItemUnreadStatusComparator>(mSortOrder->messageSortDirection(), parent->d_ptr, static_cast<MessageItem *>(item))) {
1995  attachMessageToParent(parent, static_cast<MessageItem *>(item));
1996  }
1997  } // else new/unread status changed, but it doesn't match sorting order: no need to re-sort
1998  break;
1999  case SortOrder::SortMessagesByImportantStatus:
2000  if (propertyChangeMask & ImportantStatusChanged) { // important status changed
2001  if (messageItemNeedsReSorting<ItemImportantStatusComparator>(mSortOrder->messageSortDirection(), parent->d_ptr, static_cast<MessageItem *>(item))) {
2002  attachMessageToParent(parent, static_cast<MessageItem *>(item));
2003  }
2004  } // else important status changed, but it doesn't match sorting order: no need to re-sort
2005  break;
2007  if (propertyChangeMask & AttachmentStatusChanged) { // attachment status changed
2008  if (messageItemNeedsReSorting<ItemAttachmentStatusComparator>(mSortOrder->messageSortDirection(),
2009  parent->d_ptr,
2010  static_cast<MessageItem *>(item))) {
2011  attachMessageToParent(parent, static_cast<MessageItem *>(item));
2012  }
2013  } // else important status changed, but it doesn't match sorting order: no need to re-sort
2014  break;
2015  default:
2016  // this kind of message sorting isn't affected by property changes: nothing to do.
2017  break;
2018  }
2019 
2020  return true; // parent might be affected too.
2021 }
2022 
2023 void ModelPrivate::messageDetachedUpdateParentProperties(Item *oldParent, MessageItem *mi)
2024 {
2025  Q_ASSERT(oldParent);
2026  Q_ASSERT(mi);
2027  Q_ASSERT(oldParent != mRootItem);
2028 
2029  // oldParent might have its properties changed because of the child removal.
2030  // propagate the changes up.
2031  for (;;) {
2032  // pParent is not the root item now. This is assured by how we enter this loop
2033  // and by the fact that handleItemPropertyChanges returns false when grandParent
2034  // is Item::InvisibleRoot. We could actually assert it here...
2035 
2036  // Check if its dates need an update.
2037  int propertyChangeMask;
2038 
2039  if ((mi->maxDate() == oldParent->maxDate()) && oldParent->recomputeMaxDate()) {
2040  propertyChangeMask = MaxDateChanged;
2041  } else {
2042  break; // from the for(;;) loop
2043  }
2044 
2045  // One of the oldParent properties has changed for sure
2046 
2047  Item *grandParent = oldParent->parent();
2048 
2049  // If there is no grandParent then oldParent isn't attached to the view.
2050  // Re-sorting / re-grouping isn't needed for sure.
2051  if (!grandParent) {
2052  break; // from the for(;;) loop
2053  }
2054 
2055  // The following function will return true if grandParent may be affected by the change.
2056  // If the grandParent isn't affected, we stop climbing.
2057  if (!handleItemPropertyChanges(propertyChangeMask, grandParent, oldParent)) {
2058  break; // from the for(;;) loop
2059  }
2060 
2061  // Now we need to climb up one level and check again.
2062  oldParent = grandParent;
2063  } // for(;;) loop
2064 
2065  // If the last message was removed from a group header then this group will need an update
2066  // for sure. We will need to remove it (unless a message is attached back to it)
2067  if (oldParent->type() == Item::GroupHeader) {
2068  if (oldParent->childItemCount() == 0) {
2069  mGroupHeadersThatNeedUpdate.insert(static_cast<GroupHeaderItem *>(oldParent), static_cast<GroupHeaderItem *>(oldParent));
2070  }
2071  }
2072 }
2073 
2074 void ModelPrivate::propagateItemPropertiesToParent(Item *item)
2075 {
2076  Item *pParent = item->parent();
2077  Q_ASSERT(pParent);
2078  Q_ASSERT(pParent != mRootItem);
2079 
2080  for (;;) {
2081  // pParent is not the root item now. This is assured by how we enter this loop
2082  // and by the fact that handleItemPropertyChanges returns false when grandParent
2083  // is Item::InvisibleRoot. We could actually assert it here...
2084 
2085  // Check if its dates need an update.
2086  int propertyChangeMask;
2087 
2088  if (item->maxDate() > pParent->maxDate()) {
2089  pParent->setMaxDate(item->maxDate());
2090  propertyChangeMask = MaxDateChanged;
2091  } else {
2092  // No parent dates have changed: no further work is needed. Stop climbing here.
2093  break; // from the for(;;) loop
2094  }
2095 
2096  // One of the pParent properties has changed.
2097 
2098  Item *grandParent = pParent->parent();
2099 
2100  // If there is no grandParent then pParent isn't attached to the view.
2101  // Re-sorting / re-grouping isn't needed for sure.
2102  if (!grandParent) {
2103  break; // from the for(;;) loop
2104  }
2105 
2106  // The following function will return true if grandParent may be affected by the change.
2107  // If the grandParent isn't affected, we stop climbing.
2108  if (!handleItemPropertyChanges(propertyChangeMask, grandParent, pParent)) {
2109  break; // from the for(;;) loop
2110  }
2111 
2112  // Now we need to climb up one level and check again.
2113  pParent = grandParent;
2114  } // for(;;)
2115 }
2116 
2117 void ModelPrivate::attachMessageToParent(Item *pParent, MessageItem *mi, AttachOptions attachOptions)
2118 {
2119  Q_ASSERT(pParent);
2120  Q_ASSERT(mi);
2121 
2122  // This function may be called to do a simple "re-sort" of the item inside the parent.
2123  // In that case mi->parent() is equal to pParent.
2124  bool oldParentWasTheSame;
2125 
2126  if (mi->parent()) {
2127  Item *oldParent = mi->parent();
2128 
2129  // The item already had a parent and this means that we're moving it.
2130  oldParentWasTheSame = oldParent == pParent; // just re-sorting ?
2131 
2132  if (mi->isViewable()) { // is actually
2133  // The message is actually attached to the viewable root
2134 
2135  // Unfortunately we need to hack the model/view architecture
2136  // since it's somewhat flawed in this. At the moment of writing
2137  // there is simply no way to atomically move a subtree.
2138  // We must detach, call beginRemoveRows()/endRemoveRows(),
2139  // save the expanded state, save the selection, save the current item,
2140  // save the view position (YES! As we are removing items the view
2141  // will hopelessly jump around so we're just FORCED to break
2142  // the isolation from the view)...
2143  // ...*then* reattach, restore the expanded state, restore the selection,
2144  // restore the current item, restore the view position and pray
2145  // that nothing will fail in the (rather complicated) process....
2146 
2147  // Yet more unfortunately, while saving the expanded state might stop
2148  // at a certain (unexpanded) point in the tree, saving the selection
2149  // is hopelessly recursive down to the bare leafs.
2150 
2151  // Furthermore the expansion of items is a common case while selection
2152  // in the subtree is rare, so saving it would be a huge cost with
2153  // a low revenue.
2154 
2155  // This is why we just let the selection screw up. I hereby refuse to call
2156  // yet another expensive recursive function here :D
2157 
2158  // The current item saving can be somewhat optimized doing it once for
2159  // a single job step...
2160 
2161  if (((mi)->childItemCount() > 0) // has children
2162  && mModelForItemFunctions // the UI is not actually disconnected
2163  && mView->isExpanded(q->index(mi, 0)) // is actually expanded
2164  ) {
2165  saveExpandedStateOfSubtree(mi);
2166  }
2167  }
2168 
2169  // If the parent is viewable (so mi was viewable too) then the beginRemoveRows()
2170  // and endRemoveRows() functions of this model will be called too.
2171  oldParent->takeChildItem(mModelForItemFunctions, mi);
2172 
2173  if ((!oldParentWasTheSame) && (oldParent != mRootItem)) {
2174  messageDetachedUpdateParentProperties(oldParent, mi);
2175  }
2176  } else {
2177  // The item had no parent yet.
2178  oldParentWasTheSame = false;
2179  }
2180 
2181  // Take care of perfect / imperfect threading.
2182  // Items that are now perfectly threaded, but already have a different parent
2183  // might have been imperfectly threaded before. Remove them from the caches.
2184  // Items that are now imperfectly threaded must be added to the caches.
2185  //
2186  // If we're just re-sorting the item inside the same parent then the threading
2187  // caches don't need to be updated (since they actually depend on the parent).
2188 
2189  if (!oldParentWasTheSame) {
2190  switch (mi->threadingStatus()) {
2192  if (!mi->inReplyToIdMD5().isEmpty()) {
2193  mThreadingCacheMessageInReplyToIdMD5ToMessageItem.remove(mi->inReplyToIdMD5(), mi);
2194  }
2195  if (attachOptions == StoreInCache && pParent->type() == Item::Message) {
2196  mThreadingCache.updateParent(mi, static_cast<MessageItem *>(pParent));
2197  }
2198  break;
2200  case MessageItem::ParentMissing: // may be: temporary or just fallback assignment
2201  if (!mi->inReplyToIdMD5().isEmpty()) {
2202  if (!mThreadingCacheMessageInReplyToIdMD5ToMessageItem.contains(mi->inReplyToIdMD5(), mi)) {
2203  mThreadingCacheMessageInReplyToIdMD5ToMessageItem.insert(mi->inReplyToIdMD5(), mi);
2204  }
2205  }
2206  break;
2207  case MessageItem::NonThreadable: // this also happens when we do no threading at all
2208  // make gcc happy
2209  Q_ASSERT(!mThreadingCacheMessageInReplyToIdMD5ToMessageItem.contains(mi->inReplyToIdMD5(), mi));
2210  break;
2211  }
2212  }
2213 
2214  // Set the new parent
2215  mi->setParent(pParent);
2216 
2217  // Propagate watched and ignored status
2218  if ((pParent->status().toQInt32() & mCachedWatchedOrIgnoredStatusBits) // unlikely
2219  && (pParent->type() == Item::Message) // likely
2220  ) {
2221  // the parent is either watched or ignored: propagate to the child
2222  if (pParent->status().isWatched()) {
2223  int row = mInvariantRowMapper->modelInvariantIndexToModelIndexRow(mi);
2225  mStorageModel->setMessageItemStatus(mi, row, Akonadi::MessageStatus::statusWatched());
2226  } else if (pParent->status().isIgnored()) {
2227  int row = mInvariantRowMapper->modelInvariantIndexToModelIndexRow(mi);
2229  mStorageModel->setMessageItemStatus(mi, row, Akonadi::MessageStatus::statusIgnored());
2230  }
2231  }
2232 
2233  // And insert into its child list
2234 
2235  // If pParent is viewable then the insert/append functions will call this model's
2236  // beginInsertRows() and endInsertRows() functions. This is EXTREMELY
2237  // expensive and ugly but it's the only way with the Qt4 imposed Model/View method.
2238  // Dude... (citation from Lost, if it wasn't clear).
2239 
2240  // I'm using a macro since it does really improve readability.
2241  // I'm NOT using a helper function since gcc will refuse to inline some of
2242  // the calls because they make this function grow too much.
2243 #define INSERT_MESSAGE_WITH_COMPARATOR(_ItemComparator) \
2244  if ((mSortOrder->messageSortDirection() == SortOrder::Ascending) || (pParent->type() == Item::Message)) { \
2245  pParent->d_ptr->insertChildItem<_ItemComparator, true>(mModelForItemFunctions, mi); \
2246  } else { \
2247  pParent->d_ptr->insertChildItem<_ItemComparator, false>(mModelForItemFunctions, mi); \
2248  }
2249 
2250  // If pParent is viewable then the insertion call will also set the child state to viewable.
2251  // Since mi MAY have children, then this call may make them viewable.
2252  switch (mSortOrder->messageSorting()) {
2254  INSERT_MESSAGE_WITH_COMPARATOR(ItemDateComparator)
2255  break;
2257  INSERT_MESSAGE_WITH_COMPARATOR(ItemMaxDateComparator)
2258  break;
2260  INSERT_MESSAGE_WITH_COMPARATOR(ItemSizeComparator)
2261  break;
2263  INSERT_MESSAGE_WITH_COMPARATOR(ItemSenderOrReceiverComparator)
2264  break;
2266  INSERT_MESSAGE_WITH_COMPARATOR(ItemSenderComparator)
2267  break;
2269  INSERT_MESSAGE_WITH_COMPARATOR(ItemReceiverComparator)
2270  break;
2272  INSERT_MESSAGE_WITH_COMPARATOR(ItemSubjectComparator)
2273  break;
2275  INSERT_MESSAGE_WITH_COMPARATOR(ItemActionItemStatusComparator)
2276  break;
2278  INSERT_MESSAGE_WITH_COMPARATOR(ItemUnreadStatusComparator)
2279  break;
2280  case SortOrder::SortMessagesByImportantStatus:
2281  INSERT_MESSAGE_WITH_COMPARATOR(ItemImportantStatusComparator)
2282  break;
2284  INSERT_MESSAGE_WITH_COMPARATOR(ItemAttachmentStatusComparator)
2285  break;
2287  pParent->appendChildItem(mModelForItemFunctions, mi);
2288  break;
2289  default: // should never happen
2290  pParent->appendChildItem(mModelForItemFunctions, mi);
2291  break;
2292  }
2293 
2294  // Decide if we need to expand parents
2295  bool childNeedsExpanding = (mi->initialExpandStatus() == Item::ExpandNeeded);
2296 
2297  if (pParent->initialExpandStatus() == Item::NoExpandNeeded) {
2298  switch (mAggregation->threadExpandPolicy()) {
2300  // just do nothing unless this child has children and is already marked for expansion
2301  if (childNeedsExpanding) {
2303  }
2304  break;
2305  case Aggregation::ExpandThreadsWithNewMessages: // No more new status. fall through to unread if it exists in config
2307  // expand only if unread (or it has children marked for expansion)
2308  if (childNeedsExpanding || !mi->status().isRead()) {
2310  }
2311  break;
2313  // expand only if unread, important or todo (or it has children marked for expansion)
2314  // FIXME: Wouldn't it be nice to be able to test for bitmasks in MessageStatus ?
2315  if (childNeedsExpanding || !mi->status().isRead() || mi->status().isImportant() || mi->status().isToAct()) {
2317  }
2318  break;
2320  // expand everything
2322  break;
2323  default:
2324  // BUG
2325  break;
2326  }
2327  } // else it's already marked for expansion or expansion has been already executed
2328 
2329  // expand parent first, if possible
2330  if (pParent->initialExpandStatus() == Item::ExpandNeeded) {
2331  // If UI is not disconnected and parent is viewable, go up and expand
2332  if (mModelForItemFunctions && pParent->isViewable()) {
2333  // Now expand parents as needed
2334  Item *parentToExpand = pParent;
2335  while (parentToExpand) {
2336  if (parentToExpand == mRootItem) {
2337  break; // no need to set it expanded
2338  }
2339  // parentToExpand is surely viewable (because this item is)
2340  if (parentToExpand->initialExpandStatus() == Item::ExpandExecuted) {
2341  break;
2342  }
2343 
2344  mView->expand(q->index(parentToExpand, 0));
2345 
2347  parentToExpand = parentToExpand->parent();
2348  }
2349  } else {
2350  // It isn't viewable or UI is disconnected: climb up marking only
2351  Item *parentToExpand = pParent->parent();
2352  while (parentToExpand) {
2353  if (parentToExpand == mRootItem) {
2354  break; // no need to set it expanded
2355  }
2356  parentToExpand->setInitialExpandStatus(Item::ExpandNeeded);
2357  parentToExpand = parentToExpand->parent();
2358  }
2359  }
2360  }
2361 
2362  if (mi->isViewable()) {
2363  // mi is now viewable
2364 
2365  // sync subtree expanded status
2366  if (childNeedsExpanding) {
2367  if (mi->childItemCount() > 0) {
2368  if (mModelForItemFunctions) { // the UI is not disconnected
2369  syncExpandedStateOfSubtree(mi); // sync the real state in the view
2370  }
2371  }
2372  }
2373 
2374  // apply the filter, if needed
2375  if (mFilter) {
2376  Q_ASSERT(mModelForItemFunctions); // the UI must be NOT disconnected here
2377 
2378  // apply the filter to subtree
2379  if (applyFilterToSubtree(mi, q->index(pParent, 0))) {
2380  // mi matched, expand parents (unconditionally)
2381  mView->ensureDisplayedWithParentsExpanded(mi);
2382  }
2383  }
2384  }
2385 
2386  // Now we need to propagate the property changes the upper levels.
2387 
2388  // If we have just inserted a message inside the root then no work needs to be done:
2389  // no grouping is in effect and the message is already in the right place.
2390  if (pParent == mRootItem) {
2391  return;
2392  }
2393 
2394  // If we have just removed the item from this parent and re-inserted it
2395  // then this operation was a simple re-sort. The code above didn't update
2396  // the properties when removing the item so we don't actually need
2397  // to make the updates back.
2398  if (oldParentWasTheSame) {
2399  return;
2400  }
2401 
2402  // FIXME: OPTIMIZE THIS: First propagate changes THEN syncExpandedStateOfSubtree()
2403  // and applyFilterToSubtree... (needs some thinking though).
2404 
2405  // Time to propagate up.
2406  propagateItemPropertiesToParent(mi);
2407 
2408  // Aaah.. we're done. Time for a thea ? :)
2409 }
2410 
2411 // FIXME: ThreadItem ?
2412 //
2413 // Foo Bar, Joe Thommason, Martin Rox ... Eddie Maiden <date of the thread>
2414 // Title <number of messages>, Last by xxx <inner status>
2415 //
2416 // When messages are added, mark it as dirty only (?)
2417 
2418 ModelPrivate::ViewItemJobResult ModelPrivate::viewItemJobStepInternalForJobPass5(ViewItemJob *job, QElapsedTimer elapsedTimer)
2419 {
2420  // In this pass we scan the group headers that are in mGroupHeadersThatNeedUpdate.
2421  // Empty groups get deleted while the other ones are re-sorted.
2422 
2423  int curIndex = job->currentIndex();
2424 
2425  auto it = mGroupHeadersThatNeedUpdate.begin();
2426  auto end = mGroupHeadersThatNeedUpdate.end();
2427 
2428  while (it != end) {
2429  if ((*it)->childItemCount() == 0) {
2430  // group with no children, kill it
2431  (*it)->parent()->takeChildItem(mModelForItemFunctions, *it);
2432  mGroupHeaderItemHash.remove((*it)->label());
2433 
2434  // If we were going to restore its position after the job step, well.. we can't do it anymore.
2435  if (mCurrentItemToRestoreAfterViewItemJobStep == (*it)) {
2436  mCurrentItemToRestoreAfterViewItemJobStep = nullptr;
2437  }
2438 
2439  // bye bye
2440  delete *it;
2441  } else {
2442  // Group with children: probably needs re-sorting.
2443 
2444  // Re-sorting here is an expensive operation.
2445  // In fact groups have been put in the QHash above on the assumption
2446  // that re-sorting *might* be needed but no real (expensive) check
2447  // has been done yet. Also by sorting a single group we might actually
2448  // put the others in the right place.
2449  // So finally check if re-sorting is *really* needed.
2450  bool needsReSorting;
2451 
2452  // A macro really improves readability here.
2453 #define CHECK_IF_GROUP_NEEDS_RESORTING(_ItemDateComparator) \
2454  switch (mSortOrder->groupSortDirection()) { \
2455  case SortOrder::Ascending: \
2456  needsReSorting = (*it)->parent()->d_ptr->childItemNeedsReSorting<_ItemDateComparator, true>(*it); \
2457  break; \
2458  case SortOrder::Descending: \
2459  needsReSorting = (*it)->parent()->d_ptr->childItemNeedsReSorting<_ItemDateComparator, false>(*it); \
2460  break; \
2461  default: /* should never happen */ \
2462  needsReSorting = false; \
2463  break; \
2464  }
2465 
2466  switch (mSortOrder->groupSorting()) {
2468  CHECK_IF_GROUP_NEEDS_RESORTING(ItemDateComparator)
2469  break;
2471  CHECK_IF_GROUP_NEEDS_RESORTING(ItemMaxDateComparator)
2472  break;
2474  CHECK_IF_GROUP_NEEDS_RESORTING(ItemSenderOrReceiverComparator)
2475  break;
2477  CHECK_IF_GROUP_NEEDS_RESORTING(ItemSenderComparator)
2478  break;
2480  CHECK_IF_GROUP_NEEDS_RESORTING(ItemReceiverComparator)
2481  break;
2483  needsReSorting = false;
2484  break;
2485  default:
2486  // Should never happen... just assume re-sorting is not needed
2487  needsReSorting = false;
2488  break;
2489  }
2490 
2491  if (needsReSorting) {
2492  attachGroup(*it); // it will first detach and then re-attach in the proper place
2493  }
2494  }
2495 
2496  it = mGroupHeadersThatNeedUpdate.erase(it);
2497 
2498  curIndex++;
2499 
2500  // FIXME: In fact a single update is likely to manipulate
2501  // a subtree with a LOT of messages inside. If interactivity is favored
2502  // we should check the time really more often.
2503  if ((curIndex % mViewItemJobStepMessageCheckCount) == 0) {
2504  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
2505  if (it != mGroupHeadersThatNeedUpdate.end()) {
2506  job->setCurrentIndex(curIndex);
2507  return ViewItemJobInterrupted;
2508  }
2509  }
2510  }
2511  }
2512 
2513  return ViewItemJobCompleted;
2514 }
2515 
2516 ModelPrivate::ViewItemJobResult ModelPrivate::viewItemJobStepInternalForJobPass4(ViewItemJob *job, QElapsedTimer elapsedTimer)
2517 {
2518  // In this pass we scan mUnassignedMessageListForPass4 which now
2519  // contains both items with parents and items without parents.
2520  // We scan mUnassignedMessageList for messages without parent (the ones that haven't been
2521  // attached to the viewable tree yet) and find a suitable group for them. Then we simply
2522  // clear mUnassignedMessageList.
2523 
2524  // We call this pass "Grouping"
2525 
2526  int curIndex = job->currentIndex();
2527  int endIndex = job->endIndex();
2528 
2529  while (curIndex <= endIndex) {
2530  MessageItem *mi = mUnassignedMessageListForPass4[curIndex];
2531  if (!mi->parent()) {
2532  // Unassigned item: thread leader, insert into the proper group.
2533  // Locate the group (or root if no grouping requested)
2534  attachMessageToGroupHeader(mi);
2535  } else {
2536  // A parent was already assigned in Pass3: we have nothing to do here
2537  }
2538  curIndex++;
2539 
2540  // FIXME: In fact a single call to attachMessageToGroupHeader() is likely to manipulate
2541  // a subtree with a LOT of messages inside. If interactivity is favored
2542  // we should check the time really more often.
2543  if ((curIndex % mViewItemJobStepMessageCheckCount) == 0) {
2544  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
2545  if (curIndex <= endIndex) {
2546  job->setCurrentIndex(curIndex);
2547  return ViewItemJobInterrupted;
2548  }
2549  }
2550  }
2551  }
2552 
2553  mUnassignedMessageListForPass4.clear();
2554  return ViewItemJobCompleted;
2555 }
2556 
2557 ModelPrivate::ViewItemJobResult ModelPrivate::viewItemJobStepInternalForJobPass3(ViewItemJob *job, QElapsedTimer elapsedTimer)
2558 {
2559  // In this pass we scan the mUnassignedMessageListForPass3 and try to do construct the threads
2560  // by using subject based threading. If subject based threading is not in effect then
2561  // this pass turns to a nearly-no-op: at the end of Pass2 we have swapped the lists
2562  // and mUnassignedMessageListForPass3 is actually empty.
2563 
2564  // We don't shrink the mUnassignedMessageListForPass3 for two reasons:
2565  // - It would mess up this chunked algorithm by shifting indexes
2566  // - mUnassignedMessageList is a QList which is basically an array. It's faster
2567  // to traverse an array of N entries than to remove K>0 entries one by one and
2568  // to traverse the remaining N-K entries.
2569 
2570  int curIndex = job->currentIndex();
2571  int endIndex = job->endIndex();
2572 
2573  while (curIndex <= endIndex) {
2574  // If we're here, then threading is requested for sure.
2575  auto mi = mUnassignedMessageListForPass3[curIndex];
2576  if ((!mi->parent()) || (mi->threadingStatus() == MessageItem::ParentMissing)) {
2577  // Parent is missing (either "physically" with the item being not attached or "logically"
2578  // with the item being attached to a group or directly to the root.
2579  if (mi->subjectIsPrefixed()) {
2580  // We can try to guess it
2581  auto mparent = guessMessageParent(mi);
2582 
2583  if (mparent) {
2584  // imperfect parent found
2585  if (mi->isViewable()) {
2586  // mi was already viewable, we're just trying to re-parent it better...
2587  attachMessageToParent(mparent, mi);
2588  if (!mparent->isViewable()) {
2589  // re-attach it immediately (so current item is not lost)
2590  auto topmost = mparent->topmostMessage();
2591  Q_ASSERT(!topmost->parent()); // groups are always viewable!
2592  topmost->setThreadingStatus(MessageItem::ParentMissing);
2593  attachMessageToGroupHeader(topmost);
2594  }
2595  } else {
2596  // mi wasn't viewable yet.. no need to attach parent
2597  attachMessageToParent(mparent, mi);
2598  }
2599  // and we're done for now
2600  } else {
2601  // so parent not found, (threadingStatus() is either MessageItem::ParentMissing or MessageItem::NonThreadable)
2602  Q_ASSERT((mi->threadingStatus() == MessageItem::ParentMissing) || (mi->threadingStatus() == MessageItem::NonThreadable));
2603  mUnassignedMessageListForPass4.append(mi); // this is ~O(1)
2604  // and wait for Pass4
2605  }
2606  } else {
2607  // can't guess the parent as the subject isn't prefixed
2608  Q_ASSERT((mi->threadingStatus() == MessageItem::ParentMissing) || (mi->threadingStatus() == MessageItem::NonThreadable));
2609  mUnassignedMessageListForPass4.append(mi); // this is ~O(1)
2610  // and wait for Pass4
2611  }
2612  } else {
2613  // Has a parent: either perfect parent already found or non threadable.
2614  // Since we don't end here if mi has status of parent missing then mi must not have imperfect parent.
2615  Q_ASSERT(mi->threadingStatus() != MessageItem::ImperfectParentFound);
2616  Q_ASSERT(mi->isViewable());
2617  }
2618 
2619  curIndex++;
2620 
2621  // FIXME: In fact a single call to attachMessageToGroupHeader() is likely to manipulate
2622  // a subtree with a LOT of messages inside. If interactivity is favored
2623  // we should check the time really more often.
2624  if ((curIndex % mViewItemJobStepMessageCheckCount) == 0) {
2625  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
2626  if (curIndex <= endIndex) {
2627  job->setCurrentIndex(curIndex);
2628  return ViewItemJobInterrupted;
2629  }
2630  }
2631  }
2632  }
2633 
2634  mUnassignedMessageListForPass3.clear();
2635  return ViewItemJobCompleted;
2636 }
2637 
2638 ModelPrivate::ViewItemJobResult ModelPrivate::viewItemJobStepInternalForJobPass2(ViewItemJob *job, QElapsedTimer elapsedTimer)
2639 {
2640  // In this pass we scan the mUnassignedMessageList and try to do construct the threads.
2641  // If some thread leader message got attached to the viewable tree in Pass1Fill then
2642  // we'll also attach all of its children too. The thread leaders we were unable
2643  // to attach in Pass1Fill and their children (which we find here) will make it to the small Pass3
2644 
2645  // We don't shrink the mUnassignedMessageList for two reasons:
2646  // - It would mess up this chunked algorithm by shifting indexes
2647  // - mUnassignedMessageList is a QList which is basically an array. It's faster
2648  // to traverse an array of N entries than to remove K>0 entries one by one and
2649  // to traverse the remaining N-K entries.
2650 
2651  // We call this pass "Threading"
2652 
2653  int curIndex = job->currentIndex();
2654  int endIndex = job->endIndex();
2655 
2656  while (curIndex <= endIndex) {
2657  // If we're here, then threading is requested for sure.
2658  auto mi = mUnassignedMessageListForPass2[curIndex];
2659  // The item may or may not have a parent.
2660  // If it has no parent or it has a temporary one (mi->parent() && mi->threadingStatus() == MessageItem::ParentMissing)
2661  // then we attempt to (re-)thread it. Otherwise we just do nothing (the job has already been done by the previous steps).
2662  if ((!mi->parent()) || (mi->threadingStatus() == MessageItem::ParentMissing)) {
2663  qint64 parentId;
2664  auto mparent = mThreadingCache.parentForItem(mi, parentId);
2665  if (mparent && !mparent->hasAncestor(mi)) {
2666  mi->setThreadingStatus(MessageItem::PerfectParentFound);
2667  attachMessageToParent(mparent, mi, SkipCacheUpdate);
2668  } else {
2669  if (parentId > 0) {
2670  // In second pass we have all available Items in mThreadingCache already. If
2671  // mThreadingCache.parentForItem() returns null, but returns valid parentId then
2672  // the Item was removed from Akonadi and our threading cache is out-of-date.
2673  mThreadingCache.expireParent(mi);
2674  mparent = findMessageParent(mi);
2675  } else if (parentId < 0) {
2676  mparent = findMessageParent(mi);
2677  } else {
2678  // parentId = 0: this message is a thread leader so don't
2679  // bother resolving parent, it will be moved directly to
2680  // Pass4 in the code below
2681  }
2682 
2683  if (mparent) {
2684  // parent found, either perfect or imperfect
2685  if (mi->isViewable()) {
2686  // mi was already viewable, we're just trying to re-parent it better...
2687  attachMessageToParent(mparent, mi);
2688  if (!mparent->isViewable()) {
2689  // re-attach it immediately (so current item is not lost)
2690  auto topmost = mparent->topmostMessage();
2691  Q_ASSERT(!topmost->parent()); // groups are always viewable!
2692  topmost->setThreadingStatus(MessageItem::ParentMissing);
2693  attachMessageToGroupHeader(topmost);
2694  }
2695  } else {
2696  // mi wasn't viewable yet.. no need to attach parent
2697  attachMessageToParent(mparent, mi);
2698  }
2699  // and we're done for now
2700  } else {
2701  // so parent not found, (threadingStatus() is either MessageItem::ParentMissing or MessageItem::NonThreadable)
2702  switch (mi->threadingStatus()) {
2704  if (mAggregation->threading() == Aggregation::PerfectReferencesAndSubject) {
2705  // parent missing but still can be found in Pass3
2706  mUnassignedMessageListForPass3.append(mi); // this is ~O(1)
2707  } else {
2708  // We're not doing subject based threading: will never be threaded, go straight to Pass4
2709  mUnassignedMessageListForPass4.append(mi); // this is ~O(1)
2710  }
2711  break;
2713  // will never be threaded, go straight to Pass4
2714  mUnassignedMessageListForPass4.append(mi); // this is ~O(1)
2715  break;
2716  default:
2717  // a bug for sure
2718  qCWarning(MESSAGELIST_LOG) << "ERROR: Invalid message threading status returned by findMessageParent()!";
2719  Q_ASSERT(false);
2720  break;
2721  }
2722  }
2723  }
2724  } else {
2725  // Has a parent: either perfect parent already found or non threadable.
2726  // Since we don't end here if mi has status of parent missing then mi must not have imperfect parent.
2727  Q_ASSERT(mi->threadingStatus() != MessageItem::ImperfectParentFound);
2728  if (!mi->isViewable()) {
2729  qCWarning(MESSAGELIST_LOG) << "Non viewable message " << mi << " subject " << mi->subject().toUtf8().data();
2730  Q_ASSERT(mi->isViewable());
2731  }
2732  }
2733 
2734  curIndex++;
2735 
2736  // FIXME: In fact a single call to attachMessageToGroupHeader() is likely to manipulate
2737  // a subtree with a LOT of messages inside. If interactivity is favored
2738  // we should check the time really more often.
2739  if ((curIndex % mViewItemJobStepMessageCheckCount) == 0) {
2740  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
2741  if (curIndex <= endIndex) {
2742  job->setCurrentIndex(curIndex);
2743  return ViewItemJobInterrupted;
2744  }
2745  }
2746  }
2747  }
2748 
2749  mUnassignedMessageListForPass2.clear();
2750  return ViewItemJobCompleted;
2751 }
2752 
2753 ModelPrivate::ViewItemJobResult ModelPrivate::viewItemJobStepInternalForJobPass1Fill(ViewItemJob *job, QElapsedTimer elapsedTimer)
2754 {
2755  // In this pass we scan the a contiguous region of the underlying storage (that is
2756  // assumed to be FLAT) and create the corresponding MessageItem objects.
2757  // The deal is to show items to the user as soon as possible so in this pass we
2758  // *TRY* to attach them to the viewable tree (which is rooted on mRootItem).
2759  // Messages we're unable to attach for some reason (mainly due to threading) get appended
2760  // to mUnassignedMessageList and wait for Pass2.
2761 
2762  // We call this pass "Processing"
2763 
2764  // Should we use the receiver or the sender field for sorting ?
2765  bool bUseReceiver = mStorageModelContainsOutboundMessages;
2766 
2767  // The begin storage index of our work
2768  int curIndex = job->currentIndex();
2769  // The end storage index of our work.
2770  int endIndex = job->endIndex();
2771 
2772  unsigned long msgToSelect = mPreSelectionMode == PreSelectLastSelected ? mStorageModel->preSelectedMessage() : 0;
2773 
2774  MessageItem *mi = nullptr;
2775 
2776  while (curIndex <= endIndex) {
2777  // Create the message item with no parent: we'll set it later
2778  if (!mi) {
2779  mi = new MessageItem();
2780  } else {
2781  // a MessageItem discarded by a previous iteration: reuse it.
2782  Q_ASSERT(mi->parent() == nullptr);
2783  }
2784 
2785  if (!mStorageModel->initializeMessageItem(mi, curIndex, bUseReceiver)) {
2786  // ugh
2787  qCWarning(MESSAGELIST_LOG) << "Fill of the MessageItem at storage row index " << curIndex << " failed";
2788  curIndex++;
2789  continue;
2790  }
2791 
2792  // If we're supposed to pre-select a specific message, check if it's this one.
2793  if (msgToSelect != 0 && msgToSelect == mi->uniqueId()) {
2794  // Found, it's this one.
2795  // But actually it's not viewable (so not selectable). We must wait
2796  // until the end of the job to be 100% sure. So here we just translate
2797  // the unique id to a MessageItem pointer and wait.
2798  mLastSelectedMessageInFolder = mi;
2799  msgToSelect = 0; // already found, don't bother checking anymore
2800  }
2801 
2802  // Update the newest/oldest message, since we might be supposed to select those later
2803  if (mi->date() != static_cast<uint>(-1)) {
2804  if (!mOldestItem || mOldestItem->date() > mi->date()) {
2805  mOldestItem = mi;
2806  }
2807  if (!mNewestItem || mNewestItem->date() < mi->date()) {
2808  mNewestItem = mi;
2809  }
2810  }
2811 
2812  // Ok.. it passed the initial checks: we will not be discarding it.
2813  // Make this message item an invariant index to the underlying model storage.
2814  mInvariantRowMapper->createModelInvariantIndex(curIndex, mi);
2815 
2816  // Attempt to do threading as soon as possible (to display items to the user)
2817  if (mAggregation->threading() != Aggregation::NoThreading) {
2818  // Threading is requested
2819 
2820  // Fetch the data needed for proper threading
2821  // Add the item to the threading caches
2822 
2823  switch (mAggregation->threading()) {
2825  mStorageModel->fillMessageItemThreadingData(mi, curIndex, StorageModel::PerfectThreadingReferencesAndSubject);
2826 
2827  // We also need to build the subject/reference-based threading cache
2828  addMessageToReferencesBasedThreadingCache(mi);
2829  addMessageToSubjectBasedThreadingCache(mi);
2830  break;
2832  mStorageModel->fillMessageItemThreadingData(mi, curIndex, StorageModel::PerfectThreadingPlusReferences);
2833  addMessageToReferencesBasedThreadingCache(mi);
2834  break;
2835  default:
2836  mStorageModel->fillMessageItemThreadingData(mi, curIndex, StorageModel::PerfectThreadingOnly);
2837  break;
2838  }
2839 
2840  // Perfect/References threading cache
2841  mThreadingCacheMessageIdMD5ToMessageItem.insert(mi->messageIdMD5(), mi);
2842 
2843  // Register the current item into the threading cache
2844  mThreadingCache.addItemToCache(mi);
2845 
2846  // First of all look into the persistent cache
2847  qint64 parentId;
2848  Item *pParent = mThreadingCache.parentForItem(mi, parentId);
2849  if (pParent) {
2850  // We already have the parent MessageItem. Attach current message
2851  // to it and mark it as perfect
2852  mi->setThreadingStatus(MessageItem::PerfectParentFound);
2853  attachMessageToParent(pParent, mi);
2854  } else if (parentId > 0) {
2855  // We don't have the parent MessageItem yet, but we do know the
2856  // parent: delay for pass 2 when we will have the parent MessageItem
2857  // for sure.
2858  mi->setThreadingStatus(MessageItem::ParentMissing);
2859  mUnassignedMessageListForPass2.append(mi);
2860  } else if (parentId == 0) {
2861  // Message is a thread leader, skip straight to Pass4
2862  mi->setThreadingStatus(MessageItem::NonThreadable);
2863  mUnassignedMessageListForPass4.append(mi);
2864  } else {
2865  // Check if this item is a perfect parent for some imperfectly threaded
2866  // message (that is actually attached to it, but not necessarily to the
2867  // viewable root). If it is, then remove the imperfect child from its
2868  // current parent rebuild the hierarchy on the fly.
2869  bool needsImmediateReAttach = false;
2870 
2871  if (!mThreadingCacheMessageInReplyToIdMD5ToMessageItem.isEmpty()) { // unlikely
2872  const auto lImperfectlyThreaded = mThreadingCacheMessageInReplyToIdMD5ToMessageItem.values(mi->messageIdMD5());
2873  for (const auto it : lImperfectlyThreaded) {
2874  Q_ASSERT(it->parent());
2875  Q_ASSERT(it->parent() != mi);
2876 
2877  if (!((it->threadingStatus() == MessageItem::ImperfectParentFound) || (it->threadingStatus() == MessageItem::ParentMissing))) {
2878  qCritical() << "Got message " << it << " with threading status" << it->threadingStatus();
2879  Q_ASSERT_X(false, "ModelPrivate::viewItemJobStepInternalForJobPass1Fill", "Wrong threading status");
2880  }
2881 
2882  // If the item was already attached to the view then
2883  // re-attach it immediately. This will avoid a message
2884  // being displayed for a short while in the view and then
2885  // disappear until a perfect parent isn't found.
2886  if (it->isViewable()) {
2887  needsImmediateReAttach = true;
2888  }
2889 
2890  it->setThreadingStatus(MessageItem::PerfectParentFound);
2891  attachMessageToParent(mi, it);
2892  }
2893  }
2894 
2895  // FIXME: Might look by "References" too, here... (?)
2896 
2897  // Attempt to do threading with anything we already have in caches until now
2898  // Note that this is likely to work since thread-parent messages tend
2899  // to come before thread-children messages in the folders (simply because of
2900  // date of arrival).
2901 
2902  // First of all try to find a "perfect parent", that is the message for that
2903  // we have the ID in the "In-Reply-To" field. This is actually done by using
2904  // MD5 caches of the message ids because of speed. Collisions are very unlikely.
2905 
2906  const QByteArray md5 = mi->inReplyToIdMD5();
2907  if (!md5.isEmpty()) {
2908  // Have an In-Reply-To field MD5.
2909  // In well behaved mailing lists 70% of the threadable messages get a parent here :)
2910  pParent = mThreadingCacheMessageIdMD5ToMessageItem.value(md5, nullptr);
2911 
2912  if (pParent) { // very likely
2913  // Take care of self-referencing (which is always possible)
2914  // and circular In-Reply-To reference loops which are possible
2915  // in case this item was found to be a perfect parent for some
2916  // imperfectly threaded message just above.
2917  if ((mi == pParent) // self referencing message
2918  || ((mi->childItemCount() > 0) // mi already has children, this is fast to determine
2919  && pParent->hasAncestor(mi) // pParent is in the mi's children tree
2920  )) {
2921  // Bad, bad message.. it has In-Reply-To equal to Message-Id
2922  // or it's in a circular In-Reply-To reference loop.
2923  // Will wait for Pass2 with References-Id only
2924  qCWarning(MESSAGELIST_LOG) << "Circular In-Reply-To reference loop detected in the message tree";
2925  mUnassignedMessageListForPass2.append(mi);
2926  } else {
2927  // wow, got a perfect parent for this message!
2928  mi->setThreadingStatus(MessageItem::PerfectParentFound);
2929  attachMessageToParent(pParent, mi);
2930  // we're done with this message (also for Pass2)
2931  }
2932  } else {
2933  // got no parent
2934  // will have to wait Pass2
2935  mUnassignedMessageListForPass2.append(mi);
2936  }
2937  } else {
2938  // No In-Reply-To header.
2939 
2940  bool mightHaveOtherMeansForThreading;
2941 
2942  switch (mAggregation->threading()) {
2944  mightHaveOtherMeansForThreading = mi->subjectIsPrefixed() || !mi->referencesIdMD5().isEmpty();
2945  break;
2947  mightHaveOtherMeansForThreading = !mi->referencesIdMD5().isEmpty();
2948  break;
2950  mightHaveOtherMeansForThreading = false;
2951  break;
2952  default:
2953  // BUG: there shouldn't be other values (NoThreading is excluded in an upper branch)
2954  Q_ASSERT(false);
2955  mightHaveOtherMeansForThreading = false; // make gcc happy
2956  break;
2957  }
2958 
2959  if (mightHaveOtherMeansForThreading) {
2960  // We might have other means for threading this message, wait until Pass2
2961  mUnassignedMessageListForPass2.append(mi);
2962  } else {
2963  // No other means for threading this message. This is either
2964  // a standalone message or a thread leader.
2965  // If there is no grouping in effect or thread leaders are just the "topmost"
2966  // messages then we might be done with this one.
2967  if ((mAggregation->grouping() == Aggregation::NoGrouping) || (mAggregation->threadLeader() == Aggregation::TopmostMessage)) {
2968  // We're done with this message: it will be surely either toplevel (no grouping in effect)
2969  // or a thread leader with a well defined group. Do it :)
2970  // qCDebug(MESSAGELIST_LOG) << "Setting message status from " << mi->threadingStatus() << " to non threadable (1) " << mi;
2971  mi->setThreadingStatus(MessageItem::NonThreadable);
2972  // Locate the parent group for this item
2973  attachMessageToGroupHeader(mi);
2974  // we're done with this message (also for Pass2)
2975  } else {
2976  // Threads belong to the most recent message in the thread. This means
2977  // that we have to wait until Pass2 or Pass3 to assign a group.
2978  mUnassignedMessageListForPass2.append(mi);
2979  }
2980  }
2981  }
2982 
2983  if (needsImmediateReAttach && !mi->isViewable()) {
2984  // The item gathered previously viewable children. They must be immediately
2985  // re-shown. So this item must currently be attached to the view.
2986  // This is a temporary measure: it will be probably still moved.
2987  MessageItem *topmost = mi->topmostMessage();
2988  Q_ASSERT(topmost->threadingStatus() == MessageItem::ParentMissing);
2989  attachMessageToGroupHeader(topmost);
2990  }
2991  }
2992  } else {
2993  // else no threading requested: we don't even need Pass2
2994  // set not threadable status (even if it might be not true, but in this mode we don't care)
2995  // qCDebug(MESSAGELIST_LOG) << "Setting message status from " << mi->threadingStatus() << " to non threadable (2) " << mi;
2996  mi->setThreadingStatus(MessageItem::NonThreadable);
2997  // locate the parent group for this item
2998  if (mAggregation->grouping() == Aggregation::NoGrouping) {
2999  attachMessageToParent(mRootItem, mi); // no groups requested, attach directly to root
3000  } else {
3001  attachMessageToGroupHeader(mi);
3002  }
3003  // we're done with this message (also for Pass2)
3004  }
3005 
3006  mi = nullptr; // this item was pushed somewhere, create a new one at next iteration
3007  curIndex++;
3008 
3009  if ((curIndex % mViewItemJobStepMessageCheckCount) == 0) {
3010  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
3011  if (curIndex <= endIndex) {
3012  job->setCurrentIndex(curIndex);
3013  return ViewItemJobInterrupted;
3014  }
3015  }
3016  }
3017  }
3018 
3019  if (mi) {
3020  delete mi;
3021  }
3022  return ViewItemJobCompleted;
3023 }
3024 
3025 ModelPrivate::ViewItemJobResult ModelPrivate::viewItemJobStepInternalForJobPass1Cleanup(ViewItemJob *job, QElapsedTimer elapsedTimer)
3026 {
3027  Q_ASSERT(mModelForItemFunctions); // UI must be not disconnected here
3028  // In this pass we remove the MessageItem objects that are present in the job
3029  // and put their children in the unassigned message list.
3030 
3031  // Note that this list in fact contains MessageItem objects (we need dynamic_cast<>).
3032  QList<ModelInvariantIndex *> *invalidatedMessages = job->invariantIndexList();
3033 
3034  // We don't shrink the invalidatedMessages because it's basically an array.
3035  // It's faster to traverse an array of N entries than to remove K>0 entries
3036  // one by one and to traverse the remaining N-K entries.
3037 
3038  // The begin index of our work
3039  int curIndex = job->currentIndex();
3040  // The end index of our work.
3041  int endIndex = job->endIndex();
3042 
3043  if (curIndex == job->startIndex()) {
3044  Q_ASSERT(mOrphanChildrenHash.isEmpty());
3045  }
3046 
3047  while (curIndex <= endIndex) {
3048  // Get the underlying storage message data...
3049  auto dyingMessage = dynamic_cast<MessageItem *>(invalidatedMessages->at(curIndex));
3050  // This MUST NOT be null (otherwise we have a bug somewhere in this file).
3051  Q_ASSERT(dyingMessage);
3052 
3053  // If we were going to pre-select this message but we were interrupted
3054  // *before* it was actually made viewable, we just clear the pre-selection pointer
3055  // and unique id (abort pre-selection).
3056  if (dyingMessage == mLastSelectedMessageInFolder) {
3057  mLastSelectedMessageInFolder = nullptr;
3058  mPreSelectionMode = PreSelectNone;
3059  }
3060 
3061  // remove the message from any pending user job
3062  if (mPersistentSetManager) {
3063  mPersistentSetManager->removeMessageItemFromAllSets(dyingMessage);
3064  if (mPersistentSetManager->setCount() < 1) {
3065  delete mPersistentSetManager;
3066  mPersistentSetManager = nullptr;
3067  }
3068  }
3069 
3070  // Remove the message from threading cache before we start moving up the
3071  // children, so that they don't get mislead by the cache
3072  mThreadingCache.expireParent(dyingMessage);
3073 
3074  if (dyingMessage->parent()) {
3075  // Handle saving the current selection: if this item was the current before the step
3076  // then zero it out. We have killed it and it's OK for the current item to change.
3077 
3078  if (dyingMessage == mCurrentItemToRestoreAfterViewItemJobStep) {
3079  Q_ASSERT(dyingMessage->isViewable());
3080  // Try to select the item below the removed one as it helps in doing a "readon" of emails:
3081  // you read a message, decide to delete it and then go to the next.
3082  // Qt tends to select the message above the removed one instead (this is a hardcoded logic in
3083  // QItemSelectionModelPrivate::_q_rowsAboutToBeRemoved()).
3084  mCurrentItemToRestoreAfterViewItemJobStep = mView->messageItemAfter(dyingMessage, MessageTypeAny, false);
3085 
3086  if (!mCurrentItemToRestoreAfterViewItemJobStep) {
3087  // There is no item below. Try the item above.
3088  // We still do it better than qt which tends to find the *thread* above
3089  // instead of the item above.
3090  mCurrentItemToRestoreAfterViewItemJobStep = mView->messageItemBefore(dyingMessage, MessageTypeAny, false);
3091  }
3092 
3093  Q_ASSERT((!mCurrentItemToRestoreAfterViewItemJobStep) || mCurrentItemToRestoreAfterViewItemJobStep->isViewable());
3094  }
3095 
3096  if (dyingMessage->isViewable() && ((dyingMessage)->childItemCount() > 0) // has children
3097  && mView->isExpanded(q->index(dyingMessage, 0)) // is actually expanded
3098  ) {
3099  saveExpandedStateOfSubtree(dyingMessage);
3100  }
3101 
3102  auto oldParent = dyingMessage->parent();
3103  oldParent->takeChildItem(q, dyingMessage);
3104 
3105  // FIXME: This can generate many message movements.. it would be nicer
3106  // to start from messages that are higher in the hierarchy so
3107  // we would need to move less stuff above.
3108 
3109  if (oldParent != mRootItem) {
3110  messageDetachedUpdateParentProperties(oldParent, dyingMessage);
3111  }
3112 
3113  // We might have already removed its parent from the view, so it
3114  // might already be in the orphan child hash...
3115  if (dyingMessage->threadingStatus() == MessageItem::ParentMissing) {
3116  mOrphanChildrenHash.remove(dyingMessage); // this can turn to a no-op (dyingMessage not present in fact)
3117  }
3118  } else {
3119  // The dying message had no parent: this should happen only if it's already an orphan
3120 
3121  Q_ASSERT(dyingMessage->threadingStatus() == MessageItem::ParentMissing);
3122  Q_ASSERT(mOrphanChildrenHash.contains(dyingMessage));
3123  Q_ASSERT(dyingMessage != mCurrentItemToRestoreAfterViewItemJobStep);
3124 
3125  mOrphanChildrenHash.remove(dyingMessage);
3126  }
3127 
3128  if (mAggregation->threading() != Aggregation::NoThreading) {
3129  // Threading is requested: remove the message from threading caches.
3130 
3131  // Remove from the cache of potential parent items
3132  mThreadingCacheMessageIdMD5ToMessageItem.remove(dyingMessage->messageIdMD5());
3133 
3134  // If we also have a cache for subject/reference-based threading then remove the message from there too
3135  if (mAggregation->threading() == Aggregation::PerfectReferencesAndSubject) {
3136  removeMessageFromReferencesBasedThreadingCache(dyingMessage);
3137  removeMessageFromSubjectBasedThreadingCache(dyingMessage);
3138  } else if (mAggregation->threading() == Aggregation::PerfectAndReferences) {
3139  removeMessageFromReferencesBasedThreadingCache(dyingMessage);
3140  }
3141 
3142  // If this message wasn't perfectly parented then it might still be in another cache.
3143  switch (dyingMessage->threadingStatus()) {
3146  if (!dyingMessage->inReplyToIdMD5().isEmpty()) {
3147  mThreadingCacheMessageInReplyToIdMD5ToMessageItem.remove(dyingMessage->inReplyToIdMD5());
3148  }
3149  break;
3150  default:
3151  Q_ASSERT(!mThreadingCacheMessageInReplyToIdMD5ToMessageItem.contains(dyingMessage->inReplyToIdMD5(), dyingMessage));
3152  // make gcc happy
3153  break;
3154  }
3155  }
3156 
3157  while (auto childItem = dyingMessage->firstChildItem()) {
3158  auto childMessage = dynamic_cast<MessageItem *>(childItem);
3159  Q_ASSERT(childMessage);
3160 
3161  dyingMessage->takeChildItem(q, childMessage);
3162 
3163  if (mAggregation->threading() != Aggregation::NoThreading) {
3164  if (childMessage->threadingStatus() == MessageItem::PerfectParentFound) {
3165  // If the child message was perfectly parented then now it had
3166  // lost its perfect parent. Add to the cache of imperfectly parented.
3167  if (!childMessage->inReplyToIdMD5().isEmpty()) {
3168  Q_ASSERT(!mThreadingCacheMessageInReplyToIdMD5ToMessageItem.contains(childMessage->inReplyToIdMD5(), childMessage));
3169  mThreadingCacheMessageInReplyToIdMD5ToMessageItem.insert(childMessage->inReplyToIdMD5(), childMessage);
3170  }
3171  }
3172  }
3173 
3174  // Parent is gone
3175  childMessage->setThreadingStatus(MessageItem::ParentMissing);
3176 
3177  // If the child (or any message in its subtree) is going to be selected,
3178  // then we must immediately reattach it to a temporary group in order for the
3179  // selection to be preserved across multiple steps. Otherwise we could end
3180  // with the child-to-be-selected being non viewable at the end
3181  // of the view job step. Attach to a temporary group.
3182  if (
3183  // child is going to be re-selected
3184  (childMessage == mCurrentItemToRestoreAfterViewItemJobStep)
3185  || (
3186  // there is a message that is going to be re-selected
3187  mCurrentItemToRestoreAfterViewItemJobStep && // that message is in the childMessage subtree
3188  mCurrentItemToRestoreAfterViewItemJobStep->hasAncestor(childMessage))) {
3189  attachMessageToGroupHeader(childMessage);
3190 
3191  Q_ASSERT(childMessage->isViewable());
3192  }
3193 
3194  mOrphanChildrenHash.insert(childMessage, childMessage);
3195  }
3196 
3197  if (mNewestItem == dyingMessage) {
3198  mNewestItem = nullptr;
3199  }
3200  if (mOldestItem == dyingMessage) {
3201  mOldestItem = nullptr;
3202  }
3203 
3204  delete dyingMessage;
3205 
3206  curIndex++;
3207 
3208  // FIXME: Maybe we should check smaller steps here since the
3209  // code above can generate large message tree movements
3210  // for each single item we sweep in the invalidatedMessages list.
3211  if ((curIndex % mViewItemJobStepMessageCheckCount) == 0) {
3212  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
3213  if (curIndex <= endIndex) {
3214  job->setCurrentIndex(curIndex);
3215  return ViewItemJobInterrupted;
3216  }
3217  }
3218  }
3219  }
3220 
3221  // We looped over the entire deleted message list.
3222 
3223  job->setCurrentIndex(endIndex + 1);
3224 
3225  // A quick last cleaning pass: this is usually very fast so we don't have a real
3226  // Pass enumeration for it. We just include it as trailer of Pass1Cleanup to be executed
3227  // when job->currentIndex() > job->endIndex();
3228 
3229  // We move all the messages from the orphan child hash to the unassigned message
3230  // list and get them ready for the standard Pass2.
3231 
3232  auto it = mOrphanChildrenHash.begin();
3233  auto end = mOrphanChildrenHash.end();
3234 
3235  curIndex = 0;
3236 
3237  while (it != end) {
3238  mUnassignedMessageListForPass2.append(*it);
3239 
3240  it = mOrphanChildrenHash.erase(it);
3241 
3242  // This is still interruptible
3243 
3244  curIndex++;
3245 
3246  // FIXME: We could take "larger" steps here
3247  if ((curIndex % mViewItemJobStepMessageCheckCount) == 0) {
3248  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
3249  if (it != mOrphanChildrenHash.end()) {
3250  return ViewItemJobInterrupted;
3251  }
3252  }
3253  }
3254  }
3255 
3256  return ViewItemJobCompleted;
3257 }
3258 
3259 ModelPrivate::ViewItemJobResult ModelPrivate::viewItemJobStepInternalForJobPass1Update(ViewItemJob *job, QElapsedTimer elapsedTimer)
3260 {
3261  Q_ASSERT(mModelForItemFunctions); // UI must be not disconnected here
3262 
3263  // In this pass we simply update the MessageItem objects that are present in the job.
3264 
3265  // Note that this list in fact contains MessageItem objects (we need dynamic_cast<>).
3266  auto messagesThatNeedUpdate = job->invariantIndexList();
3267 
3268  // We don't shrink the messagesThatNeedUpdate because it's basically an array.
3269  // It's faster to traverse an array of N entries than to remove K>0 entries
3270  // one by one and to traverse the remaining N-K entries.
3271 
3272  // The begin index of our work
3273  int curIndex = job->currentIndex();
3274  // The end index of our work.
3275  int endIndex = job->endIndex();
3276 
3277  while (curIndex <= endIndex) {
3278  // Get the underlying storage message data...
3279  auto message = dynamic_cast<MessageItem *>(messagesThatNeedUpdate->at(curIndex));
3280  // This MUST NOT be null (otherwise we have a bug somewhere in this file).
3281  Q_ASSERT(message);
3282 
3283  int row = mInvariantRowMapper->modelInvariantIndexToModelIndexRow(message);
3284 
3285  if (row < 0) {
3286  // Must have been invalidated (so it's basically about to be deleted)
3287  Q_ASSERT(!message->isValid());
3288  // Skip it here.
3289  curIndex++;
3290  continue;
3291  }
3292 
3293  time_t prevDate = message->date();
3294  time_t prevMaxDate = message->maxDate();
3295  bool toDoStatus = message->status().isToAct();
3296  bool prevUnreadStatus = !message->status().isRead();
3297  bool prevImportantStatus = message->status().isImportant();
3298 
3299  // The subject/reference based threading cache is sorted by date: we must remove
3300  // the item and re-insert it since updateMessageItemData() may change the date too.
3301  if (mAggregation->threading() == Aggregation::PerfectReferencesAndSubject) {
3302  removeMessageFromReferencesBasedThreadingCache(message);
3303  removeMessageFromSubjectBasedThreadingCache(message);
3304  } else if (mAggregation->threading() == Aggregation::PerfectAndReferences) {
3305  removeMessageFromReferencesBasedThreadingCache(message);
3306  }
3307 
3308  // Do update
3309  mStorageModel->updateMessageItemData(message, row);
3310  QModelIndex idx = q->index(message, 0);
3311  Q_EMIT q->dataChanged(idx, idx);
3312 
3313  // Reinsert the item to the cache, if needed
3314  if (mAggregation->threading() == Aggregation::PerfectReferencesAndSubject) {
3315  addMessageToReferencesBasedThreadingCache(message);
3316  addMessageToSubjectBasedThreadingCache(message);
3317  } else if (mAggregation->threading() == Aggregation::PerfectAndReferences) {
3318  addMessageToReferencesBasedThreadingCache(message);
3319  }
3320 
3321  int propertyChangeMask = 0;
3322 
3323  if (prevDate != message->date()) {
3324  propertyChangeMask |= DateChanged;
3325  }
3326  if (prevMaxDate != message->maxDate()) {
3327  propertyChangeMask |= MaxDateChanged;
3328  }
3329  if (toDoStatus != message->status().isToAct()) {
3330  propertyChangeMask |= ActionItemStatusChanged;
3331  }
3332  if (prevUnreadStatus != (!message->status().isRead())) {
3333  propertyChangeMask |= UnreadStatusChanged;
3334  }
3335  if (prevImportantStatus != (!message->status().isImportant())) {
3336  propertyChangeMask |= ImportantStatusChanged;
3337  }
3338 
3339  if (propertyChangeMask) {
3340  // Some message data has changed
3341  // now we need to handle the changes that might cause re-grouping/re-sorting
3342  // and propagate them to the parents.
3343 
3344  Item *pParent = message->parent();
3345 
3346  if (pParent && (pParent != mRootItem)) {
3347  // The following function will return true if itemParent may be affected by the change.
3348  // If the itemParent isn't affected, we stop climbing.
3349  if (handleItemPropertyChanges(propertyChangeMask, pParent, message)) {
3350  Q_ASSERT(message->parent()); // handleItemPropertyChanges() must never leave an item detached
3351 
3352  // Note that actually message->parent() may be different than pParent since
3353  // handleItemPropertyChanges() may have re-grouped it.
3354 
3355  // Time to propagate up.
3356  propagateItemPropertiesToParent(message);
3357  }
3358  } // else there is no parent so the item isn't attached to the view: re-grouping/re-sorting not needed.
3359  } // else message data didn't change an there is nothing interesting to do
3360 
3361  // (re-)apply the filter, if needed
3362  if (mFilter && message->isViewable()) {
3363  // In all the other cases we (re-)apply the filter to the topmost subtree that this message is in.
3364  Item *pTopMostNonRoot = message->topmostNonRoot();
3365 
3366  Q_ASSERT(pTopMostNonRoot);
3367  Q_ASSERT(pTopMostNonRoot != mRootItem);
3368  Q_ASSERT(pTopMostNonRoot->parent() == mRootItem);
3369 
3370  // FIXME: The call below works, but it's expensive when we are updating
3371  // a lot of items with filtering enabled. This is because the updated
3372  // items are likely to be in the same subtree which we then filter multiple times.
3373  // A point for us is that when filtering there shouldn't be really many
3374  // items in the view so the user isn't going to update a lot of them at once...
3375  // Well... anyway, the alternative would be to write yet another
3376  // specialized routine that would update only the "message" item
3377  // above and climb up eventually hiding parents (without descending the sibling subtrees again).
3378  // If people complain about performance in this particular case I'll consider that solution.
3379 
3380  applyFilterToSubtree(pTopMostNonRoot, QModelIndex());
3381  } // otherwise there is no filter or the item isn't viewable: very likely
3382  // left detached while propagating property changes. Will filter it
3383  // on reattach.
3384 
3385  // Done updating this message
3386 
3387  curIndex++;
3388 
3389  // FIXME: Maybe we should check smaller steps here since the
3390  // code above can generate large message tree movements
3391  // for each single item we sweep in the messagesThatNeedUpdate list.
3392  if ((curIndex % mViewItemJobStepMessageCheckCount) == 0) {
3393  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
3394  if (curIndex <= endIndex) {
3395  job->setCurrentIndex(curIndex);
3396  return ViewItemJobInterrupted;
3397  }
3398  }
3399  }
3400  }
3401 
3402  return ViewItemJobCompleted;
3403 }
3404 
3405 ModelPrivate::ViewItemJobResult ModelPrivate::viewItemJobStepInternalForJob(ViewItemJob *job, QElapsedTimer elapsedTimer)
3406 {
3407  // This function does a timed chunk of work for a single Fill View job.
3408  // It attempts to process messages until a timeout forces it to return to the caller.
3409 
3410  // A macro would improve readability here but since this is a good point
3411  // to place debugger breakpoints then we need it explicitly.
3412  // A (template) helper would need to pass many parameters and would not be inlined...
3413 
3414  if (job->currentPass() == ViewItemJob::Pass1Fill) {
3415  // We're in Pass1Fill of the job.
3416  switch (viewItemJobStepInternalForJobPass1Fill(job, elapsedTimer)) {
3417  case ViewItemJobInterrupted:
3418  // current job interrupted by timeout: propagate status to caller
3419  return ViewItemJobInterrupted;
3420  break;
3421  case ViewItemJobCompleted:
3422  // pass 1 has been completed
3423  // # TODO: Refactor this, make it virtual or whatever, but switch == bad, code duplication etc
3424  job->setCurrentPass(ViewItemJob::Pass2);
3425  job->setStartIndex(0);
3426  job->setEndIndex(mUnassignedMessageListForPass2.count() - 1);
3427  // take care of small jobs which never timeout by themselves because
3428  // of a small number of messages. At the end of each job check
3429  // the time used and if we're timeoutting and there is another job
3430  // then interrupt.
3431  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
3432  return ViewItemJobInterrupted;
3433  } // else proceed with the next pass
3434  break;
3435  default:
3436  // This is *really* a BUG
3437  qCWarning(MESSAGELIST_LOG) << "ERROR: returned an invalid result";
3438  Q_ASSERT(false);
3439  break;
3440  }
3441  } else if (job->currentPass() == ViewItemJob::Pass1Cleanup) {
3442  // We're in Pass1Cleanup of the job.
3443  switch (viewItemJobStepInternalForJobPass1Cleanup(job, elapsedTimer)) {
3444  case ViewItemJobInterrupted:
3445  // current job interrupted by timeout: propagate status to caller
3446  return ViewItemJobInterrupted;
3447  break;
3448  case ViewItemJobCompleted:
3449  // pass 1 has been completed
3450  job->setCurrentPass(ViewItemJob::Pass2);
3451  job->setStartIndex(0);
3452  job->setEndIndex(mUnassignedMessageListForPass2.count() - 1);
3453  // take care of small jobs which never timeout by themselves because
3454  // of a small number of messages. At the end of each job check
3455  // the time used and if we're timeoutting and there is another job
3456  // then interrupt.
3457  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
3458  return ViewItemJobInterrupted;
3459  } // else proceed with the next pass
3460  break;
3461  default:
3462  // This is *really* a BUG
3463  qCWarning(MESSAGELIST_LOG) << "ERROR: returned an invalid result";
3464  Q_ASSERT(false);
3465  break;
3466  }
3467  } else if (job->currentPass() == ViewItemJob::Pass1Update) {
3468  // We're in Pass1Update of the job.
3469  switch (viewItemJobStepInternalForJobPass1Update(job, elapsedTimer)) {
3470  case ViewItemJobInterrupted:
3471  // current job interrupted by timeout: propagate status to caller
3472  return ViewItemJobInterrupted;
3473  break;
3474  case ViewItemJobCompleted:
3475  // pass 1 has been completed
3476  // Since Pass2, Pass3 and Pass4 are empty for an Update operation
3477  // we simply skip them. (TODO: Triple-verify this assertion...).
3478  job->setCurrentPass(ViewItemJob::Pass5);
3479  job->setStartIndex(0);
3480  job->setEndIndex(mGroupHeadersThatNeedUpdate.count() - 1);
3481  // take care of small jobs which never timeout by themselves because
3482  // of a small number of messages. At the end of each job check
3483  // the time used and if we're timeoutting and there is another job
3484  // then interrupt.
3485  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
3486  return ViewItemJobInterrupted;
3487  } // else proceed with the next pass
3488  break;
3489  default:
3490  // This is *really* a BUG
3491  qCWarning(MESSAGELIST_LOG) << "ERROR: returned an invalid result";
3492  Q_ASSERT(false);
3493  break;
3494  }
3495  }
3496 
3497  // Pass1Fill/Pass1Cleanup/Pass1Update has been already completed.
3498 
3499  if (job->currentPass() == ViewItemJob::Pass2) {
3500  // We're in Pass2 of the job.
3501  switch (viewItemJobStepInternalForJobPass2(job, elapsedTimer)) {
3502  case ViewItemJobInterrupted:
3503  // current job interrupted by timeout: propagate status to caller
3504  return ViewItemJobInterrupted;
3505  break;
3506  case ViewItemJobCompleted:
3507  // pass 2 has been completed
3508  job->setCurrentPass(ViewItemJob::Pass3);
3509  job->setStartIndex(0);
3510  job->setEndIndex(mUnassignedMessageListForPass3.count() - 1);
3511  // take care of small jobs which never timeout by themselves because
3512  // of a small number of messages. At the end of each job check
3513  // the time used and if we're timeoutting and there is another job
3514  // then interrupt.
3515  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
3516  return ViewItemJobInterrupted;
3517  }
3518  // else proceed with the next pass
3519  break;
3520  default:
3521  // This is *really* a BUG
3522  qCWarning(MESSAGELIST_LOG) << "ERROR: returned an invalid result";
3523  Q_ASSERT(false);
3524  break;
3525  }
3526  }
3527 
3528  if (job->currentPass() == ViewItemJob::Pass3) {
3529  // We're in Pass3 of the job.
3530  switch (viewItemJobStepInternalForJobPass3(job, elapsedTimer)) {
3531  case ViewItemJobInterrupted:
3532  // current job interrupted by timeout: propagate status to caller
3533  return ViewItemJobInterrupted;
3534  case ViewItemJobCompleted:
3535  // pass 3 has been completed
3536  job->setCurrentPass(ViewItemJob::Pass4);
3537  job->setStartIndex(0);
3538  job->setEndIndex(mUnassignedMessageListForPass4.count() - 1);
3539  // take care of small jobs which never timeout by themselves because
3540  // of a small number of messages. At the end of each job check
3541  // the time used and if we're timeoutting and there is another job
3542  // then interrupt.
3543  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
3544  return ViewItemJobInterrupted;
3545  }
3546  // else proceed with the next pass
3547  break;
3548  default:
3549  // This is *really* a BUG
3550  qCWarning(MESSAGELIST_LOG) << "ERROR: returned an invalid result";
3551  Q_ASSERT(false);
3552  break;
3553  }
3554  }
3555 
3556  if (job->currentPass() == ViewItemJob::Pass4) {
3557  // We're in Pass4 of the job.
3558  switch (viewItemJobStepInternalForJobPass4(job, elapsedTimer)) {
3559  case ViewItemJobInterrupted:
3560  // current job interrupted by timeout: propagate status to caller
3561  return ViewItemJobInterrupted;
3562  case ViewItemJobCompleted:
3563  // pass 4 has been completed
3564  job->setCurrentPass(ViewItemJob::Pass5);
3565  job->setStartIndex(0);
3566  job->setEndIndex(mGroupHeadersThatNeedUpdate.count() - 1);
3567  // take care of small jobs which never timeout by themselves because
3568  // of a small number of messages. At the end of each job check
3569  // the time used and if we're timeoutting and there is another job
3570  // then interrupt.
3571  if (elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) {
3572  return ViewItemJobInterrupted;
3573  }
3574  // else proceed with the next pass
3575  break;
3576  default:
3577  // This is *really* a BUG
3578  qCWarning(MESSAGELIST_LOG) << "ERROR: returned an invalid result";
3579  Q_ASSERT(false);
3580  break;
3581  }
3582  }
3583 
3584  // Pass4 has been already completed. Proceed to Pass5.
3585  return viewItemJobStepInternalForJobPass5(job, elapsedTimer);
3586 }
3587 
3588 #ifdef KDEPIM_FOLDEROPEN_PROFILE
3589 
3590 // Namespace to collect all the vars and functions for KDEPIM_FOLDEROPEN_PROFILE
3591 namespace Stats
3592 {
3593 // Number of existing jobs/passes
3594 static const int numberOfPasses = ViewItemJob::LastIndex;
3595 
3596 // The pass in the last call of viewItemJobStepInternal(), used to detect when
3597 // a new pass starts
3598 static int lastPass = -1;
3599 
3600 // Total number of messages in the folder
3601 static int totalMessages;
3602 
3603 // Per-Job data
3604 static int numElements[numberOfPasses];
3605 static int totalTime[numberOfPasses];
3606 static int chunks[numberOfPasses];
3607 
3608 // Time, in msecs for some special operations
3609 static int expandingTreeTime;
3610 static int layoutChangeTime;
3611 
3612 // Descriptions of the job, for nicer debug output
3613 static const char *jobDescription[numberOfPasses] = {"Creating items from messages and simple threading",
3614  "Removing messages",
3615  "Updating messages",
3616  "Additional Threading",
3617  "Subject-Based threading",
3618  "Grouping",
3619  "Group resorting + cleanup"};
3620 
3621 // Timer to track time between start of first job and end of last job
3622 static QTime firstStartTime;
3623 
3624 // Timer to track time the current job takes
3625 static QTime currentJobStartTime;
3626 
3627 // Zeros the stats, to be called when the first job starts
3628 static void resetStats()
3629 {
3630  totalMessages = 0;
3631  layoutChangeTime = 0;
3632  expandingTreeTime = 0;
3633  lastPass = -1;
3634  for (int i = 0; i < numberOfPasses; ++i) {
3635  numElements[i] = 0;
3636  totalTime[i] = 0;
3637  chunks[i] = 0;
3638  }
3639 }
3640 } // namespace Stats
3641 
3642 void ModelPrivate::printStatistics()
3643 {
3644  using namespace Stats;
3645  int totalTotalTime = 0;
3646  int completeTime = firstStartTime.elapsed();
3647  for (int i = 0; i < numberOfPasses; ++i) {
3648  totalTotalTime += totalTime[i];
3649  }
3650 
3651  float msgPerSecond = totalMessages / (totalTotalTime / 1000.0f);
3652  float msgPerSecondComplete = totalMessages / (completeTime / 1000.0f);
3653 
3654  int messagesWithSameSubjectAvg = 0;
3655  int messagesWithSameSubjectMax = 0;
3656  for (const auto messages : std::as_const(mThreadingCacheMessageSubjectMD5ToMessageItem)) {
3657  if (messages->size() > messagesWithSameSubjectMax) {
3658  messagesWithSameSubjectMax = messages->size();
3659  }
3660  messagesWithSameSubjectAvg += messages->size();
3661  }
3662  messagesWithSameSubjectAvg = messagesWithSameSubjectAvg / (float)mThreadingCacheMessageSubjectMD5ToMessageItem.size();
3663 
3664  int totalThreads = 0;
3665  if (!mGroupHeaderItemHash.isEmpty()) {
3666  for (const GroupHeaderItem *groupHeader : std::as_const(mGroupHeaderItemHash)) {
3667  totalThreads += groupHeader->childItemCount();
3668  }
3669  } else {
3670  totalThreads = mRootItem->childItemCount();
3671  }
3672 
3673  qCDebug(MESSAGELIST_LOG) << "Finished filling the view with" << totalMessages << "messages";
3674  qCDebug(MESSAGELIST_LOG) << "That took" << totalTotalTime << "msecs inside the model and" << completeTime << "in total.";
3675  qCDebug(MESSAGELIST_LOG) << (totalTotalTime / (float)completeTime) * 100.0f << "percent of the time was spent in the model.";
3676  qCDebug(MESSAGELIST_LOG) << "Time for layoutChanged(), in msecs:" << layoutChangeTime << "(" << (layoutChangeTime / (float)totalTotalTime) * 100.0f
3677  << "percent )";
3678  qCDebug(MESSAGELIST_LOG) << "Time to expand tree, in msecs:" << expandingTreeTime << "(" << (expandingTreeTime / (float)totalTotalTime) * 100.0f
3679  << "percent )";
3680  qCDebug(MESSAGELIST_LOG) << "Number of messages per second in the model:" << msgPerSecond;
3681  qCDebug(MESSAGELIST_LOG) << "Number of messages per second in total:" << msgPerSecondComplete;
3682  qCDebug(MESSAGELIST_LOG) << "Number of threads:" << totalThreads;
3683  qCDebug(MESSAGELIST_LOG) << "Number of groups:" << mGroupHeaderItemHash.size();
3684  qCDebug(MESSAGELIST_LOG) << "Messages per thread:" << totalMessages / (float)totalThreads;
3685  qCDebug(MESSAGELIST_LOG) << "Threads per group:" << totalThreads / (float)mGroupHeaderItemHash.size();
3686  qCDebug(MESSAGELIST_LOG) << "Messages with the same subject:"
3687  << "Max:" << messagesWithSameSubjectMax << "Avg:" << messagesWithSameSubjectAvg;
3688  qCDebug(MESSAGELIST_LOG);
3689  qCDebug(MESSAGELIST_LOG) << "Now follows a breakdown of the jobs.";
3690  qCDebug(MESSAGELIST_LOG);
3691  for (int i = 0; i < numberOfPasses; ++i) {
3692  if (totalTime[i] == 0) {
3693  continue;
3694  }
3695  float elementsPerSecond = numElements[i] / (totalTime[i] / 1000.0f);
3696  float percent = totalTime[i] / (float)totalTotalTime * 100.0f;
3697  qCDebug(MESSAGELIST_LOG) << "----------------------------------------------";
3698  qCDebug(MESSAGELIST_LOG) << "Job" << i + 1 << "(" << jobDescription[i] << ")";
3699  qCDebug(MESSAGELIST_LOG) << "Share of complete time:" << percent << "percent";
3700  qCDebug(MESSAGELIST_LOG) << "Time in msecs:" << totalTime[i];
3701  qCDebug(MESSAGELIST_LOG) << "Number of elements:" << numElements[i]; // TODO: map of element string
3702  qCDebug(MESSAGELIST_LOG) << "Elements per second:" << elementsPerSecond;
3703  qCDebug(MESSAGELIST_LOG) << "Number of chunks:" << chunks[i];
3704  qCDebug(MESSAGELIST_LOG);
3705  }
3706 
3707  qCDebug(MESSAGELIST_LOG) << "==========================================================";
3708  resetStats();
3709 }
3710 
3711 #endif
3712 
3713 ModelPrivate::ViewItemJobResult ModelPrivate::viewItemJobStepInternal()
3714 {
3715  // This function does a timed chunk of work in our View Fill operation.
3716  // It attempts to do processing until it either runs out of jobs
3717  // to be done or a timeout forces it to interrupt and jump back to the caller.
3718 
3719  QElapsedTimer elapsedTimer;
3720  elapsedTimer.start();
3721 
3722  while (!mViewItemJobs.isEmpty()) {
3723  // Have a job to do.
3724  ViewItemJob *job = mViewItemJobs.constFirst();
3725 
3726 #ifdef KDEPIM_FOLDEROPEN_PROFILE
3727 
3728  // Here we check if an old job has just completed or if we are at the start of the
3729  // first job. We then initialize job data stuff and timers based on this.
3730 
3731  const int currentPass = job->currentPass();
3732  const bool firstChunk = currentPass != Stats::lastPass;
3733  if (currentPass != Stats::lastPass && Stats::lastPass != -1) {
3734  Stats::totalTime[Stats::lastPass] = Stats::currentJobStartTime.elapsed();
3735  }
3736  const bool firstJob = job->currentPass() == ViewItemJob::Pass1Fill && firstChunk;
3737  const int elements = job->endIndex() - job->startIndex();
3738  if (firstJob) {
3739  Stats::resetStats();
3740  Stats::totalMessages = elements;
3741  Stats::firstStartTime.restart();
3742  }
3743  if (firstChunk) {
3744  Stats::numElements[currentPass] = elements;
3745  Stats::currentJobStartTime.restart();
3746  }
3747  Stats::chunks[currentPass]++;
3748  Stats::lastPass = currentPass;
3749 
3750 #endif
3751 
3752  mViewItemJobStepIdleInterval = job->idleInterval();
3753  mViewItemJobStepChunkTimeout = job->chunkTimeout();
3754  mViewItemJobStepMessageCheckCount = job->messageCheckCount();
3755 
3756  if (job->disconnectUI()) {
3757  mModelForItemFunctions = nullptr; // disconnect the UI for this job
3758  Q_ASSERT(mLoading); // this must be true in the first job
3759  // FIXME: Should assert yet more that this is the very first job for this StorageModel
3760  // Asserting only mLoading is not enough as we could be using a two-jobs loading strategy
3761  // or this could be a job enqueued before the first job has completed.
3762  } else {
3763  // With a connected UI we need to avoid the view to update the scrollbars at EVERY insertion or expansion.
3764  // QTreeViewPrivate::updateScrollBars() is very expensive as it loops through ALL the items in the view every time.
3765  // We can't disable the function directly as it's hidden in the private data object of QTreeView
3766  // but we can disable the parent QTreeView::updateGeometries() instead.
3767  // We will trigger it "manually" at the end of the step.
3768  mView->ignoreUpdateGeometries(true);
3769 
3770  // Ok.. I know that this seems unbelieveable but disabling updates actually
3771  // causes a (significant) performance loss in most cases. This is probably because QTreeView
3772  // uses delayed layouts when updates are disabled which should be delayed but in
3773  // fact are "forced" by next item insertions. The delayed layout algorithm, then
3774  // is probably slower than the non-delayed one.
3775  // Disabling the paintEvent() doesn't seem to work either.
3776  // mView->setUpdatesEnabled( false );
3777  }
3778 
3779  switch (viewItemJobStepInternalForJob(job, elapsedTimer)) {
3780  case ViewItemJobInterrupted:
3781  // current job interrupted by timeout: will propagate status to caller
3782  // but before this, give some feedback to the user
3783 
3784  // FIXME: This is now inaccurate, think of something else
3785  switch (job->currentPass()) {
3786  case ViewItemJob::Pass1Fill:
3787  case ViewItemJob::Pass1Cleanup:
3788  case ViewItemJob::Pass1Update:
3789  Q_EMIT q->statusMessage(i18np("Processed 1 Message of %2",
3790  "Processed %1 Messages of %2",
3791  job->currentIndex() - job->startIndex(),
3792  job->endIndex() - job->startIndex() + 1));
3793  break;
3794  case ViewItemJob::Pass2:
3795  Q_EMIT q->statusMessage(i18np("Threaded 1 Message of %2",
3796  "Threaded %1 Messages of %2",
3797  job->currentIndex() - job->startIndex(),
3798  job->endIndex() - job->startIndex() + 1));
3799  break;
3800  case ViewItemJob::Pass3:
3801  Q_EMIT q->statusMessage(i18np("Threaded 1 Message of %2",
3802  "Threaded %1 Messages of %2",
3803  job->currentIndex() - job->startIndex(),
3804  job->endIndex() - job->startIndex() + 1));
3805  break;
3806  case ViewItemJob::Pass4:
3807  Q_EMIT q->statusMessage(i18np("Grouped 1 Thread of %2",
3808  "Grouped %1 Threads of %2",
3809  job->currentIndex() - job->startIndex(),
3810  job->endIndex() - job->startIndex() + 1));
3811  break;
3812  case ViewItemJob::Pass5:
3813  Q_EMIT q->statusMessage(i18np("Updated 1 Group of %2",
3814  "Updated %1 Groups of %2",
3815  job->currentIndex() - job->startIndex(),
3816  job->endIndex() - job->startIndex() + 1));
3817  break;
3818  default:
3819  break;
3820  }
3821 
3822  if (!job->disconnectUI()) {
3823  mView->ignoreUpdateGeometries(false);
3824  // explicit call to updateGeometries() here
3825  mView->updateGeometries();
3826  }
3827 
3828  return ViewItemJobInterrupted;
3829  break;
3830  case ViewItemJobCompleted:
3831 
3832  // If this job worked with a disconnected UI, Q_EMIT layoutChanged()
3833  // to reconnect it. We go back to normal operation now.
3834  if (job->disconnectUI()) {
3835  mModelForItemFunctions = q;
3836  // This call would destroy the expanded state of items.
3837  // This is why when mModelForItemFunctions was 0 we didn't actually expand them
3838  // but we just set a "ExpandNeeded" mark...
3839 #ifdef KDEPIM_FOLDEROPEN_PROFILE
3840  QTime layoutChangedTimer;
3841  layoutChangedTimer.start();
3842 #endif
3843  mView->modelAboutToEmitLayoutChanged();
3844  Q_EMIT q->layoutChanged();
3845  mView->modelEmittedLayoutChanged();
3846 
3847 #ifdef KDEPIM_FOLDEROPEN_PROFILE
3848  Stats::layoutChangeTime = layoutChangedTimer.elapsed();
3849  QTime expandingTime;
3850  expandingTime.start();
3851 #endif
3852 
3853  // expand all the items that need it in a single sweep
3854 
3855  // FIXME: This takes quite a lot of time, it could be made an interruptible job
3856 
3857  auto rootChildItems = mRootItem->childItems();
3858  if (rootChildItems) {
3859  for (const auto it : std::as_const(*rootChildItems)) {
3860  if (it->initialExpandStatus() == Item::ExpandNeeded) {
3861  syncExpandedStateOfSubtree(it);
3862  }
3863  }
3864  }
3865 #ifdef KDEPIM_FOLDEROPEN_PROFILE
3866  Stats::expandingTreeTime = expandingTime.elapsed();
3867 #endif
3868  } else {
3869  mView->ignoreUpdateGeometries(false);
3870  // explicit call to updateGeometries() here
3871  mView->updateGeometries();
3872  }
3873 
3874  // this job has been completed
3875  delete mViewItemJobs.takeFirst();
3876 
3877 #ifdef KDEPIM_FOLDEROPEN_PROFILE
3878  // Last job finished!
3879  Stats::totalTime[currentPass] = Stats::currentJobStartTime.elapsed();
3880  printStatistics();
3881 #endif
3882 
3883  // take care of small jobs which never timeout by themselves because
3884  // of a small number of messages. At the end of each job check
3885  // the time used and if we're timeoutting and there is another job
3886  // then interrupt.
3887  if ((elapsedTimer.elapsed() > mViewItemJobStepChunkTimeout) || (elapsedTimer.elapsed() < 0)) {
3888  if (!mViewItemJobs.isEmpty()) {
3889  return ViewItemJobInterrupted;
3890  }
3891  // else it's completed in fact
3892  } // else proceed with the next job
3893 
3894  break;
3895  default:
3896  // This is *really* a BUG
3897  qCWarning(MESSAGELIST_LOG) << "ERROR: returned an invalid result";
3898  Q_ASSERT(false);
3899  break;
3900  }
3901  }
3902 
3903  // no more jobs
3904 
3905  Q_EMIT q->statusMessage(i18nc("@info:status Finished view fill", "Ready"));
3906 
3907  return ViewItemJobCompleted;
3908 }
3909 
3910 void ModelPrivate::viewItemJobStep()
3911 {
3912  // A single step in the View Fill operation.
3913  // This function wraps viewItemJobStepInternal() which does the step job
3914  // and either completes it or stops because of a timeout.
3915  // If the job is stopped then we start a zero-msecs timer to call us
3916  // back and resume the job. Otherwise we're just done.
3917 
3918  mViewItemJobStepStartTime = ::time(nullptr);
3919 
3920  if (mFillStepTimer.isActive()) {
3921  mFillStepTimer.stop();
3922  }
3923 
3924  if (!mStorageModel) {
3925  return; // nothing more to do
3926  }
3927 
3928  // Save the current item in the view as our process may
3929  // cause items to be reparented (and QTreeView will forget the current item in the meantime).
3930  // This machinery is also needed when we're about to remove items from the view in
3931  // a cleanup job: we'll be trying to set as current the item after the one removed.
3932 
3933  QModelIndex currentIndexBeforeStep = mView->currentIndex();
3934  Item *currentItemBeforeStep = currentIndexBeforeStep.isValid() ? static_cast<Item *>(currentIndexBeforeStep.internalPointer()) : nullptr;
3935 
3936  // mCurrentItemToRestoreAfterViewItemJobStep will be zeroed out if it's killed
3937  mCurrentItemToRestoreAfterViewItemJobStep = currentItemBeforeStep;
3938 
3939  // Save the current item position in the viewport as QTreeView fails to keep
3940  // the current item in the sample place when items are added or removed...
3941  QRect rectBeforeViewItemJobStep;
3942 
3943  const bool lockView = mView->isScrollingLocked();
3944 
3945  // This is generally SLOW AS HELL... (so we avoid it if we lock the view and thus don't need it)
3946  if (mCurrentItemToRestoreAfterViewItemJobStep && (!lockView)) {
3947  rectBeforeViewItemJobStep = mView->visualRect(currentIndexBeforeStep);
3948  }
3949 
3950  // FIXME: If the current item is NOT in the view, preserve the position
3951  // of the top visible item. This will make the view move yet less.
3952 
3953  // Insulate the View from (very likely spurious) "currentChanged()" signals.
3954  mView->ignoreCurrentChanges(true);
3955 
3956  // And go to real work.
3957  switch (viewItemJobStepInternal()) {
3958  case ViewItemJobInterrupted:
3959  // Operation timed out, need to resume in a while
3960  if (!mInLengthyJobBatch) {
3961  mInLengthyJobBatch = true;
3962  }
3963  mFillStepTimer.start(mViewItemJobStepIdleInterval); // this is a single shot timer connected to viewItemJobStep()
3964  // and go dealing with current/selection out of the switch.
3965  break;
3966  case ViewItemJobCompleted:
3967  // done :)
3968 
3969  Q_ASSERT(mModelForItemFunctions); // UI must be no (longer) disconnected in this state
3970 
3971  // Ask the view to remove the eventual busy indications
3972  if (mInLengthyJobBatch) {
3973  mInLengthyJobBatch = false;
3974  }
3975 
3976  if (mLoading) {
3977  mLoading = false;
3978  mView->modelFinishedLoading();
3979  slotApplyFilter();
3980  }
3981 
3982  // Apply pre-selection, if any
3983  if (mPreSelectionMode != PreSelectNone) {
3984  mView->ignoreCurrentChanges(false);
3985 
3986  bool bSelectionDone = false;
3987 
3988  switch (mPreSelectionMode) {
3989  case PreSelectLastSelected:
3990  // fall down
3991  break;
3992  case PreSelectFirstUnreadCentered:
3993  bSelectionDone = mView->selectFirstMessageItem(MessageTypeUnreadOnly, true); // center
3994  break;
3995  case PreSelectOldestCentered:
3996  mView->setCurrentMessageItem(mOldestItem, true /* center */);
3997  bSelectionDone = true;
3998  break;
3999  case PreSelectNewestCentered:
4000  mView->setCurrentMessageItem(mNewestItem, true /* center */);
4001  bSelectionDone = true;
4002  break;
4003  case PreSelectNone:
4004  // deal with selection below
4005  break;
4006  default:
4007  qCWarning(MESSAGELIST_LOG) << "ERROR: Unrecognized pre-selection mode " << static_cast<int>(mPreSelectionMode);
4008  break;
4009  }
4010 
4011  if ((!bSelectionDone) && (mPreSelectionMode != PreSelectNone)) {
4012  // fallback to last selected, if possible
4013  if (mLastSelectedMessageInFolder) { // we found it in the loading process: select and jump out
4014  mView->setCurrentMessageItem(mLastSelectedMessageInFolder);
4015  bSelectionDone = true;
4016  }
4017  }
4018 
4019  if (bSelectionDone) {
4020  mLastSelectedMessageInFolder = nullptr;
4021  mPreSelectionMode = PreSelectNone;
4022  return; // already taken care of current / selection
4023  }
4024  }
4025  // deal with current/selection out of the switch
4026 
4027  break;
4028  default:
4029  // This is *really* a BUG
4030  qCWarning(MESSAGELIST_LOG) << "ERROR: returned an invalid result";
4031  Q_ASSERT(false);
4032  break;
4033  }
4034 
4035  // Everything else here deals with the selection
4036 
4037  // If UI is disconnected then we don't have anything else to do here
4038  if (!mModelForItemFunctions) {
4039  mView->ignoreCurrentChanges(false);
4040  return;
4041  }
4042 
4043  // Restore current/selection and/or scrollbar position
4044 
4045  if (mCurrentItemToRestoreAfterViewItemJobStep) {
4046  bool stillIgnoringCurrentChanges = true;
4047 
4048  // If the assert below fails then the previously current item got detached
4049  // and didn't get reattached in the step: this should never happen.
4050  Q_ASSERT(mCurrentItemToRestoreAfterViewItemJobStep->isViewable());
4051 
4052  // Check if the current item changed
4053  QModelIndex currentIndexAfterStep = mView->currentIndex();
4054  Item *currentAfterStep = currentIndexAfterStep.isValid() ? static_cast<Item *>(currentIndexAfterStep.internalPointer()) : nullptr;
4055 
4056  if (mCurrentItemToRestoreAfterViewItemJobStep != currentAfterStep) {
4057  // QTreeView lost the current item...
4058  if (mCurrentItemToRestoreAfterViewItemJobStep != currentItemBeforeStep) {
4059  // Some view job code expects us to actually *change* the current item.
4060  // This is done by the cleanup step which removes items and tries
4061  // to set as current the item *after* the removed one, if possible.
4062  // We need the view to handle the change though.
4063  stillIgnoringCurrentChanges = false;
4064  mView->ignoreCurrentChanges(false);
4065  } else {
4066  // we just have to restore the old current item. The code
4067  // outside shouldn't have noticed that we lost it (e.g. the message viewer
4068  // still should have the old message opened). So we don't need to
4069  // actually notify the view of the restored setting.
4070  }
4071  // Restore it
4072  qCDebug(MESSAGELIST_LOG) << "Gonna restore current here" << mCurrentItemToRestoreAfterViewItemJobStep->subject();
4073  mView->setCurrentIndex(q->index(mCurrentItemToRestoreAfterViewItemJobStep, 0));
4074  } else {
4075  // The item we're expected to set as current is already current
4076  if (mCurrentItemToRestoreAfterViewItemJobStep != currentItemBeforeStep) {
4077  // But we have changed it in the job step.
4078  // This means that: we have deleted the current item and chosen a
4079  // new candidate as current but Qt also has chosen it as candidate
4080  // and already made it current. The problem is that (as of Qt 4.4)
4081  // it probably didn't select it.
4082  if (!mView->selectionModel()->hasSelection()) {
4083  stillIgnoringCurrentChanges = false;
4084  mView->ignoreCurrentChanges(false);
4085 
4086  qCDebug(MESSAGELIST_LOG) << "Gonna restore selection here" << mCurrentItemToRestoreAfterViewItemJobStep->subject();
4087 
4088  QItemSelection selection;
4089  selection.append(QItemSelectionRange(q->index(mCurrentItemToRestoreAfterViewItemJobStep, 0)));
4090  mView->selectionModel()->select(selection, QItemSelectionModel::Select | QItemSelectionModel::Rows);
4091  }
4092  }
4093  }
4094 
4095  // FIXME: If it was selected before the change, then re-select it (it may happen that it's not)
4096  if (!lockView) {
4097  // we prefer to keep the currently selected item steady in the view
4098  QRect rectAfterViewItemJobStep = mView->visualRect(q->index(mCurrentItemToRestoreAfterViewItemJobStep, 0));
4099  if (rectBeforeViewItemJobStep.y() != rectAfterViewItemJobStep.y()) {
4100  // QTreeView lost its position...
4101  mView->verticalScrollBar()->setValue(mView->verticalScrollBar()->value() + rectAfterViewItemJobStep.y() - rectBeforeViewItemJobStep.y());
4102  }
4103  }
4104 
4105  // and kill the insulation, if not yet done
4106  if (stillIgnoringCurrentChanges) {
4107  mView->ignoreCurrentChanges(false);
4108  }
4109 
4110  return;
4111  }
4112 
4113  // Either there was no current item before, or it was lost in a cleanup step and another candidate for
4114  // current item couldn't be found (possibly empty view)
4115  mView->ignoreCurrentChanges(false);
4116 
4117  if (currentItemBeforeStep) {
4118  // lost in a cleanup..
4119  // tell the view that we have a new current, this time with no insulation
4120  mView->slotSelectionChanged(QItemSelection(), QItemSelection());
4121  }
4122 }
4123 
4124 void ModelPrivate::slotStorageModelRowsInserted(const QModelIndex &parent, int from, int to)
4125 {
4126  if (parent.isValid()) {
4127  return; // ugh... should never happen
4128  }
4129 
4130  Q_ASSERT(from <= to);
4131 
4132  int count = (to - from) + 1;
4133 
4134  mInvariantRowMapper->modelRowsInserted(from, count);
4135 
4136  // look if no current job is in the middle
4137 
4138  int jobCount = mViewItemJobs.count();
4139 
4140  for (int idx = 0; idx < jobCount; idx++) {
4141  ViewItemJob *job = mViewItemJobs.at(idx);
4142 
4143  if (job->currentPass() != ViewItemJob::Pass1Fill) {
4144  // The job is a cleanup or in a later pass: the storage has been already accessed
4145  // and the messages created... no need to care anymore: the invariant row mapper will do the job.
4146  continue;
4147  }
4148 
4149  if (job->currentIndex() > job->endIndex()) {
4150  // The job finished the Pass1Fill but still waits for the pass indicator to be
4151  // changed. This is unlikely but still may happen if the job has been interrupted
4152  // and then a call to slotStorageModelRowsRemoved() caused it to be forcibly completed.
4153  continue;
4154  }
4155 
4156  //
4157  // The following cases are possible:
4158  //
4159  // from to
4160  // | | -> shift up job
4161  // from to
4162  // | | -> shift up job
4163  // from to
4164  // | | -> shift up job
4165  // from to
4166  // | | -> split job
4167  // from to
4168  // | | -> split job
4169  // from to
4170  // | | -> job unaffected
4171  //
4172  //
4173  // FOLDER
4174  // |-------------------------|---------|--------------|
4175  // 0 currentIndex endIndex count
4176  // +-- job --+
4177  //
4178 
4179  if (from > job->endIndex()) {
4180  // The change is completely above the job, the job is not affected
4181  continue;
4182  }
4183 
4184  if (from > job->currentIndex()) { // and from <= job->endIndex()
4185  // The change starts in the middle of the job in a way that it must be split in two.
4186  // The first part is unaffected by the shift and ranges from job->currentIndex() to from - 1.
4187  // The second part ranges from "from" to job->endIndex() that are now shifted up by count steps.
4188 
4189  // First add a new job for the second part.
4190  auto newJob = new ViewItemJob(from + count, job->endIndex() + count, job->chunkTimeout(), job->idleInterval(), job->messageCheckCount());
4191 
4192  Q_ASSERT(newJob->currentIndex() <= newJob->endIndex());
4193 
4194  idx++; // we can skip this job in the loop, it's already ok
4195  jobCount++; // and our range increases by one.
4196  mViewItemJobs.insert(idx, newJob);
4197 
4198  // Then limit the original job to the first part
4199  job->setEndIndex(from - 1);
4200 
4201  Q_ASSERT(job->currentIndex() <= job->endIndex());
4202 
4203  continue;
4204  }
4205 
4206  // The change starts below (or exactly on the beginning of) the job.
4207  // The job must be shifted up.
4208  job->setCurrentIndex(job->currentIndex() + count);
4209  job->setEndIndex(job->endIndex() + count);
4210 
4211  Q_ASSERT(job->currentIndex() <= job->endIndex());
4212  }
4213 
4214  bool newJobNeeded = true;
4215 
4216  // Try to attach to an existing fill job, if any.
4217  // To enforce consistency we can attach only if the Fill job
4218  // is the last one in the list (might be eventually *also* the first,
4219  // and even being already processed but we must make sure that there
4220  // aren't jobs _after_ it).
4221  if (jobCount > 0) {
4222  ViewItemJob *job = mViewItemJobs.at(jobCount - 1);
4223  if (job->currentPass() == ViewItemJob::Pass1Fill) {
4224  if (
4225  // The job ends just before the added rows
4226  (from == (job->endIndex() + 1)) && // The job didn't reach the end of Pass1Fill yet
4227  (job->currentIndex() <= job->endIndex())) {
4228  // We can still attach this :)
4229  job->setEndIndex(to);
4230  Q_ASSERT(job->currentIndex() <= job->endIndex());
4231  newJobNeeded = false;
4232  }
4233  }
4234  }
4235 
4236  if (newJobNeeded) {
4237  // FIXME: Should take timing options from aggregation here ?
4238  auto job = new ViewItemJob(from, to, 100, 50, 10);
4239  mViewItemJobs.append(job);
4240  }
4241 
4242  if (!mFillStepTimer.isActive()) {
4243  mFillStepTimer.start(mViewItemJobStepIdleInterval);
4244  }
4245 }
4246 
4247 void ModelPrivate::slotStorageModelRowsRemoved(const QModelIndex &parent, int from, int to)
4248 {
4249  // This is called when the underlying StorageModel emits the rowsRemoved signal.
4250 
4251  if (parent.isValid()) {
4252  return; // ugh... should never happen
4253  }
4254 
4255  // look if no current job is in the middle
4256 
4257  Q_ASSERT(from <= to);
4258 
4259  const int count = (to - from) + 1;
4260 
4261  int jobCount = mViewItemJobs.count();
4262 
4263  if (mRootItem && from == 0 && count == mRootItem->childItemCount() && jobCount == 0) {
4264  clear();
4265  return;
4266  }
4267 
4268  for (int idx = 0; idx < jobCount; idx++) {
4269  ViewItemJob *job = mViewItemJobs.at(idx);
4270 
4271  if (job->currentPass() != ViewItemJob::Pass1Fill) {
4272  // The job is a cleanup or in a later pass: the storage has been already accessed
4273  // and the messages created... no need to care: we will invalidate the messages in a while.
4274  continue;
4275  }
4276 
4277  if (job->currentIndex() > job->endIndex()) {
4278  // The job finished the Pass1Fill but still waits for the pass indicator to be
4279  // changed. This is unlikely but still may happen if the job has been interrupted
4280  // and then a call to slotStorageModelRowsRemoved() caused it to be forcibly completed.
4281  continue;
4282  }
4283 
4284  //
4285  // The following cases are possible:
4286  //
4287  // from to
4288  // | | -> shift down job
4289  // from to
4290  // | | -> shift down and crop job
4291  // from to
4292  // | | -> kill job
4293  // from to
4294  // | | -> split job, crop and shift
4295  // from to
4296  // | | -> crop job
4297  // from to
4298  // | | -> job unaffected
4299  //
4300  //
4301  // FOLDER
4302  // |-------------------------|---------|--------------|
4303  // 0 currentIndex endIndex count
4304  // +-- job --+
4305  //
4306 
4307  if (from > job->endIndex()) {
4308  // The change is completely above the job, the job is not affected
4309  continue;
4310  }
4311 
4312  if (from > job->currentIndex()) { // and from <= job->endIndex()
4313  // The change starts in the middle of the job and ends in the middle or after the job.
4314  // The first part is unaffected by the shift and ranges from job->currentIndex() to from - 1
4315  // We use the existing job for this.
4316  job->setEndIndex(from - 1); // stop before the first removed row
4317 
4318  Q_ASSERT(job->currentIndex() <= job->endIndex());
4319 
4320  if (to < job->endIndex()) {
4321  // The change ends inside the job and a part of it can be completed.
4322 
4323  // We create a new job for the shifted remaining part. It would actually
4324  // range from to + 1 up to job->endIndex(), but we need to shift it down by count.
4325  // since count = ( to - from ) + 1 so from = to + 1 - count
4326 
4327  auto newJob = new ViewItemJob(from, job->endIndex() - count, job->chunkTimeout(), job->idleInterval(), job->messageCheckCount());
4328 
4329  Q_ASSERT(newJob->currentIndex() < newJob->endIndex());
4330 
4331  idx++; // we can skip this job in the loop, it's already ok
4332  jobCount++; // and our range increases by one.
4333  mViewItemJobs.insert(idx, newJob);
4334  } // else the change includes completely the end of the job and no other part of it can be completed.
4335 
4336  continue;
4337  }
4338 
4339  // The change starts below (or exactly on the beginning of) the job. ( from <= job->currentIndex() )
4340  if (to >= job->endIndex()) {
4341  // The change completely covers the job: kill it
4342 
4343  // We don't delete the job since we want the other passes to be completed
4344  // This is because the Pass1Fill may have already filled mUnassignedMessageListForPass2
4345  // and may have set mOldestItem and mNewestItem. We *COULD* clear the unassigned
4346  // message list with clearUnassignedMessageLists() but mOldestItem and mNewestItem
4347  // could be still dangling pointers. So we just move the current index of the job
4348  // after the end (so storage model scan terminates) and let it complete spontaneously.
4349  job->setCurrentIndex(job->endIndex() + 1);
4350 
4351  continue;
4352  }
4353 
4354  if (to >= job->currentIndex()) {
4355  // The change partially covers the job. Only a part of it can be completed
4356  // and it must be shifted down. It would actually
4357  // range from to + 1 up to job->endIndex(), but we need to shift it down by count.
4358  // since count = ( to - from ) + 1 so from = to + 1 - count
4359  job->setCurrentIndex(from);
4360  job->setEndIndex(job->endIndex() - count);
4361 
4362  Q_ASSERT(job->currentIndex() <= job->endIndex());
4363 
4364  continue;
4365  }
4366 
4367  // The change is completely below the job: it must be shifted down.
4368  job->setCurrentIndex(job->currentIndex() - count);
4369  job->setEndIndex(job->endIndex() - count);
4370  }
4371 
4372  // This will invalidate the ModelInvariantIndex-es that have been removed and return
4373  // them all in a nice list that we can feed to a view removal job.
4374  auto invalidatedIndexes = mInvariantRowMapper->modelRowsRemoved(from, count);
4375 
4376  if (invalidatedIndexes) {
4377  // Try to attach to an existing cleanup job, if any.
4378  // To enforce consistency we can attach only if the Cleanup job
4379  // is the last one in the list (might be eventually *also* the first,
4380  // and even being already processed but we must make sure that there
4381  // aren't jobs _after_ it).
4382  if (jobCount > 0) {
4383  ViewItemJob *job = mViewItemJobs.at(jobCount - 1);
4384  if (job->currentPass() == ViewItemJob::Pass1Cleanup) {
4385  if ((job->currentIndex() <= job->endIndex()) && job->invariantIndexList()) {
4386  // qCDebug(MESSAGELIST_LOG) << "Appending " << invalidatedIndexes->count() << " invalidated indexes to existing cleanup job";
4387  // We can still attach this :)
4388  *(job->invariantIndexList()) += *invalidatedIndexes;
4389  job->setEndIndex(job->endIndex() + invalidatedIndexes->count());
4390  delete invalidatedIndexes;
4391  invalidatedIndexes = nullptr;
4392  }
4393  }
4394  }
4395 
4396  if (invalidatedIndexes) {
4397  // Didn't append to any existing cleanup job.. create a new one
4398 
4399  // qCDebug(MESSAGELIST_LOG) << "Creating new cleanup job for " << invalidatedIndexes->count() << " invalidated indexes";
4400  // FIXME: Should take timing options from aggregation here ?
4401  auto job = new ViewItemJob(ViewItemJob::Pass1Cleanup, invalidatedIndexes, 100, 50, 10);
4402  mViewItemJobs.append(job);
4403  }
4404 
4405  if (!mFillStepTimer.isActive()) {
4406  mFillStepTimer.start(mViewItemJobStepIdleInterval);
4407  }
4408  }
4409 }
4410 
4411 void ModelPrivate::slotStorageModelLayoutChanged()
4412 {
4413  qCDebug(MESSAGELIST_LOG) << "Storage model layout changed";
4414  // need to reset everything...
4415  q->setStorageModel(mStorageModel);
4416  qCDebug(MESSAGELIST_LOG) << "Storage model layout changed done";
4417 }
4418 
4419 void ModelPrivate::slotStorageModelDataChanged(const QModelIndex &fromIndex, const QModelIndex &toIndex)
4420 {
4421  Q_ASSERT(mStorageModel); // must exist (and be the sender of the signal connected to this slot)
4422 
4423  int from = fromIndex.row();
4424  int to = toIndex.row();
4425 
4426  Q_ASSERT(from <= to);
4427 
4428  int count = (to - from) + 1;
4429 
4430  int jobCount = mViewItemJobs.count();
4431 
4432  // This will find out the ModelInvariantIndex-es that need an update and will return
4433  // them all in a nice list that we can feed to a view removal job.
4434  auto indexesThatNeedUpdate = mInvariantRowMapper->modelIndexRowRangeToModelInvariantIndexList(from, count);
4435 
4436  if (indexesThatNeedUpdate) {
4437  // Try to attach to an existing update job, if any.
4438  // To enforce consistency we can attach only if the Update job
4439  // is the last one in the list (might be eventually *also* the first,
4440  // and even being already processed but we must make sure that there
4441  // aren't jobs _after_ it).
4442  if (jobCount > 0) {
4443  ViewItemJob *job = mViewItemJobs.at(jobCount - 1);
4444  if (job->currentPass() == ViewItemJob::Pass1Update) {
4445  if ((job->currentIndex() <= job->endIndex()) && job->invariantIndexList()) {
4446  // We can still attach this :)
4447  *(job->invariantIndexList()) += *indexesThatNeedUpdate;
4448  job->setEndIndex(job->endIndex() + indexesThatNeedUpdate->count());
4449  delete indexesThatNeedUpdate;
4450  indexesThatNeedUpdate = nullptr;
4451  }
4452  }
4453  }
4454 
4455  if (indexesThatNeedUpdate) {
4456  // Didn't append to any existing update job.. create a new one
4457  // FIXME: Should take timing options from aggregation here ?
4458  auto job = new ViewItemJob(ViewItemJob::Pass1Update, indexesThatNeedUpdate, 100, 50, 10);
4459  mViewItemJobs.append(job);
4460  }
4461 
4462  if (!mFillStepTimer.isActive()) {
4463  mFillStepTimer.start(mViewItemJobStepIdleInterval);
4464  }
4465  }
4466 }
4467 
4468 void ModelPrivate::slotStorageModelHeaderDataChanged(Qt::Orientation, int, int)
4469 {
4470  if (mStorageModelContainsOutboundMessages != mStorageModel->containsOutboundMessages()) {
4471  mStorageModelContainsOutboundMessages = mStorageModel->containsOutboundMessages();
4472  Q_EMIT q->headerDataChanged(Qt::Horizontal, 0, q->columnCount());
4473  }
4474 }
4475 
4476 Qt::ItemFlags Model::flags(const QModelIndex &index) const
4477 {
4478  if (!index.isValid()) {
4479  return Qt::NoItemFlags;
4480  }
4481 
4482  Q_ASSERT(d->mModelForItemFunctions); // UI must be connected if a valid index was queried
4483 
4484  Item *it = static_cast<Item *>(index.internalPointer());
4485 
4486  Q_ASSERT(it);
4487 
4488  if (it->type() == Item::GroupHeader) {
4489  return Qt::ItemIsEnabled;
4490  }
4491 
4492  Q_ASSERT(it->type() == Item::Message);
4493 
4494  if (!static_cast<MessageItem *>(it)->isValid()) {
4495  return Qt::NoItemFlags; // not enabled, not selectable
4496  }
4497 
4498  if (static_cast<MessageItem *>(it)->aboutToBeRemoved()) {
4499  return Qt::NoItemFlags; // not enabled, not selectable
4500  }
4501 
4502  if (static_cast<MessageItem *>(it)->status().isDeleted()) {
4503  return Qt::NoItemFlags; // not enabled, not selectable
4504  }
4505 
4507 }
4508 
4509 QMimeData *MessageList::Core::Model::mimeData(const QModelIndexList &indexes) const
4510 {
4512  for (const QModelIndex &idx : indexes) {
4513  if (idx.isValid()) {
4514  Item *item = static_cast<Item *>(idx.internalPointer());
4515  if (item->type() == MessageList::Core::Item::Message) {
4516  msgs << static_cast<MessageItem *>(idx.internalPointer());
4517  }
4518  }
4519  }
4520  return storageModel()->mimeData(msgs);
4521 }
4522 
4524 {
4525  return d->mRootItem;
4526 }
4527 
4528 bool Model::isLoading() const
4529 {
4530  return d->mLoading;
4531 }
4532 
4534 {
4535  if (!d->mStorageModel) {
4536  return nullptr;
4537  }
4538  auto idx = d->mInvariantRowMapper->modelIndexRowToModelInvariantIndex(row);
4539  if (!idx) {
4540  return nullptr;
4541  }
4542 
4543  return static_cast<MessageItem *>(idx);
4544 }
4545 
4546 MessageItemSetReference Model::createPersistentSet(const QVector<MessageItem *> &items)
4547 {
4548  if (!d->mPersistentSetManager) {
4549  d->mPersistentSetManager = new MessageItemSetManager();
4550  }
4551 
4552  MessageItemSetReference ref = d->mPersistentSetManager->createSet();
4553  for (const auto mi : items) {
4554  d->mPersistentSetManager->addMessageItem(ref, mi);
4555  }
4556 
4557  return ref;
4558 }
4559 
4561 {
4562  if (d->mPersistentSetManager) {
4563  return d->mPersistentSetManager->messageItems(ref);
4564  }
4565  return {};
4566 }
4567 
4568 void Model::deletePersistentSet(MessageItemSetReference ref)
4569 {
4570  if (!d->mPersistentSetManager) {
4571  return;
4572  }
4573 
4574  d->mPersistentSetManager->removeSet(ref);
4575 
4576  if (d->mPersistentSetManager->setCount() < 1) {
4577  delete d->mPersistentSetManager;
4578  d->mPersistentSetManager = nullptr;
4579  }
4580 }
4581 
4582 #include "moc_model.cpp"
const QString & receiver() const
Returns the receiver associated to this item.
Definition: item.cpp:501
void append(const T &value)
@ SortMessagesByDateTime
Sort the messages by date and time.
Definition: sortorder.h:77
@ PerfectReferencesAndSubject
Thread by all of the above and try to match subjects too.
Definition: aggregation.h:69
@ GroupBySenderOrReceiver
Group by sender (incoming) or receiver (outgoing) field.
Definition: aggregation.h:40
size_t size() const
Returns the size of this item (size of the Message, mainly)
Definition: item.cpp:456
int month() const const
@ SortGroupsByReceiver
Sort groups by receiver (makes sense only with GroupByReceiver)
Definition: sortorder.h:65
const SortOrder * sortOrder() const
Returns the sort order.
Definition: model.cpp:386
The MessageItem class.
Definition: messageitem.h:34
@ GroupBySender
Group by sender, always.
Definition: aggregation.h:41
UserRole
AKONADICORE_EXPORT Collection fromIndex(const QModelIndex &index)
void setStorageModel(StorageModel *storageModel, PreSelectionMode preSelectionMode=PreSelectLastSelected)
Sets the storage model from that the messages to be displayed should be fetched.
Definition: model.cpp:742
QVariant fromValue(const T &value)
SortDirection
The "generic" sort direction: used for groups and for messages If you add values here please look at ...
Definition: sortorder.h:68
void * internalPointer() const const
The MessageList::View is the real display of the message list.
Definition: view.h:47
bool isViewable() const
Is this item attached to the viewable root ?
Definition: item.cpp:357
@ SortMessagesByReceiver
Sort the messages by receiver.
Definition: sortorder.h:81
@ SortMessagesByActionItemStatus
Sort the messages by the "Action Item" flag of status.
Definition: sortorder.h:84
int column() const const
void layoutChanged(const QList< QPersistentModelIndex > &parents, QAbstractItemModel::LayoutChangeHint hint)
@ NoGroupSorting
Don't sort the groups at all, add them as they come in.
Definition: sortorder.h:60
@ PerfectThreadingOnly
Only the data for messageIdMD5 and inReplyToMD5 is needed.
int year() const const
Qt::DayOfWeek firstDayOfWeek() const const
@ SortGroupsByDateTimeOfMostRecent
Sort groups by date/time of the most recent message.
Definition: sortorder.h:62
QIcon fromTheme(const QString &name)
static const MessageStatus statusWatched()
@ AlwaysExpandGroups
All groups are expanded as they are inserted.
Definition: aggregation.h:56
KGuiItem clear()
@ NoThreading
Perform no threading at all.
Definition: aggregation.h:66
Type type() const
Returns the type of this item.
Definition: item.cpp:342
Item * rootItem() const
Returns the hidden root item that all the messages are (or will be) attached to.
Definition: model.cpp:4523
This class is an optimizing helper for dealing with large flat QAbstractItemModel objects.
int y() const const
@ TopmostMessage
The thread grouping is computed from the topmost message (very similar to least recent,...
Definition: aggregation.h:79
@ ExpandRecentGroups
Makes sense only with GroupByDate or GroupByDateRange.
Definition: aggregation.h:55
void deletePersistentSet(MessageItemSetReference ref)
Deletes the persistent set pointed by the specified reference.
Definition: model.cpp:4568
void setStatus(Akonadi::MessageStatus status)
Sets the status associated to this Item.
Definition: item.cpp:451
void setSecsSinceEpoch(qint64 secs)
@ InvisibleRoot
This item is just Item and it's the only InvisibleRoot per Model.
Definition: item.h:59
QString standaloneMonthName(int month, QLocale::FormatType type) const const
KIOFILEWIDGETS_EXPORT QStringList list(const QString &fileClass)
Item * parent() const
Returns the parent Item in the tree, or 0 if this item isn't attached to the tree.
Definition: item.cpp:436
void setSortOrder(const SortOrder *sortOrder)
Sets the sort order.
Definition: model.cpp:381
@ PerfectOnly
Thread by "In-Reply-To" field only.
Definition: aggregation.h:67
QMetaObject::Connection connect(const QObject *sender, const char *signal, const QObject *receiver, const char *method, Qt::ConnectionType type)
@ GroupByDate
Group the messages by the date of the thread leader.
Definition: aggregation.h:38
@ NeverExpandGroups
Never expand groups during a view fill algorithm.
Definition: aggregation.h:54
@ ExpandThreadsWithUnreadMessages
Expand threads with unread messages (this includes new)
Definition: aggregation.h:93
QString displaySender() const
Display sender.
Definition: item.cpp:496
Q_GLOBAL_STATIC(Internal::StaticControl, s_instance) class ControlPrivate
@ SortMessagesBySender
Sort the messages by sender.
Definition: sortorder.h:80
@ NeverExpandThreads
Never expand any thread, this is fast.
Definition: aggregation.h:91
time_t maxDate() const
Returns the maximum date in the subtree originating from this item.
Definition: item.cpp:476
QLocale system()
void takeChildItem(Model *model, Item *child)
Removes a child from this item's child list without deleting it.
Definition: item.cpp:637
@ ImperfectParentFound
this message found an imperfect parent to attach to (might be fixed later)
Definition: messageitem.h:62
bool isWatched() const
QList< Item * > * childItems() const
Return the list of child items.
Definition: item.cpp:59
QModelIndex createIndex(int row, int column, void *ptr) const const
@ ExpandNeeded
Must expand when this item becomes viewable.
Definition: item.h:68
typedef ItemFlags
@ NoGrouping
Don't group messages at all.
Definition: aggregation.h:37
QString i18n(const char *text, const TYPE &arg...)
void dump(const QString &prefix)
Debug helper.
Definition: item.cpp:620
void dataChanged(const QModelIndex &topLeft, const QModelIndex &bottomRight, const QVector< int > &roles)
char * toString(const T &value)
bool isIgnored() const
@ AlwaysExpandThreads
Expand all threads (this might be very slow)
Definition: aggregation.h:94
Orientation
void timeout()
NETWORKMANAGERQT_EXPORT NetworkManager::Status status()
WaitCursor
QByteArray toUtf8() const const
Item * childItem(int idx) const
Returns the child item at position idx or 0 if idx is out of the allowable range.
Definition: item.cpp:64
@ NoMessageSorting
Don't sort the messages at all.
Definition: sortorder.h:76
const T & at(int i) const const
StorageModel * storageModel() const
Returns the StorageModel currently set.
Definition: model.cpp:692
@ PerfectThreadingReferencesAndSubject
All of the above plus subject stuff.
MessageItem * messageItemByStorageRow(int row) const
Returns the message item that is at the current storage row index or zero if no such storage item is ...
Definition: model.cpp:4533
QString toString(qlonglong i) const const
@ ExpandThreadsWithUnreadOrImportantMessages
Expand threads with "hot" messages (this includes new, unread, important, todo)
Definition: aggregation.h:95
bool isValid() const const
static const MessageStatus statusIgnored()
@ GroupByReceiver
Group by receiver, always.
Definition: aggregation.h:42
QString displaySenderOrReceiver() const
Display sender or receiver.
Definition: item.cpp:521
void headerDataChanged(Qt::Orientation orientation, int first, int last)
PreSelectionMode
Pre-selection is the action of automatically selecting a message just after the folder has finished l...
int childItemCount() const
Returns the number of children of this Item.
Definition: item.cpp:158
QDate currentDate()
A class which holds information about sorting, e.g.
Definition: sortorder.h:34
@ ParentMissing
this message might belong to a thread but its parent is actually missing
Definition: messageitem.h:63
QMimeData * mimeData(const QModelIndexList &indexes) const override
Called when user initiates a drag from the messagelist.
Definition: model.cpp:4509
@ SortMessagesBySize
Sort the messages by size.
Definition: sortorder.h:83
void setAggregation(const Aggregation *aggregation)
Sets the Aggregation mode.
Definition: model.cpp:370
bool useReceiver() const
Returns whether sender or receiver is supposed to be displayed.
Definition: item.cpp:526
qint64 elapsed() const const
bool isValid() const const
@ NoExpandNeeded
No expand needed at all.
Definition: item.h:69
@ SortGroupsBySenderOrReceiver
Sort groups by sender or receiver (makes sense only with GroupBySenderOrReceiver)
Definition: sortorder.h:63
@ SortMessagesByUnreadStatus
Sort the messages by the "Unread" flags of status.
Definition: sortorder.h:85
void setParent(Item *pParent)
Sets the parent for this item.
Definition: item.cpp:441
bool recomputeMaxDate()
Recompute the maximum date from the current children list.
Definition: item.cpp:323
@ MostRecentMessage
The thread grouping is computed from the most recent message.
Definition: aggregation.h:81
@ Message
This item is a MessageItem.
Definition: item.h:58
int row() const const
QString i18np(const char *singular, const char *plural, const TYPE &arg...)
QVariant data(const QModelIndex &index, int role=Qt::DisplayRole) const override
Definition: model.cpp:487
@ FavorSpeed
Do larger chunks of work, zero intervals between chunks.
Definition: aggregation.h:106
The Theme class defines the visual appearance of the MessageList.
Definition: theme.h:48
QString displayReceiver() const
Display receiver.
Definition: item.cpp:511
@ NonThreadable
this message does not look as being threadable
Definition: messageitem.h:64
bool isEmpty() const const
void start()
void rowsInserted(const QModelIndex &parent, int first, int last)
void setOverrideCursor(const QCursor &cursor)
QString & insert(int position, QChar ch)
int elapsed() const const
void rowsRemoved(const QModelIndex &parent, int first, int last)
bool isValid(QStringView ifopt)
@ SortMessagesByAttachmentStatus
Sort the messages By "Important" flags of status.
Definition: sortorder.h:87
QDate date() const const
void setPreSelectionMode(PreSelectionMode preSelect)
Sets the pre-selection mode.
Definition: model.cpp:1003
The QAbstractItemModel based interface that you need to provide for your storage to work with Message...
@ SortMessagesBySubject
Sort the messages by subject.
Definition: sortorder.h:82
@ PerfectParentFound
this message found a perfect parent to attach to
Definition: messageitem.h:61
QString i18nc(const char *context, const char *text, const TYPE &arg...)
QSet::iterator insert(const T &value)
@ BatchNoInteractivity
Do one large chunk, no interactivity at all.
Definition: aggregation.h:107
QList::iterator begin()
InitialExpandStatus initialExpandStatus() const
The initial expand status we have to honor when attaching to the viewable root.
Definition: item.cpp:347
const Akonadi::MessageStatus & status() const
Returns the status associated to this Item.
Definition: item.cpp:446
@ SortMessagesByDateTimeOfMostRecent
Sort the messages by date and time of the most recent message in subtree.
Definition: sortorder.h:78
@ FavorInteractivity
Do small chunks of work, small intervals between chunks to allow for UI event processing.
Definition: aggregation.h:105
@ ExpandExecuted
Item already expanded.
Definition: item.h:70
qint32 toQInt32() const
This class manages sets of messageitem references.
void setInitialExpandStatus(InitialExpandStatus initialExpandStatus)
Set the initial expand status we have to honor when attaching to the viewable root.
Definition: item.cpp:352
void restoreOverrideCursor()
This class is responsible of matching messages that should be displayed in the View.
Definition: filter.h:44
@ SortGroupsByDateTime
Sort groups by date/time of the group.
Definition: sortorder.h:61
time_t date() const
Returns the date of this item.
Definition: item.cpp:466
@ SortMessagesBySenderOrReceiver
Sort the messages by sender or receiver.
Definition: sortorder.h:79
int appendChildItem(Model *model, Item *child)
Appends an Item to this item's child list.
Definition: item.cpp:597
~Model() override
Destroys the mighty model along with the tree of items it manages.
Definition: model.cpp:352
A single item of the MessageList tree managed by MessageList::Model.
Definition: item.h:47
QList::iterator end()
int dayOfWeek() const const
@ SortGroupsBySender
Sort groups by sender (makes sense only with GroupBySender)
Definition: sortorder.h:64
QString standaloneDayName(int day, QLocale::FormatType type) const const
bool hasAncestor(const Item *it) const
Return true if Item pointed by it is an ancestor of this item (that is, if it is its parent,...
Definition: item.cpp:362
const QString & sender() const
Returns the sender associated to this item.
Definition: item.cpp:486
@ ExpandThreadsWithNewMessages
DEPRECATED. New message status no longer exists.
Definition: aggregation.h:92
@ PerfectAndReferences
Thread by "In-Reply-To" and "References" fields.
Definition: aggregation.h:68
void setFilter(const Filter *filter)
Sets the Filter to be applied on messages.
Definition: model.cpp:391
void setTheme(const Theme *theme)
Sets the Theme.
Definition: model.cpp:376
void setMaxDate(time_t date)
Sets the maximum date in the subtree originating from this item.
Definition: item.cpp:481
QList< MessageItem * > persistentSetCurrentMessageItemList(MessageItemSetReference ref)
Returns the list of MessageItems that are still existing in the set pointed by the specified referenc...
Definition: model.cpp:4560
A set of aggregation options that can be applied to the MessageList::Model in a single shot.
Definition: aggregation.h:28
bool isImportant() const
QObject * parent() const const
QString message
qint64 daysTo(const QDate &d) const const
@ GroupByDateRange
Use smart (thread leader) date ranges ("Today","Yesterday","Last Week"...)
Definition: aggregation.h:39
const QList< QKeySequence > & end()
const QString & subject() const
Returns the subject associated to this Item.
Definition: item.cpp:531
@ GroupHeader
This item is a GroupHeaderItem.
Definition: item.h:57
char * data()
@ PerfectThreadingPlusReferences
messageIdMD5, inReplyToMD5, referencesIdMD5
MessageItemSetReference createPersistentSet(const QVector< MessageItem * > &items)
Creates a persistent set for the specified MessageItems and returns its reference.
Definition: model.cpp:4546
bool isLoading() const
Returns true if the view is currently loading, that is it's in the first (possibly lengthy) job batch...
Definition: model.cpp:4528
This file is part of the KDE documentation.
Documentation copyright © 1996-2022 The KDE developers.
Generated on Tue May 17 2022 04:00:05 by doxygen 1.8.17 written by Dimitri van Heesch, © 1997-2006

KDE's Doxygen guidelines are available online.