Okular

How to implement a Generator

The power of Okular is its extensibility by Generator plugins.

This section will describe how to implement your own plugin for a new document type.

A Basic Generator

To provide a short overview and don't reimplementing an existing generator we'll work on a Generator for the Magic document format, a non existing, pure virtual format :)

Lets assume we have some helper class (MagicDocument) which provides the following functionality for this document format:

  • Loading a document
  • Retrieving number of pages
  • Returning a fixed size picture representation of a page

The class API looks like this

class MagicDocument
{
public:
MagicDocument();
~MagicDocument();
bool loadDocument( const QString &fileName );
int numberOfPages() const;
QSize pageSize( int pageNumber ) const;
QImage pictureOfPage( int pageNumber ) const;
private:
...
};

The methods should be self explaining, loadDocument() loads a document file and returns false on error, numberOfPages() returns the number of pages, pageSize() returns the size of the page and pictureOfPage() returns the picture representation of the page.

Our first version of our Generator is a basic one which just provides page pictures to the document class.

The API of the Generator looks like the following:

#include "magicdocument.h"
#include <okular/core/generator.h>
class MagicGenerator : public Okular::Generator
{
public:
MagicGenerator( QObject *parent, const QVariantList &args );
~MagicGenerator();
bool loadDocument( const QString &fileName, QVector<Okular::Page*> &pages );
bool canGeneratePixmap() const;
protected:
private:
MagicDocument mMagicDocument;
};

The implementation of the Generator looks like this:

#include <okular/core/page.h>
#include "magicgenerator.h"
OKULAR_EXPORT_PLUGIN(MagicGenerator, "libokularGenerator_magic.json")
MagicGenerator::MagicGenerator( QObject *parent, const QVariantList &args )
: Okular::Generator( parent, args )
{
}
MagicGenerator::~MagicGenerator()
{
}
bool MagicGenerator::loadDocument( const QString &fileName, QVector<Okular::Page*> &pages )
{
if ( !mMagicDocument.loadDocument( fileName ) ) {
emit error( i18n( "Unable to load document" ), -1 );
return false;
}
pagesVector.resize( mMagicDocument.numberOfPages() );
for ( int i = 0; i < mMagicDocument.numberOfPages(); ++i ) {
const QSize size = mMagicDocument.pageSize( i );
Okular::Page * page = new Okular::Page( i, size.width(), size.height(), Okular::Rotation0 );
pages[ i ] = page;
}
return true;
}
bool MagicGenerator::doCloseDocument()
{
return true;
}
bool MagicGenerator::canGeneratePixmap() const
{
return true;
}
void MagicGenerator::generatePixmap( Okular::PixmapRequest *request )
{
QImage image = mMagicDocument.pictureOfPage( request->pageNumber() );
image = image.scaled( request->width(), request->height(), Qt::IgnoreAspectRatio, Qt::SmoothTransformation );
request->page()->setPixmap( request->id(), new QPixmap( QPixmap::fromImage( image ) ) );
signalPixmapRequestDone( request );
}

As you can see implementing a basic Generator is quite easy. The loadDocument() method opens the document file and extracts the number of pages. For every page in the document it adds an Okular::Page object to the pages vector which is passed in as method argument. Each page is initialized with its page number, width, height and initial rotation. These page objects will be stored in the document object and act as a container for the picture representation of the pages. This code is the same for nearly every Generator. On an failure the error() signal can be emitted to inform the user about the issue. This code is the same for nearly every Generator.

In the doCloseDocument() method you should close the document and free all resources you have allocated in openDocument().

Now we come to the picture creation methods. The canGeneratorPixmap() method returns whether the Generator is currently able to handle a new pixmap generation request. For a simple Generator like our one that's always the case as it works linear, however a multithreaded Generator might return false here if it is still waiting for one of its working threads to finish. In this case the document class will try to request the pixmap later again.

The generatePixmap() method does the actual fetching of the picture for a page. The page number, requested width and height of the page is encapsulated in the passed Okular::PixmapRequest object. So the task of the Generator is to create a pixmap of the requested page in the requested size and then store this pixmap in the Okular::Page object which is associated with the page request. When this task is finished, the Generator has to call signalPixmapRequestDone() with the page request object as argument. This extra call is needed to allow the Generator to use signals and slots internally and create the pixmap asynchronously.

So now you have the code of a working Okular Generator, the next step is to tell Okular about the new plugin. Like in other places in KDE that is done by .desktop files, which are installed to the services directory.

Every Generator needs 1 .json, 3 .desktop files, and 1 .xml file:

  • libokularGenerator_<name>.json
  • okularApplication_<name>.desktop
  • okular<name>.desktop
  • org.kde.mobile.okular_<name>.desktop
  • org.kde.okular-<name>.metainfo.xml

where <name> should be the name of the document format. So for our Magic Document Generator we create the following 4 files:

  • libokularGenerator_magic.json
  • okularApplication_magic.desktop
  • okularMagic.desktop
  • org.kde.mobile.okular_magic.desktop
  • org.kde.okular-magic.metainfo.xml

where libokularGenerator_magic.json has the following content something like this

{
    "KPlugin": {
        "Authors": [
            {
                "Email": "[email protected]",
                "Name": "Proud Author",
            }
        ],
        "Copyright": "© 2042 Proud Author",
        "Id": "okular_magic",
        "License": "GPL",
        "MimeTypes": [
            "text/magic",
            "text/x-magic"
        ],
        "Name": "Magic Backend",
        "ServiceTypes": [
            "okular/Generator"
        ],
        "Version": "0.1.0"
    },
    "X-KDE-Priority": 1,
    "X-KDE-okularAPIVersion": 1,
    "X-KDE-okularHasInternalSettings": true
}

The last five fields has the special meaning to Okular

  • ServiceType Must be 'okular/Generator' for all Okular Generator Plugins
  • MimeType The mimetype or list of mimetypes of the supported document format(s)
  • X-KDE-Priority When multiple Generators for the same mimetype exists, the one with the highest priority is used
  • X-KDE-okularAPIVersion The version of the Generator Plugin API ('1' currently)
  • X-KDE-okularHasInternalSettings Is 'true' when the Generator provides configuration dialogs

The first .desktop file has the following content:

[Desktop Entry]
MimeType=application/x-magic;
Terminal=false
Name=okular
GenericName=Document Viewer
Exec=okular %U
Icon=okular
Type=Application
InitialPreference=7
Categories=Qt;KDE;Graphics;Viewer;
NoDisplay=true
X-KDE-Keywords=Magic

You can use the file as it is, you just have to adapt the mimetype. This file is needed to allow Okular to handle multiple mimetypes.

The second .desktop file looks like this:

[Desktop Entry]
Icon=okular
Name=okular
X-KDE-ServiceTypes=KParts/ReadOnlyPart
X-KDE-Library=okularpart
Type=Service
MimeType=application/x-magic;

where

  • X-KDE-Library The name of the plugin library

You can use the file as it is as well, you just have to adapt the mimetype. This file is needed to allow the Okular part to handle multiple mimetypes.

The third .desktop file contains data for the mobile version

[Desktop Entry]
MimeType=application/x-magic;
Name=Reader
GenericName=Document viewer
Comment=Viewer for various types of documents
TryExec=kpackagelauncherqml -a org.kde.mobile.okular
Exec=kpackagelauncherqml -a org.kde.mobile.okular %u
Terminal=false
Icon=okular
Type=Application
Categories=Qt;KDE;Graphics;Office;Viewer;
InitialPreference=2
NoDisplay=true
X-KDE-Keywords=Magic

And the last .xml file has the following content

<?xml version="1.0" encoding="utf-8"?>
<component type="addon">
  <id>org.kde.okular-md</id>
  <extends>org.kde.okular.desktop</extends>
  <metadata_license>CC0-1.0</metadata_license>
  <project_license>GPL-2.0+ and GFDL-1.3</project_license>
  <name>Magic</name>
  <summary>Adds support for reading Magic documents</summary>
  <mimetypes>
    <mimetype>application/magic</mimetype>
  </mimetypes>
  <url type="homepage">https://okular.kde.org</url>
</component>

The last piece you need for a complete Generator is a CMakeLists.txt which compiles and installs the Generator. Our CMakeLists.txt looks like the following:

add_definitions(-DTRANSLATION_DOMAIN="okular_magic")

macro_optional_find_package(Okular)

include_directories( ${OKULAR_INCLUDE_DIR} ${KF5_INCLUDE_DIR} ${QT_INCLUDES} )

########### next target ###############

set( okularGenerator_magic_PART_SRCS generator_magic.cpp )

target_link_libraries( okularGenerator_magic PRIVATE okularcore KF5::I18n KF5::KIOCore )

install( TARGETS okularGenerator_magic DESTINATION ${PLUGIN_INSTALL_DIR} )

########### install files ###############

install( FILES okularMagic.desktop  DESTINATION  ${KDE_INSTALL_KSERVICES5DIR} )
install( PROGRAMS okularApplication_magic.desktop org.kde.mobile.okular_magic.desktop DESTINATION ${KDE_INSTALL_APPDIR} )
install( FILES org.kde.okular-magic.metainfo.xml DESTINATION ${KDE_INSTALL_METAINFODIR} )

The macro_optional_find_package(Okular) call is required to make the ${OKULAR_INCLUDE_DIR} and ${OKULAR_LIBRARIES} variables available.

Now you can compile the Generator plugin and install it. After a restart of Okular the new plugin is available and you can open Magic documents.

A Generator with TextPage support

In this section we want to extend our Generator to support text search, text extraction and selection as well. As mentioned in Generators with Text support, the Generator must provide an Okular::TextPage object for every page which contains readable text.

Since we use the helper class MagicDocument to read the data from the document we have to extend it first, so the new API looks as the following:

class MagicDocument
{
public:
MagicDocument();
~MagicDocument();
bool loadDocument( const QString &fileName );
int numberOfPages() const;
QSize pageSize( int pageNumber ) const;
QImage pictureOfPage( int pageNumber ) const;
class TextInfo
{
public:
typedef QList<TextInfo> List;
QChar character;
qreal xPos;
qreal yPos;
qreal width;
qreal height;
};
TextInfo::List textOfPage( int pageNumber );
private:
...
};

MagicDocument has the new internal class TextInfo now, which contains a character and its absolute position on a page. Furthermore MagicDocument provides a method textOfPage() which returns a list of all TextInfo objects for a page.

That's really an optimistic API, in reality it is sometimes quite hard to find out the position of single characters in a document format.

With the extension of our helper class we can continue on extending our Generator now:

#include "magicdocument.h"
#include <okular/core/generator.h>
class MagicGenerator : public Okular::Generator
{
public:
MagicGenerator( QObject *parent, const QVariantList &args );
~MagicGenerator();
bool loadDocument( const QString &fileName, QVector<Okular::Page*> &pages );
bool canGeneratePixmap() const;
virtual bool canGenerateTextPage() const;
protected:
private:
MagicDocument mMagicDocument;
};

We have extended the MagicGenerator class by two methods canGenerateTextPage() and generateTextPage(). The first method is equal to canGeneratePixmap(), it returns whether the Generator is currently able to handle a new text page generation request. For linear Generators that should be always the case, however when the generation is done in a separated worker thread, this method might return false. In this case the document class will try to request the text page later again.

The second method will generate the Okular::TextPage object for the passed page. Depending on the capabilities of the Generator and the passed type parameter that is done synchronously or asynchronously.

Let us take a look at the implementation of these methods in our MagicGenerator:

#include <okular/core/textpage.h>
...
MagicGenerator::MagicGenerator( QObject *parent, const QVariantList &args )
: Okular::Generator( parent, args )
{
setFeature( TextExtraction );
}
bool MagicGenerator::canGenerateTextPage() const
{
return true;
}
void MagicGenerator::generateTextPage( Okular::Page *page, enum Okular::GenerationType )
{
MagicDocument::TextInfo::List characters = mMagicDocument.textOfPage( page->number() );
if ( characters.isEmpty() )
return;
for ( int i = 0; i < characters.count(); ++i ) {
qreal left = characters[ i ].xPos / page->width();
qreal top = characters[ i ].yPos / page->height();
qreal right = (characters[ i ].xPos + characters[ i ].width) / page->width();
qreal bottom = (characters[ i ].yPos + characters[ i ].height) / page->height();
textPage->append( characters[ i ].character,
new Okular::NormalizedRect( left, top, right, bottom ) );
}
page->setTextPage( textPage );
}

As you can see the generateTextPage method just iterates over the list of characters returned by our MagicDocument helper class and adds the character and its normalized bounding rect to the Okular::TextPage object. At the end the text page is assigned to the page. We don't pay attention to the GenerationType parameter here, if your Generator want to use threads, it should check here whether the request shall be done asynchronously or synchronously and start the generation according to that. Additionally we have to tell the Okular::Generator base class that we support text handling by setting this flag in the constructor.

In this state we can now search, select and extract text from Magic documents.

A Generator with Thread support

Sometimes it makes sense to do the generation of page pictures or text pages asynchronously to improve performance and don't blocking the user interface. This can be done in two ways, either by using signals and slots or by using threads. Both have there pros and cons:

  • Signals and Slots
    • Pro: Can be used with backend libraries which are not thread safe
    • Con: Sometime difficult to implement
  • Threads
    • Pro: Easy to implement as you can make synchronous calls to the backend libraries
    • Con: Backend libraries must be thread safe and you must prevent race conditions by using mutexes

The signal and slots approach can be achieved with a normal Generator by calling Okular::Generator::signalPixmapRequestDone() from a slot after pixmap generation has been finished.

When using threads you should use a slightly different API, which hides most of the thread usage, to make implementing as easy as possible.

Let's assume the pictureOfPage() and textOfPage methods in our MagicDocument helper class are thread safe, so we can use them in a multithreaded environment. So nothing prevents us from changing the MagicGenerator to use threads for better performance.

The new MagicGenerator API looks like the following:

#include "magicdocument.h"
#include <okular/core/generator.h>
class MagicGenerator : public Okular::Generator
{
public:
MagicGenerator( QObject *parent, const QVariantList &args );
~MagicGenerator();
bool loadDocument( const QString &fileName, QVector<Okular::Page*> &pages );
protected:
virtual QImage image( Okular::PixmapRequest *request );
private:
MagicDocument mMagicDocument;
};

As you can see the canGeneratePixmap() generatePixmap(), canGenerateTextPage() and generateTextPage() methods have been removed and replaced by the image() and textPage() methods.

Before explaining why, we'll take a look at the implementation:

MagicGenerator::MagicGenerator( QObject *parent, const QVariantList &args )
: Okular::Generator( parent, args )
{
setFeature( TextExtraction );
setFeature( Threaded );
}
QImage MagicGenerator::image( Okular::PixmapRequest *request )
{
QImage image = mMagicDocument.pictureOfPage( request->pageNumber() );
return image.scaled( request->width(), request->height(), Qt::IgnoreAspectRatio, Qt::SmoothTransformation );
}
Okular::TextPage* textPage( Okular::Page *page )
{
MagicDocument::TextInfo::List characters = mMagicDocument.textOfPage( page->number() );
if ( characters.isEmpty() )
return 0;
for ( int i = 0; i < characters.count(); ++i ) {
qreal left = characters[ i ].xPos / page->width();
qreal top = characters[ i ].yPos / page->height();
qreal right = (characters[ i ].xPos + characters[ i ].width) / page->width();
qreal bottom = (characters[ i ].yPos + characters[ i ].height) / page->height();
textPage->append( characters[ i ].character,
new Okular::NormalizedRect( left, top, right, bottom ) );
}
return textPage;
}

So the first obviously thing is that both methods return a value instead of modifying the page directly. The reason for this is that both methods are executed in its own thread, so the code executed in them can block as long as it wants, it won't block the GUI anyway. Additionally we have to tell the Okular::Generator base class that we can handle threads by setting the flag in the constructor.

With only a small change we made our MagicGenerator multithreaded now!

An Extended Generator

Now we want to create a new generator with some additional functionality:

  • Support for document information (author, creation date etc.)
  • Support for a table of content
  • Support for printing the document
  • Support for exporting the document as text

The new Generator shall be able to handle HTML documents. We choose this format as example, because we can use QTextDocument to load, render and print a HTML page, so a lot of code can be reused.

The API of our HTMLGenerator looks like the following:

#include <QtGui/QTextDocument>
#include <okular/core/generator.h>
class HTMLGenerator : public Okular::Generator
{
public:
HTMLGenerator( QObject *parent, const QVariantList &args );
~HTMLGenerator();
bool loadDocument( const QString &fileName, QVector<Okular::Page*> &pages );
bool canGeneratePixmap() const;
virtual bool print( KPrinter &printer );
virtual bool exportTo( const QString &fileName, const Okular::ExportFormat &format );
protected:
private:
QTextDocument *mTextDocument;
Okular::DocumentInfo mDocumentInfo;
Okular::DocumentSynopsis mDocumentSynopsis;
};

The Generator doesn't support text search and selection, as the code would be quite complex, we'll show how to do it in the next chapter (not yet written) anyway.

As you can see we have 5 new methods in the class:

  • generateDocumentInfo() Creates an Okular::DocumentInfo (which is in fact a QDomDocument) which contains document information like author, creation time etc.
  • generateDocumentSynopsis() Creates an Okular::DocumentSynopsis (which is a QDomDocument as well) which contains the table of content.
  • print() Prints the document to the passed printer.
  • exportFormats() Returns the supported export formats.
  • exportTo() Exports the document to the given file in the given format.

Now that you know what the methods are supposed to do, let's take a look at the implementation:

#include <QFile>
#include <QAbstractTextDocumentLayout>
#include <QPrinter>
#include <okular/core/document.h>
#include <okular/core/page.h>
#include "htmlgenerator.h"
#include <KLocalizedString>
OKULAR_EXPORT_PLUGIN(HTMLGenerator, "libokularGenerator_html.json")
HTMLGenerator::HTMLGenerator( QObject *parent, const QVariantList &args )
: Okular::Generator( parent, args ),
mTextDocument( 0 )
{
}
HTMLGenerator::~HTMLGenerator()
{
delete mTextDocument;
}
bool HTMLGenerator::loadDocument( const QString &fileName, QVector<Okular::Page*> &pages )
{
QFile file( fileName );
if ( !file.open( QIODevice::ReadOnly ) ) {
emit error( i18n( "Unable to open file" ), -1 );
return false;
}
const QString data = QString::fromUtf8( file.readAll() );
file.close();
mTextDocument = new QTextDocument;
mTextDocument->setHtml( data );
mTextDocument->setPageSize( QSizeF( 600, 800 ) );
pages.resize( mTextDocument->pageCount() );
for ( int i = 0; i < mTextDocument->pageCount(); ++i ) {
Okular::Page * page = new Okular::Page( i, 600, 800, Okular::Rotation0 );
pages[ i ] = page;
}
mDocumentInfo.set( "author", "Tobias Koenig", i18n( "Author" ) );
mDocumentInfo.set( "title", "The Art of Okular Plugin Development", i18n( "Title" ) );
Okular::DocumentViewport viewport = ... // get the viewport of the chapter
QDomElement item = mDocumentSynopsis.createElement( "Chapter 1" );
item.setAttribute( "Viewport", viewport.toString() );
mDocumentSynopsis.appendChild( item );
viewport = ... // get the viewport of the subchapter
QDomElement childItem = mDocumentSynopsis.createElement( "SubChapter 1.1" );
childItem.setAttribute( "Viewport", viewport.toString() );
item.appendChild( childItem );
return true;
}
bool HTMLGenerator::doCloseDocument()
{
delete mTextDocument;
mTextDocument = 0;
return true;
}
bool HTMLGenerator::canGeneratePixmap() const
{
return true;
}
void HTMLGenerator::generatePixmap( Okular::PixmapRequest *request )
{
QPixmap *pixmap = new QPixmap( request->width(), request->height() );
pixmap->fill( Qt::white );
p.begin( pixmap );
qreal width = request->width();
qreal height = request->height();
p.scale( width / 600, height / 800 );
const QRect rect( 0, request->pageNumber() * 800, 600, 800 );
p.translate( QPoint( 0, request->pageNumber() * -800 ) );
d->mDocument->drawContents( &p, rect );
p.end();
request->page()->setPixmap( request->id(), pixmap );
signalPixmapRequestDone( request );
}
Okular::DocumentInfo HTMLGenerator::generateDocumentInfo( const QSet<Okular::DocumentInfo::Key> &keys ) const
{
return mDocumentInfo;
}
const Okular::DocumentSynopsis* HTMLGenerator::generateDocumentSynopsis()
{
if ( !mDocumentSynopsis.hasChildNodes() )
return 0;
else
return &mDocumentSynopsis;
}
bool HTMLGenerator::print( KPrinter &printer )
{
QPainter p( &printer );
for ( int i = 0; i < mTextDocument->pageCount(); ++i ) {
if ( i != 0 )
printer.newPage();
QRect rect( 0, i * 800, 600, 800 );
p.translate( QPoint( 0, i * -800 ) );
mTextDocument->drawContents( &p, rect );
}
}
Okular::ExportFormat::List HTMLGenerator::exportFormats() const
{
}
bool HTMLGenerator::exportTo( const QString &fileName, const Okular::ExportFormat &format )
{
QFile file( fileName );
if ( !fileName.open( QIODevice::WriteOnly ) ) {
emit error( i18n( "Unable to open file" ), -1 );
return false;
}
if ( format.mimeType()->name() == QLatin1String( "text/plain" ) )
file.writeBlock( mTextDocument->toPlainText().toUtf8() );
file.close();
return true;
}

Let's take a closer look at the single methods. In the loadDocument() method we try to open the passed file name and read all the content into the QTextDocument object. By calling QTextDocument::setPageSize(), the whole document is divided into pages of the given size. In the next step we create Okular::Page objects for every page in the QTextDocument and fill the pages vector with them.

Afterwards we fill our Okular::DocumentInfo object with data. Since extracting the HTML meta data would need a lot of code we work with static data here. [to be continued]

This file is part of the KDE documentation.
Documentation copyright © 1996-2020 The KDE developers.
Generated on Mon Jul 6 2020 22:35:33 by doxygen 1.8.11 written by Dimitri van Heesch, © 1997-2006

KDE's Doxygen guidelines are available online.