Python Plugins with Yapsy

I have a little Python/GTK+ project that I work on here and there. It's a little text editor that I've re-written a dozen times (first few versions were written in C actually). Really, it's more of a platform for me to experiment with GTK+ and various GUI design techniques. I decided to scrap the plugin system which I had hacked together from scratch and replace it with something based on Yapsy (Yet Another Plugin SYstem).

Yapsy isn't a complete plugin framework but a rather simple system from which to build your own complete plugin system. Using only the Python standard library, Yapsy wasn't too far off from what I was tring to implement on my own. But, I liked some of it's approaches much better than what I had come up with. Since there are not many examples out there, I wrote an Example Python/GTK+ Application using Yapsy to demonstrate some of the techniques I'm playing with and put a TON of comments throughout the code.

Screenshot of yapsy-gtk-example Window

The Plugin Manager

The Yapsy PluginManager class handles locating, loading, and activating plugins. However, Yapsy also provides a singleton version of the class. I prefer using the singleton pattern for the plugin manager rather than passing around the instance to all the components of my application.

from yapsy.PluginManager import PluginManagerSingleton
# ...
manager = PluginManagerSingleton.get()

The PluginManager is given a list of directories in which to search for plugins. These directories contain special INI-style information files for each plugin as well as the Python code for the actual plugins. By passing the PluginManager directories based on the XDG Base Directory Specification, the end user could install plugins by simply copying them to their home folder data directory (eg. ~/.local/share/my-application/plugins) yet your application can also install plugins to the system data directory (eg. /usr/share/my-application/plugins).

import os
from xdg.BaseDirectory import xdg_data_dirs
from yapsy.PluginManager import PluginManagerSingleton

# ...

places = []
[places.append(os.path.join(path, "my-application", "plugins")) for path 
    in xdg_data_dirs]
manager = PluginManagerSingleton.get()
manager.setPluginPlaces(places)

Now, you may be wondering what happens if the same plugin exists in different directories. Perhaps even different versions of the same plugin. Fear not, Yapsy has a PluginManagerDecorator for just that purpose. Prior to getting the manager PluginManager instance, you can call the setBehavior() method to install the VersionedPluginManager decorator.

from yapsy.PluginManager import PluginManagerSingleton
from yapsy.VersionedPluginManager import VersionedPluginManager

# ...

PluginManagerSingleton.setBehaviour([
    VersionedPluginManager,
])
manager = PluginManagerSingleton.get()

Now the PluginManager will only load the most recent version of a plugin.

Another handy PluginManagerDecorator is ConfigurablePluginManager which will allow the plugin manager to load/save which plugins have been activated to
a configuration file (as well as plugin settings). The ConfigurablePluginManager is installed in the same way as the VersionedPluginManager.

PluginManagerSingleton.setBehaviour([
    VersionedPluginManager,
    ConfigurablePluginManager,
])

In order for the PluginManager to save the activated plugins and re-load them upon startup, a ConfigParser must be passed to the manager. The XDG Base Directory Specification comes in handy once again when saving your configuration file.

from xdg.BaseDirectory import save_config_path
from ConfigParser import SafeConfigParser
from yapsy.PluginManager import PluginManagerSingleton
from yapsy.ConfigurablePluginManager import ConfigurablePluginManager

# ...

PluginManagerSingleton.setBehaviour([
    ConfigurablePluginManager,
])
manager = PluginManagerSingleton.get()

config_path = save_config_path("my-application")
config_file = os.path.join(config_path, "my-application.conf")
parser = SafeConfigParser()
parser.read(config_file)
manager.setConfigParser(parser)

# ...

f = open(config_file, "w")
self.config.write(f)
f.close()

Once the PluginManager is setup, the plugins can be located and loaded using the collectPlugins() method.

manager.collectPlugins()

If the manager was setup with ConfigurablePluginManager then the plugins that were last activated by the user will also be automatically activated.

The Plugins

Each of the plugins manager.collectPlugins() consists of at least 2 files. First, the plugin information file provides some meta information about the plugin, which I use in a Gtk.About dialog, and a Python module that defines a class which extends the Yapsy IPlugin interface. It's this class that extends IPlugin that is the plugin code. What that plugin can do is up to you.

from yapsy.IPlugin import IPlugin

class MyPlugin(IPlugin):
    def activate(self):
        super(MyPlugin, self).activate()
        print "I've been activated!"

    def deactivate(self):
        super(MyPlugin, self).deactivate()
        print "I've been deactivated!"

In a real world application you would actually create your base plugin class(es) which would extend IPlugin and then your application's plugins would instead extend your base plugin class. But, just directly extending IPlugin is a good way to get started.

Now, in a basic python application you would iterate over PluginManager.getAllPlugins() or PluginManager.getPluginsOfCategory() to call pre-determined methods on any activated plugins at specific places in your application code. However, in a GTK+ application you can simply give the plugins access to specific objects and let the plugins connect to GObject signals.

For example, if your application extends Gtk.Application, a plugin may then connect to the "window-added" signal to add it's GUI components to any application window. The plugin would override the IPlugin.activate() method to add GUI components/connect to signals and the IPlugin.deactivate() method to remove those GUI components and disconnect the signal handlers. This concept is pretty well demonstrated in my Example Python/GTK+ Application using Yapsy

The only tricky part is giving your plugins access to your objects. Since the PluginManager instantiates the plugin and calls it's activate() and deactivate() methods, you cannot specify additional arguments. You could extend PluginManager with your own manager class and that would indeed be an ideal solution for some situations. However, my little text editor only needs the plugins to have access to my application object which has an API for everything they would need. My solution is to simply give the PluginManager a reference to the application object, and let the plugins get it from the manager singleton.

class MyApplication(object):
     def __init__(self):
        # ...
        manager = PluginManagerSingleton.get()
        manager.app = self
        #...

class MyPlugin(IPlugin):
    def __init__(self):
        super(MyPlugin, self).__init__()
        manager = PluginManagerSingleton.get()
        self.app = manager.app

Plugin List Widget

My example project includes a couple of widgets in a file aptly named widgets.py for displaying the list of installed plugins. The plugins can be activated and deactivated via checkboxes in the list and an about dialog shows the meta information for the plugin.

The list (which is a Gtk.TreeView) is populated by iterating PluginManager.getAllPlugins(). This method gives you a list of PluginInfo objects which contain the meta information for the plugin (name, version, author, description, etc.) as well as the instance of the plugin itself in a property named plugin_object.

manager = PluginManagerSingleton.get()
for info in manager.getAllPlugins():
    print info.plugin_object # an instance of the class you extended from IPlugin
    print info.name
    print info.version
    print info.website
    # etc.

When the checkbox in the PluginTreeView is toggled, PluginManager.activatePluginByName() or PluginManager.deactivatePluginByName() is called. Since PluginManager was setup with ConfigurablePluginManager, the ConfigParser is changed to reflect which plugins are active.

Viola--it's that simple. If I end up actually using these widgets in my project, I will clean them up and make them a little more versatile and release them separately.

Final Thoughts

All in all I'm pretty impressed with Yapsy. Right out of box (so to speak) it is handling locating, loading, activating/deactivating, versions, and configurations for my plugin system. And it does all this with only the Python standard library. This allows me to focus on how the plugins will play with my application, what API they will have access to, how to handle exceptions within plugins, etc.

Did you enjoy Python Plugins with Yapsy? If you would like to help support my work, A donation of a buck or two would be very much appreciated.
blog comments powered by Disqus
Linux Servers on the Cloud IN MINUTES