Firstly, printerd is experimental and is very far from being a functional print spooler. It doesn’t yet run any filters, for instance, and has no backends of its own to transport jobs to devices. So far it is just a test of what a Linux print spooler would look like if it were written today.

There are several problems printerd aims to solve. Some of the solutions come automatically from implementing it as a polkit-enabled D-Bus system service.  I mentioned one of them in the original post about printer: the fact of having an asynchronous client API. All D-Bus services can be used asynchronously thanks to the D-Bus client library. This approach means the print dialog will be able to use printerd without blocking (and without having to start another thread to use it).

Another benefit is the fact that security/authentication can be made much cleaner and more well integrated into a desktop system if polkit is the basis for policy decisions. Although there is some support for adding polkit support to CUPS in the shape of cups-pk-helper, the way this works is to effectively bypass the standard CUPS policy mechanism. In order to completely limit some operation from users, two different policies must be changed: the built-in CUPS policy in cupsd.conf, and the polkit policy for cups-pk-helper.

In contrast, printerd’s only interface is D-Bus and it uses polkit to authenticate operations. If extra security policy is added in future that cannot be expressed using polkit (for instance, printer-specific policy rules like CUPS has), that extra policy layer will be in addition to polkit, not an alternative. Currently, for example, cancelling a job requires that the user is permitted to use the org.freedesktop.printerd.job-cancel action (a polkit check), as well as being the user that submitted the job in the first place (an additional check). Both requirements must be fulfilled.

One more benefit of printerd is the idea of splitting the IPP server out from the local spooler. Currently printerd only spools files and does not provide IPP services but the idea is that when it does, this will be implemented out-of-process by a program that acts as a client to printerd. It will use printerd’s D-Bus interface, just like any other client. That way, users that want to print to local printers but are not interested in sharing those printers on the network don’t even have to run the IPP server. In fact, as printerd is an activatable system service, the spooler itself won’t even be running unless there is something for it to do.

By accepting only one format for printing, some complexity can be removed from the spooler. Generally people print from applications that generate PDF for printing. By using PDF as the required format, the task of selecting pages and arranging several pages onto each side (i.e. number-up) is hopefully made a little easier, as the structure of PDF makes this task more straightforward than PostScript does. Increasingly printers are able to understand PDF natively themselves. This means converting between formats can be kept to a minimum.

Finally, printerd is experimental, so ideas can be tested and developed. For example, I hope to get printerd to improve the latency between cancelling a job and having the printer stop feeding paper, by splitting the backend out of the filter pipeline, killing its input, and telling it to discard its send buffer. Another idea related to separating the filter pipeline from the backend is to begin filtering the next job before the backend finishes clearing out the last of its send buffer. I’m sure there are other areas for improvement that can be played around with in a project like printerd.

It is a polkit-enabled D-Bus system service, written using the GLib object system. Although modelled on concepts from IPP (Internet Printing Protocol), printerd is not in itself an IPP server. Its only interface is D-Bus, although the aim is to be able to implement an IPP server on top of the D-Bus API as a separate process. Having a D-Bus interface means that applications wanting to print automatically get to use printerd asynchronously.

As a design decision, the range of input formats accepted by printerd will be very limited: essentially only PDF. The existing CUPS drivers and backends will be compatible with printerd.

There isn’t much written yet aside from the basic framework and a very simple command line tool.

Feel free to take a look around: http://gitorious.org/printerd

$ pd-client -v print-files myprinter ~/Documents/portrait-a4.pdf
TI:15:09:58    getting printerd manager
TI:15:09:58    Getting printer /org/freedesktop/printerd/printer/myprinter
TI:15:09:58    Job created: /org/freedesktop/printerd/job/1
TI:15:09:58    Document added
TI:15:09:58    Job started
Job path is /org/freedesktop/printerd/job/1

# printerd -v
TI:15:02:46    Entering main event loop
TI:15:02:46    add device usb://HP/DESKJET%20990C?serial=US05N1J00XLG [...]
TI:15:02:46    Connected to the system bus
TI:15:02:46    Acquired the name org.freedesktop.printerd on the system message bus
TI:15:08:52    Handling GetDevices
TI:15:08:57    Handling GetDevices
TI:15:09:12    Checking authorization of :1.642 for org.freedesktop.printerd.printer-add
TI:15:09:12    Authorized
TI:15:09:12    Creating printer from device HEWLETT_PACKARD_DESKJET_990C_US05N1J00XLG
TI:15:09:12    add printer myprinter
TI:15:09:58    Checking authorization of :1.647 for org.freedesktop.printerd.job-add
TI:15:09:58    Authorized
TI:15:09:58    Creating job for printer myprinter
TI:15:09:58    New job path is /org/freedesktop/printerd/job/1
TI:15:09:58    Created job path is /org/freedesktop/printerd/job/1
TI:15:09:58    [Job 1] Adding document
TI:15:09:58    [Job 1] Got file descriptor: 10
TI:15:09:58    [Job 1] Starting job
TI:15:09:58    [Job 1] Spooling
TI:15:09:58    [Job 1]   Created temporary file /tmp/printerd-spool-AHFWDW
TI:15:09:58    [Job 1]   Set job state to pending
TI:15:09:58    Job 1 changed state: pending
TI:15:09:58    Printer for job 1 idle so starting job
TI:15:09:58    [Job 1] Starting to process job
TI:15:09:58    [Job 1] Using device URI usb://HP/DESKJET%20990C?serial=US05N1J00XLG
TI:15:09:58    [Job 1] Executing /usr/lib/cups/backend/usb
TI:15:09:58    [Job 1]  Env: DEVICE_URI=usb://HP/DESKJET%20990C?serial=US05N1J00XLG
TI:15:09:58    [Job 1]  Arg: usb://HP/DESKJET%20990C?serial=US05N1J00XLG
TI:15:09:58    [Job 1]  Arg: 1
TI:15:09:58    [Job 1]  Arg: :1.647
TI:15:09:58    [Job 1]  Arg: job 1
TI:15:09:58    [Job 1]  Arg: 1
TI:15:09:58    [Job 1]  Arg:
TI:15:09:58    [Job 1] Read 1024 bytes from spool file
TI:15:09:58    Job 1 changed state: processing
TI:15:09:58    [Job 1] Wrote 1024 bytes to backend
TI:15:09:58    [Job 1] Read 1024 bytes from spool file
TI:15:09:58    [Job 1] Wrote 1024 bytes to backend
TI:15:09:58    [Job 1] Read 1024 bytes from spool file
TI:15:09:58    [Job 1] Wrote 1024 bytes to backend
TI:15:09:58    [Job 1] Read 1024 bytes from spool file
TI:15:09:58    [Job 1] Wrote 1024 bytes to backend
TI:15:09:58    [Job 1] Read 1024 bytes from spool file
TI:15:09:58    [Job 1] Wrote 1024 bytes to backend
TI:15:09:58    [Job 1] Read 1024 bytes from spool file
TI:15:09:58    [Job 1] Wrote 1024 bytes to backend
TI:15:09:58    [Job 1] Read 1024 bytes from spool file
TI:15:09:58    [Job 1] Wrote 1024 bytes to backend
TI:15:09:58    [Job 1] Read 1024 bytes from spool file
TI:15:09:58    [Job 1] Wrote 1024 bytes to backend
TI:15:09:58    [Job 1] Read 564 bytes from spool file
TI:15:09:58    [Job 1] Wrote 564 bytes to backend
TI:15:09:58    [Job 1] Spool finished
TI:15:09:58    [Job 1] backend: STATE: +connecting-to-device
TI:15:09:58    [Job 1] backend: DEBUG: Printer using device file "/dev/usb/lp0"...
TI:15:09:58    [Job 1] backend: STATE: -connecting-to-device
TI:15:09:58    [Job 1] backend: DEBUG: backendRunLoop(print_fd=0, device_fd=3, snmp_fd=-1, addr=(nil), use_bc=1, side_cb=0x7f053adc6c30)
TI:15:09:58    [Job 1] backend: DEBUG: Read 8192 bytes of print data...

Where I can see it can make some sense to have printing entirely in the user session is for PDF printing to smart services hosted elsewhere: e.g. the office CUPS server, or Google Cloud Print.  Applications produce PDF, so for printing to these types of service there is nothing to do but send the PDF (along with any print options).

A little about Google Cloud Print: this is a service that allows you to register your local printers with their service, and share those printers with other people.  It currently works as part of Chrome: it maintains an XMPP connection to Google so that incoming jobs can be received and passed on to the local CUPS queue.  For printing jobs to it, it is simply an authenticated web service.

The GTK print dialog supports multiple print backends, and currently ‘cups’ is the one we use for printing.  This communicates with the locally-running cupsd.

Printing is more than simply getting capabilities and submitting jobs though: we also need to monitor the status of submitted jobs, and perform actions on those jobs such as pause, resume, and cancel.

Currently job status monitoring is performed in gsd-printer, which displays notifications when a job has completed etc, and job management actions are implemented in the “Printers” panel of the Systems Settings
application.

Adding support for printing to PDF-capable print services directly from the session could be implemented as follows.

A new session service for printing could be created, providing methods for obtaining a list of printers, explicitly adding/removing printers, and with properties for finding out the current state of each printer and whether it is reporting problems.  It could also provide methods for retrieving the list of jobs for each printer, performing actions on those jobs, and have properties for the state of each job.

This service could have plug-ins for dealing with the locally-running cupsd; with CUPS/IPP servers on the network; and with Google Cloud Print.

For the “network IPP” plug-in, it could discover available CUPS (and IPP Everywhere) queues using Avahi, and show in the list of printers only those queues that handle PDF.  Additionally, the print dialog could
allow IPP print servers to be added by hostname (for CUPS) or URI (for network printers which speak IPP and handle PDF).

For the “cloudprint” plug-in, it could interrogate the Google Cloud Print server to determine the list of available queues. (The “Online Accounts/Google” part of System Settings knows how to log in.)

For job status feedback, gsd-printer could instead query the new service. (Or perhaps the service would be implemented in gsd-printer?)

For job management, the printer panel in System Settings could perform actions through the new service.

The solution we’ve been using for a while is portreserve, and it works by having each privileged service (e.g. cupsd, spamd) provide a file in /etc/portreserve saying which ports it uses. At boot portreserve starts before portmap and binds to those ports, to prevent portmap (or anything else) getting them. When each privileged service starts, it calls portrelease first in order to tell portreserve to let go of its ports.

This works well enough from a simplistic point of view, but doesn’t go far enough. There is a race condition when a privileged service starts: between calling portrelease and the service actually binding to the now-freed port, it could potentially be grabbed by some other service. Also portreserve is a once-per-boot thing: if you stop or restart a protected service after boot there is no protection for its ports.

A really race-proof solution appears to now exist in systemd. It provides port-based socket activation, meaning that it can allocate ports that will later be required during the boot, stopping portmap from getting them. When the relevant service starts, systemd hands the socket file descriptor directly to the service, with no race condition. It even retains the port when the service stops.

What’s the problem? Services don’t always want to be activated on demand. In the case of CUPS, there are two ports: TCP 631 (for IPP) and UDP 631 (for CUPS Browsing). The UDP port is simple for listening out for periodic announcements of network shared CUPS queues. When a packet arrives, there is no need to start CUPS — but the port needs to be protected from portmap, or else CUPS Browsing will mysteriously fail from time to time.

Proposal: separate activation from port reservation

My proposal to fix this is for systemd to separate this socket activation feature from the more fundamental one of reserving ports for services. One way of doing this would be to add ListenStreamNoActivate and ListenDatagramNoActivate configuration directives.

Back in 2007 CUPS became an Apple project.  Now the parts that are not relevant on Mac OS are being dropped, with some of the Linux-relevant parts being gathered together in a separate project, cups-filters.

The main part that is being dropped completely is CUPS Browsing. This is currently the primary mechanism for CUPS-to-CUPS printer queue discovery on Linux. It works by having each CUPS server periodically broadcast UDP packets on port 631 announcing its available queues, and listening for broadcasts from other CUPS servers.

This discovery method is being dropped because DNS-SD is preferred upstream. Support for it has been upstream in CUPS for a while, and it is what CUPS uses on Mac OS X, but it uses Apple’s libdns_sd library and not Avahi. I have added support for this in Fedora, and the patch is submitted upstream.

So in CUPS 1.6, automatic CUPS queue discovery will require Avahi to be running on both the server (i.e. the system hosting the CUPS queue) and the clients (i.e. the systems wanting to print to it).

Of course, you will be able to run CUPS without having Avahi running — but you won’t have automatic CUPS queue discovery in that case.  Clients will have to have queues explicitly configured, or else use the BrowsePoll configuration setting to periodically query a particular CUPS server for its queues.

Several other filters will be dropped from CUPS in 1.6, to be picked up by the new cups-filters package. Information about this package is on the OpenPrinting web site and a beta release is now available.

This new package restores the filters that will be dropped in CUPS 1.6 and also adds new filters to support PDF as the baseline document format rather than PostScript.

This has already been reported in Bugzilla, and a test update is available.

There is also an Ask Fedora answer about it.

The Test Day page has instructions on what you need and how to test.

I recently added support in system-config-printer for determining the best driver to use for a particular printer.  This is an expensive operation, largely because of the time it takes to get a list of available drivers from CUPS, and the Python program providing the D-Bus service also provides other services.  I wanted the program to be able to deal with other callers while the CUPS operation was in progress.  Here’s how that was done.

Firstly, I already had a class for asynchronous communication with CUPS.  When using PolicyKit to talk to CUPS, the calls use D-Bus which provides an asynchronous API for method calls anyway.  For calls not provided that way the fallback is to queue requests for a worker thread to deal with.

Asynchronous D-Bus calls are made in Python by supplying “reply_handler” and “error_handler” named options in the method call, like this:

self._cupsconn.getPPDs2 (reply_handler=self._cups_getppds_reply,
                         error_handler=self._cups_error)

That call returns right away, but the getPPDs2 operation continues in the background.  When it is done, my reply_handler function is called.  Alternatively, if there is an error, the error_handler function is called.  One or the other will be called, at which point the operation is finished with.

So far so good from the client side, but the question is how to implement a D-Bus service in an asynchronous manner.  Here is a reminder of what a synchronous D-Bus service implementation looks like.

#!/usr/bin/python
import gobject
import dbus.service
import time

BUS='com.example.Timer'
PATH='/com/example/Timer'
IFACE='com.example.Timer'
START_TIME=time.time ()

class Timer(dbus.service.Object):
    def __init__ (self):
        self.bus = dbus.SessionBus ()
        bus_name = dbus.service.BusName (BUS, bus=self.bus)
        dbus.service.Object.__init__ (self, bus_name, PATH)

    @dbus.service.method(dbus_interface=IFACE,
                         in_signature='i',
                         out_signature='i')
    def Delay (self, seconds):
        print "Sleeping for %ds" % seconds
        time.sleep (seconds)
        return seconds

def heartbeat():
    print "Still alive at", time.time () - START_TIME
    return True

from dbus.glib import DBusGMainLoop
DBusGMainLoop (set_as_default=True)
loop = gobject.MainLoop ()
# Start the heartbeat
handle = gobject.timeout_add_seconds (1, heartbeat)
# Start the D-Bus service
timer = Timer ()
loop.run ()

The Timer class is the D-Bus service.  It has a single method, Delay, which delays for a number of seconds and returns that same number.  The program sets a repeating 1-second timer to print “Still alive”, and the D-Bus calls into the Timer service are handled by the D-Bus main loop.

How does this program behave?  Here is its output.  While it was running I used D-Feet to call Delay(3).

Still alive at 1.02512407303
Still alive at 2.0262401104
Still alive at 3.02633500099
Still alive at 4.02575397491
Sleeping for 3s
Still alive at 7.0756611824
Still alive at 8.02573609352
Still alive at 9.02583909035
Still alive at 10.0259339809
^CTraceback (most recent call last):
  File "/tmp/demo.py", line 36, in <module>
    loop.run ()
KeyboardInterrupt

As you can see, while it was handling the call to the Delay method it could not do anything else.  Here is a new version of the Delay function, this time implemented using asynchronous callbacks.

    @dbus.service.method(dbus_interface=IFACE,
                         in_signature='i',
                         out_signature='i',
                         async_callbacks=('reply_handler',
                                          'error_handler'))
    def Delay (self, seconds, reply_handler, error_handler):
        print "Sleeping for %ds" % seconds
        gobject.timeout_add_seconds (seconds,
                                     lambda: reply_handler (seconds))

You’ll notice that the D-Bus function decorator now contains an async_callbacks keyword.  This keyword declares the method keywords that the function uses for reply and error handlers.  Here, I’ve stuck with the usual “reply_handler” and “error_handler” names, and added those same names to the definition of the Delay function on the next line.

This time, when the D-Bus main loop calls the Delay method it will also provide the reply and error callbacks.  When the Delay method returns, its return value is ignored.  The D-Bus call is only ended by calling one of the callbacks.  In this very simple implementation, I’ve arranged for that to happen by setting a timeout.

How does the program behave now?

Still alive at 1.07246303558
Still alive at 2.07272696495
Still alive at 3.07237696648
Still alive at 4.07276511192
Sleeping for 3s
Still alive at 5.07263112068
Still alive at 6.07277011871
Still alive at 7.07274699211
Still alive at 8.07311701775
Still alive at 9.07332015038
Still alive at 10.0734181404
Still alive at 11.0735199451
^CTraceback (most recent call last):
  File "/tmp/demo-async.py", line 38, in <module>
    loop.run ()
KeyboardInterrupt

Much better.

Fedora packages for F-16 and rawhide have been built.

Fedora 15 has been released for a little while now, including GNOME 3.  One of the great new features in this release of GNOME is the System Settings window.  It is easily accessed from the system menu in the top right corner of the desktop.

This shows a System Settings window containing an overview of all the various tweakable settings for the system, including personal preferences.  They are shown as icons, such as “Keyboard”, “Background”, “Printers” etc, organised into groups: Personal, Hardware, System, and Other.  Clicking on one of them changes the window so it shows the settings relating to that topic.  So if you click on Printers, you get this:

It’s great to have printer configuration in GNOME, and this interface is nice and simple.  There are a couple of things that it needs to learn to do though.

One of them is choosing a good driver when adding a printer.  This is something that is not easy, and system-config-printer has a fair amount of logic to deal with it.  Now that logic is exposed via D-Bus, so the GNOME System Settings program will be able to use it in future.

node "/org/fedoraproject/Config/Printing" {
  interface "org.fedoraproject.Config.Printing" {
    method "GetBestDrivers"      (
      in s device_id
      in s device_make_and_model
      in s device_uri
      out a(ss) drivers
    )
  }
}

Given the IEEE 1284 Device ID of the printer, and/or a string describing the make and model, and the CUPS device URI to be used, this method returns a list of drivers that should work.  They are ordered with the “best” choice first, and with descriptions of how good a fit each driver is for this model (e.g. exact, close, etc).

Two other methods have been exposed using D-Bus.  The first is a method for examining a PPD and deciding whether it has unmet requirements.  Often a PPD will containing information about what commands to run to convert from various different formats into the format required for the printer.  This method parses those and returns a list of executables that are missing.

node "/org/fedoraproject/Config/Printing" {
  interface "org.fedoraproject.Config.Printing" {
    method "MissingExecutables"      (
      in s ppd_filename
      out as missing_executables
    )
  }
}

The final method is for grouping together CUPS device URIs that address the same physical device.  It is often the case that there is a choice of backends to use for a printer, and this method helps to sort out which devices are just different choices for the same printer.

node "/org/fedoraproject/Config/Printing" {
  interface "org.fedoraproject.Config.Printing" {
    method "GroupPhysicalDevices"      (
      in a{sa{ss}} devices
      out aas grouped_devices
    )
  }
}

It takes a dictionary (keyed by device URI) of IPP attribute dictionaries describing devices, and returns a list of physical devices, each one represented by a list of device URIs that refer to it.

These methods will be in system-config-printer 1.3.5.