Firstly, printerd is experimental and is very far from being a functional print spooler. It doesn’t yet run any filters, for instance, and has no backends of its own to transport jobs to devices. So far it is just a test of what a Linux print spooler would look like if it were written today.

There are several problems printerd aims to solve. Some of the solutions come automatically from implementing it as a polkit-enabled D-Bus system service.  I mentioned one of them in the original post about printer: the fact of having an asynchronous client API. All D-Bus services can be used asynchronously thanks to the D-Bus client library. This approach means the print dialog will be able to use printerd without blocking (and without having to start another thread to use it).

Another benefit is the fact that security/authentication can be made much cleaner and more well integrated into a desktop system if polkit is the basis for policy decisions. Although there is some support for adding polkit support to CUPS in the shape of cups-pk-helper, the way this works is to effectively bypass the standard CUPS policy mechanism. In order to completely limit some operation from users, two different policies must be changed: the built-in CUPS policy in cupsd.conf, and the polkit policy for cups-pk-helper.

In contrast, printerd’s only interface is D-Bus and it uses polkit to authenticate operations. If extra security policy is added in future that cannot be expressed using polkit (for instance, printer-specific policy rules like CUPS has), that extra policy layer will be in addition to polkit, not an alternative. Currently, for example, cancelling a job requires that the user is permitted to use the org.freedesktop.printerd.job-cancel action (a polkit check), as well as being the user that submitted the job in the first place (an additional check). Both requirements must be fulfilled.

One more benefit of printerd is the idea of splitting the IPP server out from the local spooler. Currently printerd only spools files and does not provide IPP services but the idea is that when it does, this will be implemented out-of-process by a program that acts as a client to printerd. It will use printerd’s D-Bus interface, just like any other client. That way, users that want to print to local printers but are not interested in sharing those printers on the network don’t even have to run the IPP server. In fact, as printerd is an activatable system service, the spooler itself won’t even be running unless there is something for it to do.

By accepting only one format for printing, some complexity can be removed from the spooler. Generally people print from applications that generate PDF for printing. By using PDF as the required format, the task of selecting pages and arranging several pages onto each side (i.e. number-up) is hopefully made a little easier, as the structure of PDF makes this task more straightforward than PostScript does. Increasingly printers are able to understand PDF natively themselves. This means converting between formats can be kept to a minimum.

Finally, printerd is experimental, so ideas can be tested and developed. For example, I hope to get printerd to improve the latency between cancelling a job and having the printer stop feeding paper, by splitting the backend out of the filter pipeline, killing its input, and telling it to discard its send buffer. Another idea related to separating the filter pipeline from the backend is to begin filtering the next job before the backend finishes clearing out the last of its send buffer. I’m sure there are other areas for improvement that can be played around with in a project like printerd.

It is a polkit-enabled D-Bus system service, written using the GLib object system. Although modelled on concepts from IPP (Internet Printing Protocol), printerd is not in itself an IPP server. Its only interface is D-Bus, although the aim is to be able to implement an IPP server on top of the D-Bus API as a separate process. Having a D-Bus interface means that applications wanting to print automatically get to use printerd asynchronously.

As a design decision, the range of input formats accepted by printerd will be very limited: essentially only PDF. The existing CUPS drivers and backends will be compatible with printerd.

There isn’t much written yet aside from the basic framework and a very simple command line tool.

Feel free to take a look around: http://gitorious.org/printerd

$ pd-client -v print-files myprinter ~/Documents/portrait-a4.pdf
TI:15:09:58    getting printerd manager
TI:15:09:58    Getting printer /org/freedesktop/printerd/printer/myprinter
TI:15:09:58    Job created: /org/freedesktop/printerd/job/1
TI:15:09:58    Document added
TI:15:09:58    Job started
Job path is /org/freedesktop/printerd/job/1

# printerd -v
TI:15:02:46    Entering main event loop
TI:15:02:46    add device usb://HP/DESKJET%20990C?serial=US05N1J00XLG [...]
TI:15:02:46    Connected to the system bus
TI:15:02:46    Acquired the name org.freedesktop.printerd on the system message bus
TI:15:08:52    Handling GetDevices
TI:15:08:57    Handling GetDevices
TI:15:09:12    Checking authorization of :1.642 for org.freedesktop.printerd.printer-add
TI:15:09:12    Authorized
TI:15:09:12    Creating printer from device HEWLETT_PACKARD_DESKJET_990C_US05N1J00XLG
TI:15:09:12    add printer myprinter
TI:15:09:58    Checking authorization of :1.647 for org.freedesktop.printerd.job-add
TI:15:09:58    Authorized
TI:15:09:58    Creating job for printer myprinter
TI:15:09:58    New job path is /org/freedesktop/printerd/job/1
TI:15:09:58    Created job path is /org/freedesktop/printerd/job/1
TI:15:09:58    [Job 1] Adding document
TI:15:09:58    [Job 1] Got file descriptor: 10
TI:15:09:58    [Job 1] Starting job
TI:15:09:58    [Job 1] Spooling
TI:15:09:58    [Job 1]   Created temporary file /tmp/printerd-spool-AHFWDW
TI:15:09:58    [Job 1]   Set job state to pending
TI:15:09:58    Job 1 changed state: pending
TI:15:09:58    Printer for job 1 idle so starting job
TI:15:09:58    [Job 1] Starting to process job
TI:15:09:58    [Job 1] Using device URI usb://HP/DESKJET%20990C?serial=US05N1J00XLG
TI:15:09:58    [Job 1] Executing /usr/lib/cups/backend/usb
TI:15:09:58    [Job 1]  Env: DEVICE_URI=usb://HP/DESKJET%20990C?serial=US05N1J00XLG
TI:15:09:58    [Job 1]  Arg: usb://HP/DESKJET%20990C?serial=US05N1J00XLG
TI:15:09:58    [Job 1]  Arg: 1
TI:15:09:58    [Job 1]  Arg: :1.647
TI:15:09:58    [Job 1]  Arg: job 1
TI:15:09:58    [Job 1]  Arg: 1
TI:15:09:58    [Job 1]  Arg:
TI:15:09:58    [Job 1] Read 1024 bytes from spool file
TI:15:09:58    Job 1 changed state: processing
TI:15:09:58    [Job 1] Wrote 1024 bytes to backend
TI:15:09:58    [Job 1] Read 1024 bytes from spool file
TI:15:09:58    [Job 1] Wrote 1024 bytes to backend
TI:15:09:58    [Job 1] Read 1024 bytes from spool file
TI:15:09:58    [Job 1] Wrote 1024 bytes to backend
TI:15:09:58    [Job 1] Read 1024 bytes from spool file
TI:15:09:58    [Job 1] Wrote 1024 bytes to backend
TI:15:09:58    [Job 1] Read 1024 bytes from spool file
TI:15:09:58    [Job 1] Wrote 1024 bytes to backend
TI:15:09:58    [Job 1] Read 1024 bytes from spool file
TI:15:09:58    [Job 1] Wrote 1024 bytes to backend
TI:15:09:58    [Job 1] Read 1024 bytes from spool file
TI:15:09:58    [Job 1] Wrote 1024 bytes to backend
TI:15:09:58    [Job 1] Read 1024 bytes from spool file
TI:15:09:58    [Job 1] Wrote 1024 bytes to backend
TI:15:09:58    [Job 1] Read 564 bytes from spool file
TI:15:09:58    [Job 1] Wrote 564 bytes to backend
TI:15:09:58    [Job 1] Spool finished
TI:15:09:58    [Job 1] backend: STATE: +connecting-to-device
TI:15:09:58    [Job 1] backend: DEBUG: Printer using device file "/dev/usb/lp0"...
TI:15:09:58    [Job 1] backend: STATE: -connecting-to-device
TI:15:09:58    [Job 1] backend: DEBUG: backendRunLoop(print_fd=0, device_fd=3, snmp_fd=-1, addr=(nil), use_bc=1, side_cb=0x7f053adc6c30)
TI:15:09:58    [Job 1] backend: DEBUG: Read 8192 bytes of print data...

Where I can see it can make some sense to have printing entirely in the user session is for PDF printing to smart services hosted elsewhere: e.g. the office CUPS server, or Google Cloud Print.  Applications produce PDF, so for printing to these types of service there is nothing to do but send the PDF (along with any print options).

A little about Google Cloud Print: this is a service that allows you to register your local printers with their service, and share those printers with other people.  It currently works as part of Chrome: it maintains an XMPP connection to Google so that incoming jobs can be received and passed on to the local CUPS queue.  For printing jobs to it, it is simply an authenticated web service.

The GTK print dialog supports multiple print backends, and currently ‘cups’ is the one we use for printing.  This communicates with the locally-running cupsd.

Printing is more than simply getting capabilities and submitting jobs though: we also need to monitor the status of submitted jobs, and perform actions on those jobs such as pause, resume, and cancel.

Currently job status monitoring is performed in gsd-printer, which displays notifications when a job has completed etc, and job management actions are implemented in the “Printers” panel of the Systems Settings
application.

Adding support for printing to PDF-capable print services directly from the session could be implemented as follows.

A new session service for printing could be created, providing methods for obtaining a list of printers, explicitly adding/removing printers, and with properties for finding out the current state of each printer and whether it is reporting problems.  It could also provide methods for retrieving the list of jobs for each printer, performing actions on those jobs, and have properties for the state of each job.

This service could have plug-ins for dealing with the locally-running cupsd; with CUPS/IPP servers on the network; and with Google Cloud Print.

For the “network IPP” plug-in, it could discover available CUPS (and IPP Everywhere) queues using Avahi, and show in the list of printers only those queues that handle PDF.  Additionally, the print dialog could
allow IPP print servers to be added by hostname (for CUPS) or URI (for network printers which speak IPP and handle PDF).

For the “cloudprint” plug-in, it could interrogate the Google Cloud Print server to determine the list of available queues. (The “Online Accounts/Google” part of System Settings knows how to log in.)

For job status feedback, gsd-printer could instead query the new service. (Or perhaps the service would be implemented in gsd-printer?)

For job management, the printer panel in System Settings could perform actions through the new service.

Back in 2007 CUPS became an Apple project.  Now the parts that are not relevant on Mac OS are being dropped, with some of the Linux-relevant parts being gathered together in a separate project, cups-filters.

The main part that is being dropped completely is CUPS Browsing. This is currently the primary mechanism for CUPS-to-CUPS printer queue discovery on Linux. It works by having each CUPS server periodically broadcast UDP packets on port 631 announcing its available queues, and listening for broadcasts from other CUPS servers.

This discovery method is being dropped because DNS-SD is preferred upstream. Support for it has been upstream in CUPS for a while, and it is what CUPS uses on Mac OS X, but it uses Apple’s libdns_sd library and not Avahi. I have added support for this in Fedora, and the patch is submitted upstream.

So in CUPS 1.6, automatic CUPS queue discovery will require Avahi to be running on both the server (i.e. the system hosting the CUPS queue) and the clients (i.e. the systems wanting to print to it).

Of course, you will be able to run CUPS without having Avahi running — but you won’t have automatic CUPS queue discovery in that case.  Clients will have to have queues explicitly configured, or else use the BrowsePoll configuration setting to periodically query a particular CUPS server for its queues.

Several other filters will be dropped from CUPS in 1.6, to be picked up by the new cups-filters package. Information about this package is on the OpenPrinting web site and a beta release is now available.

This new package restores the filters that will be dropped in CUPS 1.6 and also adds new filters to support PDF as the baseline document format rather than PostScript.

The Test Day page has instructions on what you need and how to test.

Fedora 15 has been released for a little while now, including GNOME 3.  One of the great new features in this release of GNOME is the System Settings window.  It is easily accessed from the system menu in the top right corner of the desktop.

This shows a System Settings window containing an overview of all the various tweakable settings for the system, including personal preferences.  They are shown as icons, such as “Keyboard”, “Background”, “Printers” etc, organised into groups: Personal, Hardware, System, and Other.  Clicking on one of them changes the window so it shows the settings relating to that topic.  So if you click on Printers, you get this:

It’s great to have printer configuration in GNOME, and this interface is nice and simple.  There are a couple of things that it needs to learn to do though.

One of them is choosing a good driver when adding a printer.  This is something that is not easy, and system-config-printer has a fair amount of logic to deal with it.  Now that logic is exposed via D-Bus, so the GNOME System Settings program will be able to use it in future.

node "/org/fedoraproject/Config/Printing" {
  interface "org.fedoraproject.Config.Printing" {
    method "GetBestDrivers"      (
      in s device_id
      in s device_make_and_model
      in s device_uri
      out a(ss) drivers
    )
  }
}

Given the IEEE 1284 Device ID of the printer, and/or a string describing the make and model, and the CUPS device URI to be used, this method returns a list of drivers that should work.  They are ordered with the “best” choice first, and with descriptions of how good a fit each driver is for this model (e.g. exact, close, etc).

Two other methods have been exposed using D-Bus.  The first is a method for examining a PPD and deciding whether it has unmet requirements.  Often a PPD will containing information about what commands to run to convert from various different formats into the format required for the printer.  This method parses those and returns a list of executables that are missing.

node "/org/fedoraproject/Config/Printing" {
  interface "org.fedoraproject.Config.Printing" {
    method "MissingExecutables"      (
      in s ppd_filename
      out as missing_executables
    )
  }
}

The final method is for grouping together CUPS device URIs that address the same physical device.  It is often the case that there is a choice of backends to use for a printer, and this method helps to sort out which devices are just different choices for the same printer.

node "/org/fedoraproject/Config/Printing" {
  interface "org.fedoraproject.Config.Printing" {
    method "GroupPhysicalDevices"      (
      in a{sa{ss}} devices
      out aas grouped_devices
    )
  }
}

It takes a dictionary (keyed by device URI) of IPP attribute dictionaries describing devices, and returns a list of physical devices, each one represented by a list of device URIs that refer to it.

These methods will be in system-config-printer 1.3.5.

For those unfamiliar with printing, this sounds easy enough. To make the user interface as easy to use as that takes more work than you would think at first glance. It’s the direction I’ve been moving towards in system-config-printer.

Here is a description of the issues involved.

1. Once you’ve found a printer, you then have to choose a driver for it.
2. Once you have a driver, it may need adjusting so that the installable
   options are correct.

Choosing a driver

The thing to know about choosing a driver is that it is not well-defined. CUPS takes care not to take sides or pick favourites, and leaves it to the client application to decide which driver to use and indeed which drivers are likely to work for the printer.

There is no way to be certain you have a complete list of drivers that will work for a given printer. The most reliable way of matching up printers and drivers is to have each PPD file list all of the IEEE 1284 Device IDs for devices that it supports. This practice is not wide-spread, although there has been progress towards this in Fedora.

In response to a CUPS-Get-Devices IPP request, CUPS will have given you attributes describing the device: device-id, device-make-and-model, device-info and so on. Likewise, each driver has attributes in the response to CUPS-Get-PPDs describing itself, including ppd-make-and-model. However, device-make-and-model is not quite the same type of thing as ppd-make-and-model.

The device-make-and-model describes the make and model of the device. The ppd-make-and-model describes the make and model of the printer that the driver is written for, as well as the driver itself.

There is no canonical list of printer manufacturer names, and printer manufacturers routinely write their names differently, e.g. HP, Hewlett-Packard, Hewlett Packard, HEWLETT PACKARD, etc.

There are very often discrepancies between how the device describes its model name and how PPDs describe it, especially when it comes to drivers that support a range of models.

Since Fedora 13 we’ve had special RPM tags on driver packages which can be used to give the user the opportunity to install any packages containing drivers that are known to support this device, once we know the device-id. They are tagged with the make and model of the device as reported in its Device ID.

There isn’t really an easy way to pick the right driver. What system-config-printer does is filter the available drivers using a variety of methods to find a list of the drivers that might support the printer, and then sort them into a preference order based on some XML-coded rules.  Ways of finding candidate drivers include:

• Matching MFG and MDL fields of the Device ID (but not all PPDs contain Device IDs)
• Converting the ppd- and device-make-and-model strings at runtime into semi-canonical representations in hopes of getting a match that way (but this will not always be successful)
• Looking for what look like model numbers and making this successively generic, e.g. 2215… 2210… 2200… 2000 (but the driver stands less chance of working the more generic the match)
• Matching CMD fields of the Device ID (but this is only good for finding “generic” drivers, e.g. PCL)

After this, some PPDs are eliminated: those that seem to match but which depend on an optional PostScript module which is not present in this device. If the Device IDs match in the MFG and MDL fields, but the PPD’s CMD field has POSTSCRIPT and the device’s CMD field does not, the PPD is likely no good. (This is a recent convention but has been adopted by Ricoh.)

The reason for sorting PPDs into a preference order is that there are lots of situations where one driver is most likely better than another.  The situations are numerous but include:

• If the manufacturer has supplied a driver, use that.
• Otherwise, if we can send PostScript straight to the printer, we should probably do that rather than sending raster data.
• …except if the printer does not come with a large amount of memory, in which case raster drivers may be more reliable.
• Gutenprint is usually a high quality driver to use, so use that when possible.
• …except if we’re printing to a colour laserjet, because Gutenrprint only provides black and white output at the moment.
• Native CUPS drivers are generally preferred to the older-style Foomatic drivers.

…and so on.  I wrote about the mechanism used in system-config-printer a while ago.  The current rules are here.

Note that there may not be any driver you can choose that will work. There is no “fail-safe” driver that will work for all printers — not even sending them ASCII text or some sort of bitmap.

Setting installable options

Now, assuming you have chosen the right driver it may need some tweaking once it has been assigned. This is because printers have optional parts, e.g. extra memory, duplexer units, binders, staplers etc.

CUPS provides a way to ask the printer to tell CUPS what those settings are: it’s a special print job of type CUPS-Command that says “AutoConfigure”. There are attributes that will tell you whether CUPS-Command jobs are accepted, and if so whether the AutoConfigure command job is accepted. If this is the case, sending this small job should complete the set-up of the PPD.

Most (if not all) non-PostScript printer drivers do not support the CUPS-Command format, however. If there are installable options in this case (which you can detect by seeing if there is an InstallableOptions group in the PPD), the only way to avoid asking the user about them is to hope the defaults are safe to use as they are.

You can help put this right, whether you are running Fedora 13 Alpha, or Fedora 11/12.

Information on collecting Device IDs is given on the Printing Test Day page but here is a good way of doing it:

1. Make sure you have the foomatic, gutenprint-cups, and hpijs packages installed
2. Connect and switch on your printer(s) — yes, even network ones

3. git clone http://git.fedoraproject.org/git/system-config-printer.git
   cd system-config-printer
   su -c ./check-device-ids.py

4. If you see “MISSING” in the output, file a bug in Bugzilla against the relevant driver

Incidentally, some statistics on Device IDs in Fedora 13:

• We have at least 10,100 drivers
• Of those, over 6,200 have Device IDs associated with them
• That means we’re missing Device IDs for nearly 40% of all our printer drivers

$ lpinfo -l -m | grep 'device-id = ' | wc -l
10135
$ lpinfo -l -m | grep 'device-id = $' | wc -l
3922

UPDATED: Changed instructions to run check-device-ids.py using su.

One of the new features in Fedora 13 is the automatic installation of printer drivers when USB printers are connected.  For this feature to work, specific information about each printer model needs to be stored in the relevant driver packages.  Some of this is done but we need your help to test it and to collect more Device ID strings.

Please come along and join in!