Did you know that you can edit an existing test update?  Doing this keeps the list of fixed bugs, keeps any user comments about the update, and keeps the karma from previous testing.

After clicking Edit, make your changes but be sure to choose Testing or Stable for the Request field.

So, for my own reference as much as anything else, here is how to map an encrypted logical volume back to an actual disk partition.

Take a look at the mounted devices with “mount”.  In my case, I can see that /dev/mapper/vg_worm01-LogVol00 is mounted at the filesystem root /.  This logical volume belongs to a volume group vg_worm01 (it’s right there in the name).

Next, take a look at which physical volumes are collected into this volume group using “pvdisplay”.  For me, that’s just one: /dev/dm-1.

Finally, “cryptsetup status /dev/dm-1″ says which actual device underlies the whole lot: /dev/sda3.

The 3.9.10 release contains a completely re-written print filter, 27 thousand lines of new code, all of it written by HP with no outside review.  There is no public source code repository for HPLIP so 3.9.10 is the first time this code has seen the light of day.

The lpstat -o command shows jobs that are in the CUPS scheduler’s ‘active jobs
list’.  The CUPS web interface also uses this list when displaying the job queue; so does the system-config-printer job viewer.

In CUPS 1.3 and earlier you should never see cancelled jobs in queue.

In CUPS 1.4 the only time you should see cancelled jobs in the queue is when those jobs have been cancelled but one or more filters or backends have not yet finished.

When a job is cancelled, each process in the job pipeline is sent a SIGTERM signal.  If there is no signal handler, this kills the process there and then.

However, the CUPS backends ignore that signal and just wait for the filters to finish sending data.  The reason for this is that some devices may require sort of reset code to be sent in order to be back in their ‘normal’ state.

In contrast to CUPS 1.3, in 1.4 a job is not immediately removed from the active jobs list at the time it is cancelled.  This is in order to implement a job kill delay: after the configurable ‘JobKillDelay’ (default is 300 seconds), any remaining filters or backends associated with cancelled jobs on the active jobs list are sent SIGKILL, and the job is removed from the active jobs list.

CUPS 1.4 is available in Fedora 11.

Before PolicyKit awareness was added there was already adjustable policy controlling IPP operations.  This ‘CUPS policy’ mechanism is provided by the CUPS scheduler, cupsd, and its configuration is in the file /etc/cups/cupsd.conf.  This controls which IP addresses are allowed to access various HTTP locations corresponding to the CUPS configuration actions (such as http://cups-server:631/admin), and which IP address, users, groups, and classes of users can carry out the various IPP operations such as cancelling jobs.  It also specifies whether and how users must authenticate themselves.

This CUPS policy is great for managing CUPS servers remotely but is not consistent with the method used by the rest of the desktop environment, PolicyKit.  Although the CUPS policy can be adjusted using a fairly easy to use list of check-boxes (available in both the CUPS web interface and via System→Administration→Printing), it is not fine-grained enough to be as useful as PolicyKit can be.  The CUPS policy itself is very fine-grained, but this power is only unleashed by editing configuration files.

With PolicyKit the policy can be adjusted easily using a graphical tool, to make it easier to perform certain tasks for those needing to do so.  The intention is for Fedora “spins” to have appropriate default policies depending on their target audience, so that a desktop spin might allow the active console user to perform all administration tasks without asking for authentication for example.

We have two types of policy then, CUPS policy and PolicyKit policy, for controlling print administration tasks.  The CUPS policy in Fedora is unmodified from the upstream CUPS releases, and is fairly restrictive.  The default PolicyKit policy for printing administration in Fedora is also fairly restrictive, and the overall policy is only as restrictive as the most relaxed of the two.

The implementation consists of a package called cups-pk-helper (created by Vincent Untz) which provides the D-Bus system service which is the mechanism for the policy — it performs the administration tasks on behalf of its clients by talking to the CUPS scheduler using IPP.  It runs as root so the default CUPS policy allows it to perform all administration tasks.  It can authenticate itself as root by connecting over a UNIX domain socket (CUPS will use peer credentials to discover the requesting user) or, if that is not possible, by proving that it can read a certificate file only readable by root.

At present the only client for this system service is the printer configuration tool accessible from System→Administration→Printing.  The CUPS web interface and command line tools only use IPP directly.

How fine-grained is the policy?  Here are the defined operations:

• Set a printer as system default printer
• Enable/disable a printer
• Add/remove/edit a local printer
• Add/remove/edit a remote printer
• Add/remove/edit a class
• Get/set server settings
• Restart/cancel/edit a job you own
• Restart/cancel/edit a job owned by another user
• Get the list of available devices (available in Fedora 11 soon)

For each of these operations, the policy depends on who is attempting to carry out the operation:

• Someone on the active console
• Someone on the console but inactive
• Anyone else

For each of these groups of users, the policy can be that they are disallowed altogether, that they are allowed without authentication, or that they require authentication either as themselves or with the root password.  If authentication is required it can be:

• Required every time
• Required once but not needed for the rest of the time the configuration program is running
• Required once per session
• Required once and not again

Here is a screenshot of the program used for editing this policy, accessible from System→Preferences→Authorizations:

The interface is simple:

bus com.redhat.PrinterConfig
object /com/redhat/PrinterConfig
interface com.redhat.PrinterConfig

UsbPrinterAdd (STRING devpath, STRING deviceid)
UsbPrinterRemove (STRING devpath)

Next step: a D-Bus service for finding an appropriate PPD file for a given IEEE 1284 Device ID. This would allow the PrinterConfig implementation to avoid running a Python helper script to actually add the queue. The same goes for bluetooth devices.

As hal will be going away shortly — that, and the fact that hal-cups-utils doesn’t really work very well — I have had a go at re-writing it as a udev rule over the last few days.

The way it works is as follows. We have two ACTION==”add” triggers: one for the usb_device and one for the USB class driver (usblp) device. We need both of them because:

• at least one, and soon two, of the CUPS backends will most likely unload the usblp module if it is loaded, in order to use libusb. This means that the usb_device is the only reliable object we can track to see when printers have been disconnected.
• if the usblp module is already loaded, libusb won’t work because usblp holds open the raw device.
• the usblp module may be blacklisted altogether, as most printer device users use libusb now.

We have a single ACTION==”remove” trigger for the usb_device.

On “add” we obtain the Device ID by fair means or foul, depending on whether the usblp module is loaded. We match this against devices reported by CUPS using the device-id attribute, and keep a note of this mapping of USB devpath to device URIs. Note that if we fail to contact CUPS we just exit — the idea is that the CUPS initscript will replay the udev events after cupsd is started.

Next, we examine the existing CUPS queues to see if any of them use any of the device URIs associated with our device. If so, and they are disabled, and we’d disabled them, we enable them.

If no queues match, we invoke udev-add-printer to create a queue.

On “remove” we look up the list of device URIs associated with our USB devpath, then examine the existing CUPS queues to see if any of them use it. If so they are disabled.

A walk in the park.

(I know which I’d prefer.)

The code is in the udev branch of system-config-printer.

I used the cProfile/pstats modules to do this. It’s quite easy to do. I replace the part where my program says:

gtk.main()

with:

import cProfile
cProfile.run("gtk.main()", "profile.out")

After running the program I used an interactive Python shell to display the profiling statistics, ordered by cumulative time taken in each function:

$ python
>>> import pstats
>>> p=pstats.Stats("profile.out")
>>> p.strip_dirs().sort_stats("cumulativ").print_stats()
Wed Jul 15 17:57:41 2009    profile.out

         1962150 function calls (1959323 primitive calls) in 21.377 CPU seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000   21.377   21.377 <string>:1(<module>)
        1    3.968    3.968   21.377   21.377 {gtk._gtk.main}
        1    0.000    0.000   12.303   12.303 system-config-printer.py:4109(on_btnNPForward_clicked)
        1    0.000    0.000   12.303   12.303 system-config-printer.py:4112(nextNPTab)
        1    0.000    0.000    6.549    6.549 system-config-printer.py:4052(loadPPDs)
        1    0.001    0.001    6.549    6.549 system-config-printer.py:4018(fetchPPDs)
       60    6.515    0.109    6.515    0.109 {time.sleep}
        1    0.000    0.000    4.631    4.631 system-config-printer.py:3360(on_new_printer_activate)
        1    0.000    0.000    4.539    4.539 system-config-printer.py:3720(init)
        1    0.000    0.000    4.342    4.342 ppds.py:475(getPPDNameFromDeviceID)
        1    0.024    0.024    4.279    4.279 system-config-printer.py:4850(fillDeviceTab)
        1    0.000    0.000    4.208    4.208 system-config-printer.py:4538(fetchDevices)
        1    0.000    0.000    4.096    4.096 cupshelpers.py:511(getDevices)
        1    0.000    0.000    4.096    4.096 authconn.py:184()
        1    2.069    2.069    4.096    4.096 authconn.py:186(_authloop)
        4    0.083    0.021    3.985    0.996 ppds.py:816(_init_makes)
    11133    0.753    0.000    3.933    0.000 ppds.py:34(ppdMakeModelSplit)
   202792    0.463    0.000    2.056    0.000 re.py:144(sub)
...

As system-config-printer is an interactive graphical application, a lot of time was spent waiting for me to react to the interface. The functions at the top of the list are there for that reason. However, getPPDNameFromDeviceID is not interactive yet is taking an enormous amount of time: over four seconds! This time does not include the time taken to retrieve the list of available PPDs from CUPS (see loadPPDs), but is entirely spent on processing that data and trying to find a matching driver for the printer I was using.

In a lot of cases it would not take as long as this. In this instance, however, there is no exact match for my printer so it has to perform a lot more processing in order to find a “close” match. Part of that processing involves splitting the (often messy) ppd-make-and-model attributes for the PPDs into manufacturer names and model names.

The reason this was taking so long is apparent from the profiling, in conjunction with reading the code. It is the fact that regular expressions are used extensively to process the ppd-make-and-model attributes. This is confirmed by running p.print_callers(20):

...
Function                  was called by...
                              ncalls  tottime  cumtime
...
re.py:144(sub)            <-  202791    0.463    2.055  ppds.py:34(ppdMakeModelSplit)
...

One way of making this more efficient would be to "compile" the regular expressions once, then execute the compiled expressions on each call. However, by using simple string operations instead of regular expressions in that function, the cumulative time used by ppdMakeModelSplit was reduced from 3.93s to 0.98s.

The lesson: regular expressions are expensive!

The coreutils project has been creating its ChangeLog file from the git commit logs for a little while now, and I decided to take a look at how that’s done. Now, at last, system-config-printer can do that too. Here is how it works.

Step 1 happened a while ago, and that was to modify the style of my git commit log entries. I now write a single line at the top giving a summary of the overall change (as before), so that I get nice output from tools that want to abbreviate the commit log to one line. Then I leave a blank line, and below that my ChangeLog-style entry with no indentation.

Firstly, the gitlog-to-changelog script from gnulib has been added to the repository. This is the build tool Jim Meyering wrote which does all the work.

Next, I renamed the ChangeLog file “ChangeLog-OLD”, because I’d like to keep that history around without any alterations.

I use automake in system-config-printer, but it gets upset if there is no ChangeLog file. To pacify it I modified the “bootstrap” script to create an empty ChangeLog file. I added “/ChangeLog” to the .gitignore file so that I don’t get continually told about this dummy file not being under revision control.

Finally, to generate the ChangeLog file from git commit logs whenever a new release is built I added a “dist-hook” rule to the automake file, getting it to run the gitlog-to-changelog script.

These changes were made on the 1.1.x branch. The older 1.0.x branch still has a normal ChangeLog file in the source repository, and when changes are made on that branch and merged into 1.1.x the ChangeLog entries get merged into ChangeLog-OLD automatically (git is fantastic for merging changes across renamed files).

The actual changes are here.

Prior to 1.3.10 it was based on Xpdf but with some changes to make it work as a CUPS filter. This meant that every PDF security vulnerability had to be fixed separately in both Xpdf and in CUPS’s pdftops program, not to mention all the other PDF implementations (ghostscript, poppler, etc).

The new CUPS PDF filter just runs the pdftops program that comes with poppler, or alternatively ghostscript depending on the CUPS build configuration (in Fedora we use poppler for this).

In future if any new PDF security vulnerabilities are discovered, CUPS in Fedora will not need to be patched for them. Yay!