[Printing-architecture] Removal of features in CUPS

[Printing-architecture] Removal of features in CUPS

[Till Kamppeter]

Hi,

as mentioned in the phone meeting Apple/Mike Sweet is dropping features 
which are important for Linux but not much interesting for Mac OS X. 
This means that others have to overtake maintainership to assure that 
printing under Linux continues to work. Even though CUPS is still the 
best maintained printing environment for Linux/Unix and its core 
continues as free software project supplied and maintained by Apple.

Here are the dropped features:


1. Filters go to the new cups-filters project
---------------------------------------------

All filters and backends not used by Mac OS X are dropped. These 
filters, together with the filters for the PDF printing workflow are now 
hosted as the cups-filters project at OpenPrinting. For a first release 
only bannertopdf needs to get finished (by Lars Uebernickel).


2. The Linux implementation of the USB backend looses attention
---------------------------------------------------------------

USB printers can get accessed with two methods: The usblp kernel module 
and libusb. The former is the classic way. It is simple: Data is sent to 
the printer by writing it to a device file as a simple serial data 
stream. Reading the same device file allows bi-di access, at least for 
things like reading out ink levels, status, or printer capability 
information (PJL). It works very well for simple printers, but for 
multi-function devices (printer/scanner) it is not suitable and 
manufacturers like HP supply their own backends.

Formerly, under Unix everything was a file, but now, with printers being 
more complex, one moves to general USB device access as supplied via 
libusb. HP's CUPS backend as shipped with HPLIP for example is 
libusb-based and detaches printers from the usblp kernel module. On HP's 
multi-function devices one can do things like simultaneous, independent 
printing and scanning.

CUPS added a libusb-based USB backend for Linux to access printers via 
libusb, but it is not possible to compile and install both the 
usblp-kernel-module-based one and the libusb-based one at the same time. 
One has to select one by an option for the "./configure" call. Having 
usblp loaded makes only the usblp-based backend see the printers and not 
having the module loaded makes only the libusb-based backend seeing the 
printers. If the module is loaded and some printers get detached from 
the module (HPLIP's backend detaches the printer which it accesses), 
then the not detached printers are visible only by the usblp-based and 
the detached printers only by the libusb-based backend.

To avoid any confusion I have taken care that in Ubuntu all USB printer 
detection and access is always done by both methods to be sure that 
every printer is accessible. This leads to awkward, complex code 
especially for the Plug'n'Print part (automatic queue creation) of 
system-config-printer and for the USB backend of CUPS I have written a 
patch to use CUPS' code for the two USB backend flavors to get one 
hybrid USB backend working with both the usblp and libusb methods so 
that all printers are acessible independent what the usblp kernel module 
is doing.

I proposed my patch to CUPS upstream but Mike rejected it and told me to 
use libusb-only. So in Oneiric I blacklisted the kernel module and built 
CUPS with the libusb version of the backend, not knowing that this 
backend was not really ready for prime time. Several smaller bughs I got 
fixed and issued patched CUPS versions for Oneiric, but now there are 
still several bug reports, especially also due to the libusb-based 
backend not being capable of doing bi-dim which is required by some 
proprietary manufacturer-supplied drivers even for simple printing. I 
asked Mike whether he could fix that but he told that he has no time and 
patches are welcome.

So I am currently trying to improve the backend, I never programmed USB 
stuff before. First I will switch the backend over to use libusb 1.0.x 
instead of the deprecated 0.1.x which is not maintained any more. After 
that I will try to get bi-di in.

Any help is welcome.


3. CUPS Raster output driver in Ghostscript dropped 5 years ago
---------------------------------------------------------------

The first piece of code where Mike dropped maintainership was the CUPS 
Raster output device of Ghostscript. It got into the Ghostscript 
upstream source code right after my merger of ESP Ghostscript into GPL 
Ghostscript in the middle of 2007. From then on Mike did not do any 
change on the CUPS Raster device any more and it was in a rather bad 
state, crashing a lot. I have worked a lot on it to fix its bugs and to 
stabilize it. Now it is working really well.


4. PPD concept will be dropped in CUPS
--------------------------------------

If all printers would be IPP Everywhere when CUPS 1.6.0 comes out, there 
would be no problem, capability and option info comes just 
out-of-the-printer and drivers are not needed. Unfortunately in real 
life we have a lot of printers from the good old times before IPP 
Everywhere and a lot of cheapo printers with rather dumb 
proprietary-raster-only input which would never be capable of telling 
about its capabilities in a standard way. So PPDs and PPD handling is 
still needed. There will be a lot of problems if one cannot place PPD 
generators in /usr/lib/cups/driver any more and if there is no code in 
CUPS any more which generates PPDs from .drv files /usr/share/cups/drv.


All this someone else has to take care of/maintain now. So a lot of 
extra work, especially for OpenPrinting. Here anyone is welcome to 
volunteer, especially a USB expert to get the USB CUPS backend up to the 
state of the art would be great.

    Till

Re: [Printing-architecture] Removal of features in CUPS

[Tim Waugh]

[-- Attachment #1: Type: text/plain, Size: 523 bytes --]

Hi Till,

I think that CUPS Browsing is also being removed in 1.6, leaving Avahi
as the sanest of the remaining discovery methods.

The current state of the Avahi support patches is that service
announcement is working in the scheduler, and service discovery is
working in the dnssd backend -- i.e. by manual user intervention.

What is currently missing is automatic service discovery in the
scheduler, so that shared queues can be discovered by CUPS itself as
used to happen with CUPS Browsing.

Tim.
*/


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 482 bytes --]

Re: [Printing-architecture] Removal of features in CUPS

[Till Kamppeter]

On 01/12/2012 11:51 AM, Tim Waugh wrote:
> I think that CUPS Browsing is also being removed in 1.6, leaving Avahi
> as the sanest of the remaining discovery methods.
>
> The current state of the Avahi support patches is that service
> announcement is working in the scheduler, and service discovery is
> working in the dnssd backend -- i.e. by manual user intervention.
>
> What is currently missing is automatic service discovery in the
> scheduler, so that shared queues can be discovered by CUPS itself as
> used to happen with CUPS Browsing.

Does libdnssd-based upstream CUPS do automatic service discovery by the 
scheduler, meaning that your Avahi patch is not yet complete in this 
sense or is automatic service discovery by the scheduler even not 
implemented at all upstream, meaning that by removing IPP browsing one 
of the most important features of CUPS gets lost?

    Till

Re: [Printing-architecture] Removal of features in CUPS

[Tim Waugh]

[-- Attachment #1: Type: text/plain, Size: 663 bytes --]

On Thu, 2012-01-12 at 12:09 +0100, Till Kamppeter wrote:
> Does libdnssd-based upstream CUPS do automatic service discovery by the 
> scheduler, meaning that your Avahi patch is not yet complete in this 
> sense or is automatic service discovery by the scheduler even not 
> implemented at all upstream, meaning that by removing IPP browsing one 
> of the most important features of CUPS gets lost?

My Avahi patch was a straight port of the existing features in CUPS to
Avahi.  Upstream CUPS does not currently perform DNS-SD discovery in the
scheduler but -- given that CUPS Browsing is to be removed -- I expect
that it will in 1.6.0.

Tim.
*/


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 482 bytes --]

Re: [Printing-architecture] Removal of features in CUPS

[Michael Sweet]

[-- Attachment #1: Type: text/plain, Size: 2871 bytes --]

[Sorry, I was just made aware of this message thread or I would have responded sooner since much of this information just isn't correct...]

Till Kamppeter wrote:
> ...
> Here are the dropped features:
> 
> 
> 1. Filters go to the new cups-filters project
> ---------------------------------------------
> 
> All filters and backends not used by Mac OS X are dropped. These 
> filters, together with the filters for the PDF printing workflow are now 
> hosted as the cups-filters project at OpenPrinting. For a first release 
> only bannertopdf needs to get finished (by Lars Uebernickel).

This part is correct and thank you Till for hosting these filters!
> 2. The Linux implementation of the USB backend looses attention
> ---------------------------------------------------------------

We continue to develop and support the USB backend, although my personal development time is (unfortunately) quite limited.  I will be integrating Till's libusb 1.0 patch in the near future, so at least the current functionality will be usable with both the old and new libusb.

For bidirectional printing support, we need to adopt the method used for the Darwin/OS X USB backend where there is a separate read thread that just does a bulk-in every 200ms. That provides good latency for drivers that need it while avoiding the performance hit for printers that just return a copy of their 1284 Device ID with status info every time.

Depending on my time constraints and whether libusb 1.0 has the necessary support (I think so, but it has been a while since I looked), we may be able to get bidi support in CUPS 1.6.  The big hurdle was moving to libusb 1.0, and Till has already done that work.

> 4. PPD concept will be dropped in CUPS
> --------------------------------------
This is just incorrect.  PPDs are being *deprecated* in CUPS 1.6, but this just means that we will no longer be extending support for PPDs in CUPS.  The APIs are still there, and PPD-based drivers will continue to be supported.

That said, the long-term replacement *is* IPP Everywhere, and to support that we are adding new APIs in CUPS 1.6 (many still just stubs in the current trunk) to deal with dynamic printer availability and capabilities.  The goal is to have IPP equivalents for what we currently provide in the PPD APIs, and in 1.6 they will use the information the scheduler (cupsd) already generates from and maps to PPD files since CUPS 1.4.

CUPS 1.6 is just a baby step towards PPD independence, but the new APIs will allow applications (and the common print dialog) to make the transition so that when we are able to pull the plug they won't notice a thing.  And in the meantime we will be able to support things like media sensing for new printers on the market...

_________________________________________________________
Michael Sweet, Senior Printing System Engineer, PWG Chair


[-- Attachment #2: Type: text/html, Size: 4870 bytes --]

Re: [Printing-architecture] Removal of features in CUPS

[Michael Sweet]

[-- Attachment #1: Type: text/plain, Size: 1462 bytes --]

Till Kamppeter wrote:
> On 01/12/2012 11:51 AM, Tim Waugh wrote:
> > I think that CUPS Browsing is also being removed in 1.6, leaving Avahi
> > as the sanest of the remaining discovery methods.
> >
> > The current state of the Avahi support patches is that service
> > announcement is working in the scheduler, and service discovery is
> > working in the dnssd backend -- i.e. by manual user intervention.
> >
> > What is currently missing is automatic service discovery in the
> > scheduler, so that shared queues can be discovered by CUPS itself as
> > used to happen with CUPS Browsing.
> 
> Does libdnssd-based upstream CUPS do automatic service discovery by the 
> scheduler, meaning that your Avahi patch is not yet complete in this 
> sense or is automatic service discovery by the scheduler even not 
> implemented at all upstream, meaning that by removing IPP browsing one 
> of the most important features of CUPS gets lost?

No, the scheduler does not maintain a constant browse for shared printers - that is done in libcups, and there are new APIs for dealing with dynamically-available printers in CUPS 1.6.  You can also manually add specific shared printers you use, and the CUPS 1.6 web interface will automagically use the "raw" driver for shared printers.
The only functionality that is lost is the notion of implicit classes.
_________________________________________________________
Michael Sweet, Senior Printing System Engineer, PWG Chair


[-- Attachment #2: Type: text/html, Size: 3197 bytes --]

Re: [Printing-architecture] Removal of features in CUPS

[Till Kamppeter]

On 02/03/2012 05:32 PM, Michael Sweet wrote:
>
> We continue to develop and support the USB backend, although my personal
> development time is (unfortunately) quite limited. I will be integrating
> Till's libusb 1.0 patch in the near future, so at least the current
> functionality will be usable with both the old and new libusb.
>
> For bidirectional printing support, we need to adopt the method used for
> the Darwin/OS X USB backend where there is a separate read thread that
> just does a bulk-in every 200ms. That provides good latency for drivers
> that need it while avoiding the performance hit for printers that just
> return a copy of their 1284 Device ID with status info every time.
>
> Depending on my time constraints and whether libusb 1.0 has the
> necessary support (I think so, but it has been a while since I looked),
> we may be able to get bidi support in CUPS 1.6. The big hurdle was
> moving to libusb 1.0, and Till has already done that work.

As mentioned, I have done the port of the USB backend from libusb 0.1.x 
to libusb 1.0.x (http://www.cups.org/str.php?L3477, scheduled for CUPS 
1.6). It was no problem to do this port. libusb 1.0.x contains all 
needed functions. The new libusb-1.0.x-based backend performs as well as 
the old one not better, not worse, but is now backed by a fully 
upstream-supported library.

Now I am thinking about adding bi-di support to the USB backend. My 
first approach was using the new asynchronous API of libusb 1.0.x, doing 
something similar to the usbmuxd (this is a daemon for Linux to 
communicate with iOS devices on USB), but comparing the complexity of 
usbmuxd compared to the Darwin version of the USB CUPS backend 
(backend/usb-darwin.c) I am tending to do the approach of three threads, 
generated by libpthread: Job transport, back channel and side channel, 
including the reading of the back-channel of the device being done only 
every 200ms and the reading only ebding several seconds after the job 
transport has finished, to assure to get all the printer's answers and 
the printer getting back into its initial state for the next job.

Using the libpthread approach will also make the libusb-based backend 
very similar to the Darwin backend, which should make maintenance of 
CUPS easier.

Mike, what do you think about that?

I am working on this to get this done before Ubuntu's Feature Freeze on 
Feb 16, as in Oneiric (11.10) we have lost the USB bi-di feature by 
deprecating the usblp kernel module, causing bugs with several 
closed-source manufacturer-supplied drivers.

    Till

Re: [Printing-architecture] Removal of features in CUPS

[Michael Sweet]

On Feb 7, 2012, at 10:07 AM, Till Kamppeter wrote:
> ...
> Using the libpthread approach will also make the libusb-based backend very similar to the Darwin backend, which should make maintenance of CUPS easier.
> 
> Mike, what do you think about that?

That is exactly how I would do it.  And thanks again for working on this!

_________________________________________________________
Michael Sweet, Senior Printing System Engineer, PWG Chair

Re: [Printing-architecture] Removal of features in CUPS

[Till Kamppeter]

On 02/08/2012 05:19 PM, Michael Sweet wrote:
> On Feb 7, 2012, at 10:07 AM, Till Kamppeter wrote:
>> ...
>> Using the libpthread approach will also make the libusb-based backend very similar to the Darwin backend, which should make maintenance of CUPS easier.
>>
>> Mike, what do you think about that?
>
> That is exactly how I would do it.  And thanks again for working on this!

I have implemented it this way now. The result is in

http://www.cups.org/str.php?L2890

With this patch applied to CUPS 1.5.2

echo -en '#CUPS-COMMAND\nAutoConfigure\n' | lpr -P <printer>

works now also for USB-connected PostScript printers when using the 
libusb-based USB backend.

    Till

Re: [Printing-architecture] Removal of features in CUPS

[Michael Sweet]

Till,

On Feb 8, 2012, at 1:58 PM, Till Kamppeter wrote:
> ...
> I have implemented it this way now. The result is in
> 
> http://www.cups.org/str.php?L2890
> 
> With this patch applied to CUPS 1.5.2
> 
> echo -en '#CUPS-COMMAND\nAutoConfigure\n' | lpr -P <printer>
> 
> works now also for USB-connected PostScript printers when using the libusb-based USB backend.


Perfect, I'll merge the changes in for 1.5.3 and 1.6 when I get back from this week's PWG meeting.

Thanks!

_________________________________________________________
Michael Sweet, Senior Printing System Engineer, PWG Chair