hpoj reference: Device protocols

This document briefly summarizes the various device protocols used by the hpoj software and the peripherals that it supports, and where possible gives pointers to additional information and/or official specifications. Some of the standards mentioned below have additional features not discussed here which are not used by the hpoj software or the devices it supports.

IEEE standards documents may be ordered in hard-copy form from the IEEE Standards Association.

Certain HP standards and protocols described below are available from the HP Developer Solutions website. Others are "available on request" from the hpoj project maintainer for use in conjunction with the hpoj project. In both cases, the documents may not be redistributed verbatim (for example, posted on a web or FTP site), but it is permissible to use them to develop Free/open-source software, such as the hpoj project.

Parallel: IEEE 1284

IEEE 1284 specifies a "Standard Signaling Method for a Bidirectional Parallel Peripheral Interface for Personal Computers".

It specifies the following physical signal lines:

Depending on the mode, data may be transferred in one direction: It specifies the following basic modes (among others): It specifies the following signalling sequences (among others): Depending on the particular mode in use, the signal lines may have different names in the standard than the compatibility-mode names listed above.

At the physical level, IEEE 1284 defines:

Parallel: Microsoft ECP register interface

The Microsoft Extended Capabilities Port Protocol and ISA Interface Standard defines the register set used in PC parallel ports, specifically ones with ECP capabilities. This register set has its roots in that of the unidirectional parallel ports in the original IBM PC; it therefore carries forward some rather confusing inverted register bits relative to the value on the wire, which is hidden deep within ptal-mlcd's ParPort class. This standard provides for hardware-assisted compatibility-mode and ECP-mode transfers, which ptal-mlcd now uses to speed up ECP transfers in most cases.

Parallel: IEEE 1284.3

IEEE 1284.3 specifies "Standard Interface and Protocol Extensions to IEEE Std 1284 Compliant Peripherals and Host Adapters".

Among other things, it introduces a "bounded ECP" mode with added restrictions over regular IEEE 1284 ECP mode, to address design problems with the Microsoft ECP register interface specification.

Parallel: meta-modes

HP peripherals typically support communication in some or all of the following basic combinations of IEEE 1284 modes: The following kinds of data may be transferred over the above modes:

USB: Universal Serial Bus

USB (Universal Serial Bus) provides an easier and more generalized way to connect a wider variety of peripherals to your computer, compared to traditional serial and parallel ports. By using hubs in a star-like configuration, you can theoretically connect up to 127 devices to a single bus.

The hardware, signalling, and protocol specification may be downloaded from the USB Implementor's Forum. Since the protocol is largely implemented in hardware and in the kernel, we won't go into much detail here. Suffice it to say that devices may advertise various kinds of interfaces, some of which may be used concurrently and others of which are "alternate settings" of the same interface. Interface descriptors advertise a "Class/Subclass/Protocol" code, such as "7/1/3". There are also "endpoints" of various types (control and bulk for our purposes, and also sometimes interrupt and isochronous), which are basically the channels over which data is transferred. Endpoints may be "OUT" for forward data or "IN" for reverse data.

USB printer class

For our purposes we are concerned with the USB printer class. It specifies three possible interfaces, which are usually mutually-exclusive alternate settings when a device supports more than one of them: Normally, 7/1/1 and 7/1/2 are for raw print data, and 7/1/3 is for MLC and 1284.4 packets. Some HP peripherals support a vendor-specific command that simulates switching ECP channels as described above, so the result of switching a 7/1/2 interface to channel 77 would be the same as if you had a 7/1/3 interface.

The printer class also supports reading the peripheral's device ID string, just like IEEE 1284 (described above).

Composite USB

Some newer devices have a "composite USB" approach, where certain device functions are available on separate USB interfaces outside of an MLC/1284.4 (see below) transport session. For example, a PSC 2510 advertises the following interfaces (all in hexadecimal): Certain models, such as the ones using the LIDIL print language (see below), do not support printing through an MLC/1284.4 channel, so raw printing over 7/1/2 is the only way to print to them. Therefore, if you don't compile hpoj for libusb support, then you will be able to scan (for example) but not print through hpoj on these models.

HP MLC and IEEE 1284.4

HP's MLC (Multiple Logical Channels) transport protocol and the IEEE 1284.4 protocol based on MLC both allow multiple data channels to be multiplexed over a single bidirectional link, such as ECP channel 77 or a USB 7/1/3 interface or 7/1/2 interface that has been switched to channel 77.

There is a command channel that is open all the time, and other channels may be opened and closed over time using request/reply transactions over the command channel. A channel is defined as a pair of socket IDs, one for each side (referred to as "host"/"peripheral" socket IDs in MLC and "primary"/"secondary" socket IDs in 1284.4). Each data packet is prepended with a 6-byte header, which among other things contains the two socket IDs (so it can be routed to the correct channel) and the packet length (including the header).

Data flow control for a given channel is accomplished where each side issues "credit" to the other side, which is the number of packets (of a previously negotiated maximum size) that are allowed to be transmitted before more credit is given. Most MLC/1284.4 hosts and peripherals are considered "credit gushers", where they automatically supply credit to the other side. However, a few older MLC peripherals are "credit misers," where the host must always request credit whenever it needs some.

MLC and 1284.4 support the following request/reply transactions over the command channel (among others that are generally not used in practice):

In addition to granting or requesting credit over the command channel, there's a "piggyback credit" field in the header that may be used as a lower-overhead way to grant more credit to the other side at the same time when sending a data packet. This feature is normally used only on bidirectional data channels, including the command channel. (However, for MLC the command-channel piggyback credit is implicit, meaning that the piggyback-credit field is set to zero for command packets.)

The HP MLC specification is "available on request" (see above). The IEEE 1284.4 standard document may be ordered in hard-copy form from the IEEE Standards Association. The final IEEE 1284.4 draft specification may also still be downloadable.

PML -- Peripheral Management Language

PML is a request/reply protocol which provides control and status of a device. Each setting or piece of status information in a device is encapsulated in an "object", and all of the objects are grouped into a tree-like hierarchy called a MIB (Management Information Base). Each object has a dotted-decimal "object identifier" or OID, such as "1.2.2.1.12".

PML supports the following operations:

Control objects are typically read/write, and status objects are typically read-only. Depending on the object, writing to some control objects changes a setting in the device, and writing to others causes an action to be taken, such as starting a scan operation.

The PML protocol specification is available from the HP Developer Solutions web site (see above). A partial list of PML objects that may or may not be supported by HP multi-function peripherals is available. In this table, the OID_ "object name" is provided only as a hint of the meaning, and you must enter the numerical object ID on the command line. Some more specific information about the meaning or valid values of certain objects is "available on request" (see above), or you can just look at the source code to ptal-hp and/or libsane-hpoj.

HP JetDirect

HP JetDirect "external" (parallel-port or USB) or LIO print servers use TCP/IP ports to access most stream services that would be accessible as MLC/1284.4 channels in a local-connect case. In the following list, the standard port number is the first number in the set of three, and the other two numbers are for the second and third ports on a multi-port JetDirect, such as the 500X: JetDirect encapsulates PML within SNMP (Simple Network Management Protocol), with one of several prefixes before the PML OID (object ID). SNMP is also needed for retrieving the device ID string from the peripheral.

See the file ptal/ptal-hpjd.c in the hpoj source-code package for more information.

Printing protocols

Most HP printers support some form of PCL, HP's Printer Control Language. PCL can be as simple as some escape sequences added to an otherwise ASCII print job, or it can be as complex as an entire job pre-rendered as a raster image. There are various levels of PCL, such as PCL3, PCL3GUI, PCL5, PCLXL, etc., but of course, not all devices support all levels.

There is also PJL, HP's Printer Job Language, which depending on the level of device support, consists of job control and status commands sent between jobs. Some commands on some devices can even result in data coming back from the device. PJL is not currently used by the hpoj software, other than the UEL (universal escape language) sequence sent by ptal-printd.

More information on PCL and PJL may be found on the HP Developer Solutions web site (see above).

Some HP printers also support Adobe's PostScript language, which is a (mostly) device-independent page description language (PDL). Most Unix applications generate PostScript data when asked to print, which would then be converted into PCL or another PDL for non-PostScript printers. HP's Enhanced Print System can be used in conjunction with a device-specific "PPD" file to access special features of a PostScript printer, such as selecting paper trays, duplex printing, and stapling. PostScript is well-documented in books from Adobe and others.

Some newer low-end HP printers support instead of PCL a "host-based" protocol known as LIDIL (Lightweight Imaging Device Interface Language), which is advertised with LDL in the CMD: (command set) field of the device ID string. Such models do not support printing ASCII text.

Scanning protocols

Scanning protocols for supported HP multi-function peripherals generally fall into two categories: Some models are capable of returning JPEG-compressed grayscale and color scan data, and some models can only return grayscale and color as JPEG. The book JPEG Still Image Data Compression Standard by Pennebaker and Mitchell provides a thorough explanation and even includes the official specification as an appendix.

Most JPEG files you encounter are stored in the JPEG File Interchange Format (JFIF), which is described in a separate document you can find in various formats on many websites (just do a Google search on JFIF). This document assumes familiarity with the base JPEG specification, however.

Storage protocols

HP multi-function peripherals with photo-card reader support use either one of two slightly different versions of a pair of storage protocol modes.

"Disk access" mode, which is currently implemented by ptal-photod, involves reading or writing individual "sectors" on the photo card as if it were a disk drive. It therefore is well-suited for use in conjunction with mtools.

"Streaming save" mode is not currently implemented. It is generally only usable after a "save" operation has been requested from the device's front panel. Entire files (presumably photos) are then read from the device. This protocol does not allow writing or deleting of files or subdirectories.

In addition, some newer models provide a mountable "USB mass-storage" interface, which is discussed above under "Composite USB."