JPEG Totally Explained

Everything about Jpeg totally explained

In computing, JPEG (pronounced JAY-peg; ) is a commonly used method of compression for photographic images. The degree of compression can be adjusted, allowing a selectable tradeoff between storage size and image quality. JPEG typically achieves 10 to 1 compression with little perceivable loss in image quality.
JPEG is the most common image format used by digital cameras and other photographic image capture devices, and is the most common format for storing and transmitting photographic images on the World Wide Web.
The MIME media type for JPEG is image/jpeg (defined in RFC 1341).

The JPEG standard

The name "JPEG" stands for Joint Photographic Experts Group, the name of the committee that created the standard. The group was organized in 1986, issuing a standard in 1992, which was approved in 1994 as ISO 10918-1. JPEG is distinct from MPEG (Moving Picture Experts Group), which produces compression schemes for video.
The JPEG standard specifies both the codec, which defines how an image is compressed into a stream of bytes and decompressed back into an image, and the file format used to contain that stream.

Recommended usage

The JPEG compression algorithm is at its best on photographs and paintings of realistic scenes with smooth variations of tone and color. For web usage in particular, where the bandwidth used by an image is important, JPEG is the ideal photographic image format.
On the other hand, JPEG is not as well suited for line drawings and other textual or iconic graphics, where the sharp contrasts between adjacent pixels cause noticeable artifacts. Such images are better saved in TIFF format (for local usage) or in GIF or PNG format (for web usage).
JPEG is also not well suited to files that will undergo multiple edits, as some image quality will usually be lost each time the image is decompressed and recompressed. It is preferable to use a non-lossy format such as TIFF while working on an image, with the final image saved as JPEG after all editing is complete.

JPEG compression

The compression method is usually lossy compression, meaning that some visual quality is lost in the process and can't be restored. There are variations on the standard baseline JPEG that are lossless, however these are not yet widely supported.
There is also an interlaced "Progressive JPEG" format, in which data is compressed in multiple passes of progressively higher detail. This is ideal for large images that will be displayed while downloading over a slow connection, allowing a reasonable preview after receiving only a portion of the data. However, progressive JPEGs are not as widely supported.
There are also many medical imaging systems that create and process 12-bit JPEG images. The 12-bit JPEG format has been part of the JPEG specification for some time, but again, this format isn't as widely supported.

Lossless editing

A number of alterations to a JPEG image can be performed losslessly (that is, without recompression and the associated quality loss) as long as the image size is a multiple 1 MCU block (Minimum Coded Unit) (usually 16 pixels in both directions, for 4:2:0).
   Blocks can be rotated in 90 degree increments, flipped in the horizontal, vertical and diagonal axes and moved about in the image. Not all blocks from the original image need to be used in the modified one.
   The top and left of a JPEG image must lie on a block boundary, but the bottom and right need not do so. This limits the possible lossless crop operations, and also what flips and rotates can be performed on an image whose edges don't lie on a block boundary for all channels.
   When using lossless cropping, if the bottom or right side of the crop region isn't on a block boundary then the rest of the data from the partially used blocks will still be present in the cropped file and can be recovered relatively easily by anyone with a hex editor and an understanding of the format.
   It is also possible to transform between baseline and progressive formats without any loss of quality, since the only difference is the order in which the coefficients are placed in the file.

JPEG files

The file format is known as 'JPEG Interchange Format' (JIF), as specified in Annex B of the standard. However, this "pure" file format is rarely used, primarily because of the difficulty of programming encoders and decoders that fully implement all aspects of the standard and because of certain shortcomings of the standard:

Color Space definition
Component Sub-Sampling Registration definition
Pixel Aspect Ratio definition

A couple of additional standards have evolved to address these issues. The first of these, released in 1992, was JPEG File Interchange Format (or JFIF), followed in recent years by Exchangeable image file format (Exif) and ICC color profiles.
There is some confusion between the original 'JPEG Interchange Format' (JIF) and the similarly titled 'JPEG File Interchange Format' (JFIF). In some ways JFIF is a cutdown version of the JIF standard in that it specifies certain constraints (such as standard color space), while in other ways it's an extension of JIF due to the standard Application Segment header. The documentation for the original JFIF standard states: ť JPEG File Interchange Format is a minimal file format which enables JPEG bitstreams to be exchanged between a wide variety of platforms and applications. This minimal format doesn't include any of the advanced features found in the TIFF JPEG specification or any application specific file format. Nor should it, for the only purpose of this simplified format is to allow the exchange of JPEG compressed images.

Image files that employ JPEG compression are commonly called "JPEG files". Most image capture devices (such as digital cameras) and most image editing software programs that write to a "JPEG file" are actually creating a file in the JFIF and/or Exif format.
Strictly speaking, the JFIF and Exif standards are incompatible because they each specify that their header appears first. In practice, most JPEG files in Exif format contain a small JFIF header that precedes the Exif header. This allows older readers to correctly handle the older format JFIF header, while newer readers also decode the following Exif header.

JPEG file extensions

The most common filename extensions for files employing JPEG compression are .jpg and .jpeg, though .jpe, .jfif and .jif are also used. It is also possible for JPEG data to be embedded in other file types - TIFF encoded files often embed a JPEG image as a thumbnail of the main image.

Color profile

Many JPEG files embed an ICC color profile (color space). Commonly used color profiles include sRGB and Adobe RGB. Because these color spaces use a non-linear transformation, the dynamic range of an 8-bit JPEG file is about 11 stops.
However, a large number of applications are not able to deal with JPEG color profiles and simply ignore them. (eg: The GIMP and all web browsers, excluding Apple Safari).

Syntax and structure

A JPEG image contains a sequence of markers, each of which begins with a 0xFF byte followed by a byte indicating what kind of marker it is. Some markers consist of just those two bytes; others are followed by two bytes indicating the length of marker-specific payload data that follows. (The length includes the two bytes for the length, but not the two bytes for the marker.) Some markers are followed by entropy-coded data; the length of such a marker doesn't include the entropy-coded data.
Within the entropy-coded data, after any 0xFF byte, a 0x00 byte is inserted by the encoder before the next byte, so that there doesn't appear to be a marker where none is intended. Decoders must skip this 0x00 byte. This technique, called byte stuffing, is only applied to the entropy-coded data, not to marker payload data.

Short name	Bytes	Payload	Name	Comments
SOI	0xFFD8	none	Start Of Image
SOF0	0xFFC0	variable size	Start Of Frame (Baseline DCT)	Indicates that this is a baseline DCT-based JPEG, and specifies the width, height, number of components, and component subsampling (for example, 4:2:0).
SOF2	0xFFC2	variable size	Start Of Frame (Progressive DCT)	Indicates that this is a progressive DCT-based JPEG, and specifies the width, height, number of components, and component subsampling (for example, 4:2:0).
DHT	0xFFC4	variable size	Define Huffman Table(s)	Specifies one or more huffman tables.
DQT	0xFFDB	variable size	Define Quantization Table(s)	Specifies one or more quantization tables.
DRI	0xFFDD	2 bytes	Define Restart Interval	Specifies the interval between RSTn markers, in macroblocks.
SOS	0xFFDA	variable size	Start Of Scan	Begins a top-to-bottom scan of the image. In baseline DCT JPEG images, there's generally a single scan. Progressive DCT JPEG images usually contain multiple scans. This marker specifies which slice of data it'll contain, and is immediately followed by entropy-coded data.
RSTn	0xFFDn	variable size	Restart	Inserted every r macroblocks, where r is the restart interval set by a DRI marker. Not used if there was no DRI marker. n, the low 4 bits of the marker code, cycles from 0 to 7.
APPn	0xFFEn	variable size	Application-specific	For example, an Exif JPEG file uses an APP1 marker to store metadata, laid out in a structure based closely on TIFF.
COM	0xFFFE	variable size	Comment	Contains a text comment.
EOI	0xFFD9	none	End Of Image

There are other Start Of Frame markers that introduce other kinds of JPEG.
Since several vendors might use the same APPn marker type, application-specific markers often begin with a standard or vendor name (for example, "Exif" or "Adobe") or some other identifying string.
At a restart marker, block-to-block predictor variables are reset, and the bitstream is synchronized to a byte boundary. Restart markers provide means for recovery after bitstream error. Since the runs of macroblocks between restart markers may be independently decoded, these runs may be decoded in parallel.

JPEG codec example

Although a JPEG file can be encoded in various ways, most commonly it's done with JFIF encoding. The encoding process consists of several steps:

The representation of the colors in the image is converted from RGB to YCbCr, consisting of one luma component (Y), representing brightness, and two chroma components, (Cb and Cr), representing color. This step is sometimes skipped.

The resolution of the chroma data is reduced, usually by a factor 2. This reflects the fact that the eye is less sensitive to fine color details than to fine brightness details.

The image is split into blocks of 8×8 pixels, and for each block, each of the Y, Cb, and Cr data undergoes a discrete cosine transform (DCT). A DCT is similar to a Fourier transform in the sense that it produces a kind of spatial frequency spectrum.

The amplitudes of the frequency components are quantized. Human vision is much more sensitive to small variations in color or brightness over large areas than to the strength of high-frequency brightness variations. Therefore, the magnitudes of the high-frequency components are stored with a lower accuracy than the low-frequency components. The quality setting of the encoder (for example 50% or 95%) affects to what extent the resolution of each frequency component is reduced. If an excessively low quality setting is used, the high-frequency components are discarded altogether.

The resulting data for all 8×8 blocks is further compressed with a loss-less algorithm, a variant of Huffman encoding. The decoding process reverses these steps. In the remainder of this section, the encoding and decoding processes are described in more detail.

Encoding

Many of the options in the JPEG standard are not commonly used, and as mentioned above, most image software uses the simpler JFIF format when creating a JPEG file, which among other things specifies the encoding method. Here is a brief description of one of the more common methods of encoding when applied to an input that has 24 bits per pixel (eight each of red, green, and blue). This particular option is a lossy data compression method.

Color space transformation

First, the image should be converted from RGB into a different color space called YCbCr. It has three components Y, Cb and Cr: the Y component represents the brightness of a pixel, the Cb and Cr components represent the chrominance (split into blue and red components). This is the same color space as used by digital color television as well as digital video including video DVDs, and is similar to the way color is represented in analog PAL video and MAC but not by analog NTSC, which uses the YIQ color space. The YCbCr color space conversion allows greater compression without a significant effect on perceptual image quality (or greater perceptual image quality for the same compression). The compression is more efficient as the brightness information, which is more important to the eventual perceptual quality of the image, is confined to a single channel, more closely representing the human visual system.
This conversion to YCbCr is specified in the JFIF standard, and should be performed for the resulting JPEG file to have maximum compatibility. However, some JPEG implementations in "highest quality" mode don't apply this step and instead keep the color information in the RGB color model, where the image is stored in separate channels for red, green and blue luminance. This results in less efficient compression, and wouldn't likely be used if file size were an issue.

Downsampling

Due to the densities of color- and brightness-sensitive receptors in the human eye, humans can see considerably more fine detail in the brightness of an image (the Y component) than in the color of an image (the Cb and Cr components). Using this knowledge, encoders can be designed to compress images more efficiently.
The transformation into the YCbCr color model enables the next step, which is to reduce the spatial resolution of the Cb and Cr components (called "downsampling" or "chroma subsampling"). The ratios at which the downsampling can be done on JPEG are (no downsampling), (reduce by factor of 2 in horizontal direction), and most commonly (reduce by factor of 2 in horizontal and vertical directions). For the rest of the compression process, Y, Cb and Cr are processed separately and in a very similar manner. Downsampling the chroma components saves 33% or 50% of the space taken by the image without drastically affecting perceptual image quality.

Block splitting

After subsampling, each channel must be split into 8×8 blocks (of pixels). If the data for a channel doesn't represent an integer number of blocks then the encoder must fill the remaining area of the incomplete blocks with some form of dummy data:

filling the edge pixels with a fixed color (typically black) creates dark artifacts along the visible part of the border

repeating the edge pixels is a common but non-optimal technique that avoids the visible border, but it still creates artifacts with the colorimetry of the filled cells

a better strategy is to fill pixels using colors that preserve the DCT coefficients of the visible pixels, at least for the low frequency ones (for example filling with the average color of the visible part will preserve the first DC coefficient, but best fitting the next two AC coefficients will produce much better results with less visible 8×8 cell edges along the border).

Discrete cosine transform

Next, each component (Y, Cb, Cr) of each 8×8 block is converted to a frequency-domain representation, using a normalized, two-dimensional type-II discrete cosine transform (DCT).
As an example, one such 8×8 8-bit subimage might be:
ť

egin
The mid-quality photo uses only one sixth the storage space but has little noticeable loss of detail or visible artifacts. However, once a certain threshold of compression is passed, compressed images show increasingly visible defects. See the article on rate distortion theory for a mathematical explanation of this threshold effect.

Potential patent issues

In 2002 Forgent Networks asserted that it owned and would enforce patent rights on the JPEG technology, arising from a patent that had been filed on October 27, 1986, and granted on October 6, 1987 . The announcement created a furor reminiscent of Unisys' attempts to assert its rights over the GIF image compression standard.
   The JPEG committee investigated the patent claims in 2002 and were of the opinion that they were invalidated by prior art. Others also concluded that Forgent didn't have a patent that covered JPEG. Nevertheless, between 2002 and 2004 Forgent was able to obtain about US$105 million by licensing their patent to some 30 companies. In April 2004, Forgent sued 31 other companies to enforce further license payments. In July of the same year, a consortium of 21 large computer companies filed a countersuit, with the goal of invalidating the patent. In contrast to other major computer companies such as Sony and Philips, Microsoft launched a major lawsuit against Forgent. In February 2006, the United States Patent and Trademark Office agreed to re-examine Forgent's JPEG patent at the request of the Public Patent Foundation. On May 26, 2006 the USPTO found the patent invalid based on prior art. The USPTO also found that Forgent knew about the prior art, and didn't tell the Patent Office, making any appeal to reinstate the patent highly unlikely to succeed.
   Forgent also possesses a similar patent granted by the European Patent Office in 1994, though it's unclear how enforceable it is.
   As of October 27, 2006, the U.S. patent's 20-year term appears to have expired, and in November 2006, Forgent agreed to abandon enforcement of patent claims against use of the JPEG standard.
   The JPEG committee has as one of its explicit goals that their standards (in particular their baseline methods) be implementable without payment of license fees, and they've secured appropriate license rights for their upcoming JPEG 2000 standard from over 20 large organizations.
   Beginning in August 2007, another company, Global Patent Holdings, LLC claimed that its patent, is infringed by the downloading of JPEG images on either a website or through e-mail. If not invalidated, this patent could apply to any website that displays JPEG images. The patent emerged in July 2007 following a seven-year reexamination by the U.S. Patent and Trademark Office in which all of the original claims of the patent were revoked, but an additional claim (claim 17) was confirmed. In its first two lawsuits following the reexamination, both filed in Chicago, Illinois, Global Patent Holdings sued the Green Bay Packers, CDW, Motorola, Apple, Orbitz, Officemax, Caterpillar, Kraft and Peapod as defendants. A third lawsuit was filed on December 5, 2007 in Southern Florida against ADT Security Services, AutoNation, Florida Crystals Corp., HearUSA, MovieTickets.com, Ocwen Financial Corp. and Tire Kingdom, and a fourth lawsuit on January 8, 2008 in Southern Florida against the Boca Raton Resort & Club. A fifth lawsuit was filed against Global Patent Holdings in Nevada. That lawsuit was filed by Zappos.com, Inc., who was allegedly threatened by Global Patent Holdings, and seeks a judicial declaration that the '341 patent is invalid and not infringed. The patent owner has also used the patent to sue or threaten outspoken critics of broad software patents, including Gregory Aharonian and the anonymous operator of a website blog known as the "Patent Troll Tracker." On December 21, 2007, patent lawyer Vernon Francissen of Chicago asked the U.S. Patent and Trademark Office to reexamine the sole remaining claim of the '341 patent on the basis of new prior art. On March 5, 2008, the U.S. Patent and Trademark Office agreed to reexamine the '341 patent, finding that the new prior art raised substantial new questions regarding the patent's validity. In light of the reexamination, the accused infringers in four of the five pending lawsuits have moved to suspend their cases pending the completion of the U.S. Patent and Trademark Office's review of the '341 patent. Niro, Scavone, Haller & Niro is the law firm that's representing Global Patent Holdings.

Standards

JPEG (lossy and lossless): ITU-T T.81, ISO/IEC IS 10918-1

JPEG (extensions): ITU-T T.84

JPEG-LS (lossless, improved): ITU-T T.87, ISO/IEC IS 14495-1

JBIG (black and white pictures): ITU-T T.82, ISO/IEC IS 11544-1

JPEG 2000 (successor of JPEG/JPEG-LS): ITU-T T.800, ISO/IEC IS 15444-1

JPEG-2000 (extensions): ITU-T T.801Further Information

Get more info on 'Jpeg'.

External Link Exchanges

Do you know how hard it is to get a link from a large encyclopaedia? Well we're different and will prove it. To get a link from us just add the following HTML to your site on a relevant page:

<a href="http://jpeg.totallyexplained.com">JPEG Totally Explained</a>

Then simply click through this link from your web page. Our crawlers will verify your link, extract the title of your web page and instantly add a link back to it. If you like you can remove the words Totally Explained and embed the link in article text.
As long as your link remains in place, we'll keep our link to you right here. Please play fair - our crawlers are watching. Your site must be closely related to this one's topic. Any kind of spamming, dubious practises or removing the link will result in your link from us being dropped and, potentially, your whole site being banned.