DETAILED DESCRIPTION OF THE INVENTION

[0010] 1. Technical Background

[0011] It is well known that ringing and blocking artifacts are consequential image components that result from traditionally employed JPEG compression algorithms. With respect to looking at and thinking about a color image as one that is made up of a plurality of 8×8 blocks of pixels, the format in which JPEG performs its compression, when high compression is required, the traditional JPEG algorithm greatly quantizes the information that is contained inside such basic 8×8 blocks. In many instances, the artifact-creating effects of this activity are extremely visible in computer-based color images, and there is a substantial desire to provide simple and elegant solutions, which will effectively remove such artifact effects from visibility.

[0012] 2 . The Invention Generally

[0013] The present invention offers a solution to artifact removal, in the setting discussed generally above, by implementing two novel stages of activity, the first of which involves performing an assessment with respect to a selected JPEG-compressed color image regarding whether or not to invoke artifact removal, and the second of which comes to play when the decision to invoke artifact removal is positive. This second stage, or phase, involves examining the luminance channel of an image to characterize different 8×8 pixel blocks with regard to certain definitive characterizations which reside in three categories. These categories are referred to herein as uniform, transitional, and busy. The uniform characterization is one which is applied where, in terms of pixel variance, a block is seen to possess substantial pixel uniformity within itself and with respect to neighboring pixel blocks. The transitional characterization is one which is employed particularly at the edge of an image where there is a region containing pixel blocks that have higher variance and an adjacent region containing pixel blocks that have lower variance associated with a uniform area. This same (transitional) characterization can also apply to image-internal regional transitions which are somewhat like image edge transitions. The busy characterization involves pixel blocks, and neighbors of these blocks, wherein there is very high degree of pixel variance complexity, in which kind of a visual field it is unlikely that even fairly pronounced JPEG ringing and blocking artifacts will be visually detected.

[0014] When a decision has been made to perform artifact removal, on the basis of an examination of an image from any one of three different specific manners proposed to herein for performing such an assessment (described below), the single luminance channel of the image is effectively separated from the two chrominance channels so as to enable the creation of three kinds of luminance-based images from which a final artifact-removed output image will be created. One of these luminance-based images is exactly the unfiltered luminance-channel image of the original image per se. Another is one that has been created by applying a de-blocking filter to the original luminance channel image, the details of which filter will be explained shortly below. The third type of luminance-based image is one that is created by applying a de-ringing filter to the original luminance-channel image, and this filter will be described in greater detail below also.

[0015] In accordance with a preferred manner of practicing the invention, these three kinds of luminance-based images are generated, and also generated is what is referred to herein as a variance map which describes the character of the overall luminance-based image in terms of the three characteristics mentioned above, namely, the characteristics given the names uniform, transitional and busy. It should be understood that these characterizations are singular with respect to each pixel block. In other words, no pixel block is characterized in two of these ways.

[0016] With the variance map created, and the three kinds of luminance-based processed images readied, what is referred to herein as a logic map is employed to define precisely how a final artifact-removed image will be constructed. For assembly of such a final image, each pixel block is examined for its characterization as being either uniform, transitional or busy, and depending upon this characterization, its positional counterpart in one of the three luminance-based images is employed directly in the final image. Very specifically, if a pixel block is determined to have the characteristic called busy, the positionally-matching counterpart pixel block which exists in the unfiltered original luminance-based image is employed in the final image. If, however, such a pixel block is characterized as uniform, then its positional counterpart in the de-blocking luminance image is selected for incorporation into the final image. Finally, if a pixel block's characterization is determined to be transitional, then its positional counterpart in the de-ringing luminance-based image is selected for use in the final image.

[0017] The preferred manner of practicing the invention involves the full creation of the three luminance-based images mentioned above, and the full characterization of each 8×8 pixel block for the entire image as being uniform, transitional or busy, before any final image-assembly takes place. On route to such a final assembly, the chosen, processed luminance-based image for use in the final image is re-combined with the counterpart two chrominance channels which were separated at the time that the original full image was first investigated, and also along this route, certain image enhancements, as will be further described, may be employed.

[0018] An alternative manner of practicing the artifact-removal portion of the invention, which manner is practiced by a modified form of the invention, analysis involves a chain of pixel-block by pixel-block serial activity. In this approach, each 8×8 pixel block is characterized as either being uniform, transitional, or busy, and depending upon this characterization is then filtered or not filtered, along the lines just mentioned above, and then directly placed into a location wherein the final output artifact-removed image is to be created. In other words, in this approach toward practicing the invention, the overall and full image analyses and characterizations described earlier are not performed before final output-image assembly begins.

[0019] 3. The FIGS. 1 and 2 Embodiment

[0020] With reference now to FIG. 1 in the drawings, here there is shown, in block form and schematically, the preferred system and method of practicing the present invention. This system and related methodology are designated generally at 10 in FIG. 1 . Generally speaking now, what is shown in FIG. 1 will be, for the sake of convenience and simplicity, described mostly in systemic terms.

[0021] Within system 10 , a block 12 represents a JPEG-compressed color image which has been selected for examination, and for possible artifact removal. A block 14 , which is operationally downstream from block 12 in FIG. 1 , represents a structure and an activity for determining both the file size (F) and the pixel count (R) of image 12 in kilobytes. A block 16 in FIG. 1 represents structure and methodology for selecting a final-image magnification factor, and a block 18 represents structure and methodology for calculating the end result produced by multiplying the file size by the pixel count, and dividing this by the selected magnification number. Further shown in FIG. 1 are a Yes/No determination block 20 which represents structure and methodology essentially for asking the question whether the calculation produced by block 18 is equal to or less than a predetermined threshold value (θ), and a block 22 which brings the activity to an end if the answer to the question raised in block 20 is No. Also illustrated is a block 24 which represents structure and methodology for implementing artifact removal if the answer to the question asked by block 20 is Yes. The logical flow of activity pictured in FIG. 1 is clearly illustrated by the connecting arrow-headed lines.

[0022] Thus FIG. 1 illustrates the preferred manner of assessing a full JPEG-compressed color image to determine whether or not artifact removal should be invoked. While different threshold values (θ) can be selected, a value which has been found, by experience with the invention, to be a very good value for this number is 100,000.

[0023] As was mentioned earlier, while what is specifically shown in FIG. 1 represents a preferred manner of characterizing an image as being or not being an appropriate candidate for artifact removal, two other specifically useful approaches are also contemplated by the invention. One of these approaches is simply based upon deciding artifact-removal candidacy based purely upon file size. Here, an appropriate file-size number in kilobytes has been found to be about 300,000. The other alternative manner of assessing an original image for candidacy involves determining both file size and pixel count in kilo pixels, and by then multiplying these two together to produce a product which is then compared with another kind of threshold value. Here, an appropriate value has been found to be about 500,000.

[0024] No matter which of these techniques is employed to qualify an image as being or not being a candidate for artifact removal, when an image is determined to be appropriate for artifact removal, then the second phase of the invention, namely specific artifact removal activity, is performed.

[0025] Turning attention now to FIG. 2 in the drawings, this figure illustrates the preferred structure and methodology proposed by the present invention for performing artifact removal. Included as shown in the block/schematic diagram of FIG. 2 are a Select Image block 12 (which is like block 12 pictured in FIG. 1 ), a Transform block 26 , a 2 Chrominance Channel block 28 , a 1-Luminance channel block 30 , a multi-function block 32 which involves image pixel-block characterizations in accordance with the terms uniform, transitional and busy, a No Filter block 34 , a De-Blocking Filter block 36 , a De-Ringing filter block 38 , a Build New Luminance Image block 40 , an Enhance block 42 , and finally a Build Final Image block 44 .

[0026] Associated preferably with, and effectively disposed operatively within, Deblocking filter block 36 , is a low-pass filter which is based upon a Gaussian spread function:

[0027] Kernel=e EXP-(x ² +y ² )/2σ ² , where σ=1 in terms of pixels. The filter is multiplied by 1024, and rounded to the nearest integer to give: 1 $\mathrm{kernel}=\left[\begin{array}{ccccccc}0& 2& 7& 11& 7& 2& 0\\ 2& 19& 84& 139& 84& 19& 2\\ 7& 84& 377& 621& 377& 84& 7\\ 11& 139& 621& 1024& 621& 139& 11\\ 7& 84& 377& 621& 377& 84& 7\\ 2& 19& 84& 139& 84& 19& 2\\ 0& 2& 7& 11& 7& 2& 0\end{array}\right]/6436$

[0028] Associated preferably with, and effectively disposed operatively within, De-ringing filer block 38 , is another low-pass filter, as follows: 2 $\mathrm{kernel}=\left[\begin{array}{ccc}1& 3& 1\\ 3& 6& 3\\ 1& 3& 1\end{array}\right]/22$

[0029] Other very specific details about the preferred manner of implementing artifact removal are described below. At this point, however, the operational flow of artifact removal in accordance with the system and methodology represented in FIG. 2 can be expressed generally in the following manner.

[0030] An image which has been selected for invocation of artifact removal in accordance, for example, with what is shown in FIG. 1 herein, is transformed in block 26 which effectively separates the whole image into its two constituent chrominance channels (block 28 ) and into its single luminance channel (block 30 ). Blocks 34 , 36 , 38 effectively act upon the original luminance channel to make available to block 40 three different luminance-based images which were mentioned briefly earlier herein. Block 34 simply passes an unfiltered version of the luminance-channel image directly to block 40 . De-blocking filter block 36 applies the above described low-pass de-blocking filter to the original luminance-channel image to create, and to make available to block 40 , what has been referred to herein as a de-blocking luminance-based image. Similarly, the de-ringing low-pass filter which is operationally present in block 38 acts upon the original luminance channel image to create, and to present to block 40 , a de-ringing luminance-based image.

[0031] The luminance-channel image present and made available by block 30 is also made available to block 32 wherein, on a pixel-block-by-pixel-block basis, the luminance-channel image is characterized with respect to the characteristic designators called uniform, transitional and busy. One will note that block 32 is divided into three sections, each of which bears a different character of area shading. This shading selves to distinguish these three characteristics, and to relate these three characteristics to activities which take place respectively with regard to blocks 34 , 36 , 38 . One will see that the surface areas of blocks 34 , 36 , 38 are respectively area-shaded in manners which give a visual direct relationship between what is performed in these three blocks and what takes place in the characterization areas respectively in block 32 . Pixel blocks which are identified as uniform, busy and transitional, respectively, are appropriately made available to block 40 .

[0032] The earlier mentioned logic map which describes the architecture of final assembly to produce an artifact-cleared output image is prepared on the basis of the three characterizations which are performed on the whole luminance-based image within block 32 . Effectively, this logic map describes, on a pixel-block-by-pixel-block basis, just which pixel blocks in the final image will be drawn from one of the original luminance-based image, the de-blocking luminance-based image, and the de-ringing luminance-based image.

[0033] From this information, block 40 builds a precursor to the final output image, which precursor is referred to herein as a processed luminance image.

[0034] From block 40 the processed luminance image is furnished to enhancement block 42 which produces and supplies to block 44 what is referred to herein as an enhanced processed luminance image. Considering what takes place within block 42 to perform enhancement, based on the size of the image and the final print size (say 4-inches in the short dimension) a suitable enhancement filter and kernel is calculated based on well known criteria. Based on the final print size, the printer dpi and the pixel resolution, a 3×3, 5×5 or 7×7 is picked for enhancement. A fixed enhancement MTF is used (in the plane of the final print) and based on the kernel size, the appropriate number of frequency samples is taken form 0 to 6 cycles/mm; it is assumed that most observers will not see enhancement above 6-cycles/mm on the print. This results in first a one-dimensional FIR, and then using eight-way symmetry the two-dimensional FIR filter is generated. Note, as cited above, if avgvar (the average variance of the whole image) is greater than 600 the “boost factor” (roughly the boost factor is the gain in excess of unity at the peak of the frequency response) is reduced from 1.0 to 0.5 to insure a natural look. The kernal is then applied to the image resulting in an enhanced, processed luminance-channel image. The processed luminance-channel image is then recombined with the two chrominance channels to form the final image.

[0035] The enhanced processed luminance image which is fed to block 44 is therein combined with the two previously separated chrominance channels (block 28 ) to form the final artifact-purged output color image.

[0036] Presented now immediately below is a full description of the algorithm which is implemented in the practice or the system and methodology illustrated so far with regard to FIGS. 1 and 2 .

[0037] Accordingly, a novel system and methodology is provided by the present invention for dealing with JPEG-compressed artifacts of the usual blocking and ringing types. The invention begins with a first stage investigation (see FIG. 1 ) regarding whether a selected original color image is a suitable candidate for artifact removal, and if so, triggers the invocation of one of the two forms of artifact removal described above. Original image pixels, and more specifically, the luminance-channel components of those images, are examined to characterize them as being either busy, uniform or transitional, with this information then being employed to create correspondingly positioned output pixels for incorporation in a final image. While several modifications of the system and methodology of the invention have been illustrated and described herein, it is appreciated that variation and modification may be may without departing from the spirit of the invention.