BACKGROUND OF THE INVENTION

[0001] The present invention relates to an image-capturing apparatus such as a digital camera, an image processing method for optimizing an image quality of a visual image formed on an outputting medium in respect to digital image data obtained by such the image-capturing apparatus, an image-processing apparatus employing the abovementioned image processing method, an image-recording apparatus that forms an visual image on the outputting medium by employing the abovementioned image processing method, a computer program for executing the abovementioned image processing method and a recording medium that stores the computer program and is readable by an computer.

[0002] At present, the digital image data captured by an image-capturing apparatus is distributed through such a memory device as a CD-R (Compact Disk Recordable), floppy disk (registered trade name) and memory card or the Internet, and is displayed on such a display monitor as a CRT (Cathode Ray Tube), liquid crystal display and plasma display or a small-sized liquid crystal monitor display device of a cellular phone, or is printed out as a hard copy image using such an output device as a digital printer, inkjet printer and thermal printer. In this way, display and print methods have been diversified in recent years.

[0003] When digital image data is displayed and output for viewing, it is a common practice to provide various types of image processing typically represented by gradation adjustment, brightness adjustment, color balancing and enhancement of sharpness to ensure that a desired image quality is obtained on the display monitor used for viewing or on the hard copy.

[0004] In response to such varied display and printing methods, efforts have been made to improve the general versatility of digital image data captured by an image-capturing apparatus. As part of these efforts, an attempt has been made to standardize the color space represented by digital RGB (Red, Green and Blue) signals into the color space that does not depend on characteristics of an image-capturing apparatus. At present, large amounts of digital image data have adopted the sRGB (See Multimedia Systems and Equipment—Color Measurement and Management—Part 2-1: Color Management—Default RGB Color Space—sRGB” IEC61966-2-1) as a standardized color space. The color space of this sRGB has been established to meet the color reproduction area for a standard CRT display monitor.

[0005] Generally, a digital camera is equipped with an image-capturing device (CCD type image-capturing device, hereinafter referred to simply as “CCD”) having a photoelectric conversion function with color sensitivity provided by a combination of a CCD (charge coupled device), a charge transfer device and a mosaic color filter. The digital image data output from the digital camera is obtained after the electric original signal gained by conversion via the CCD has been corrected by the photoelectric conversion function of the image capturing device, and processing of file conversion and compression into the predetermined data format standardized to permit reading and display by image editing software.

[0006] Correction by the photoelectric conversion function of the image capturing device includes, for example, gradation correction, spectral sensitivity, cross-talk correction, dark current noise control, sharpening, white balance adjustment and color saturation adjustment. The above-mentioned standardized data format widely known includes Baseline Tiff Rev. 6.0 RGB Full Color Image adopted as a non-compressed file of the Exif (Exchangeable Image File Format) file and compressed data file format conforming to the JPEG format.

[0007] The Exif file conforms to the above-mentioned sRGB, and the correction of the photoelectric conversion function of the above-mentioned image-capturing element is established so as to ensure the most suitable image quality on the display monitor conforming to the sRGB.

[0008] For example, if a digital camera has the function of writing into the header of the digital image data the tag information for display in the standard color space (hereinafter referred to as “monitor profile”) of the display monitor conforming to the sRGB signal, and accompanying information indicating the device dependent information such as the number of pixels, pixel arrangement and number of bits per pixel as meta-data as in the case of Exif format, and if only such a data format is adopted, then the tag information can be analyzed by the image edit software (e.g. Photoshop by Abode for displaying the above-mentioned digital image data on the digital display monitor, conversion of the monitor profile into the sRGB can be prompted, and modification can be processed automatically. This capability reduces the differences in apparatus characteristics among different displays, and permits viewing of the digital image data photographed by a digital camera under the optimum condition.

[0009] In addition to the above-mentioned information dependent on device type, the above-mentioned accompanying information includes;

[0010] information directly related to the camera type (device type) such as a camera name and code number,

[0011] information on photographing conditions such as exposure time, shutter speed, f-stop number (F number), ISO sensitivity, brightness value, subject distance range, light source, on/off status of a stroboscopic lamp, subject area, white balance, zoom scaling factor, subject configuration, photographing scene type, the amount of reflected light of the stroboscopic lamp source and color saturation for photographing, and tags (codes) for indicating the information related to a subject. The image edit software and output device have a function of reading the above-mentioned accompanying information and making the quality of hardware image more suitable.

[0012] The image displayed on such a display device as a CRT display monitor and the hard copy image printed by various printing devices have different color reproduction areas depending on the configuration of the phosphor or color material to be used. For example, the color reproduction area of the CRT display monitor corresponding to the sRGB standard space has a wide bright green and blue area. It contains the area that cannot be reproduced by the hard copy formed by a silver halide photographic printer, inkjet printer and conventional printer. Conversely, the cyan area of the conventional printing or inkjet printing and the yellow area of the silver halide photographic printing contain the area that cannot be reproduced by the CRT display monitor corresponding to the sRGB standard color space. (For example, see “Fine imaging and digital photographing” edited by the Publishing Commission of the Japan Society of Electrophotography, Corona Publishing Co., P. 444). In the meantime, some of the scenes of the subject to be photographed may contain the color in the area that cannot be reproduced in any of these areas for color reproduction.

[0013] As described above, the color space (including the sRGB) optimized on the basis of display and printing by a specific device is accompanied by restrictions in the color gamut where recording is possible. So when recording the information picked up by a photographing device, it is necessary to make adjustment of mapping by compressing the information into the color gamut where recording is allowed. The simplest way is provided by clipping where the color chromaticity point outside the color gamut where recording is possible is mapped onto the boundary of the nearest color gamut. This causes the gradation outside the color gamut to be collapsed, and the image will give a sense of incompatibility to the viewer. To avoid this problem, non-liner compression method is generally used. In this method, the chromaticity point in the area where chroma is high in excess of an appropriate threshold value is compressed smoothly according to the size of the chroma. As a result, chroma is compressed and recording is carried out even at the chromaticity point inside the color gamut where recording is possible. (For the details of the procedure of mapping the color gamut, see “Fine imaging and digital photographing” edited by the Publishing Commission of the Japan Society of Electrophotography, Corona Publishing Co., P. 447, for example).

[0014] The image displayed on such a display device as a CRT display monitor, the hard copied image printed by various types of printing devices, or color space (including sRGB) optimized on the basis of display and printing by these devices are restricted to the conditions where the area of brightness that allows recording and reproduction is of the order of about 100 to 1. By contrast, however, the scene of the subject to be photographed has a wide area of brightness, and it often happens that the order of several thousands to 1 is reached outdoors. (See “Handbook on Science of Color, New Version, 2nd Print” by Japan Society for Science of Colors, Publishing Society of the University of Tokyo, P. 926, for example). Accordingly, when recording the information gained by the image capturing device, compression is also necessary for brightness. In this processing, adequate conditions must be set for each image in conformity to the dynamic range of the scene to be photographed, and the range of brightness for the main subject in the scene to be photographed.

[0015] However, when compression has been carried out for the color gamut and brightness area as described above, information on gradation prior to compression or information prior to clipping is lost immediately due to the principle of the digital image to be recorded in terms of the discrete value. The original state cannot be recovered. This imposes a big restriction on the general versatility of high-quality digital image.

[0016] For example, when the image recorded in the sRGB standard color space is printed, mapping must be carried out again based on the differences between the sRGB standard color space and the area for color reproduction of the printing device. For the image recorded in the sRGB standard color space, however, the information on gradation in the area compressed at a time of recording is lost. So the smoothness of gradation is deteriorated as compared to the case where the information captured by the photographing device is mapped directly in the area for color reproduction of the printing device. Further, if gradation compression conditions are not adequate at a time of recording, and there are problems such as a whitish picture, dark face, deformed shadow and conspicuous white skipping in the highlighted area, improvement is very inadequate as compared to the case where the new image is created again from the information gained by the photographing device, even if the gradation setting is changed to improve the image. This is because information on gradation prior to compression, and information on the portion subjected to deformation or white skipping have been already lost.

[0017] As a solution of the above-mentioned problems, the art of storing the process of image editing as a backup data and returning it to the state prior to editing whenever required has long been known. For example, Patent Document 1 (listed later) discloses a backup device wherein, when the digital image is subjected to local modification by image processing, the image data on the difference between the digital image data before image processing and that after image processing is saved as backup data. Patent Document 2 (listed later) discloses a method for recovering the digital image data before editing, by saving the image data on the difference between the digital image data before image processing and that after image processing. The technologies, set forth in Patent Document 1 and Patent Document 2, are effective from the viewpoint of preventing information from being lost, but the number of sheets that can be photographed by a camera is reduced with the increase in the amount of data recorded in the media.

[0018] Other than the image recording in the sRGB standard color space mentioned above, an automatic white-balance adjustment would be cited as an processing including a compression of the subject information at the time of the recording. Conventional digital camera requires cumbersome operations for adjusting frame by frame according to the manual compensating function in order to apply the white-balance adjustment. Accordingly, it has been impossible to appropriately reflect the photographer's tastes onto the reproduced image.

[0019] To overcome the abovementioned problem, a digital camera that can select a color tone converting means for outputting an image having a color tone just same as the photographer's perception by considering the adaptability of human's eyes, or another color tone converting means for outputting an image having a color tone under the daylight source, is set forth in Patent Document 3. Further, a digital camera that can omit the cumbersome adjusting operations by making it possible for the photographer to select a degree of the white-balance adjustment is set forth in Patent Document 4. There are many cases that the tastes for finished color tones just same as the viewer's perception and under the daylight source with respect to various light sources, such as a sunset light, a tungsten light, a candle light, etc., would vary among the photographer just capturing the image, a person creating a print of the image and viewers of the image. In addition, the light source at the time of the image-capturing is not limited to a single kind of light source, but there are many cases that a mixed light mixed with two or more than two different kinds of light sources is employed for the image-capturing operation, and therefore, an accurate white-balance adjustment is not always applied.

[0020] On the other hand, a method for adjusting the white balance, in which a captured image is divided into small image areas, and the small image areas are grouped into plural groups for every same color temperature, namely every same photographic light source, by creating a histogram of estimated color temperature for every small image area, and then, the white balance is adjusted for a respective one of the plural groups, is set forth in Patent Document 5. However, since, in the information to which the white-balance adjustment processing was already applied at the time of the image-capturing operation, either B-information or R-information already processed with a heavy compression processing, only an insufficient improvement could be achieved even if the digital image data, to which the white-balance adjustment was once applied, is intended to readjust.

[0021] The problems introduced above are caused by the procedure where the information on the wide color gamut and brightness area gained by a photographing device is recorded after having being compressed into the output-referred image data in the state optimized by assuming an image to be viewed. By contrast, if the information on the wide color gamut and brightness area gained by a photographing device is recorded as scene-referred image data that is not compressed, then inadvertent loss of information can be prevented. The standard color space suited to record such scene-referred image data is proposed, for example, by RIMM RGB (Reference Input Medium Metric RGB) and ERIMM RGB (Extended Reference Input Medium Metric RGB). (See the Journal of Imaging Science and Technology, Vol. 45 p p. 418 to 426 (2001)).

[0022] However, the data expressed in the standard color space like the one described above, is not suitable for being displayed directly on the display monitor and viewed. Generally, a digital camera has a built-in display monitor or is connected to it in order for the user to check the angle of view before photographing or to check the photographed image after photographing. When photographed data is recorded as output referred image data like the sRGB, it can be displayed directly on the display monitor, without the data being converted. Despite this advantage, when the photographed data is recorded as scene-referred image data, the data must be subjected to the processing of re-conversion into the output-referred image data in order to display that data. Such double processing of conversion inside the camera increases the processing load and power consumption, and causes the continuous shooting capability to be reduced, and imposes restrictions on the number of sheets to be shot in the battery mode.

[0023] To solve the abovementioned problems, Patent Document 6 discloses an image processing apparatus characterized by two modes; a mode of recording in the form of an image signal displayed on the display means and a mode of recording in the form of captured image signal. The form of image signal in the latter case is generally called RAW data. Using the special-purpose application software (called “development software”), such digital image data can be converted into output-referred image data of the above-mentioned Exif file or the like for display or printing (called “electronic development” or simply “development”).

[0024] Patent Document 1: Tokkaihei 7-57074

[0025] Patent Document 2: Tokkai 2001-94778

[0026] Patent Document 3: Tokkaihei 10-4458

[0027] Patent Document 4: Tokkai 2002-218495

[0028] Patent Document 5: Tokkai 2002-271638

[0029] Patent Document 6: Tokkaihei 11-261933

[0030] Since the RAW data retains all information at a time of photographing, it permits output-referred image data to be remade. If other color system files such as CMYK are created directly, there will no inadvertent modification of the color system due to the difference in color gamut from the display monitor (sRGB). However, the RAW data is recorded according to the color space based on the spectral sensitivity characteristics inherent to the type of a photographing apparatus and the file format inherent to the type of a photographing apparatus. Accordingly, image suitable to display and printing can be obtained only when special-purpose development software inherent to the type of the photographing apparatus is used. Further, it is impossible to conduct the white-balance adjustment, which reflects the tastes of the photographer, with respect to the RAW data as it is.

SUMMARY OF THE INVENTION

[0031] To overcome the abovementioned drawbacks in conventional image-processing apparatus, image-processing methods and image-recording apparatus, it is an object of the present invention to provide an image-capturing apparatus, which makes it possible to store digital image data by employing a general purpose method without losing captured image information and to apply a white-balance adjustment reflecting tastes of the photographer onto the digital image data, and to provide a novel environment for processing a high quality image by employing the abovementioned image-capturing apparatus and the digital image data stored in it.

[0032] Accordingly, to overcome the cited shortcomings, the abovementioned object of the present invention can be attained by image-capturing apparatus, image-processing apparatus, image-processing methods and image-recording apparatus described as follow.

[0033] (1) An apparatus for capturing an image, comprising: a scene-referred raw data generating section to generate scene-referred raw data, which directly represent the image while depending on image-capturing characteristics of the apparatus; a reproduction-auxiliary data generating section to generate reproduction-auxiliary data, based on which an image-capturing characteristic compensation processing is to be applied to the scene-referred raw data generated by the scene-referred raw data generating section, so as to generate scene-referred image data in a standardized format from the scene-referred raw data; a designating section to designate a degree of white-balance adjustment; and a storage controlling section to attach the reproduction-auxiliary data and white-balance data indicating the degree of the white-balance adjustment to the scene-referred raw data in order to store all of them into a storage medium.

[0034] (2) The apparatus of item 1, further comprising: an image-capturing data generating section to generate image-capturing data, which represent image-capturing conditions established at a time of capturing the image; wherein the storage controlling section attaches the reproduction-auxiliary data, the white-balance data and the image-capturing data to the scene-referred raw data in order to store all of them into the storage medium.

[0035] (3) The apparatus of item 1, wherein the image-capturing data includes a photographing EV value established at a time of capturing the image.

[0036] (4) An apparatus for processing data, comprising: a receiving section to receive scene-referred raw data, which directly represent an image captured by an image-capturing apparatus while depending on image-capturing characteristics of the image-capturing apparatus, and to receive reproduction-auxiliary data in respect to the scene-referred raw data, and to receive white-balance data indicating a degree of white-balance adjustment; a scene-referred image data generating section to generate scene-referred image data from the scene-referred raw data received by the receiving section, based on the reproduction-auxiliary data received by the receiving section, by applying an image-capturing characteristic compensation processing to the scene-referred raw data; and an output data generating section to generate output data by attaching the white-balance data to the scene-referred image data.

[0037] (5) The apparatus of item 4, wherein the scene-referred image data are generated in a standardized format from the scene-referred raw data.

[0038] (6) The apparatus of item 4, wherein the receiving section also receives image-capturing data, which represent image-capturing conditions established at a time of capturing the image; and wherein the output data generating section generates the output data by attaching the white-balance data and the image-capturing data to the scene-referred image data.

[0039] (7) The apparatus of item 5, further comprising: an output-referred image data generating section to generate output-referred image data, based on which a reproduced image is formed on an outputting medium, by applying an image-processing for optimizing the reproduced image to the scene-referred image data generated by the scene-referred image data generating section; wherein contents of the image-processing are determined on the basis of the white-balance data received by the receiving section.

[0040] (8) The apparatus of item 6, further comprising: an output-referred image data generating section to generate output-preferred image data, based on which a reproduced image is formed on an outputting medium, by applying an image-processing for optimizing the reproduced image to the scene-referred image data generated by the scene-referred image data generating section; wherein contents of the image-processing are determined on the basis of the white-balance data and the image-capturing data, both received by the receiving section.

[0041] (9) The apparatus of item 4, wherein the receiving section receives a photographing EV value established at a time of capturing the image.

[0042] (10) The apparatus of item 7, further comprising: an applying amount determining section to determine an applying amount of the white-balance adjustment to be applied to the scene-referred image data, based on the white-balance data indicating the degree of the white-balance adjustment; wherein the output-referred image data generating section includes: a white-balance adjusting section to apply the applying amount of the white-balance adjustment, determined by the applying amount determining section, to the scene-referred image data.

[0043] (11) The apparatus of item 10, wherein the white-balance adjusting section is provided with an image area dividing function for dividing a whole image area of the scene-referred image data into a plurality of small image areas.

[0044] (12) The apparatus of item 11, wherein the white-balance adjusting section is further provided with a ratio calculating function for calculating a R/G ratio and a B/G ratio for a respective one of the plurality of small image areas, divided by the image area dividing function; and wherein the R/G ratio represents a ratio between an integrated value of R (Red) signals and another integrated value of G (Green) signals within each of the plurality of small image areas, while the B/G ratio represents a ratio between an integrated value of B (Blue) signals and another integrated value of G (Green) signals within each of the plurality of small image areas.

[0045] (13) The apparatus of item 12, wherein the white-balance adjusting section is further provided with a light-source estimating function for estimating a kind of a photographic light source for a respective one of the plurality of small image areas, by plotting the R/G ratio and the B/G ratio, calculated by the ratio calculating function, onto a light-source estimating map in which light-source area frames, indicating ranges of various combinations of the R/G ratio and the B/G ratio corresponding to various kinds of light-sources.

[0046] (14) The apparatus of item 13, wherein the white-balance adjusting section is further provided with a light-source determining function for determining a kind of a photographic light source under which the scene-referred image data are acquired by employing a number of small image areas plotted within one of the light-source area frames, or a membership function in which a photographic EV value is a variable.

[0047] (15) The apparatus of item 11, wherein the white-balance adjusting section is further provided with a color-temperature estimating function for estimating a color temperature of a photographic light source for a respective one of the plurality of small image areas by employing a least squares method.

[0048] (16) The apparatus of item 15, wherein the white-balance adjusting section is further provided with a histogram creating function for creating a histogram based on an emerging frequency of the color temperature of the photographic light source for a respective one of the plurality of small image areas.

[0049] (17) The apparatus of item 16 , wherein the white-balance adjusting section is further provided with a group-wise white-balance adjusting function for dividing the whole image area of the scene-referred image data into plural groups, based on the histogram created by the histogram creating function, so as to apply a different white-balance adjustment to a respective one of the plural groups, the different white-balance adjustment is one of various white-balance adjustments being different relative to each other corresponding to a respective one of the plural groups.

[0050] (18) The apparatus of item 10, wherein the applying amount determining section can arbitrarily establish a relationship between the white-balance data, indicating the degree of the white-balance adjustment, and an applying amount of the white-balance adjustment to be applied in practice.

[0051] (19) An apparatus for outputting a reproduced image onto an outputting medium, comprising: a receiving section to receive scene-referred raw data, which directly represent an image captured by an image-capturing apparatus while depending on image-capturing characteristics of the image-capturing apparatus, and to receive reproduction-auxiliary data in respect to the scene-referred raw data, and to receive white-balance data indicating a degree of white-balance adjustment; a scene-referred image data generating section to generate scene-referred image data from the scene-referred raw data received by the receiving section, based on the reproduction-auxiliary data received by the receiving section, by applying an image-capturing characteristic compensation processing to the scene-referred raw data; an output-referred image data generating section to generate output-referred image data, based on which a reproduced image is formed on an outputting medium, by applying an image-processing for optimizing an image quality of the reproduced image to the scene-referred image data generated by the scene-referred image data generating section; and an image-forming section to form the reproduced image on the outputting medium, based on the output-referred image data; wherein contents of the image-processing for optimizing the image quality of the reproduced image are determined on the basis of the white-balance data indicating the degree of white-balance adjustment.

[0052] (20) The apparatus of item 19, wherein the scene-referred image data are generated in a standardized format from the scene-referred raw data.

[0053] (21) The apparatus of item 19, wherein the receiving section also receives image-capturing data, which represent image-capturing conditions established at a time of capturing the image.

[0054] (22) The apparatus of item 19, wherein the receiving section receives a photographing EV value established at a time of capturing the image.

[0055] (23) The apparatus of item 19, further comprising: an applying amount determining section to determine an applying amount of the white-balance adjustment to be applied to the scene-referred image data, based on the white-balance data indicating the degree of the white-balance adjustment; wherein the output-referred image data generating section includes: a white-balance adjusting section to apply the applying amount of the white-balance adjustment, determined by the applying amount determining section, to the scene-referred image data.

[0056] (24) The apparatus of item 23, wherein the white-balance adjusting section is provided with an image area dividing function for dividing a whole image area of the scene-referred image data into a plurality of small image areas.

[0057] (25) The apparatus of item 24, wherein the white-balance adjusting section is further provided with a ratio calculating function for calculating a R/G ratio and a B/G ratio for a respective one of the plurality of small image areas, divided by the image area dividing function; and wherein the R/G ratio represents a ratio between an integrated value of R (Red) signals and another integrated value of G (Green) signals within each of the plurality of small image areas, while the B/G ratio represents a ratio between an integrated value of B (Blue) signals and another integrated value of G (Green) signals within each of the plurality of small image areas.

[0058] (26) The apparatus of item 25, wherein the white-balance adjusting section is further provided with a light-source estimating function for estimating a kind of a photographic light source for a respective one of the plurality of small image areas, by plotting the R/G ratio and the BIG ratio, calculated by the ratio calculating function, onto a light-source estimating map in which light-source area frames, indicating ranges of various combinations of the R/G ratio and the B/G ratio corresponding to various kinds of light-sources.

[0059] (27) The apparatus of item 26, wherein the white-balance adjusting section is further provided with a light-source determining function for determining a kind of a photographic light source under which the scene-referred image data are acquired by employing a number of small image areas plotted within one of the light-source area frames, or a membership function in which a photographic EV value is a variable.

[0060] (28) The apparatus of item 24, wherein the white-balance adjusting section is further provided with a color-temperature estimating function for estimating a color temperature of a photographic light source for a respective one of the plurality of small image areas by employing a least squares method.

[0061] (29) The apparatus of item 28, wherein the white-balance adjusting section is further provided with a histogram creating function for creating a histogram based on an emerging frequency of the color temperature of the photographic light source for a respective one of the plurality of small image areas.

[0062] (30) The apparatus of item 29, wherein the white-balance adjusting section is further provided with a group-wise white-balance adjusting function for dividing the whole image area of the scene-referred image data into plural groups, based on the histogram created by the histogram creating function, so as to apply a different white-balance adjustment to a respective one of the plural groups, the different white-balance adjustment is one of various white-balance adjustments being different relative to each other corresponding to a respective one of the plural groups.

[0063] (31) The apparatus of item 23, wherein the applying amount determining section can arbitrarily establish a relationship between the white-balance data, indicating the degree of the white-balance adjustment, and an applying amount of the white-balance adjustment to be applied in practice.

[0064] (32) A method for processing data, comprising the steps of: applying an image-capturing characteristic compensation processing to scene-referred raw data, which directly represent an image captured by an image-capturing apparatus while depending on image-capturing characteristics of the image-capturing apparatus, based on reproduction-auxiliary data in respect to the scene-referred raw data, in order to generate scene-referred image data from the scene-referred raw data; and generating output-referred image data, based on which a reproduced image is formed on an outputting medium, by applying an image-processing for optimizing an image quality of the reproduced image to the scene-referred image data generated in the applying step; wherein contents of the image-processing for optimizing the image quality of the reproduced image are determined on the basis of white-balance data indicating a degree of white-balance adjustment.

[0065] (33) The method of item 32, wherein the scene-referred image data are generated in a standardized format from the scene-referred raw data.

[0066] (34) The method of item 32, wherein the contents of the image-processing for optimizing the image quality of the reproduced image are determined on the basis of the white-balance data indicating the degree of white-balance adjustment and image-capturing data representing image-capturing conditions established at a time of capturing the image.

[0067] (35) The method of item 32, further comprising the steps of: determining an applying amount of the white-balance adjustment to be applied to the scene-referred image data, based on the white-balance data indicating the degree of the white-balance adjustment; and applying the applying amount of the white-balance adjustment, determined in the determining step, to the scene-referred image data.

[0068] (36) The method of item 35, further comprising the step of: dividing a whole image area of the scene-referred image data into a plurality of small image areas.

[0069] (37) The method of item 36, further comprising the step of: calculating a R/G ratio and a B/G ratio for a respective one of the plurality of small image areas, divided in the dividing step; wherein the R/G ratio represents a ratio between an integrated value of R (Red) signals and another integrated value of G (Green) signals within each of the plurality of small image areas, while the B/G ratio represents a ratio between an integrated value of B (Blue) signals and another integrated value of G (Green) signals within each of the plurality of small image areas.

[0070] (38) The method of item 37, further comprising the step of: estimating a kind of a photographic light source for a respective one of the plurality of small image areas, by plotting the R/G ratio and the B/G ratio, calculated in the calculating step, onto a light-source estimating map in which light-source area frames, indicating ranges of various combinations of the R/G ratio and the B/G ratio corresponding to various kinds of light-sources.

[0071] (39) The method of item 38, further comprising the step of: determining a kind of a photographic light source under which the scene-referred image data are acquired by employing a number of small image areas plotted within one of the light-source area frames, or a membership function in which a photographic EV value is a variable.

[0072] (40) The method of item 36, further comprising the step of: estimating a color temperature of a photographic light source for a respective one of the plurality of small image areas by employing a least squares method.

[0073] (41) The method of item 40, further comprising the step of: creating a histogram based on an emerging frequency of the color temperature of the photographic light source for a respective one of the plurality of small image areas.

[0074] (42) The method of item 41, further comprising the step of: dividing the whole image area of the scene-referred image data into plural groups, based on the histogram created in the creating step, so as to apply a different white-balance adjustment to a respective one of the plural groups, the different white-balance adjustment is one of various white-balance adjustments being different relative to each other corresponding to a respective one of the plural groups.

[0075] (43) The method of item 35, wherein a relationship between the white-balance data indicating the degree of the white-balance adjustment and an applying amount of the white-balance adjustment to be applied in practice can be arbitrarily established in the determining step for determining the applying amount of the white-balance adjustment.

[0076] (44) The method of item 32 , further comprising the step of: receiving a photographing EV value established at a time of capturing the image.

[0077] (45) A computer program for executing data processing operations, comprising the functional steps of: applying an image-capturing characteristic compensation processing to scene-referred raw data, which directly represent an image captured by an image-capturing apparatus while depending on image-capturing characteristics of the image-capturing apparatus, based on reproduction-auxiliary data in respect to the scene-referred raw data, in order to generate scene-referred image data from the scene-referred raw data; and generating output-referred image data, based on which a reproduced image is formed on an outputting medium, by applying an image-processing for optimizing an image quality of the reproduced image to the scene-referred image data generated in the applying step; wherein contents of the image-processing for optimizing the image quality of the reproduced image are determined on the basis of white-balance data indicating a degree of white-balance adjustment.

[0078] (46) A recording medium that stores a computer program for executing data processing operations, wherein the computer program comprises the functional steps of: applying an image-capturing characteristic compensation processing to scene-referred raw data, which directly represent an image captured by an image-capturing apparatus while depending on image-capturing characteristics of the image-capturing apparatus, based on reproduction-auxiliary data in respect to the scene-referred raw data, in order to generate scene-referred image data from the scene-referred raw data; and generating output-referred image data, based on which a reproduced image is formed on an outputting medium, by applying an image-processing for optimizing an image quality of the reproduced image to the scene-referred image data generated in the applying step; and wherein contents of the image-processing for optimizing the image quality of the reproduced image are determined on the basis of white-balance data indicating a degree of white-balance adjustment.

[0079] Further, to overcome the abovementioned problems, other image-capturing apparatus, image-processing apparatus, image-processing processing methods and image-recording apparatus, embodied in the present invention, will be described as follow:

[0080] ( 47 ) An image-capturing apparatus, characterized in that

[0081] the image-capturing apparatus is provided with:

[0082] a scene-referred raw data generating means for generating scene-referred raw data dependent on the image-capturing characteristics of the image-capturing apparatus by an image capturing operation;

[0083] a reproduction-auxiliary data generating means for generating reproduction-auxiliary data based on which image-capturing characteristic compensation processing is to be applied to the above-mentioned scene-referred image data generated by the above-mentioned scene-referred raw data generating means, so as to generate scene-referred raw data standardized in respect to the scene-referred raw data;

[0084] a designating means for designating a degree of white-balance adjustment; and

[0085] a storage controlling means for attaching the reproduction-auxiliary data generated by the reproduction-auxiliary data generating means and data representing the degree of the white-balance adjustment designated by the designating means to the scene-referred raw data generated by the scene-referred raw data generating means, and further for storing them into a storage medium.

[0086] The term “generate” appearing in the description of the present Specification refers to the act of a new image signal or data being produced by a program and processing circuit working in the image-capturing apparatus, image processing apparatus and image recording apparatus according to the present invention. The term “create” may be used synonymously with it.

[0087] The “image-capturing apparatus” denotes an apparatus equipped with an image-capturing element (image sensor), and includes a so-called digital camera and scanner. The above-mentioned image-capturing element is exemplified by a CCD type image-capturing element with color sensitivity added through a combination of a Charge Coupled Device (CCD), a charge transfer device and a colored mosaic filter, and a CMOS type image-capturing device. The output current from those image-capturing devices is digitized by an analog-to-digital converter. The contents in each color channel in this phase represent signal intensities based on the spectral sensitivity inherent to the image-capturing device.

[0088] The above-mentioned “scene-referred raw data dependent on the image-capturing characteristics” denotes a raw signal directly outputted from the image-capturing apparatus with information on a subject being faithfully recorded. It refers to the data digitized by the analog-to-digital converter and the same data having been subjected to correction of such a noise as fixed pattern noise and dark current noise. It includes the above-mentioned RAW data. This scene-referred raw data is characterized by omission of the image processing for modifying the contents of data to improve such effects in image viewing as gradation conversion, sharpness enhancement and color saturation enhancement, and processing of mapping the signal intensify of each color channel based on the spectral sensitivity inherent to the image-capturing device, onto the standardized color space such as the above-mentioned RIMM and sRGB. The amount of information on the scene-referred raw data (e.g. number of gradations) is preferred to be equal to greater than that of the information required by the output-referred data (e.g. number of gradations), in conformity to the performances of the above-mentioned analog-to-digital converter. For example, when the number of gradations for the output-referred data is 8 bits per channel, the number of gradations for the scene-referred raw data is preferred to be 12 bits or more. It is more preferred to be 14 bits or more, and still more preferred to be 16 bits or more.

[0089] “Standardized scene-referred image data” signifies the image data characterized in that at least the signal intensity of each color channel based on the spectral sensitivity of the image-capturing device has been already mapped onto the above-mentioned standard color space such as RIMM RGB and ERIMM RGB, wherein this image data is further characterized by omission of image processing for modifying the data contents in order to improve such effects in viewing the image as gradation conversion, sharpness enhancement and color saturation enhancement. It is preferred that the scene-referred raw data be subjected to correction (opto-electronic conversion function defined in ISO1452, e.g. “Fine imaging and digital photographing” edited by the Publishing Commission of the Japan Society of Electrophotography, Corona Publishing Co., P. 479 of the photoelectric conversion characteristics of the image-capturing apparatus. The amount of the standardized scene-referred image data (e.g. number of gradations) is preferred to be equal to or greater than that of the information (e.g. number of gradations) required by the output-referred image data, in conformity to the above-mentioned analog-to-digital converter performances. For example, when the number of gradations for the output-referred image data is 8 bits per channel, then the number of gradations for the scene-referred image data is preferred to be equal to or greater than 12 bits. It is more preferred to be equal to or greater than 14 bits, and is still more preferred to be equal to or greater than 16 bits.

[0090] “Image-capturing characteristic compensation processing (also referred to as image-capturing device characteristic compensation processing) for generating the standardized scene-referred image data” is defined as the process of converting the above-mentioned “scene-referred raw data dependent on the image-capturing characteristics (also referred to as image-capturing device characteristic) of an image-capturing apparatus” into the “standardized scene-referred image data”. This processing depends on the state of “scene-referred raw data dependent on the image-capturing characteristics of the image-capturing apparatus”, and includes the step of mapping at least the signal intensity of each color channel based on the spectral sensitivity of the image-capturing device, onto the above-mentioned standard color space such as RIMM RGB and ERIMM RGB. For example, when the “scene-referred raw data dependent on the image-capturing characteristics of an image-capturing apparatus” is not subjected to interpolation processing based of the color filter arrangement, execution of this processing is essential. (For the details of the interpolation processing based of the color filter arrangement, see “Fine imaging and digital photographing” edited by the Publishing Commission of the Japan Society of Electrophotography, Corona Publishing Co., P. 51). This will provide “standardized scene-referred raw data” where the differences of signal values among different image-capturing apparatuses are corrected, while almost the same amount of information as that of “scene-referred raw data” is retained.

[0091] The “reproduction-auxiliary data based on which image-capturing characteristic compensation processing is applied” denotes the data based on which image-capturing characteristic compensation processing defined in the previous item can be applied, using only the information described in the data. This requires description of the information that permits mapping at least the signal intensity of each color channel based on the spectral sensitivity of the image-capturing device, onto the above-mentioned standard color space such as RIMM RGB and ERIMM RGB. To put it another way, this requires description of the matrix coefficient to be used for conversion into the specific standard color space such as RIMM RGB. For example, when only the device type of the image-capturing apparatus is described, it is possible that the image processing apparatus and image recording apparatus for executing this processing do not have a table showing correspondence between the device type name and the above-mentioned matrix coefficient. This cannot be said to be sufficient data. Further, even if there is no direct description of sufficient information for the execution of this processing, for example, the data can be said to be sufficient data if it contains description of the URL indicating the position of this information on the Internet. The “reproduction-auxiliary data based on which image-capturing characteristic compensation processing is applied” is preferred to be recorded as tag information to be written on the header of an image file.

[0092] If the above-mentioned “reproduction-auxiliary data based on which image-capturing characteristic compensation processing is applied” is stored in the medium, independently of the “scene-referred raw data”, information for associating the “reproduction-auxiliary data based on which image-capturing characteristic compensation processing is applied” with “scene-referred raw data” must be attached to both or either of them. Alternatively, a separate status information file containing the information on their relation must be attached to both or either of the two.

[0093] The “designating means for designating a degree of white-balance adjustment” (also referred to as designating section) denotes a structure that inputs a degree of white-balance adjustment to be applied to digital image data acquired by image-capturing operation continuously or in stepwise, corresponding to tastes of the photographer, from an operating switch or a liquid-crystal display having a touch panel equipped in the image-capturing apparatus. The structure of the “designating means for designating a degree of white-balance adjustment” can be anyone of various structures, such as an inputting unit coupled through an electric wire in a wired mode, an independent or a remote inputting unit coupled through a communication line or Internet, etc.

[0094] The term of “degree of white-balance adjustment” means at least a selected candidate in a range from the “non-compensation”, at which the color of the photographed light source is outputted as it is without applying the white-balance adjustment to the image signals acquired through an image-capturing element, to the “maximum compensation”, at which the white-balance adjustment is fully applied to the image signals at maximum. When the photographer wishes to remain the feeling of the photographed light source, he would select the “non-compensation”, while conversely, when the photographer wishes to completely compensate for the photographed light source, he would select the “maximum compensation”.

[0095] Further, the term of “designating” means that the photographer manually sets inputting data, by means of the operating switch or the liquid-crystal display having a touch panel equipped in the image-capturing apparatus as mentioned above.

[0096] Although it is applicable that the “data indicating said degree of said white-balance adjustment” (also referred to as white-balance data) are stored into the recording medium independently form the “scene-referred raw data”, it is specifically desirable that the white-balance data are stored in the image file as a form of tag information written into the header area.

[0097] In case that the “data indicating the degree of the white-balance adjustment” are stored into the recording medium independently form the “scene-referred raw data”, for at least one of the white-balance data and the “scene-referred raw data”, information correlating them with each other should be attached, or a status file including related information should be attached.

[0098] The “medium” is defined as a storage medium used to store “scene-referred raw data”, “reproduction-auxiliary data based on which image-capturing characteristic compensation processing is applied”, etc. It can be any one of the compact flash (registered trademark), memory stick, smart media, multi-media card, hard disk, floppy (registered trademark) disk, magnetic storage medium (MO) and CD-R. The unit for writing on the storage medium can be integral with the image-capturing apparatus, a wired write unit connected via a cord, or a wireless unit installed independently or at a remote site connected through a communications line or via the Internet. It is also possible to provide such a function that, when the image-capturing apparatus is connected with the write unit for writing on the storage medium, “reproduction-auxiliary data based on which image-capturing characteristic compensation processing is applied” and “required data” are read directly from the image processing apparatus or image recording apparatus. When “stored into a storage medium”, it is preferred that the standardized general-purpose file format such as TIFF, JPEG and Exif—not the format inherent to the image-capturing apparatus—be used.

[0099] (48) An image-processing apparatus, characterized in that

[0100] the image-processing apparatus is provided with:

[0101] an input means for inputting scene-referred raw data dependent on image-capturing characteristics of an image-capturing apparatus, reproduction-auxiliary data based on which image-capturing characteristic compensation processing is applied to the scene-referred raw data so as to generate the scene-referred image data standardized in respect to the scene-referred raw data, and data indicating a degree of white-balance adjustment;

[0102] a scene-referred image data generating means for generating the standardized scene-referred image data by applying image-capturing characteristic compensation processing to the scene-referred raw data inputted by the input means, based on the reproduction-auxiliary data for application of image-capturing characteristic compensation processing; and

[0103] an output data generating means for generating output data by attaching the data indicating the degree of the white-balance adjustment to the generated scene-referred image data.

[0104] The “input” described in item 48 indicates the process in which the “scene-referred raw data”, the “reproduction-auxiliary data based on which image-capturing characteristic compensation processing is applied” and the “data indicating the degree of the white-balance adjustment” are transmitted from an image-capturing apparatus to the image processing apparatus of the present invention.

[0105] For example, when a image-capturing apparatus is connected with the above-mentioned unit for writing into the storage medium, and the image processing apparatus has also a function of reading the “scene-referred raw data”, the “reproduction-auxiliary data based on which image-capturing characteristic compensation processing is applied” and the “data indicating the degree of the white-balance adjustment” directly from the image-capturing apparatus, then the image processing apparatus of the present invention has means for, connection with the image-capturing apparatus, and this connection means corresponds to the input section of the present invention. Further, when a portable “medium” such as a compact flash (registered trademark), memory stick, smart media, multi-media card, floppy (registered trademark) disk, photomagnetic storage medium (MO) or CD-R is used, then the image processing apparatus of the present invention has corresponding reading means, and this reading means corresponds to the input section of the present invention. Further, when the write unit is installed in a wireless state independently or at a remote site connected through a communications line or via the Internet, then the image processing apparatus of the present invention has communication means for connection with a communications line or the Internet, and this communications means corresponds to the input section of the present invention.

[0106] Further, in the embodiment of the present invention, it is applicable to employ the “data indicating the degree of the white-balance adjustment”, which are directly inputted from the software screen of the image-processing apparatus by the operator of the image-processing apparatus (sometimes, being different from the photographer concerned), without employing other “data indicating the degree of the white-balance adjustment”, which are designated by the photographer concerned and are inputted through the medium. In this case, it is applicable that only the “data indicating the degree of the white-balance adjustment” inputted from the image-processing apparatus by the operator are attached to the “scene-referred image data”, or both the “data indicating the degree of the white-balance adjustment” inputted from the image-processing apparatus and the other “data indicating the degree of the white-balance adjustment” designated by the photographer concerned are attached to the “scene-referred image data”.

[0107] (49) An image-capturing apparatus, characterized in that

[0108] the image-capturing apparatus is provided with:

[0109] a scene-referred raw data generating means for generating scene-referred raw data dependent on the image-capturing characteristics of the image-capturing apparatus by an image capturing operation;

[0110] a reproduction-auxiliary data generating means for generating reproduction-auxiliary data based on which image-capturing characteristic compensation processing is to be applied to the above-mentioned scene-referred image data generated by the above-mentioned scene-referred raw data generating means, so as to generate scene-referred raw data standardized in respect to the scene-referred raw data;

[0111] an image-capturing information data generating means for generating image-capturing information data being an image-capturing condition setting at a time of an image-capturing operation;

[0112] a designating means for designating a degree of white-balance adjustment; and

[0113] a storage controlling means for attaching the reproduction-auxiliary data generated by the reproduction-auxiliary data generating means, the image-capturing information data generated by the image-capturing information data generating means and data representing the degree of the white-balance adjustment designated by the designating means to the scene-referred raw data generated by the scene-referred raw data generating means, and further for storing them into a storage medium.

[0114] The “output-referred image data” (also referred to as “visual image referred image data”) denotes digital image data that is used by such a display device as CRT, liquid crystal display and plasma display, or by the output device for generation of a hard copy image on such an outputting medium as silver halide photographic paper, inkjet paper and thermal printing paper. The output-referred image data is provided with “optimization processing” in order to obtain the optimum image on such a display device as CRT, liquid crystal display and plasma display, or such an outputting medium as silver halide photographic paper, inkjet paper and thermal printing paper.

[0115] The “image-capturing data” (also referred to as “image-capturing information data”) described in the present invention is a record representing photographing conditions at a time of photographing. It may contain the same as the tag information written into the header of the Exif file. To put it more specifically, it denotes the tag (code) representing the exposure time, shutter speed, f-stop number (F number), ISO sensitivity, brightness value, subject distance range, light source, on/off status of a stroboscopic lamp, subject area, white balance, zoom scaling factor, subject configuration, photographing scene type, the amount of reflected light of the stroboscopic lamp source and color saturation for photographing.

[0116] The above-mentioned “image-capturing data” can be divided into (1) the value captured at a time of photographing by a sensor mounted on the camera for automating the exposure setting and focusing functions of the image-capturing apparatus, (2) the data obtained by processing the value captured by the sensor, and (3) photographing conditions of the camera set on the basis of the value captured by the sensor. In addition to these, it also includes the information manually set by a user on the photographing mode dial (e.g. portrait, sports and macro photographing mode) and the setting switch for forced lighting of a stroboscopic lamp.

[0117] The “image-capturing data” can be arranged to be stored on the medium independently of “scene-referred raw data”. It is particularly preferred to be recorded in the image file in the form of tag information that can be written on the header.

[0118] When the “image-capturing data” is configured to be stored in the medium independently of the “scene-referred raw data”, information for associating “image-capturing data” with “scene-referred raw data” must be attached to both or either of them. Alternatively, a separate status information file containing the information on their relation must be attached to both or either of the two.

[0119] (50) An image-processing apparatus, characterized in that

[0120] the image-processing apparatus is provided with:

[0121] an input means for inputting scene-referred raw data dependent on image-capturing characteristics of an image-capturing apparatus, reproduction-auxiliary data based on which image-capturing characteristic compensation processing is applied to the scene-referred raw data so as to generate the scene-referred image data standardized in respect to the scene-referred raw data, image-capturing information data being an image-capturing condition setting at a time of an image-capturing operation and data indicating a degree of white-balance adjustment;

[0122] a scene-referred image data generating means for generating the standardized scene-referred image data by applying image-capturing characteristic compensation processing to the scene-referred raw data inputted by the input means, based on the reproduction-auxiliary data for application of image-capturing characteristic compensation processing; and

[0123] an output data generating means for generating output data by attaching the image-capturing information data being the image-capturing condition setting at the time of the image-capturing operation and the data indicating the degree of the white-balance adjustment to the generated scene-referred image data.

[0124] (51) The image-processing apparatus, described in item 48 characterized in that

[0125] the image-processing apparatus is further provided with:

[0126] an output-referred image data generating means for generating output-referred image data, by applying an image-processing, contents of which are determined on the basis of the data indicating the degree of the white-balance adjustment and which optimizes an image quality of a visual image formed on an outputting medium, to the standardized scene-referred image data.

[0127] The “outputting medium” appearing in the present invention is defined as including such a display device as CRT, liquid crystal display and plasma display, or such paper for generation of a hard copy image as silver halide photographic paper, inkjet paper and thermal printing paper.

[0128] The “image-processing, which optimizes an image quality of a visual image formed on an outputting medium” is provided to ensure the optimum image on such display device as CRT, liquid crystal display and plasma display, or such an outputting medium as silver halide photographic paper, inkjet paper and thermal printing paper. For example, when display is given on the CRT display monitor conforming to the sRGB standard, processing is provided in such a way that the optimum color reproduction can be gained within the color gamut of the sRGB standard. When the data is to be outputted on silver halide photographic paper, processing is provided in such a way that the optimum color reproduction can be gained within the color gamut of silver halide photographic paper. In addition to compression of the above-mentioned color gamut, compression of gradation from 16 to 8 bits, reduction in the number of output pixels, and processing in response to the output characteristics (LUT) of the output device are also included. Further, it goes without saying that such processing as noise control, sharpening, white balance adjustment, color saturation adjustment or dodging is carried out.

[0129] Further, in the image-processing apparatus embodied in the present invention, it is applicable to employ the “data indicating the degree of the white-balance adjustment”, which are directly inputted from the image-processing apparatus by the operator of the image-processing apparatus, without employing other “data indicating the degree of the white-balance adjustment”, which are designated by the photographer concerned and are inputted through the medium. In addition, it is desirable that the “data indicating the degree of the white-balance adjustment” are attached to the “scene-referred image data” generated by the image-processing apparatus embodied in the present invention.

[0130] The “data indicating the degree of the white-balance adjustment” can be arranged to be stored on the medium independently of the “scene-referred raw data”, the “scene-referred image data” and the “output-referred image data”. It is particularly preferred to be recorded in the image file in the form of tag information that can be written on the header.

[0131] In case that the “data indicating the degree of the white-balance adjustment” are stored into the recording medium independently form the “scene-referred raw data”, the “scene-referred image data” and the “output-referred image data”, information correlating the “data indicating the degree of the white-balance adjustment” with one of the “scene-referred raw data”, the “scene-referred image data” and the “output-referred image data” should be attached to one of them or both of them, or a status file including related information should be attached.

[0132] It is applicable that only the “data indicating the degree of the white-balance adjustment” inputted from the image-processing apparatus by the operator are attached to the “scene-referred image data”, the “scene-referred image data” and the “output-referred image data”, or both the “data indicating the degree of the white-balance adjustment” designated by the photographer and inputted from the image-processing apparatus and the other “data indicating the degree of the white-balance adjustment” designated by the photographer concerned are attached to the “scene-referred image data”, the “scene-referred image data” and the “output-referred image data”.

[0133] (52) The image-processing apparatus, described in item 50 characterized in that

[0134] the image-processing apparatus is further provided with:

[0135] an output-referred image data generating means for generating output-referred image data, by applying an image-processing, contents of which are determined on the basis of the image-capturing information data and the data indicating the degree of the white-balance adjustment and which optimizes an image quality of a visual image formed on an outputting medium, to the standardized scene-referred image data.

[0136] The following shows an example of optimizing the output-referred image data using the “image-capturing data”:

[0137] “Subject configuration” information allows color saturation enhancement processing to be partially carried out, and permits dodging to be carried out for the scene containing an extensive dynamic range.

[0138] “Photographing scene type” allows special adjustment of color balance to be made by reducing the degree of white balance adjustment is loosened, in photographing of a night view, for example.

[0139] The distance between the photographer and subject can be estimated from the information in the “amount of reflected light from a stroboscopic light source”. The result can be effectively utilized in setting the conditions for image processing in order to reduce the white skipping of the skin of the subject, for example.

[0140] The information on “subject type” allows the degree of sharpness to be reduced and processing of smoothening to be enhanced in a people photograph, for example, thereby making the wrinkles on the skin less conspicuous.

[0141] In order to compensate for information on “image-capturing data”, “subject configuration”, “photographing scene type”, “amount of reflected light from a stroboscopic light source” and “subject type”, it is possible to use information on “exposure time”, “shutter speed”, “f-stop number (F number)”, “ISO sensitivity”, “brightness value”, “subject distance range”, “light source”, “on/off status of a stroboscopic lamp”, “subject area”, “white balance”, “zoom scaling factor”, etc. for supplementary purpose. Further, the amount of noise control processing can be adjusted based on the “ISO sensitivity” information, and the “light source” information can be used for readjustment of white balance.

[0142] (53) An image-recording apparatus, characterized in that

[0143] the image-recording apparatus is provided with:

[0144] an input means for inputting scene-referred raw data dependent on image-capturing characteristics of an image-capturing apparatus, reproduction-auxiliary data based on which image-capturing characteristic compensation processing is applied to the scene-referred raw data so as to generate the scene-referred image data standardized in respect to the scene-referred raw data, and data indicating a degree of white-balance adjustment;

[0145] a scene-referred image data generating means for generating the standardized scene-referred image data by applying image-capturing characteristic compensation processing to the scene-referred raw data inputted by the input means, based on the reproduction-auxiliary data for application of image-capturing characteristic compensation processing;

[0146] an output-referred image data generating means for generating output-referred image data, by applying an image-processing, contents of which are determined on the basis of the data indicating the degree of the white-balance adjustment and which optimizes an image quality of a visual image formed on an outputting medium, to the standardized scene-referred image data generated by the scene-referred image data generating means; and

[0147] an image-forming means for forming a visual image on a outputting medium by using the output-referred image data generated by the output-referred image data generating means.

[0148] (54) An image-recording apparatus, characterized in that

[0149] the image-recording apparatus is provided with:

[0150] an input means for inputting scene-referred raw data dependent on image-capturing characteristics of an image-capturing apparatus, reproduction-auxiliary data based on which image-capturing characteristic compensation processing is applied to the scene-referred raw data so as to generate the scene-referred image data standardized in respect to the scene-referred raw data, data indicating a degree of white-balance adjustment and image-capturing information data being an image-capturing condition setting at a time of an image-capturing operation;

[0151] a scene-referred image data generating means for generating the standardized scene-referred image data by applying image-capturing characteristic compensation processing to the scene-referred raw data inputted by the input means, based on the reproduction-auxiliary data for application of image-capturing characteristic compensation processing;

[0152] an output-referred image data generating means for generating output-referred image data, by applying an image-processing, contents of which are determined on the basis of the data indicating the degree of the white-balance adjustment and the image-capturing information data being the image-capturing condition setting at the time of the image-capturing operation, and which optimizes an image quality of a visual image formed on an outputting medium, to the standardized scene-referred image data generated by the scene-referred image data generating means; and

[0153] an image-forming means for forming a visual image on a outputting medium by using the output-referred image data generated by the output-referred image data generating means.

[0154] The image recording apparatus of the present invention can be equipped with a film scanner for inputting the frame image information of the photosensitive material recorded by an analog camera including a color film, color reversal film, black-and-white negative and black-and-white reversal film, or a flat head scanner for inputting the image information reproduced on color paper as silver halide photographic paper, in addition to the mechanism of applying image processing of the present invention to the digital image data obtained by the image-capturing apparatus of the present invention. It can be equipped with means for reading digital image data obtained by a digital camera other than the image-capturing apparatus of the present invention and stored in the known portable “medium” including a compact flash (registered trademark), memory stick, smart media, multimedia card, floppy (registered trademark) disk, photomagnetic storage medium (MO) or CD-R. Alternatively, it can be equipped with processing means for forming an output-referred image on a display device such as a CRT, liquid crystal display and plasma display, and a storage medium of any known type including silver halide photographic paper, inkjet paper and thermal printing paper, by obtaining digital image data from a remote place through communications means such as the Internet.

[0155] (55) The image-capturing apparatus, described in item 47 or item 49, characterized in that

[0156] the storage controlling means attaches a photographing EV value at a time of capturing the image to the scene-referred image data and stores them into the medium.

[0157] In the present invention, as described in item 55, it is desirable that the image-capturing apparatus itself calculates the “photographing EV value” at the time when the release switch of the image-capturing apparatus is depressed up to its half-stroke, and stores it with the “data indicating the degree of the white-balance adjustment” into the medium.

[0158] The “photographing EV value” can be arranged to be stored on the medium independently of the “scene-referred raw data” as well as the “data indicating the degree of the white-balance adjustment” and the “image-capturing information data”. It is particularly preferred to be recorded in the image file in the form of tag information that can be written on the header of the “scene-referred raw data”.

[0159] In case that the “photographing EV value” is stored into the recording medium independently of the “scene-referred raw data”, for at least one of the “photographing EV value” and the “scene-referred raw data”, information correlating them with each other should be attached, or a status file including related information should be attached.

[0160] (56) The image-processing apparatus, described in anyone of items 47, 50, 51 and 52, characterized in that

[0161] the input means input a photographing EV value at a time of capturing the image.

[0162] In the image-processing apparatus embodied in the present invention, it is applicable that the “photographing EV value” can be inputted from the image-capturing apparatus through the medium, or the “photographing EV value” can be inputted from the image-processing apparatus by the operator. Further, it is desirable that the “photographing EV value” is attached to the “scene-referred image data” and the “output-referred image data” generated by the image-processing apparatus embodied in the present invention. It is also possible to attach the “photographing EV value” to the “scene-referred raw data”. At this time, it is particularly preferred for the “photographing EV value” to be recorded in the image file in the form of tag information that can be written on each of the headers of the “scene-referred raw data”, the “scene-referred image data” and the “output-referred image data”.

[0163] In case that the “photographing EV value” is stored into the recording medium independently of the “scene-referred raw data”, the “scene-referred image data” and the “output-referred image data”, information correlating the “photographing EV value” with one of the “scene-referred raw data”, the “scene-referred image data” and the “output-referred image data” should be attached to one of them or both of them, or a status file including related information should be attached.

[0164] (57) The image-processing apparatus, described in anyone of items 51, 52 and 56, characterized in that

[0165] the image-processing apparatus is further provided with:

[0166] an applying amount determining means for determining an applying amount of the white-balance adjustment to be applied to the scene-referred image data, based on the data indicating the degree of the white-balance adjustment; and

[0167] the output-referred image data generating means has a white-balance adjusting means for applying the applying amount of the white-balance adjustment, determined in the above, to the scene-referred image data.

[0168] (58) The image-processing apparatus, described in item 57, characterized in that

[0169] the white-balance adjusting section is provided with an image area dividing means for dividing a whole image area of the scene-referred image data into a plurality of small image areas.

[0170] (59) The image-processing apparatus, described in item 58, characterized in that

[0171] the white-balance adjusting section is provided with a R/G, B/G calculating means for finding a R/G ratio between an integrated value of R (Red) signals and another integrated value of G (Green) signals, and a B/G ratio between an integrated value of B (Blue) signals and another integrated value of G (Green) signals, for a respective one of the plurality of small image areas.

[0172] (60) The image-processing apparatus, described in item 59, characterized in that

[0173] the white-balance adjusting section is provided with a first light-source judging means for estimating a kind of a photographic light source for a respective one of the plurality of small image areas, by plotting the R/G and the B/G, calculated in the above, onto a light-source estimating map in which light-source area frames, indicating ranges of various combinations of the R/G and the B/G corresponding to various kinds of photographing light-sources.

[0174] (61) The image-processing apparatus, described in item 60, characterized in that

[0175] the white-balance adjusting section is provided with a second light-source judging means for judging a kind of a photographic light source, under which the scene-referred image data are acquired, by employing a number of small image areas plotted within one of the light-source area frames, or a membership function in which a photographic EV value is a variable.

[0176] (62) The image-processing apparatus, described in item 58, characterized in that

[0177] the white-balance adjusting section is provided with a color-temperature estimating means for estimating a color temperature of a photographic light source for a respective one of the plurality of small image areas by employing a least squares method.

[0178] (63) The image-processing apparatus, described in item 62, characterized in that

[0179] the white-balance adjusting section is provided with a histogram creating means for creating a histogram based on an emerging frequency of the color temperature of the photographic light source for a respective one of the plurality of small image areas.

[0180] (64) The image-processing apparatus, described in item 63, characterized in that

[0181] the white-balance adjusting section is provided with a group-wise white-balance adjusting means for dividing the whole image area of the scene-referred image data into equal to or more than two groups, based on the histogram created in the above, so as to apply a white-balance adjustment, being different for every group, to a respective one of the plural groups.

[0182] (65) The image-processing apparatus, described in anyone of items 57-64, characterized in that

[0183] the applying amount determining section can arbitrarily establish a relationship between the data indicating the degree of the white-balance adjustment, and an applying amount of the white-balance adjustment in practice.

[0184] (66) The image-recording apparatus, described in item 53 or item 54, characterized in that

[0185] the input means inputs a photographing EV value at a time of capturing the image.

[0186] In the image-recording apparatus embodied in the present invention, it is applicable that the “photographing EV value” is inputted for use from the image-capturing apparatus and the image-processing apparatus, or the “photographing EV value” is inputted by the operator from the image-recording apparatus.

[0187] (67) The image-recording apparatus, described in anyone of items 53, 54 and 66, characterized in that

[0188] the image-recording apparatus is further provided with:

[0189] an applying amount determining means for determining an applying amount of the white-balance adjustment to be applied to the scene-referred image data, based on the data indicating the degree of the white-balance adjustment; and

[0190] the output-referred image data generating means has a white-balance adjusting means for applying the applying amount of the white-balance adjustment, determined in the above, to the scene-referred image data.

[0191] (68) The image-recording apparatus, described in item 67, characterized in that

[0192] the white-balance adjusting section is provided with an image area dividing means for dividing a whole image area of the scene-referred image data into a plurality of small image areas.

[0193] (69) The image-recording apparatus, described in item 68, characterized in that

[0194] the white-balance adjusting section is provided with a R/G, B/G calculating means for finding a R/G ratio between an integrated value of R (Red) signals and another integrated value of G (Green) signals, and a B/G ratio between an integrated value of B (Blue) signals and another integrated value of G (Green) signals, for a respective one of the plurality of small image areas.

[0195] (70) The image-recording apparatus, described in item 69, characterized in that

[0196] the white-balance adjusting section is provided with a first light-source judging means for estimating a kind of a photographic light source for a respective one of the plurality of small image areas, by plotting the R/G and the B/G, calculated in the above, onto a light-source estimating map in which light-source area frames, indicating ranges of various combinations of the R/G and the B/G corresponding to various kinds of photographing light-sources.

[0197] (71) The image-recording apparatus, described in item 70, characterized in that

[0198] the white-balance adjusting section is provided with a second light-source judging means for judging a kind of a photographic light source, under which the scene-referred image data are acquired, by employing a number of small image areas plotted within one of the light-source area frames, or a membership function in which a photographic EV value is a variable.

[0199] (72) The image-recording apparatus, described in item 68, characterized in that

[0200] the white-balance adjusting section is provided with a color-temperature estimating means for estimating a color temperature of a photographic light source for a respective one of the plurality of small image areas by employing a least squares method.

[0201] (73) The image-recording apparatus, described in item 72, characterized in that

[0202] the white-balance adjusting section is provided with a histogram creating means for creating a histogram based on an emerging frequency of the color temperature of the photographic light source for a respective one of the plurality of small image areas.

[0203] (74) The image-recording apparatus, described in item 73, characterized in that

[0204] the white-balance adjusting section is provided with a group-wise white-balance adjusting means for dividing the whole image area of the scene-referred image data into equal to or more than two groups, based on the histogram created in the above, so as to apply a white-balance adjustment, being different for every group, to a respective one of the plural groups.

[0205] (75) The image-recording apparatus, described in anyone of items 67-74, characterized in that

[0206] the applying amount determining section can arbitrarily establish a relationship between the data indicating the degree of the white-balance adjustment, and an applying amount of the white-balance adjustment in practice.

[0207] (76) An image-processing method, characterized in that

[0208] the image-processing method includes:

[0209] a scene-referred image data generating process for generating the standardized scene-referred image data by applying an image-capturing characteristic compensation processing to the scene-referred raw data, based on reproduction-auxiliary data for application of the image-capturing characteristic compensation processing; and

[0210] an output-referred image data generating process for generating output-referred image data, by applying an image-processing, contents of which are determined on the basis of data indicating the degree of the white-balance adjustment and which optimizes an image quality of a visual image formed on an outputting medium, to the scene-referred image data.

[0211] (77) An image-processing method, characterized in that

[0212] the image-processing method includes:

[0213] a scene-referred image data generating process for generating the standardized scene-referred image data by applying an image-capturing characteristic compensation processing to the scene-referred raw data, based on reproduction-auxiliary data for application of the image-capturing characteristic compensation processing; and

[0214] an output-referred image data generating process for generating output-referred image data, by applying an image-processing, contents of which are determined on the basis of image-capturing information data and data indicating the degree of the white-balance adjustment and which optimizes an image quality of a visual image formed on an outputting medium, to the scene-referred image data.

[0215] (78) The image-processing method, described in item 76 or item 77 , characterized in that

[0216] the image-processing method includes:

[0217] an applying amount determining process for determining an applying amount of the white-balance adjustment, based on the data indicating the degree of the white-balance adjustment; and

[0218] a white-balance adjusting process for applying the applying amount of the white-balance adjustment, determined in the above, to the scene-referred image data.

[0219] (79) The image-processing method, described in item 78, characterized in that

[0220] the white-balance adjusting process includes an image area dividing process for dividing a whole image area of the scene-referred image data into a plurality of small image areas.

[0221] (80) The image-processing method, described in item 79, characterized in that

[0222] the white-balance adjusting process includes a R/G, B/G calculating process for finding a R/G ratio between an integrated value of R (Red) signals and another int