Plaque It!
|
| 4965619 | Image stabilizing device | October, 1990 | Shikaumi et al. | 396/55 |
| 5479236 | Image stabilizing apparatus | December, 1995 | Tanaka | 396/55 |
| 5946503 | Image processing apparatus wherein an image stabilizing device is also used for another operation | August, 1999 | Washisu | 396/55 |
| 5974268 | Image blur prevention apparatus | October, 1999 | Washisu | 396/55 |
| 6091448 | Apparatus for preventing an image blur prevention device from changing states during an image recording operation | July, 2000 | Washisu et al. | 348/208 |
| 6154611 | Image-shake compensation apparatus | November, 2000 | Washisu | 396/55 |
| 6332060 | Camera with vibration compensation device varying the compensating action in accordance with the focal length of a lens and the distance to the subject | December, 2001 | Miyamoto et al. | 396/55 |
| 6670986 | Apparatus for orthogonal movement of a CCD sensor and a method of light sampling therewith | December, 2003 | Ben Shoshan et al. | 348/219.1 |
| 7239342 | Image processing apparatus and method | July, 2007 | Kingetsu et al. | 348/208.4 |
| 7268336 | Image detecting module | September, 2007 | Hsieh et al. | 250/208.1 |
| 20020163581 | Imaging apparatus, and method and device for shake correction in imaging apparatus | November, 2002 | Kitazawa et al. | 348/208.6 |
| JP0H41992-95932 | March, 1992 | |||
| JP0H91997-261526 | October, 1997 | |||
| JP2720955 | November, 1997 | |||
| JP20047220 | January, 2004 |
1. An image-taking apparatus comprising: an image pickup element which photoelectrically converts an object image formed by an image-taking optical system into an electrical signal; a detection sensor which detects vibration; and a control circuit which controls driving of a correcting lens unit being disposed in the image-taking optical system and corrects image blurring by moving the correcting lens unit within a plane substantially orthogonal to an optical axis of the image-taking optical system, wherein in a case where it is not possible to move the correcting lens unit according to a result of the detection by the detection sensor after starting an image-taking operation, the control circuit (a) stops the correcting lens unit at a present position, (b) stops a first exposure of the image pickup element in the image-taking operation to generate first image data based on an output signal of the image pickup element before stopping the correcting lens unit, (c) carries out successively a second exposure at a predetermined period while the correcting lens unit is positioned at the present position to generate at least one second image data to be synthesized together with the first image data, and (d) stops the second exposure when total period of the first and second exposures reaches a proper exposure period determined before the image-taking operation.
2. The image-taking apparatus according to claim 1, wherein in a case where a remaining period representing a difference between the proper exposure period and a period of the first exposure is shorter than a predetermined period, the control circuit prohibits the restart of the image-taking operation the second exposure.
3. The image-taking apparatus according to claim 1, further comprising: a detection circuit which detects a displacement between a reference image and another image among the first and second image data; a coordinate conversion circuit which applies coordinate conversion to the another image so as to conform the another image to the reference image based on a result of the detection by the detection circuit; and a synthesis circuit which produces a synthesized image by synthesizing the another image to which the coordinate conversion is applied and the reference image.
4. The image-taking apparatus according to claim 1, further comprising: a recording circuit which records the second image data on a recording medium in a form enabling identification.
5. The image-taking apparatus according to claim 3, wherein the reference image is an image obtained through exposure with illumination light.
6. The image-taking apparatus according to claim 3, wherein in a stage of image-taking, the control circuit can start the second exposure according to the result of the detection by the detection sensor, and in a stage of object observation, the control circuit retains the correcting lens unit at the reference position and changes a region read out in each image obtained by using the image pickup element.
7. An image-taking system comprising: the image-taking apparatus according to claim 1; and a lens apparatus which is attached to the image-taking apparatus and has the image-taking optical system including the correcting lens unit.
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates to an image-taking apparatus which improves accuracy of taken images by correcting image blurring caused by vibration.
2. Description of Related Art
In cameras currently used, operations important for image-taking operation such as exposure determination and focusing, etc., are all automated, and it is very rare that even a person who is not skilled in the camera operations fails image-taking. Recently, a system that represses vibration applied to the camera has also been studied and factors that cause a photographer to fail image-taking have almost been eliminated.
Herein, a vibration control system that represses image blurring (image deflection on an image surface) is briefly described. Vibration of a camera when taking an image is normally vibration with a frequency of 1 Hz through 10 Hz, and as a basic idea that makes it possible to take an image without image blurring even when such vibration occurs during exposure, when an optical vibration control system is taken as an example, it is required that vibration of a camera due to hand vibration is detected and according to the results of this detection a correcting optical unit (comprising a correcting lens and a supporting frame, etc.) is displaced within an optical axis orthogonal plane. Namely, in order to take an image without image blurring even when the camera vibrates, first, the camera vibration must be accurately detected, and second, the optical axis change due to the vibration must be corrected.
In principle, image blurring can be corrected by mounting a vibration detection unit which detects the acceleration, the angular acceleration, the angular velocity, and the angular displacement, by using an accelerometer, vibration gyro, or laser gyro, and carries out proper calculation with respect to the results of this detection, for the camera. Image blurring is corrected by driving a correcting optical unit that makes the image-taking optical axis eccentric depending on the camera vibration detection data supplied from the vibration detection unit.
As the vibration control system, in addition to the above-described optical vibration control system, an electronic vibration control system (electronic correcting system) has also been proposed which repeats image-taking operation a plurality of times with an exposure period the length of which does not cause image blurring, and synthesizes images taken by the plurality of times of image-taking operations while correcting deviations of the images to obtain a taken image (synthesized image) with a long exposure period. This technique has been disclosed in Japanese Patent Application Laid-Open No. H9(1997)-261526.
Recent digital cameras have become smaller than silver-halide compact cameras, and have been so reduced in size that they can be built in a portable electronic apparatus (for example, a portable phone). Under these circumstances, in order to install the above-described optical vibration control system in the camera, the correcting optical unit must be made smaller. However, there is a limitation in downsizing the correcting optical unit since it must support a correcting lens and drive the correcting lens with high accuracy in a large stroke according to vibration.
On the other hand, in the electronic vibration control system disclosed in Japanese Patent Application Laid-Open No. H9 (1997)-261526, the exclusive correcting lens, etc., used in the above-described optical vibration control system becomes unnecessary, so that the entire product can be downsized. Furthermore, the method in which the exposure period for one time image-taking operation is shortened and images obtained through a plurality of times of image-taking operations are synthesized has an advantage in that the vibration control accuracy increases as the exposure period for one time image-taking operation becomes shorter. However, as the exposure period for one time image-taking operation becomes shorter, underexposure increases and image data accuracy lowers, so that it is not possible to obtain an excellent image. In addition, in this method, deviations among the images are detected from characteristic point changes on the images and are corrected, however, in a case where the image exposure is not sufficient at all, characteristic points on the images cannot be reliably identified, and accuracy of correction of deviations among the images lowers and it becomes impossible to synthesize an excellent image. Namely, in the electronic vibration control system, a great vibration control effect cannot be expected.
SUMMARY OF THE INVENTION
One aspect of an image-taking apparatus according to the invention comprises an image pickup element which photoelectrically converts an object image formed by an image-taking optical system; a detection sensor which detects vibration; and a control circuit which controls driving of a correcting lens unit being disposed in the image-taking optical system and correcting image blurring by moving within a plane substantially orthogonal to the optical axis. Here, in a case where it is not possible to move the correcting lens unit according to a result of the detection by the detection sensor during an image-taking operation, the control circuit stops the image-taking operation, and restarts the image-taking operation after moving the correcting lens unit to a reference position.
Another aspect of an image-taking apparatus comprises an image pickup element which photoelectrically converts an object image formed by an image-taking optical system, a drive mechanism which moves the image pickup element within a plane substantially orthogonal to the optical axis, a detection sensor which detects vibration, and a control circuit which corrects image blurring by controlling driving of the drive mechanism based on a result of the detection by the detection sensor. Here, in a case where it is not possible to move the image pickup element according to a result of the detection by the detection sensor during an image-taking operation, the control circuit stops the image-taking operation, and restarts the image-taking operation after moving the image pickup element to a reference position.
The characteristics of the image-taking apparatus of the invention will be clarified by the following detailed description of embodiments with reference to the drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a diagram showing the structure of a camera according to Embodiment 1 of the invention.
FIG. 2 is a diagram showing driving waveforms of a correcting optical unit in Embodiment 1 of the invention.
FIG. 3 is a diagram for explaining a coordinate conversion processing in Embodiment 1 of the invention.
FIGS. 4A and 4B are diagrams for explaining a characteristic extraction region in Embodiment 1 of the invention.
FIG. 5 is a diagram for explaining an image synthesis processing in Embodiment 1 of the invention.
FIGS. 6A and 6B are flowcharts showing image-taking operations in Embodiment 1 of the invention.
FIG. 7 is a timing chart showing image-taking processing in Embodiment 2 of the invention.
FIGS. 8A and 8B are flowcharts showing image-taking operations in Embodiment 2 of the invention.
FIG. 9 is a diagram showing a connection state of a personal computer and a camera in Embodiment 3 of the invention.
FIG. 10 is a diagram showing a file storing status in Embodiment 3 of the invention.
FIG. 11 is a diagram showing a file organizing status in Embodiment 3 of the invention.
FIGS. 12A to 12C are flowcharts showing image-taking operations in Embodiment 3.
FIG. 13 is a flowchart showing image synthesis processing in Embodiment 3 of the invention.
FIG. 14 is a diagram showing a structure of a camera according to Embodiment 4.
FIGS. 15A to 15C are structural diagrams of a correcting optical unit using an image pickup element in Embodiment 4 of the invention.
FIG. 16 is a structural diagram of a correcting optical unit using an image pickup element in Embodiment 5 of the invention.
FIG. 17 is a structural diagram showing a structure of a camera according to Embodiment 5 of the invention.
FIGS. 18A to 18C are flowcharts showing image-taking operations in Embodiment 5 of the invention.
FIG. 19 is a diagram showing a file storing status in Embodiment 5 of the invention.
FIG. 20 is a flowchart showing a image synthesis processing in Embodiment 5 of the invention.
FIG. 21 is a flowchart showing receiving processing of the camera according to Embodiment 5 of the invention.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
Embodiment 1
FIG. 1 is a schematic view showing a structure of a camera (image-taking apparatus) according to Embodiment 1 of the present invention. In FIG. 1, an incident light flux (image-taking light) from image-taking lenses 11 a and 11 b is limited in light amount by a stop 13 a , and then passes through a shutter 12 a , and the image thereof is formed on an image pickup element 19 a . The image-taking lens 11 b is a correcting lens which corrects image blurring by moving within a plane orthogonal to the optical axis L. The correcting lens 11 b and a correcting lens drive mechanism 11 c constitute a correcting optical unit 11 d . An image pickup element 19 a consists of an MOS or a CCD, and is driven by an image pickup device driving circuit 19 b.
The image-taking lenses 11 a and 11 b are further formed of a plurality of optical lens units, and a part or all of these lens units move in the optical axis L direction by receiving a driving force from an AF drive motor 14 a and stop at predetermined in-focus positions to carry out focusing. The AF drive motor 14 a is driven by receiving a drive signal from a focusing drive circuit 14 b . Furthermore, a part of the optical lens units of the image-taking lenses move in the optical axis L direction by receiving a driving force from a zooming drive motor 15 a and stop at predetermined zooming positions to change the angle of image-taking field. The zooming drive motor 15 a is driven by receiving a drive signal from a zooming drive circuit 15 b.
A vibration detection sensor 11 f is formed of a vibration gyro, etc., which detects the angular velocity of vibration applied to the camera, and an output of the vibration detection sensor 11 f is input into an image-taking control circuit 18 described later. The correcting optical unit 11 d is driven by a image blurring correcting drive circuit 11 e that is controlled by the image-taking control circuit 18 to repress image blurring. The stop 13 a has a plurality of stop blades, and these stop blades move by receiving a driving force from a stop drive unit 13 b to change the aperture area (stop diameter) of a light passing aperture. The shutter 12 a has a plurality of shutter blades, and these shutter blades move with respect to an aperture through which the image-taking light flux pass by receiving a driving force from a shutter drive unit 12 b to control the light amount of the light flux to be made incident on the image pickup element 19 a . Furthermore, according to conditions (object luminance, etc.) when taking an image, an illumination unit 16 a is driven (emits light) by receiving a drive signal from an illumination drive circuit 16 b . Furthermore, a speaker 17 a is driven (produces a sound) by receiving a drive signal from a sound producing drive circuit 17 b for informing a photographer of the image-taking operations.
The driving of the image blurring correcting drive circuit 11 e , the focusing drive circuit 14 b , the zooming drive circuit 15 b , the stop drive unit 13 b , the shutter drive unit 12 b , the illumination drive circuit 16 b , and the sound producing drive circuit 17 b is controlled by the image-taking control circuit 18 , respectively.
Into the image-taking control circuit 18 , operation signals from a release operation member 12 c , a stop operation member 13 c , a zooming operation member 15 c , a illumination operation member 16 c , and a vibration control operation member 120 described later are input, and according to the image-taking status of the camera, the operation signals are supplied to the focusing drive circuit 14 b , the zooming drive circuit 15 b , the stop drive unit 13 b , the shutter drive unit 12 b , and the illumination drive circuit 16 b and the image-taking conditions are set, and thereafter, image-taking operation is carried out.
Furthermore, the aperture diameter of the stop 13 a and the emission of the illumination unit 16 a are normally automatically set by the camera side when taking an image, so that the stop operation member 13 c and the flashing drive circuit 16 b are not necessary, however, they are provided so that a photographer can arbitrarily set the aperture diameter and the emission.
The image-taking control circuit 18 measures the object luminance (photometry) on the basis of the image signals taken-in by a signal processing circuit 111 described later, and sets the stop diameter of the stop 13 a and the closing timing of the shutter 12 a (exposure period) on the basis of the results of this photometry. The image-taking control circuit 18 calculates the in-focus position of the image-taking lens (focusing lens) 11 a on the basis of an output from the signal processing circuit 111 while driving the focusing drive circuit 14 b.
Signals output from the image pickup element 19 a are converted into digital signals by an A/D converter 110 and input into the signal processing circuit 111 . The signal processing circuit 111 applies signal processing to the input signals to form luminance signals or color signals, thereby forming color video signals. Thereafter, the video signals produced by the signal processing circuit 111 are input into a second image correcting circuit 117 b via a signal switching circuit 112 . In the second image correcting circuit 117 b , gamma correction processing and compression processing are applied to the input signals. Output signals of the second image correcting circuit 117 b are input into a display unit 118 and a recording circuit 119 , whereby the taken image is displayed on the display unit 118 and recorded on the recording circuit 119 .
Herein, when a photographer depresses halfway the release operation member 12 c as a release button (turns sw1 on) in a case where a vibration control mode is set by operating the vibration control operation member 120 , the vibration detection sensor 11 f starts operating and data on vibration applied to the camera is input into the image-taking control circuit 18 . For example, when an angular velocity meter such as a vibration gyro is used as the vibration detection sensor 11 f , signals in which a low band (for example, a band equal to and lower than 0.2 Hz) has been attenuated in the vibration angular velocity data are input into the image-taking control circuit 18 . This is for eliminating the low frequency band of the angular velocity meter since the low frequency band is low in signal accuracy
The input angular velocity data is subjected to proper signal processing such as integration processing by the image-taking control circuit 18 and then output to the image blurring correcting circuit 11 e . The image blurring correcting circuit 11 e drives the correcting optical unit 11 d according to the input signals. Thereby, image blurring due to vibration applied to the camera is optically repressed.
Furthermore, when a photographer depresses halfway the release operation member 12 c , in addition to the abovementioned operations, image-taking preparation operation (including focusing and photometry) is started. Namely, when the image-taking preparation operation such as photometry and focusing is carried out, since optical vibration control has already acted, it becomes easy to capture an object and focusing accuracy is also increased. Then, on the basis of the photometric value obtained by the photometry, the closing timing of the shutter 12 a (exposure period) and the stop diameter of the stop 13 a are set. The diameter of the stop 13 a is changed to a proper diameter at this point.
Next, when the photographer fully depresses the release operation member 12 c (turns the sw2 on), exposure for image recording is started. Concretely, when the photographer fully depresses the release operation member 12 c , charges accumulating in the image pickup element 19 are all temporarily reset, and charge storage operation is started again with a set image size. Then, at the point where the exposure period elapses, which was determined through photometry when the release operation member 12 c was depressed halfway, the shutter 12 a is closed, and while the shutter 12 a is closed, charge transfer is carried out and after charge transfer, the shutter 12 a is opened. Herein, the correcting optical unit 11 d is continuously driven during exposure. In a case where the exposure period extends long (for example, ¼ seconds) and a great vibration occurs, the correction stroke of the correcting optical unit 11 d (the correcting lens 11 b ) may become deficient.
FIG. 2 shows waveforms for explaining this, wherein the horizontal axis 11 g indicates the elapsed time and the vertical axis 11 h indicates the amount of vibration or the amount of movement of the correcting optical unit 11 d . Herein, the actual vibration waveform 11 j within the total exposure period 11 i exceeds the correction range 11 l of the correcting optical unit 11 d after the timing 11 k . Therefore, in exposure after the timing 11 k , optical vibration control becomes impossible, and therefore, an image with blur is generated.
In this embodiment, when the optical vibration control becomes impossible as described above, exposure is temporarily stopped and the correcting optical unit 11 d (the correcting lens 11 b ) is returned to its initial position, and then exposure is started again, and finally, a plurality of taken images are synthesized to compensate underexposure.
Hereinafter, a detailed description is given. In FIG. 2, when the vibration exceeds the correction range 11 l of the correcting optical unit 11 d at the timing 11 k , the shutter 12 a is closed at this point and accumulated charges are readout, and the correcting optical unit 12 a is returned to its initial position as shown by the waveform 11 q . Here, the period 11 n denotes the time required for reading out and initialization of the correcting optical unit 11 d . Thereafter, the shutter 12 a is opened again and charge storage operation is started again, and then the correcting optical unit 11 d is caused to perform vibration control operation.
The second time exposure period (second exposure period 11 o ) is a remaining exposure period obtain by subtracting the first exposure period 11 m during which exposure has already been carried out from the total exposure period 11 i that provides proper exposure. Then, after the second exposure period elapses, the shutter 12 a is closed and the accumulated charges are readout, and then the shutter 12 a is opened and the exposure sequence is finished. In addition, when the vibration exceeds the correction range 11 l of the correcting optical unit 11 d within the second exposure period, exposure stopping, accumulated charge reading out, initialization of the correcting optical unit 11 d , and exposure are repeated. Storing of the image data obtained by each exposure into a memory is carried out simultaneously with the abovementioned series of operations. When the image-taking operations are continued as mentioned above, the number of taken images changes according to the vibration magnitude, that is, the number of times of the vibration exceeded the correcting range 11 l.
When vibration is slight or the total exposure period 11 i is not so long, image-taking operation is finished without transition to re-image-taking operation so that the operation range of the correcting optical unit 11 d does not exceed the correction range 11 l during exposure. On the other hand, in a case where a great vibration, that is, vibration that cannot be corrected by the correcting operation of the correcting optical unit 11 d frequently occurs, image-taking operation is carried out a plurality of times. After finishing image-taking operation a plurality of times, a plurality of taken images are stored in the memory.
The method of processing with respect to the plurality of images is described below.
As described above, in a case where the total exposure period is divided into the plurality of exposure period by the vibration exceeding the correction range 11 l of the correcting optical unit 11 d , each image obtained through each exposure is underexposed, however, vibration influence on the images is small. Then, exposure is improved by synthesizing the plurality of images into one image after finishing image-taking operation. However, when a plurality of images are taken, even when the images obtained through a plurality of times of image-taking operations are not influenced by vibration, the compositions in the respective images may slightly deviate due to the vibration during continuous image-taking operations. Therefore, if these images are synthesized as they are, the synthesized image is blurred due to the composition deviations among the images.
In response to the above-described continuous image-taking operations, plurality of image signals which are output for each image-taking operation from the image pickup element 19 a are subjected to signal processing in the signal processing circuit 111 after being converted into digital signals by the A/D converter 110 .
When the image-taking control circuit 18 is informed that the vibration control system is to be started by operating the vibration control operation member 120 and the vibration nearly exceeds the correction range 11 l of the correcting optical unit 11 d during exposure, the image data from the signal processing circuit 111 is input into an image storing circuit 113 via the signal switching circuit 112 (see FIG. 1). Namely, inputting into the second image correcting circuit 117 b is interrupted. The image storing circuit 113 stores the entire plurality of taken images. A displacement detection circuit 114 extracts characteristic points in the images stored from the respective image storing circuit 113 and calculates the position coordinates of the characteristic points within the image-taking plane.
For example, as shown in FIG. 3, in the frame 121 a , it is assumed that an image of a person 122 a standing against a building 123 a is taken. In this case, when vibration exceeding the correction range 11 l of the correcting optical unit 11 d occurs and a plurality of times of image-taking operations are carried out, like the frame 121 b , an image in which the composition deviates from the frame 121 a may be taken.
The displacement detection circuit 114 extracts the edge 125 a of a window 124 a that is a point with high luminance in the building 123 a positioned on the periphery of the image plane as a characteristic point by means of edge detection. Then, the characteristic point 125 a is compared with the characteristic point 125 b (the edge of a window 124 b ) in the frame 121 b corresponding to the characteristic point 125 a , and the difference between these is corrected (coordinate conversion). In FIG. 3, the coordinates of the frame 121 b are converted so that the characteristic point 125 b of the frame 121 b conforms the characteristic point 125 a of the frame 121 a as shown by the arrow 126 . Herein, the reason why the characteristic point is extracted from the periphery of the image-taking plane is described below.
In most of image-taking, a main object is positioned close to the center of the image plane and the main object is a person. In such a case, selection of the main object as a characteristic point poses a problem due to vibration of the main object. Namely, when image-taking is carried out a plurality of times, not only camera shaking but also object vibration influence the images obtained through the plurality of times of image-taking operations, so that the coordinate conversion processing is applied to the images on the basis of the object vibration. In this case, it seems that a preferable image is obtained since the coordinate conversion processing is applied so as to make the composition of the main object proper, however, the motion of a person is generally complicated and deviation detection accuracy is greatly influenced by the location where a characteristic point is extracted. For example, when an eye of a main object (person) is selected as a characteristic point, blinking influences, and when the tip of a hand is selected as a characteristic point, the hand easily moves and causes vibration different from the actual vibration of the entire main object. Even when image coordinate conversion processing is carried out by thus using a point of a person as a characteristic point, the entirety of the person is not always properly converted in coordinates, and in the case where a plurality of images are subjected to coordinate conversion processing and synthesized, the positions of the coordinates in the images scatter, so that a preferable image cannot be obtained.
Therefore, a more preferable image is obtained by selecting a motionless object such as a background as a characteristic point and applying image coordinate conversion processing as in this embodiment. However, in this case, the influence of the abovementioned object vibration appears.
Therefore, in this embodiment, for only the first one of the plurality of frames to be taken, the object is irradiated with light of the illumination unit 16 a . Namely, the illumination unit 16 a is made to emit light at the time of the first image-taking operation, and thereafter, when the second image-taking operation and subsequent image-taking operation are carried out in response to vibration exceeding the correction range 11 l of the correcting optical unit 11 d , the illumination unit 16 a is not made to emit light. Herein, the image taken by using the illumination unit 16 a is defined as the first image, and a plurality of images taken without using the illumination unit 16 a are defined as a second image group. In this case, there is the following difference between the first image and the respective images of the second image group in addition to the composition deviations described above. That is, the brightness in the object region which the illumination light has reached in the first image is different from that of the same region in the respective images of the second image group. For the object which the illumination light has reached in the first image, sufficient exposure is obtained, and exposure of the background which the illumination light has not reached is insufficient. The reason for this is that, generally, the illumination light reaches the main object such as a person since the main object is near the camera, and on the other hand, the illumination light does not reach the background since the background is far from the camera. The underexposure of the background is compensated by synthesis of the first image and the second image group while correcting the composition deviations of the second image group from the composition of the first image.
FIGS. 4A and 4B are diaphragm for explaining the selecting method for an extraction region of a characteristic point by the displacement detection circuit 114 . When the first image 127 taken by using the illumination unit 16 a and the second image group (one image 128 is shown as an example) taken without using the illumination unit 16 a are compared, the illumination light has reached the person 122 a in the first image 127 , and the person 122 b in the second image 128 is dark since the illumination light has not reached it. On the other hand, in the background which the illumination light has not reached, the characteristic points in the buildings 123 a and 123 b have no change in brightness between the first image 127 and the second image 128 .
Thus, in the background region without change in brightness, the illumination light does not reach and exposure is insufficient, so that this background region is considered as a point of image synthesis, and composition deviations are corrected by selecting this background region as the characteristic point extraction region.
In FIG. 4, in the buildings 123 a and 123 b at the periphery of the image plane without a change in brightness between the first image 127 and the second image 128 as described above, the edges 125 a and 125 b of the window with high luminance are extracted as characteristic points by means of edge detection. Then, as described in FIG. 3, the characteristic point 125 a in the first image 127 and the characteristic point 125 b in the second image 128 are compared, and the difference between these is corrected (coordinate conversion). Namely, the coordinate conversion circuit 115 applies coordinate conversion processing to the second image 128 so that the characteristic point 125 b of the second image 128 conforms the characteristic point 125 a of the first image 127 . Then, for the images of the second image group subsequent from the second one, coordinates of the characteristic points 125 b are calculated respectively, and the coordinate conversion circuit 115 converts the coordinates of the images (second image group) so that the calculated coordinates of the characteristic points 125 b conforms the coordinates of the characteristic point 125 a in the first image 127 .
Herein, the coordinates of the characteristic points of the respective images are calculated for the sake of explanation, however, in actuality, the first image 127 and the first one 128 of the second image group are subjected to correlation calculation processing, and the displacement detection circuit 114 calculates a change of pixels corresponding to each other among images as a motion vector and uses them as a characteristic point change. Then, for the images subsequent from the second one of the second image group, characteristic point changes are also calculated by means of correlation calculation processing with the first image 127 , and thereafter, characteristic point changes of the images are determined in the same manner.
It is also possible that, in place of selection of only one point as a characteristic point, a plurality of points are selected and an average or the minimum scalar of motion vector of these points is used as a characteristic point change. Use of the minimum value as the characteristic point change is for selecting the most immovable characteristic point since a characteristic point selected at the periphery of the image plane may move by itself.
The images subjected to coordinate conversion processing by the coordinate conversion circuit 115 are output to an image synthesis circuit 116 and synthesized into one image. Thus, in this embodiment, the first image 127 taken by using the illumination unit 16 a is used as a reference (center), and the respective images of the second image group 128 are subjected to coordinate conversion processing so as to conform the first image.
Herein, the reason why the first image 127 is a reference is described.
When two images with compositions deviating from each other as shown in FIG. 4 are synthesized, a region 129 in which the two images do not conform may be created as shown in FIG. 5. The image synthesis circuit 116 cuts the region 129 and complements by spreading only the region in which the two images conform each other to the original frame size. Therefore, in the images of the second image group, the periphery of the image plane is cut according to the direction and degree of the composition deviation. In the first image 127 and the second image group, the most excellent image is the first image 127 taken by using the illumination unit 16 a.
Therefore, in order not to cut the periphery of the first image 127 , it is preferable that the first image 127 is used as a reference image and the images of the second image group are made to conform the first image 127 .
In the case of digital images, even when only one underexposed image is taken, exposure can be corrected by increasing the gain, however, if the gain is increased, noise also increases and an undesirable image is obtained. However, as in this embodiment, in the case where many images are synthesized to increase the gain in total, the noises in the images are averaged, so that an image with a high S/N ratio can be obtained, and as a result, noise is reduced and exposure is made proper.
In another respect, for example, image-taking operation is carried out a plurality of times by setting a high sensitivity of the image pickup element 19 a while allowing noise, and averaging processing is applied to the obtained images, whereby the random noise contained in the images is reduced.
Image data synthesized by the image synthesis circuit 116 is input into the first image correcting circuit 117 a and subjected to gamma correction processing and compression processing, and furthermore, end deficient portions (corresponding to the region 129 in FIG. 5) of the images produced by synthesis processing are cut, and the portion reduced in image size is complemented by spreading. Namely, the region 129 in which the images do not conform in FIG. 5 is cut, and only the region in which the images conform is extracted and complemented by spreading. Thereafter, the synthesized image is recorded onto the recording circuit 119 .
In the description given above, the deviations among the images are corrected by detecting the characteristic points of the images and deviations thereof, however, in the camera of this embodiment, since a vibration detection sensor 11 f is provided, synthesis processing can be carried out after the coordinates of the images are converted on the basis of the amount of vibration between the first image-taking and the second image-taking (for example, the amount shown by the arrow 11 r in FIG. 2) or the amounts of vibrations between the subsequent image-taking frames. With this structure, since the deviation detection processing of the displacement detection circuit 114 becomes unnecessary, the period of time to completion of the synthesized image can be shortened, and synthesis failures due to characteristic point detection failures can be repressed.
The point of Embodiment 1 is as described above, and other than this, this embodiment has the following new structural points.
-
- 1) When the vibration correctable stroke of the correcting optical unit 11 d (correcting lens 11 b ) is reduced, vibration control changes from optical vibration control to electronic vibration control involving image synthesis processing (described above). Reduction in vibration correctable stroke of the correcting optical unit 11 d includes not only the case where the operation of the correcting optical unit 11 d according to vibration exceeds the correction range 11 l as in this embodiment, but also the case where the operation nearly exceeds the correction range 11 l.
- 2) In the case where re-image-taking operation (second and subsequent image-taking) is carried out, when the remaining exposure period necessary for proper exposure is very short, re-image-taking operation is not carried out.
- 3) When image-taking operation is carried out while panning, re-image-taking operation is not carried out.
- 4) The initially taken image is displayed on the display unit 118 of the camera and is not renewed.
The above 1) has already been described, so that 2) and the rest are described below.
As described in FIG. 2, when vibration exceeding the correction range 11 l of the correcting optical unit 11 d occurs during exposure, the exposure is temporarily stopped, the correcting optical unit 11 d (the correcting lens 11 b ) is returned to its initial position and image-taking is carried out again. A second exposure period 11 o in a case where image-taking is carried out again is a period of time determined by subtracting a first exposure period 11 m of the first image-taking operation from a total exposure period 11 i necessary for proper exposure.
However, when the second exposure period 11 o is a very short remaining time, it is not necessary to carry out the second image-taking operation and image synthesis processing. The reason for this is that using the time required for the second image-taking operation to image-taking operation of the next object is better for improvement in mobility of the camera. As a matter of course, the image is slightly underexposed due to omission of the second image-taking operation.
For example, in a case where the total exposure period is 400 ms, the first exposure period 11 m is 300 ms, and the second exposure period 11 o is 100 ms, that is, the ratio of the first and second periods is 3 to 1, or the ratio of the second exposure period 11 o to the first exposure period 11 m is ⅓ or less, underexposure is not so great, so that the second image-taking operation is not carried out. Even in the case where image-taking operation is repeated not two times but as many as three or four times more, when the ratio of the remaining exposure period to the first exposure period 11 m becomes equal to or less than the above-mentioned ratio of ⅓, re-image-taking operation is discontinued.
Furthermore, if re-image-taking operation is carried out during image-taking operation while panning the camera as in the case of camera panning shooting, the composition greatly changes between the first image-taking operation and the second image-taking operation due to the time difference. When a plurality of images are synthesized, deviation correction is carried out so that the characteristic points in the backgrounds of the images are made to conform, however, as in this case, the background deviates due to panning, the main object in the synthesized image deviates. In addition, when the characteristic point is extracted from the main object region and image synthesis processing is carried out, although the main object does not deviate, the background deviates between two images, so that a background image without image blurring by conforming images cannot be obtained. Therefore, in this image-taking condition, a more excellent image is obtained by correction by increasing the gain of the first image than by complement of the exposure by conforming the images. Therefore, when the vibration detection sensor 11 f detects vibration of the camera during exposure and it is judged that the camera is being panned on the basis of the detection result, re-image-taking operation is not carried out.
Generally, in the case of a digital camera, a taken image is displayed on a back face liquid crystal monitor (display unit 118 ) of the camera after image-taking operation. However, as in this embodiment, when images are synthesized to complement exposure, it takes time from creation to display of a synthesized image. Therefore, in this embodiment, on the display unit 118 of the camera, an image before being synthesized is displayed. The reason for this is that, display of an underexposed image upon increasing the gain does not pose a problem in observation since the display unit 118 of the camera is small. In this case, when the illumination unit 16 a is used, an image taken by using the illumination unit 16 a is continuously displayed. Namely, in the case where the strobe is used for the first image-taking operation, an image obtained by the first image-taking operation is continuously displayed from the finish of the second and subsequent image-taking operation to the finish of image confirmation.
When continuous shooting is carried out with a normal digital camera, a taken image of the last frame in continuous shooting is continuously displayed on the display unit 118 . However, in this embodiment, an image taken by using the illumination unit such as the first image (when rear-curtain flash sync is employed, the last image) is displayed by priority. The reason for this is that an image taken without the illumination is greatly different from the image to be actually obtained (synthesized image).
In a case where an image recorded on the recording circuit 119 is replayed, a display prohibiting signal from the recording prohibiting circuit 118 a input into the display unit 118 of FIG. 1. As described above, when a plurality of images are taken and synthesized, a specified image is displayed immediately after image-taking operation. On the other hand, when an image is replayed after synthesis processing is completed, the display prohibiting circuit 118 a does not display the specified image but displays the image after being completely synthesized so that the actual image can be confirmed.
Furthermore, in a case of the image playback mode, generally, all image data in the recording circuit 119 can be viewed, however, in the case where image-taking operation is carried out a plurality of times and images obtained through the plurality of times of image-taking operation are synthesized, the display prohibiting circuit 118 a prevents the images before being synthesized from being viewed. The reason for this is that, if the plurality of images to be synthesized (the first image and the second image group) are displayed during playback, many images that the photographer does not recognize are displayed, not only taking a long time to browse the images, but also causing confusion. However, among the plurality of images to be synthesized, only the first image taken by using the illumination unit 16 a is capable of being displayed during playback so that it is compared with the synthesized image.
FIGS. 6A and 6B are flowcharts showing the image-taking operations of the camera according to Embodiment 1 of the invention, and this flow starts when the power source of the camera is turned on.
In Step S 1001 , the camera is checked and various functions are initialized. Concretely, the power source status and communications status are checked, and when the results of checking are not suitable for image-taking operation, processing such as indication is carried out. Furthermore, initializations processings including initialization of pulse encoders to be used for focusing control and initialization of calculation (resetting) of vibration control are carried out. In the next Step S 1002 , the process waits until the sw1 is turned on in response to half depression of the release operation member 12 c by a photographer, and until the sw1 is turned on, the process returns to Step S 1001 and continues camera checking processing and initializations processings. Then, when the sw1 is turned on, the process advances to Step S 1003 .
In Step S 1003 , it is determined whether or not the photographer has turned the vibration control operation member 120 on, and when the vibration control operation member 120 is turned on, the process advances to Step S 1004 . In Step S 1004 , the vibration detection sensor 11 f is started and the output thereof is processed by the image-taking control circuit 18 and driving of the correcting optical unit 11 d is started by the image blurring correcting circuit 11 e . Then the process advances to Step S 1005 . On the other hand, when the vibration control operation member 120 is off, the process advances to Step S 1005 by skipping Step S 1004 . In this case, the vibration detection sensor 11 f is not started and the correcting optical unit 11 d is not driven, either.
In Step S 1005 , image is taken by using the image pickup element 19 a . The image-taking control circuit 18 drives the AF drive motor 14 a by using the focusing drive circuit 14 b to move the image-taking lens 11 a in the optical axis L direction while detecting the contrast of the image on the basis of the output from the signal processing circuit 111 . Then, when the contrast becomes highest, the driving of the image-taking lens 11 a is stopped to obtain a focused status of the image-taking optical system (AF by means of the hill-climbing method). Focusing can also be carried out by using phase difference detection method. The image-taking control circuit 18 calculates the brightness of the object on the basis of the output of the image pickup element 19 a , simultaneously. Then, according to the image-taking conditions, an exposure period of the image-taking operation is calculated.
The image-taking conditions mentioned herein include the following four items.
-
- Brightness of the object
- Focal length of the image-taking optical system
- Brightness (f-number) of the image-taking optical system
- Sensitivity of the image pickup element
For example, it is assumed that the sensitivity of the image pickup element 19 a has been set to ISO speed 200 . The brightness of an object is measured (photometry), and in order to obtain proper exposure on the basis of the results of this photometry, the stop 13 a is fully opened (for example, f2.8) and the closing timing of the shutter 12 a , that is, the exposure period is set to ⅛ seconds.
In the next Step S 1006 , the image picked up in Step S 1005 is displayed on an external liquid crystal monitor (display unit 118 ) or the like of the camera. Then in the next Step S 1007 , the process waits while circulating Steps S 1002 through S 1007 until the sw2 is turned on in response to full depression of the release operation member 12 c . Then, when the sw2 is turned on, the process advances to Step S 1008 .
In Step S 1008 , by driving the speaker 17 a via the sound producing drive circuit 17 b , a sound informing the start of image-taking is produced. This sound may be an electronic bleep sound, or a shutter opening sound or a mirror rising sound. In the next Step S 1009 , charges that have been accumulated in the image pickup element 19 a up to this point are all reset, and charge storage operation is started again. Thereby, real exposure is started.
In Step S 1010 , a deficiency in correct ability when the vibration control is turned on is detected during exposure. More concretely, when vibration detected by the vibration detection sensor 11 f during exposure reaches the limit value of the correction range 11 l (FIG. 2) of the correcting optical unit 11 d (timing 11 k of the waveform 11 p of FIG. 2), the process advances from Step S 1010 to Step S 1018 . On the other hand, when there is no deficiency in correct ability, the process advances to Step S 1011 . The reason for using the expression “correction ability” is that, when the correcting optical unit 11 d cannot sufficiently correct image blurring, not only is the correction range 11 l deficient, but also there is a possibility that the vibration speed is high and the correcting optical unit 11 d cannot follow the speed. For detection of the correction range 11 l , a position detector is provided in the correcting optical unit 11 d and the correcting position (movement position of the correcting lens 11 b ) is always monitored, or the image blurring correction target value to be supplied to the correcting optical unit 11 d is monitored. Furthermore, the vibration speed may be determined, for example, on the basis of the derivative value of the image blurring correction target value. When the correction range 11 l is not exceeded, the process advances to Step S 1011 as described above.
In Step S 1011 , it is determined whether or not the exposure period has completely elapsed, and when the exposure period obtained in Step S 1005 is reached, the process advances to Step S 1012 , and otherwise the process returns to Step S 1010 and this flow is circulated until the exposure is completed while checking the correction ability of the correcting optical unit 11 d.
In Step S 1010 , when the operation of the correcting optical unit 11 d nearly exceeds the vibration correction range 11 l during exposure, as described above, the process advances to Step S 1018 and it is determined whether or not the camera is being panned. Concretely, when a higher angular velocity than a predetermined value is added for a predetermined period during exposure depending on the output of the vibration detection sensor 11 f , it is determined that panning is being carried out. Actual detection of panning is not started in Step S 1018 but is started prior to Step S 1018 . Then, when it is determined in Step S 1018 that panning is being carried out, the process advances to Step S 1020 , and when it is determined that panning is not being carried out, the process advances to Step S 1019 .
When the camera mode is set to rear-curtain flash sync, exposure must be continued since the illumination unit has not emitted light yet. Therefore, in the rear-curtain flash sync mode, the process advances to Step S 1019 without condition. Then, in Step S 1019 , it is determined whether or not the exposure is to be restarted according to the ratio of the remaining time until the finish of exposure to the total exposure period, and for example, when the ratio of the remaining exposure period to the total exposure period is less than ⅓, the process advances to Step S 1020 , and otherwise the process advances to Step S 1021 .
When the process advances to Step S 1020 , the gain of the image is increased according to omission of the remaining exposure period, and then the process advances to Step S 1012 .
On the other hand, when the remaining exposure period is long in Step S 1019 and the remaining exposure must be carried out, the process advances to Step S 1021 , the shutter 12 a is closed, and charges accumulated in the image pickup element 19 a are transferred to the signal processing circuit 111 . When this transfer is completed, the shutter 12 a is opened. Then, in the next Step S 1022 , obtained image data is stored in the image storing circuit 113 . In the next Step S 1023 , the correcting optical unit 11 d (the correcting lens 11 b ) is moved to its initial position. Simultaneously, the shutter 12 a is opened and the image of the light flux from the object is formed on the image pickup element 19 a . Next, in Step S 1024 , the process waits while circulating Steps S 1023 and S 1024 until the correcting optical unit 11 d is moved to its initial position. Then, when the correcting optical unit 11 d moves at its initial position, the process returns to Step S 1009 and exposure is started again.
When the correcting optical unit 11 d thus nearly exceeds the correction range 11 l during exposure, image data obtained through image-taking operation until this point is temporarily stored, and image-taking operation is started again (S 1010 → S 1019 to S 1024 → S 1009 . . . ). When the remaining exposure period is short or panning is carried out (when the answer of S 1018 is NO, the answer of S 1019 is NO), image-taking operation is not carried out again. This is for giving the mobility of the camera priority and repressing lowering in reliability of background image synthesis.
When the process advances from Step S 1011 to Step S 1012 , by driving the speaker 17 a via the sound producing drive circuit 17 b , a sound informing completion of image-taking is produced. This sound may be an electronic bleep sound which is different from the electronic bleep sound in Step S 1008 , a shutter closing sound; or a mirror lowering sound. Thus, when a plurality of images are taken, the sounds informing the operations of image-taking are one set of sounds (each produced once when starting exposure of the first image-taking and when completing the exposure of the last image-taking), so that the sounds do not give a photographer discomfort even when image-taking is carried out a plurality of times. Namely, the number of sounds is equal in the case of normal image-taking operation (one exposure) and in the case of a plurality of times of image-taking operation, thereby preventing a photographer from feeling discomfort during image-taking operation.
In the next Step S 1013 , the shutter 12 a is closed so as to block light reaching the image pickup element 19 a , and then charges accumulated in the image pickup element 19 a are transferred to the signal processing circuit 111 via the A/D converter 110 . Thereafter, the correcting optical unit 11 d is returned to the initial position for preparation for the next image-taking operation, and when charge transfer processing is finished, the shutter 12 a is opened. In the next Step S 1014 , in the case where the vibration control system is operating, it is determined whether or not image-taking operation has been carried out a plurality of times due to the condition where the correcting optical unit 11 d nearly exceeds the correction range 11 l during exposure. Herein, when image-taking operation is carried out only one time, the process advances to Step S 1015 , and when image-taking operation is carried out two or more times, the process advances to Step S 1025 .
When it is determined that image-taking operation has been carried out only one time and the process advances to Step S 1015 , the second image correcting circuit 117 b applies gamma correction processing and compression processing to input signals (data of one image). Then, in the next Step S 1016 , the output signals of the second image correcting circuit 117 b are input into the display unit 118 and displayed on the display unit 118 provided on the back face of the camera. Thereafter, the process advances to Step S 1017 , and the output signals of the second image correcting circuit 117 b are input into the recording circuit 119 . Then, the process returns to Step S 1002 .
Furthermore, in the case where image-taking operation has been carried out a plurality of times and the process advances from Step S 1014 to Step S 1025 , in Step S 1025 , the image taken by using the illumination unit 16 a among the images obtained through the plurality of times of image-taking operation (the first image when the strobe is not used or front-curtain sync is set, and the last image when the rear-curtain flash sync is set) is readout from the image storing circuit 113 . Then, in the next Step S 1026 , the readout image is corrected by the second image correcting circuit 117 b and the gain is increased. When image-taking operation is carried out a plurality of times, each image is underexposed, so that the gain is increased for compensating underexposure when the image is displayed on the display unit 118 .
In the next Step S 1027 , the image that has been corrected is input into the display unit 118 and displayed on the display unit 118 . When the image is thus displayed on the display unit 118 provided on the back face of the camera, not the image obtained by synthesizing the images obtained through a plurality of times of image-taking operation, but one image among the images (the first image when images are taken without using the illumination unit 16 a , and when the illumination unit 16 a is used, the first image in the case of front-curtain flash sync, and the last image in the case of rear-curtain flash sync) is displayed to improve the mobility. Furthermore, displaying of only one image makes it possible to downsize the display unit 118 and this contributes to downsizing of the camera. While the image is displayed, in Step S 1028 , the plurality of taken images stored in the image storing circuit 113 are successively readout.
The processes from Step S 1028 to Step S 1033 are sections of electronic operations which make apparent exposure proper by synthesizing the images obtained through a plurality of image-taking operation (images stored in Step S 1022 ) while conforming their characteristic points.
In Step S 1029 , the characteristic points in the respective images are extracted. In the next Step S 1030 , the coordinate conversion circuit 115 converts the coordinates of the images (the second image group) on the basis of the coordinates of extracted characteristic points. Herein, only the first image (the first image taken by using the illumination unit 16 a ) is not subjected to coordinate conversion processing. Namely, the first image is used as a reference image for coordinate conversion processing. In the subsequent Step S 1031 , the process waits while circulating Steps S 1028 through S 1031 until the coordinate conversion processing of all the images is finished, and when the coordinate conversion processing of all the images is finished, the process advances to Step S 1032 .
When the process advances to Step S 1032 , the first image and the images of the second image group that have been subjected to coordinate conversion processing are synthesized. Herein, image synthesis processing is carried out by averaging of the signals of the coordinates corresponding to each of the images, and random noise in the images is reduced through averaging. Then, the gain of the image with the reduced noise is increased to make the exposure proper. In the next Step S 1033 , the first image correcting circuit 117 a applies gamma correction processing and compression processing to the image data, and furthermore, cuts the end deficient portions of the images produced when the images are synthesized, and complements the reduction in image size by spreading. Namely, the region 129 in which the images do not conform each other in FIG. 5 is cut, and only the region in which the images conform each other is extracted and compensated by spreading.
Thereafter, the process advances to Step S 1017 , and image data obtained in Step S 1021 is recorded on the recording circuit 119 and a recording medium that consists of a semiconductor memory, and can be attached to and detached from the camera, and the process returns to Step S 1002 .
Furthermore, when the release operation member 12 c is still continuously depressed halfway and the sw1 is still on after Step S 1017 , the process advances in the flow in sequence again, to Steps S 1002 , S 1003 , S 1004 , and S 1005 .
Furthermore, in the case where the half-depression of the release operation member 12 c has been released, the process returns from Step S 1002 to Step S 1001 , and elements of the camera are reset to their initial statuses.
Furthermore, when the release operation member 12 c is fully depressed and the sw2 is on in Step S 1017 , the process does not return to start, but waits in Step S 1017 . That is, the process does not change into the next image-taking.
Thus, when the correction range 11 l of the correcting optical unit 11 d is exceeded, by cooperative operation of the optical vibration control and electronic vibration control by synthesizing images, the correcting optical unit 11 d can be downsized.
Normally, when the correcting optical unit is designed, the correction range 11 l is set by assuming the maximum vibration that may be applied to the camera during exposure. When the correction range 11 l of the correcting optical unit is increased, the size of the correcting optical unit significantly increases. The reason for this increase is that the correction range 11 l is required to have an extent around the center (initial position) corresponding to vibrations vibrating around the center, and accordingly, a driving space that has twice the area of the correction range 11 l becomes necessary, and as the correction range 11 l increases, the energy required for driving increases, and the size of the driving part is accordingly increased. However, the excessive increase in the size of the optical vibration control system for the purpose of coping with a great vibration that rarely occurs is not desirable and especially deteriorates the portability of the camera.
As in this embodiment, by employing the cooperative operation with the electronic vibration control system (coordinate conversion processing and synthesis processing of images) according to the amount of correction of the correcting optical unit 11 d , the correcting optical unit can be made small, thereby realizing a camera easy to use. Conversely, even a small-sized correcting optical unit can cope with a great vibration, and even with a conventional large correcting optical unit, greater vibration can be corrected, whereby the vibration control effect is increased.
Furthermore, as seen in the flowchart of FIG. 6, in the stage of object observation from Step S 1002 to Step S 1006 , that is, when a photographer aims an object to be taken by looking at the monitor (electronic view finder) or finder (optical view finder), the correcting optical unit 11 d operates on the basis of the signals of the vibration detection sensor 11 f . This not only reduces the influence of vibration and makes aiming easier, but also improves focusing accuracy. In this stage of object observation, when the correction range 11 l of the correcting optical unit 11 d is nearly exceeded, image-taking operation is interrupted, the correcting optical unit 11 d is returned to its initial position and the image reading out position is displaced (by the arrow 11 r of FIG. 2). Then, until a next great vibration is applied, the correcting optical unit 11 d is driven without changing this image reading out position, the system can thereby cope with great vibrations. However, with this structure, there is a possibility that the image reading out position greatly deviates until the end of the stage of object observation, and in the stage of image recording (in the stage of real image-taking operation from Step S 1007 to Step S 1011 ), image-taking operation with full-screen becomes impossible. Conversely, for carrying out image-taking operation while effectively using the image plane area in the stage of image recording, a photographer can more stably hold the camera by being informed of composition changes caused by a great vibration in the stage of object observation.
For this reason, in this embodiment, in the stage of object observation, only the optical vibration control is carried out, and in the stage of image recording, image reading out (coordinate conversion) and synthesis processings are carried out in addition to the optical vibration control to realize vibration control with high accuracy.
Furthermore, in this embodiment, usability after finishing image-taking operation is also considered, and when the image is replayed after image-taking operation is finished, the plurality of images to be synthesized are prevented from being replayed (displayed), whereby image browsing is smoothly carried out. Furthermore, at the time of this playback, a sufficient period of time has elapsed until image synthesis is completed, so that the image after being synthesized processing is displayed. Immediately after image-taking operation, one of the plurality of images before being synthesized is displayed. Then, a specified one image among the plurality of images to be synthesized, that is, the image taken by using the illumination unit 16 a is allowed to be displayed for comparison with the synthesized image. Only the specified one image is displayed, so that the display unit 118 can be downsized, and this contributes to downsizing of the camera.
In the above-described embodiment, the matter that the vibration control effect is improved by cooperative operation of the optical vibration control system and the electronic vibration control system is noted, and furthermore, these vibration control system are used in a balanced manner, whereby the correcting optical unit is significantly downsized.
Embodiment 2
Next, the camera according to Embodiment 2 of the invention is described. First, the differences from the above-described Embodiment 1 are described below.
-
- 1) Coordinate conversion processing and synthesis processing of images obtained through re-image-taking operation are carried out simultaneously with image-taking operation. Namely, during the period of the initial position returning operation of the correcting optical unit 11 d , image processing (coordinate conversion processing and synthesis processing) is carried out.
- 2) In a case where re-image-taking operation is carried out, the taken and synthesized image is displayed on the monitor on the back face of the camera.
In the flow of Embodiment 1 shown in FIGS. 6A and 6B, after all the images are taken, the images are read out from the image storing circuit 113 , subjected to coordinate conversion processing, and synthesized. Here, when the operation of the correcting optical unit 11 d nearly exceeds the correction range 11 l and re-image-taking operation is carried out, the operation to move the correcting optical unit 11 d (the correcting lens 11 b ) to its initial position (Steps S 1023 and S 1024 of FIG. 6A) is required. During this operation, it is possible that coordinate conversion processing is carried out so that the characteristic point in the image readout from the image pickup element 19 a conforms the characteristic point in the prior image and the images are synthesized. In this case, at the end of image-taking, image synthesis processing is finished, so that the image after being synthesized can be immediately displayed on the display unit 118 such as the back face monitor of the camera.
FIG. 7 is a timing chart describing the structure in which the image coordinate conversion processing and synthesis processing are carried out simultaneously with current image-taking operation. In response to the exposure operation f1 through f4 (f1 is exposure operation by using the illumination unit 16 a ), signal photoelectrically converted by the image pickup element 19 a are readout as image signal, and simultaneously with reading out of the image signal F 2 , correlation of the previous image signal F 1 and the current image signal F 2 is calculated to determine a characteristic point variation, and the two image signals F 1 and F 2 are synthesized to obtain a synthesized image signal C 2 .
Next, simultaneously with reading out of the image signal F 3 , correlation of the previous synthesized image signal C 2 and the current image signal F 3 is calculated to determine a characteristic point variation, and the two image signals C 2 and F 3 are synthesized to obtain a synthesized image signal C 3 . Then, simultaneously with reading out of the image signal F 4 , correlation of the previous synthesized image signal C 3 and the current image signal F 4 is calculated to determine a characteristic point variation, and the two image signals C 3 and F 4 are synthesized to obtain a synthesized image signal C 4 . Then, the calculated synthesized image signal C 4 is displayed on the liquid crystal display on the camera back face and the synthesized image signal C 4 is recorded on a recording medium.
FIGS. 8A and 8B are flowcharts describing the operations mentioned above, and image storing of Step S 1022 in the flowchart of FIG. 6A is omitted. In Step S 1023 , after the operation of moving the correcting optical unit 11 d to its initial position is started, in the processing of Steps S 1029 through S 1032 , the images are synthesized.
The reason for omitting the image storing processing of Step S 1022 is that, in Embodiment 2 of the invention, since each taken image is synthesized with the previous taken image simultaneously with image-taking, only one synthesized image is sufficient to be stored at the point of Step S 1032 of image synthesis processing, and other taken images are not necessary. Namely, the synthesized image is renewed every image-taking operation, so that it is not necessary to store the taken images. Therefore, the image storing circuit 113 shown in FIG. 1 becomes unnecessary.
The details of the flowchart are described below. The flow of FIGS. 8A and 8B starts when the power source of the camera is turned on.
In Step S 1001 , camera checking is carried out and various functions are initialized. Concretely, the power source status and the communications status are checked, and when the results of checking are not suitable for image-taking operation, processing such as indication is carried out. Furthermore, initialization including initialization of pulse encoders to be used for focusing control and initialization of vibration control operation (resetting) are carried out. In the next Step S 1002 , the process waits until the sw1 is turned on in response to half-depression of the release operation member 12 c by a photographer, and until the sw1 is turned on, the process returns to Step S 1001 and continues camera checking and initialization. Then, when the sw1 is turned on, the process advances to Step S 1003 .
In Step S 1003 , it is determined whether or not the photographer has turned the vibration control operation member 120 on, and when it is on, the process advances to Step S 1004 , the vibration detection sensor 11 f is started, and the output thereof is processed by the image-taking control circuit 18 and the driving of the correcting optical unit 11 d is started by the image blurring correcting circuit 11 e . On the other hand, when the vibration control operation member 120 is off, the process advances to Step S 1005 by skipping Step S 1004 . Therefore, in this case, the vibration detection sensor 11 f is not started, and the correcting optical unit 11 d is not driven, either.
In Step S 1005 , image is taken by using the image pickup element 19 a . The image-taking control circuit 18 drives the AF drive motor 14 a by the focusing drive circuit 14 b to move the image-taking lens 11 a in the optical axis L direction while detecting the contrast of the image on the basis of the output from the signal processing circuit 111 . Then, when the contrast becomes highest, the driving of the image-taking lens 11 a is stopped to obtain a focused status of the image-taking optical system (AF by means of a hill-climbing method). It is also possible that focusing is carried out by using the phase difference detection method. The image-taking control circuit 18 calculates the brightness of the object on the basis of the output of the image pickup element 19 a , simultaneously. According to the image-taking conditions, an image-taking exposure period is determined.
The image-taking conditions mentioned herein include the following four items.
-
- Brightness of the object
- Focal length of the image-taking optical system
- Brightness of the image-taking optical system (f-number)
- Sensitivity of the image pickup element
For example, it is assumed that the sensitivity of the image pickup element 19 a has been set to ISO speed 200. The brightness of an object is measured (photometry), and in order to carry out proper exposure on the basis of the results of this photometry, the stop 13 a is fully opened (for example, f2.8) and the closing timing of the shutter 12 a , that is, the exposure period is set to ⅛ seconds.
In Step S 1006 , the image taken in Step S 1005 is displayed on the external liquid crystal monitor (display unit 118 ) of the camera. Then, in the next Step S 1007 , the process waits while circulating Steps S 1002 through S 1007 until the sw2 is turned on in response to full-depression of the release operation member 12 c . Then, when the sw2 is turned on, the process advances to Step S 1008 .
In Step S 1008 , by driving the speaker 17 a via the sound producing drive circuit 17 b , a sound informing the start of image-taking operation is produced. This sound may be, for example, an electronic bleep sound, a shutter opening sound, or a mirror rising sound. In the next Step S 1009 , charges that have been accumulated in the image pickup element 19 a are all reset, and change storage operation is started again. Thereby, real exposure is started.
When the vibration control operates, and a deficient of correction ability is detected during exposure in Step S 1010 . More concretely, when the operation of the correcting optical unit 11 d nearly exceeds the correction range during exposure (timing 11 k of the waveform 11 p of FIG. 2), the process advances to Step S 1018 , and otherwise the process advances to Step S 1011 .
Herein, the reason for using the expression “correction ability” is that, when the correcting optical unit 11 d cannot sufficiently correct image blurring, not only is the correction range 11 l deficient, but also there is a possibility that the vibration speed is high and the correcting optical unit 11 d cannot follow the speed. For detection of the correction range 11 l , a position detector is provided in the correcting optical unit 11 d and the correcting position is always monitored, or the image blurring correction target value to be supplied to the correcting optical unit 11 d is monitored. Detection of the vibration speed may be made by, for example, the derivative value of the image blurring target value. When the correction range 11 l is not exceeded, the process advances to Step S 1011 as described above.
In Step S 1011 , it is determined whether or not the exposure period has completely elapsed, and when the exposure period elapses, the process advances to Step S 1012 , and otherwise the process returns to Step S 1010 and circulates this flow while checking the correction ability of the correcting optical unit until exposure is completed.
When the correcting optical unit 11 d nearly exceeds the vibration correction range 11 l in Step S 1010 during exposure, it is determined in Step S 1018 whether or not the camera is being panned as described above. Concretely, when a great angular velocity is added to the camera for a predetermined period of time during exposure depending on the output of the vibration detection sensor 11 f , it is determined that the camera is being panned. Actual panning detection is not started in Step S 1018 , but is started prior to Step S 1018 . When it is determined in Step S 1018 that the camera is being panned, the process advances to Step S 1020 , and when it is determined that panning is not being carried out, the process advances to Step S 1019 . When the camera mode is set to rear-curtain flash sync, since the illumination unit has not emitted light, exposure must be continued. Therefore, in the case of the rear-curtain flash sync mode, the process advances to Step S 1019 without condition.
In Step S 1019 , it is determined whether or not exposure is started again according to the ratio of the remaining time until the end of exposure to the total exposure period, and for example, when the ratio of the remaining exposure period to the total exposure period is less than ⅓, the process advances to Step S 1020 , and otherwise the process advances to Step S 1021 . When the camera mode is set to rear-curtain flash sync, the illumination unit has not emitted light, so that the exposure must be continued. Therefore, in the case of the rear-curtain flash sync mode, the process advances to Step S 1021 without condition.
When the process advances to Step S 1020 , the gain of the image is increased according to the omission of the remaining exposure period, and the process advances to Step S 1012 .
When the remaining exposure period is long and re-image-taking operation is necessary in Step S 1019 , the process advances to Step S 1021 , and herein, the shutter 12 a is closed and charges accumulated in the image pickup element 19 are transferred to the signal processing circuit 111 . Then, when this transfer is finished, the shutter 12 a is opened. In the subsequent Step S 1023 , the correcting optical unit 11 d starts moving to its initial position. In addition, simultaneously with this, the shutter 12 a is opened and the image of a light flux from the object is formed on the image pickup element 19 a.
The Steps S 1029 through S 1032 are sections of electronic correcting operations which make the apparent exposure proper by synthesizing images obtained through re-image-taking operation (images readout in Step S 1021 ) while conforming their characteristic points.
In Step S 1029 , characteristic point extraction from the image readout in Step S 1021 is started. In the next Step S 1030 , the coordinates of the extracted characteristic point is compared with the coordinates of the characteristic point in the image previously taken, and the coordinate conversion circuit 115 starts coordinate conversion processing for the readout image so that the coordinates thereof match those of the previous image. Herein, only the first image (the first image taken by using the illumination unit 16 a ) is not subjected to coordinate conversion processing. Namely, this first image is used as a reference image for coordinate conversion processing. In the next Step S 1032 , synthesis processing of the image obtained by synthesizing the images previously taken and the image that has been readout and subjected to coordinate conversion processing at this time is started. Herein, image synthesis processing is carried out by averaging the signals of the coordinates corresponding to each other in the images, and random noise in the images is reduced through averaging processing. Then, the gain of the image with reduced noise is increased to make exposure proper.
In the next Step S 1024 , the process waits while circulating Steps S 1023 and S 1024 until the correcting optical unit 11 d operates at its initial position. Then, when the correcting optical unit 11 d (the correcting lens 11 b ) moves at its initial position, the process returns to Step S 1009 and exposure is started again.
When the correcting optical unit 11 d is returned to its initial position as described above, the calculation load on the image-taking control circuit 18 is small, so that the operation of Step S 1029 through Step S 1032 can be carried out a high rate, and the originally required initialization period of the correcting optical unit 11 d can be effectively used. The operations of Step S 1029 through Step S 1032 only command the operation start in each step, and do not include transition to the next operation from the end of each operation. Namely, even when the process advances from Step S 1024 to Step S 1009 and re-image-taking operation is started, operations of Step S 1029 through Step S 1032 can be carried out simultaneously until the process advances to Step S 1021 from Step S 1009 .
As described above, when the correcting optical unit 11 d nearly exceeds the correction range 11 l during exposure, the image-taking data obtained until this point is readout, image-taking operation is started again, and simultaneously, synthesis processing with the previous image is carried out. However, when the remaining exposure period is short or panning is carried out, re-image-taking operation is not carried out.
This is for giving priority to the mobility of the camera and for repressing lowering in reliability of background image synthesis.
Thereafter, when the process advances to Step S 1012 , by driving the speaker 17 a via the sound producing drive circuit 17 b , a sound informing the completion of image-taking operation is produced. This sound may be an electronic bleep sound which is different from the electronic bleep sound in Step S 1008 , a shutter closing sound, or a mirror lowering sound. Thus, in the case where image-taking operation is carried out a plurality of times, sounds informing the operations are one set of sounds (each produced once when starting exposure of the first image-taking operation and when completing the exposure of the last image-taking operation), and therefore, the sounds do not give a photographer discomfort even when image-taking operation is carried out a plurality of times. Namely, the number of sounds to be produced is equal in the case of normal image-taking operation (one exposure) and the plurality of times of image-taking operations, and this represses the photographer from feeling discomfort when taking images.
In the next Step S 1013 , the shutter 12 a is closed so as to block light reaching the image pickup element 19 a , and then charges in the image pickup element 19 a are transferred to the signal processing circuit 111 via the A/D converter 110 . Thereafter, the correcting optical unit 11 d is returned to its initial position for preparation for the next image-taking operation, and when the charge transfer is finished, the shutter 12 a is opened. In the next Step S 1014 , it is determined whether or not image-taking operation has been carried out a plurality of times due to the movement of the correcting optical unit 11 d exceeding the correction range 11 l during exposure when the vibration system operates, and when image-taking operation is carried out only one time, the process advances to Step S 1015 , and when image-taking operation is carried out two or more times, the process advances to Step S 1033 .
When the process advances to Step S 1015 , the second image correcting circuit 117 b applies gamma correction processing and compression processing to input signals. In the next Step S 1016 , the output signals of the second image correcting circuit 117 b are input into and displayed on the display unit 118 provided on the back face of the camera. In Step S 1017 , the output signals of the second image correcting circuit 117 are input into the recording circuit 119 . Then, the process returns to Step S 1002 .
On the other hand, when the process advances to Step S 1033 since it is determined that image-taking operation has been carried out a plurality of times in Step S 1014 , the first image correcting circuit 117 a applies gamma correction processing and compression processing to the image data, and furthermore, cuts the end deficient portions of the respective images produced when being synthesized and complements the reduction in image by spreading. Namely, the region 129 in which the images do not conform in FIG. 5 is cut, and only the region in which the images conform each other is extracted and subjected to complement by spreading. Then, in the next Step S 1016 , the signal of the first image correcting circuit 117 a is input into and displayed on the display unit 118 provided on the back face of the camera. In the next Step S 1017 , the image data produced in Step S 1021 is recorded on a recording medium (recording circuit 119 ) which consists of a semiconductor memory, and can be attached to and detached from the camera. Then, the process returns to Step S 1002 .
When the release operation member 12 c is still continuously depressed halfway and the sw1 is still on after Step S 1017 , the process advances in the flow in sequence again, to Steps S 1002 , S 1003 , S 1004 , and S 1005 .
On the other hand, when the half-depression of the release operation member 12 c is released, the process returns from Step S 1002 to Step S 1001 , and the elements of the camera are reset to their initial statuses. Furthermore, when the release operation member 12 c is fully depressed and the sw2 is on in Step S 1017 , the process does not return to start but waits in Step S 1017 .
In Embodiment 2 of the invention, image synthesis processing is carried out simultaneously with image-taking operation by effectively using the period during which the correcting optical unit 11 d is returned to its initial position. During this period, the calculation load on the camera is small, so that image coordinate conversion processing and synthesis processing can be carried out at a high rate. Thus, by carrying out image synthesis processing simultaneously with image-taking operation, the storage section that stores the plurality of images can be made unnecessary. In addition, this provides an advantage in that the actual synthesized image can be confirmed when image-taking operation is finished.
Embodiment 3
Next, Embodiment 3 of the invention is described. First, the differences from Embodiment 1 given above are described.
-
- 1) When the correcting optical unit 11 d nearly exceeds the correction range 11 l , the correcting optical unit 11 d is fixed at this position. Then, the image-taking operation is temporarily stopped and re-image-taking operation is carried out in the same manner as in Embodiment 1, and at this point, the correcting optical unit 11 d is not operated, and instead, image-taking operation is carried out with an exposure period that does not cause image blurring, and if the total exposure period is deficient, image-taking operation is repeated again. When the interval of continuous shooting is short, the period during which the correcting optical unit 11 d is returned to its initial position becomes a problem, however, high-speed continuous shooting is made possible by the above-described structure.
- 2) The obtained plurality of images are not synthesized within the camera, but are synthesized by using an external apparatus such as a personal computer after they are taken. Thereby, the camera calculation time is reduced so as to repress the right moment of releasing the shutter from being lost.
- 3) In the stage of object observation, vibration control is carried out by means of reading out of the images. Thereby, even in the stage of image recording, sufficient allowance is obtained in the correction range 11 l of the correcting optical unit 11 d , and electric power saving can be realized in the stage of object observation.
As described above, Embodiment 3 of the invention realizes high-speed image-taking operation, which improves high-speed continuous shooting and image-taking mobility.
The reason why the driving of the correcting optical unit 11 d is stopped during re-image-taking operation is that, when the re-image-taking operation is increased in speed by omitting the period of returning the correcting optical unit 11 d to its initial position, the correcting optical unit 11 d is not positioned at its initial position when starting re-image-taking operation, and the driving of the correcting optical unit 11 d is not necessary (exceeds the correcting range 11 l soon). Then, the correcting optical unit 11 d is fixed at the position where it is positioned at the end of the first image-taking operation and waits there until all image-taking operations are finished. The reason for this is that image blurring is caused if the correcting optical unit 11 d is returned to its initial position during image-taking operation.
As described above, during re-image-taking operation, the correcting optical unit 11 is not driven for image blurring correction, so that the exposure period must be shortened so as not to cause image blurring.
Image deterioration due to vibration is determined by the magnitude of the vibration, the focal length of the image-taking optical system and the exposure period when taking the image. Generally, in the case of an image size of 135 format, when image-taking operation is carried out with the exposure period equal to one over the focal length (for example, 1/300 when the focal length is 300 mm), image blurring does not greatly stand out even in normal size printing. Therefore, when the conditions require re-image-taking operation, image-taking operation is carried out for the exposure period determined as mentioned above, and when image-taking operation for a longer period is required, image-taking operation is stopped again, and image-taking operation is carried out again.
Image-taking operation is thus repeated a plurality of times with a short exposure period, and the plurality of obtained images are synthesized while conforming their characteristic points to improve exposure.
As described above, in Embodiment 3, the processing of conforming the characteristic points and synthesis operation are not performed in the camera, but performed with a personal computer by using application software supplied with the camera after the image-taking operation. In this case, the following method is used for making this processing easier.
The taken images are recorded on a recording medium (recording circuit 119 ) that consists of a semiconductor memory or the like and can be attached to and detached from the camera. Among the recorded images, an image obtained by normal image-taking operation, that is, an image obtained by one image-taking operation without setting the vibration control mode, an image obtained by one image-taking operation even in the vibration control mode since the correcting optical unit 11 d is driven within the correction range 11 l and image-taking operation is not repeated a plurality of times, and images obtained through a plurality of times of image-taking operations, are mixed. Selection of images obtained through a plurality of times of image-taking operations from these images becomes remarkably troublesome as the number of recorded images increases.
Therefore, in this embodiment, the images taken in a plurality of times are recorded so as to be identified in the recording circuit 119 . The application software supplied with the camera recognizes the identification information and extracts only the images taken in a plurality of times and automatically synthesizes the plurality of images by conforming the positions. The synthesized image is recorded again on the recording circuit 119 , and the plurality of images that have been recorded on the recording circuit 119 are erased. Thereby, space is created in the recording capacity of the recording circuit 119 .
Furthermore, in Embodiment 3, in the stage of object observation, optical vibration control by driving the correcting optical unit 11 d is not carried out. This is for effectively using the correction range 11 l by setting the correcting optical unit 11 d at the initial position when starting image-taking operation. If the correcting optical unit 11 d is driven in the stage of object observation, when image-taking operation is started, the correcting optical unit 11 d may be positioned close to the limit of correction range 11 l for image blurring correction. In this case, re-image-taking operation may be carried out immediately after starting image-taking operation.
In Embodiment 3 the object of which is to improve the continuous shooting speed, it is desired to reduce as much as possible the occasion for change into re-image-taking, so that the correcting optical unit 11 d is stopped at its initial position until image-taking operation is started. Alternatively, on the basis of motion vector information obtained from the images and information of the vibration detection sensor 11 f , image information is displayed on the back face monitor (display unit 118 ) of the camera by changing the position of extracting the image information from the image pickup element 19 a for each frame, whereby image blurring correction processing is carried out. In this case, there is a possibility that the position of extracting also deviates when starting image-taking operation. Therefore, the extraction region may be limited. However, image extraction processing is not carried out when starting image-taking operation, that is, in actuality, the region in which the images obtained through re-image-taking operation and the synthesized image do not conform each other is cut, so that the extraction position limitation does not reduce the image blurring correction amount. In addition, in the stage of object observation, the correcting optical unit 11 d is not driven, and this reduces the power consumption.
FIG. 9 is a connection view of the personal computer and the camera, wherein the digital camera 31 is cable-connected by using a USB cable 32 to the notebook computer 33 . By this connection, the image data in the digital camera 31 can be transmitted to the notebook computer 33 . It is also possible that, for data control, in place of cable connection using the USB cable, a recording medium (a recording circuit 119 ) attached to the digital camera 31 is detached from the camera main body and inserted into a slot 34 formed in the notebook computer 33 .
In the notebook computer 33 , application software relating to this embodiment is installed in advance. Operations of this application software are described below.
-
- 1) From images recorded on the recording circuit 119 of the digital camera 31 , a plurality of images recorded for image synthesis processing are identified and automatically taken into the notebook computer 33 . At this point, since the image data are moved, the images to be synthesized in the recording circuit 119 are erased.
- 2) In the notebook computer 33 , deviations among the images taken by continuous shooting are corrected by means of coordinate conversion processing and synthesized to improve exposure.
- 3) The synthesized image is subjected to gamma correction processing and compression processing, and furthermore, the end deficient portions of the images generated at the point of synthesis processing are cut, and the reduction in image size is compensated by spreading.
- 4) The synthesized image is returned to the recording circuit 119 again via the cable 32 , etc.
These operations are described in detail.
As shown in FIG. 10, the inside of the folder 35 of the recording circuit 119 inside the camera is divided into two folders including a normal image folder 36 for storing images taken by normal image-taking operation and a synthesizing image folder 37 for storing images to be synthesized continuously taken when the correcting optical unit 11 d exceeds the correction range 11 l during image-taking operation.
In the normal image folder 36 , images 39 , 310 , 311 a , 312 , 313 d , and 314 are stored in order of image-taking operation. Herein, one image (image taken by using the illumination unit 16 a ) 311 a ( 003 a ) among the images to be synthesized continuously taken when the correcting optical unit 11 d exceeds the correction range 11 l during image-taking operation is also stored in the normal image folder 36 . The image number of the image 312 next to the image 311 a skips one number (skips 004 ).
Likewise, the image 313 d ( 006 d ) is one of the plurality of images taken for synthesis processing, and in this case, image-taking operation is carried out in order from the image 006 a , however, rear-curtain flash sync is selected, and the illumination unit 16 a is used for the last frame, and therefore, this last frame is stored in the normal image folder 36 . In addition, the next image number ( 007 ) is skipped.
In the synthesizing image folder 37 , all the images taken by continuous shooting for synthesis processing are recorded, and the camera 31 transmits the image data together with the folder 37 into the notebook computer 33 . Then, the positions of the images 311 a through 311 e taken by continuous shooting are conformed.
In this case, the image 311 a ( 003 a ) taken by using the illumination unit 16 a is used as a reference image. Images whose positions have been conformed are synthesized to complement exposure, subjected to gamma correction processing and compression processing, and the end deficient portions of the images generated when the images are synthesized are cut, and the reduction in image size is complemented by spreading, and then the image is temporarily stored in the memory of the notebook computer 33 .
Likewise, the images 313 a through 313 d obtained by continuous shooting are conformed in position. At this point, the image 313 d ( 006 d ) taken by using the illumination unit 16 a is used as a reference image. The images whose positions have been conformed are synthesized to complement exposure, subjected to gamma correction processing and compression processing, and the end deficient portions of the images generated when the images are synthesized are cut, and the reduction in image size is complemented by spreading, and then the image is temporarily stored in the memory of the notebook computer 33 .
The images thus recorded in the memory of the notebook computer 33 are collectively transmitted to the recording circuit 119 of the digital camera 31 , and are stored in the normal image folder 36 in order of image-taking operation. Namely, no image data is left in the notebook computer 33 .
In the normal image folder 36 of the recording circuit 119 after the processing mentioned above, images (synthesized images) 315 and 316 are stored next to the images 311 a and 313 d that have been taken by using the illumination unit 16 a and recorded in advance as shown in FIG. 11. Therefore, a photographer can compare the synthesized image and the image that was taken by using the illumination unit 16 a and has the same composition although it has not been synthesized, and if either one is not necessary, it can be easily erased.
FIGS. 12A to 12C are flowcharts describing the operations of the camera according to this embodiment, and this flow starts when the power source of the camera is turned on.
In Step S 1001 , camera checking is ca