A Comprehensive Methodology for Optimizing Read-Out Timing and Reference DAC Offset in High Frame Rate Image Sensing Systems

Abstract

This paper presents a comprehensive timing optimization methodology for power-efficient high-resolution image sensors with column-parallel single-slope analog-to-digital converters (ADCs). The aim of the method is to optimize the read-out timing for each period in the image sensor’s operation, while considering various factors such as ADC decision time, slew rate, and settling time. By adjusting the ramp reference offset and optimizing the amplifier bandwidth of the comparator, the proposed methodology minimizes the power consumption of the amplifier array, which is one of the most power-hungry circuits in the system, while maintaining a small color linearity error and ensuring optimal performance. To demonstrate the effectiveness of the proposed method, a power-efficient 108 MP 3-D stacked CMOS image sensor with a 10-bit column-parallel single-slope ADC array was implemented and verified. The image sensor achieved a random noise of 1.4 e⁻rms, a column fixed-pattern noise of 66 ppm at an analog gain of 16, and a remarkable figure-of-merit (FoM) of 0.71 e⁻·nJ. This timing optimization methodology enhances energy efficiency in high-resolution image sensors, enabling higher frame rates and improved system performance. It could be adapted for various imaging applications requiring optimized performance and reduced power consumption, making it a valuable tool for designers aiming to achieve optimal performance in power-sensitive applications.

1. Introduction

Various types of sensing systems are used in the era of the Internet of Everything [1,2,3,4,5,6,7,8,9,10]. With the advent of the Fourth Industrial Revolution, the demand for image sensor-related products has been increasing rapidly [5,6,7,8,9,10]. This enormous demand is not limited to consumer products and is continually expanding into defense, security, privacy, autonomous driving, and space science.

As the applications of image sensor systems become more diverse, they require extreme characteristics that are difficult to achieve, such as 200-megapixel (MP) resolution, 140 dB dynamic range, ultra-compact multi-functionality, and invisible ray cameras [11,12,13]. Additionally, increased power consumption and heat generation are issues as more image functions are required for high-resolution cameras, such as fast auto-focus (<0.3 s) and slow-motion video with ultra-high frame rates (>240 frames/s) [13,14,15,16]. Moreover, there has been a recent demand for ultra-low-power characteristics for always-on-display capabilities in imaging systems.

There are many ways to read out the output of a pixel array, but in most cases, an array of thousands of analog-to-digital converters (ADCs) is integrated into a column-parallel architecture and used to digitize the pixel output. A single high-precision ADC must be implemented with a sub-micron pitch (<1 μm) to realize a high-resolution image sensing system with low-noise characteristics. Therefore, single-slope ADCs with relatively simple structures are commonly used as pixel digitizers.

When utilizing a column-parallel ADC array to digitize the output of a pixel array, it is crucial to cancel out the dark noise of the pixels to obtain a high-quality image. To suppress the low-frequency noise of a pixel, most state-of-the-art image sensing systems use the digital correlated-double sampling (CDS) technique, which subtracts two digitized outputs of a pixel before and after it receives external light [17,18]. The digital CDS technique requires twice as many ADC operations, making the system timing budget insufficient for modern image systems with high resolution and a high frame rate. Furthermore, there are many other complex considerations for read-out timing, such as auto-zeroing (AZ), analog CDS, pixel reset, and the shutter.

This paper proposes a read-out timing optimization methodology utilizing an optimal reference offset for high-resolution, high-frame-rate image sensing systems. It includes considerations for the pixel array, the digital-to-analog converter (DAC) for the ramp reference of a single-slope ADC, and both analog and digital CDS techniques. With this timing optimization methodology, the amplifier bandwidth of the power-hungry comparator array can also be optimized, enabling energy-efficient image sensing. The rest of this article is organized as follows: Section 2 describes the architecture of modern image sensing systems. The proposed read-out timing optimization methodology is discussed in Section 3. Section 4 presents an implementation example with the proposed timing optimization. This paper concludes in Section 5.

2. Image Sensor Architecture

An imaging system has an inevitable trade-off between system performance and power consumption. To optimize this complex timing budget, the first step is to thoroughly understand how advanced image sensors are configured. Image sensors have evolved to implement pixel arrays and digitizer arrays on separate chips in stacks of three-dimensional (3-D) integrated circuits (ICs) using through-silicon via (TSV) or Cu-Cu connection techniques to achieve a small form factor [7], as shown in Figure 1. With the 3-D stacked architecture, an upper chip for the pixel array and a lower chip for the digitizer array can be separately implemented using optimal process technologies. Therefore, the rest of this section describes the structure of pixel and digitizer arrays for read-out timing analysis and system optimization.

2.1. Pixel Structure

Figure 2 shows a simplified active pixel sensor (APS) structure with one pinned photodiode and four transistors (4-T) for a CMOS image sensor (CIS) [19,20]. A photodiode in a pixel acts as a light-to-electron converter. When incident light is applied, a photodiode in the pixel produces electrons proportional to the intensity of the light. The four MOS transistors consist of a row selection transistor (SEL), a pixel reset gate (RG), a charge transfer gate (TG), and a source follower (SF) buffer. The output of the pixels is read out row-by-row with the rolling shutter method, so the SEL transistor is used to select the pixel row to digitize. After row selection, a reset sequence is required to eliminate residual electrons by turning on the RG before using the pixel as a sensor.

Once the pixel reset is completed, electrons are generated by the photodiode receiving incident light and transferred to a floating diffusion (FD) node by turning on the TG. An FD node has a capacitance on the order of fF or smaller, and electron-to-voltage conversion with a conversion gain (CG) occurs during this photodiode-to-FD charge transfer process. Furthermore, the FD node voltage becomes the output voltage of the pixel through the in-pixel SF buffer, which is digitized by the following ADC.

Figure 3 shows the timing diagram of the 4-T pixel with a digital CDS technique. To cancel out the pixel output variation, including pixel reset noise, a digital CDS technique is widely used. For a digital CDS function, two digitizations are performed, and the digital difference is equivalent to the perceived intensity of the light. Therefore, the dark signal is read before the TG is turned on, and the light signal is read after the TG is turned on.

2.2. Read-Out IC Structure

The SF elements in the pixel array on the top chip require a current load (I_PL) to function properly, which is usually implemented on the bottom chip. As shown in Figure 4, once the pixel reset (period A) is complete, the auto-zero (AZ) operation (period B) of the ADC can be started. During the AZ phase, DC offset and flicker noise are stored for the analog CDS operation, and a self-bias network is operated to determine the operating bias of the amplifier. When the AZ operation is completed, the digitizer reads out the data before and after receiving the incident light and then finds the difference to obtain the result of the digital CDS (periods C to I). In the single-slope counting sections (periods E and I), a comparator compares the pixel output with the reference voltage, which is the ramp signal implemented based on the DAC. Additionally, the reference offset (OFF_RAMP) can be added before the start of the ramping to prevent missing the dark signal, and the added offset is naturally canceled out with the digital CDS technique.

3. Read-Out Timing Optimization Methodology

A simplified block diagram of an image sensor is shown in Figure 5. To achieve a column-parallel ADC architecture, a comparator must be composed of a simple structure, which is a 5-transistor first amplifier and a common-source second amplifier. The two-stage amplifier with an open-loop topology is well-used in an ADC array structure [7,8,18,21]. The read-out sequence of the image sensor from period A to I (one-row read-out time) is repeated until the entire pixel array has been read row by row. Therefore, the one-row read-out time can be determined based on the pixel resolution and the target frame rate of the image sensing system. For example, if a 100 MP image sensor (10,000 × 10,000) is to be digitized at a target of 10 fps, the one-row read-out time would be 10 μs in a single ADC per single pixel column (1 ADC/col) structure. The one-row read-out time should be carefully distributed from period A to period I without any redundant or bottleneck periods. In this paper, an advanced read-out timing optimization methodology is proposed with an optimum reference offset.

3.1. Period A: Reset

At the beginning of every horizontal read-out time, a pixel row for digitization should be selected using the SEL transistor. Additionally, a reset operation at the FD node should be completed to empty the FD capacitor C_FD. The kT/C noise generated during the reset period is suppressed by the digital CDS technique. This reset and selection of the pixel are relatively independent of the image resolution and can be defined as an absolute time interval according to a pixel structure.

3.2. Periods B and C: AZ

Before starting the AZ operation, the RG is turned off, which causes a voltage fluctuation (ΔRG_OFF) that is transferred to the bottom digitizer chip through the 3-D chip-to-chip connection. During the AZ period, therefore, the effect of the voltage fluctuation from the top pixel chip should be sufficiently settled, as should the operation of the amplifier to determine the DC bias and store low-frequency noise. In this period, the amplifier of the single-slope ADC is in a very fast unity-gain configuration and has a very small time constant. Therefore, the settling bottleneck induced by the ΔRG_OFF is the output of the pixel, which is the input of the ADC. With the negligible time constant of the ADC, the resistance for the RC time constant is determined by the transconductance of the pixel source follower, the on-resistance of the pixel selection transistor, and the metal line resistance of the pixel output. The time constant and slew rate of the SF can be obtained as follows:

τ_SF = (1/g_m,SF + R_SEL + R_LINE)·C_LINE,

(1)

SR_SF = I_PL/C_LINE

(2)

where g_m,SF is the transconductance of the SF, R_SEL is the on-resistance of the SEL, and R_LINE is the line resistance from the pixel to the ADC, including the chip-to-chip connection line. In the worst-case settling situation, slewing is caused by the condition τ_SF·SR_SF < ΔRG_OFF, and the required settling and slewing voltage can be given by:

ΔV_SETTLE,B = τ_SF·SR_SF,

(3)

ΔV_SLEW,B = ΔRG_OFF − τ_SF·SR_SF.

(4)

Then, the required time for the slewing and settling can be derived as follows:

T_SLEW,B = ΔV_SLEW,B/SR_SF,

(5)

T_SETTLE,B = τ_SF·ln(ΔV_SETTLE,B/E_TARG,B)

(6)

where E_TARG,B is the target achieved settling error in period B, and the timing budget for the period can be obtained as T_SLEW,B + T_SETTLE,B.

For period C, the voltage fluctuation induced by the turn-off signal of the AZ is well suppressed by the pseudo-differential amplifier topology of the ADC. Therefore, the timing budget for this period can be defined as a small absolute value for non-overlapping clock timing.

3.3. Period D: Reference Offset and Its Counting

In the ideal case, the ADC decision time of the dark signal is the end of period D. However, the pixel output has a wide output variation, so it can be missed without the ramp offset (OFF_RAMP). Therefore, in general, the dark ramping period should be long enough to include the output variation before digital CDS. However, if the ramp settling at the output of the amplifier is not sufficient until the time of the ADC decision, this settling error cannot be suppressed by the digital CDS technique. To achieve the high color linearity (CL) characteristic of an imager, the linearity relative to the ideal dark signal should be constant with respect to the light intensity. The color linearity (CL) error can be expressed as follows:

$$\left(1-\frac{\left(O_{\mathrm{X}}-O_0\right)/\left(O_{\mathrm{X},\mathrm{IDEAL}}-O_{0,\mathrm{IDEAL}}\right)}{\left(O_{\mathrm{REF}}-O_0\right)/\left(O_{\mathrm{REF},\mathrm{IDEAL}}-O_{0,\mathrm{IDEAL}}\right)}\right)·100$$

(7)

where O_X is the digitized output with the external incident light equivalent to the X LSB input, O₀ is the output with no input, and O_REF is the output with the high code LSB input for the ratio calculation. In addition, O_X,IDEAL, O_0,IDEAL, and O_REF,IDEAL represent the ideal output values without any settling error. In high-resolution image sensors, the remaining settling error can thus degrade the CL error.

In this paper, a read-out timing optimization methodology is proposed to find the optimal reference DAC offset with the optimal settling error. There are two factors that contribute to settling errors in this period. The first factor is the voltage fluctuation due to OFF_RAMP. With the reference offset and the target achieved settling error in period D (E_TARG,D), the timing budget for this period is given by:

$${\tau }_{\mathrm{OTA}1} ln\left(OF{F}_{\mathrm{RAMP}}/E_{\mathrm{TARG},\mathrm{D}}\right)$$

(8)

where τ_OTA1 is the time constant of the first amplifier, which is directly related to the bandwidth of the amplifier and can be approximated by the time constant of the single-slope ADC (τ_ADC).

The second factor that affects settling error in this period is ramp settling. Figure 6 shows ideal and realistic reference ramp waveforms, where t_CCLK is the unit-time step of the counter clock frequency. The reference ramp signal with a finite amplifier bandwidth of the following ADC causes a time-variable delay at the output of the ADC. At the start of the ramp, this time-variable delay is zero, which is the minimum delay. After sufficient settling time, the ramp delay gradually increases to the time constant of the amplifier, which is the maximum delay. In a single-slope ADC, the decision time is directly digitized by a following counter; thus, this ramp settling error must be well suppressed before the ADC decision.

For an ideal ramp, the time to count OFF_RAMP with an input signal of X LSB is given by:

$$\left(\raisebox{1ex}{$OF{F}_{\mathrm{RAMP}}$}\!\left/ \!\raisebox{-1ex}{$LSB$}\right.+X \right)·t_{\mathrm{CCLK}}.$$

(9)

For a realistic ramp, however, the time taken for a decision can be determined by finding the zero-crossing solution of the following equation:

$$\frac{-LSB}{t_{\mathrm{CCLK}}}·\left(t-{\tau }_{\mathrm{OTA}1}·(1-e^{\frac{-t}{{\tau }_{\mathrm{OTA}1}}})\right)+X+OF{F}_{\mathrm{RAMP}}.$$

(10)

Using (9) and (10), the CL in (7) can be estimated with an X LSB input and a reference input.

Figure 7 shows the CL error estimation versus the settling time budget for OFF_RAMP with an input of 10 LSBs and a reference of 256 LSBs. As an example, to ensure linearity characteristics above 99%, the minimum time for period D′ can be chosen as a relative value of 3τ_OTA1. As shown in Figure 7, with a sufficient settling time of more than 4τ_OTA1, the CL error becomes relatively independent of the ramp offset.

3.4. Period E: Dark Counting

Although the ideal decision timing of the dark signal is at the end of period D, the ramping period should be longer to include the peak-to-peak variation of the pixel output. If the ramping period is too short to include all pixel output, the fixed-pattern noise (FPN) of the output image is severely degraded. Therefore, it is important to budget the ramping period to the appropriate time, which can be iteratively determined between Monte-Carlo simulation of the ADC and the timing optimization method presented in this paper. After the iterations, the timing budget for period E can be defined.

3.5. Periods F, G, and H: TG

After the reset counting, the TG of the pixel must be turned on to transfer the electrons accumulated at the PN junction of the photodiode to the FD node for light counting. After reset counting, a small timing margin before turning on TG is required to avoid clock overlapping, so allocating a small absolute time is enough for period F.

For period G, the on-time of the TG should be long enough to allow sufficient photodiode-to-FD charge transfer. By comprehensively considering the structure and process of the pixel array, including back deep trench isolation (BDTI)/front deep trench isolation (FDTI), and front-side illumination (FSI)/back-side illumination (BSI), the timing budget for the on-time of the TG can be defined, which is independent of the ADC.

When the TG is turned off, a voltage fluctuation (ΔTG_OFF), which is similar to ΔRG_OFF in period B, is induced and transferred to the digitizer chip. Since the ramping time in period Hʹ is the same as that in period Dʹ, which is used for OFF_RAMP ramping, the settling time for ΔTG_OFF can be optimized with period H*. At the pixel output, the required settling and slewing voltages can be derived as follows:

ΔV_SETTLE,H = τ_SF·SR_SF,

(11)

ΔV_SLEW,H = ΔTG_OFF − τ_SF·SR_SF.

(12)

The minimum time budgets for the slewing and settling voltages are then given by:

T_SLEW,H = ΔV_SLEW,H/SR_SF,

(13)

T_SETTLE,H = ln(ΔV_SETTLE,H/E_TARG,H)

(14)

where E_TARG,H is the target achieved settling error in period H, and the minimum time budget for period H* can be calculated by:

(T_SLEW,H + T_SETTLE,H) − T_D′

(15)

where T_D′ is a chosen time budget for period Dʹ, considering the result shown in Figure 7.

3.6. Period I: Light Counting

For the single-slope counting of the light digitization, the ADC decision timing is dependent on the light intensity. If there is no light coming into the pixel chip, the ADC decision occurs with the same timing as the dark digitization. If there is detectable light, stronger light intensity leads to a later ADC decision. Therefore, the light counting period should sufficiently cover the pixel output range, and then the count of this period must be longer than 2^N LSB with an N-bit single-slope ADC. The timing budget for this period is then given by:

(2^N + COUNT_MARGIN)/t_CCLK

(16)

where COUNT_MARGIN is a single-slope counting margin that considers the system offset, mismatch, noise, and PVT variation.

3.7. Timing Optimization

Based on the timing analysis of each read-out period, an optimal timing diagram for a high-resolution image sensor can be derived. By utilizing the proposed timing optimization methodology with an optimal offset of a ramp reference, an optimized time for each period can be assigned, and an optimal ramp offset and amplifier bandwidth can also be achieved.

For example, consider a 7680 × 4320 pixel array (4K) that needs to be digitized with a 12-bit ADC array at 23 fps. The ADC array needs to process the pixel output of 4320 rows 23 times in 1 s, and a one-row read-out time is then 10 μs. With a specific pixel structure, system architecture design, and circuit simulation results, design parameters for a high-resolution image sensor can be achieved, as shown in

Table 1. Design parameters for timing optimization examples.

Design Parameter	Value
Full-scale range (FSRADC)	1 V
Maximum analog gain	16 V/V
Counter clock (1/tCCLK)	1 GHz
Reset-on (TA)	1 μs
ΔRGOFF	1 V
𝜏SF	0.05 μs
ILOAD	5 μA
ETARG,B	5 LSB
ETARG,D = ETARG,H	0.05 LSB
Timing margin (TC = TF)	50 ns
Settling for OFFRAMP	3τOTA1
Dark counting (TE)	200 LSB
TG-on (TG)	1 μs
ΔTGOFF	1 V
COUNTMARGIN	256 LSB

. With the design parameters, the settling time for ramp offset and the time constant of the amplifier versus ramp offset can be calculated, as shown in Figure 8. A large ramp offset is required to ensure a sufficient ramp offset settling time, which in turn requires a small time constant, which increases power consumption.

Through the iterative calculation based on the other parameters in Table 1 and the equations in Section 3, optimized time budget results can be achieved, as shown in

Table 2. Time budget results of the example.

Period	Time Budget
A	1000 ns
B	851 ns
C	50 ns
D	885 ns
(D’)	360 ns
E	200 ns
F	50 ns
G	1000 ns
H	1635 ns
(H*)	1275 ns
(H′)	360 ns
I	4328 ns
Sum (one-row)	10,000 ns (=10 μs)

. With the proposed timing optimization methodology, an optimal reference offset of 360 LSB was achieved. Furthermore, an optimal amplifier time constant of 112.3 ns is also derived, which is equivalent to a bandwidth of 1.42 MHz. Without optimizing the reference offset as proposed in this paper, the power efficiency of an image sensing system becomes very poor. For example, an amplifier bandwidth of 2.12 MHz would be required to maintain the same CL error with an unoptimized reference offset of 240 LSB.

With this approach, the power consumption of the amplifier array, which is one of the most power-hungry circuits, can be minimized. This can increase the system’s energy efficiency or frame rate by minimizing one-row read-out timing.

4. Implementation and Experimental Results

A power-efficient digitizer array for verifying the proposed time budgeting method is implemented in a 28-nanometer process. The prototype digitizer is designed with an optimal reference ramp offset and a 10-bit column-parallel single-slope ADC array. Figure 9 shows an annotated microphotograph of the digitizer chip, which can be stacked with a pixel chip. The comparator array and counter array are operated with a supply voltage of 2.8 V and 1 V, respectively. The peripheral blocks include a DAC for reference ramp signal generation, a voltage doubler for the pixel chip, and reference current generation.

The digitizer array chip is connected to a 0.7 μm 108 MP pixel array chip in a 3-D stacked configuration for its performance verification, and the low-frequency noise is suppressed using the digital CDS technique [7]. Figure 10 shows the measured random noise (RN) and column FPN. The sample image captured using the 3-D stacked CIS at 20 lux and 10 fps is shown in Figure 11. An RN of 1.4 e⁻rms and a column FPN of 66 ppm are measured at an analog gain of 16. The 108 MP imager consumes only 551 mW and also achieves a remarkable figure-of-merit (FoM) of 0.71 e⁻·nJ based on the common FoM equation for image sensor applications [10]. In

Table 3. Performance Summary and Comparison.

Parameter	This Work [7]	[5]	[11]	[13]	[14]	[16]
Pixel pitch	0.7 μm	1.5 μm	2.45 μm	1.1 μm	1.1 μm	2.7 μm
# of pixels	108 MP	246 MP	133 MP	13.5 MP	33.8 MP	1.38 MP
Frame rate	10 fps	5 fps	60 fps	34 fps	240 fps	120 fps
RN	1.4 e−rms	7.1 e−rms	7.7 e−rms	1.8 e−rms	3.6 e−rms	3.5 e−rms
HN	0.03 e−rms	–	–	–	–	–
Column FPN	66 ppm	–	–	–	–	–
Power Consumption	551 mW	1970 mW	11,000 mW	258 mW	3000 mW	205 mW
FoM 1	0.71 e−∙nJ	11.36 e−∙nJ	10.61 e−∙nJ	1.01 e−∙nJ	1.36 e−∙nJ	4.33 e−∙nJ

¹ FoM (e⁻·nJ) = (Power × Noise)/(# of Pixels × Frame Rate).

, the performance of the 108 MP imager is summarized and compared with previously published works [5,11,12,14,16]. Compared to other image sensors, this work shows a remarkable FoM with a low RN.

5. Conclusions

This work presents a timing optimization methodology for power-efficient high-resolution image sensors with column-parallel single-slope ADCs. By optimizing the ramp reference offset and amplifier bandwidth, the power consumption in the amplifier array is reduced without compromising performance. The methodology has been successfully applied to a 108 MP 3-D stacked CMOS image sensor, resulting in a random noise of 1.4 e⁻rms, column fixed-pattern noise of 66 ppm, and FoM of 0.71 e⁻·nJ. The importance of this work lies in its ability to enhance energy efficiency in high-resolution image sensors, which allows for higher frame rates and improved overall system performance. The proposed design methodology is versatile and could be adapted for a wide range of imaging applications that demand optimized performance and reduced power consumption.