脉冲连续摄影原理与超高速高动态成像验证
Spiking Continuous Photographing Principle and Demonstration on Ultrahigh Speed and High Dynamic Imaging
查看参考文献14篇
|
文摘
|
传统相机采用定时曝光方式获得静态图像或图像序列形式的视频,不能有效表达极高速的光子流过程,而且导致高速和高动态相互对立的"两难困境".基于光电传感器像素独立的特点,提出了连续摄影原理:每个像素各自把接收的光子流转换成电子流,连续测量光电流并转换成数字信息流,实现像素级别的连续表达,再按照像素空间排布组成序列阵列,就是对像素平面入射光子流过程的连续表达.截取序列阵列任何一个时刻的状态就可得到该时刻的图像,从而实现连续成像.进而,提出了把光电子流调制为脉冲序列的脉冲连续摄影原理:像素从清空状态开始积累电荷,达到额定阈值时产生一个脉冲作为积满标志并自动复位重新开始累积,如此重复.一个脉冲积满所经历的时间称为它的脉宽,与这个时段的光强成反比,据此可以估计这一时段的光强.脉冲按照自然时序排列而成的脉冲序列就是对光电子流过程的数字化表达.各像素产生的脉冲流按照像素空间分布排列而成的脉冲流阵列称为视象,蕴含了光过程丰富的时空信息,从中可以生成任意时刻的图像,实现超高速、高动态、无模糊连续成像,解决了定时曝光成像的"两难困境".脉冲连续摄影原理的唯一参数是累积阈值Q,对应积满一个脉冲所需光子/电子数,唯一变量是脉冲累积时长τ,是最容易准确测量的物理量,并可以利用自然时序简练表达.任意时刻的光强为Q/τ,由所处脉冲的宽度τ决定,τ的取值范围是(0,∞),因此理论上可以表达任意强度的光,动态范围无穷大.实际物理实现中,强光表达的极限取决于电路最短读出时间,暗光表达极限就是暗电流强度,它累积为一个脉冲的时长τ̂,τ̂/τ̌即相机的动态范围,采用常规光电器件和电路就能实现160 dB甚至180 dB超高动态成像.在要求成像时间灵敏度小于τ̂时,可通过对累积电压亚阈值量化,实现超高动态成像.光子流和光电子流是符合泊松分布的离散随机过程,往往是一个"分段线性"过程光强瞬时突变,突变之间光强稳定.提出了实现脉冲流最简无损压缩的首脉冲编码方法:只在光强变化时输出首脉冲发放时刻及其脉宽,之后不输出脉冲就表示重复.采用首脉冲序列编码,可在几乎不增加输出数据量条件下,大幅提高时间计量精度,显著提升系统的动态范围和时间灵敏度.采用成熟CMOS光电器件和标准工艺,研制了两款脉冲连续摄影芯片和脉冲相机,空间分辨率分别为10万像素和100万像素,采用4万赫兹同步脉冲输出,即最短脉宽25 μs.实拍试验验证了脉冲连续摄影原理的可实现性和超高速高动态无模糊成像性能.脉冲连续摄影视象蕴含了光子流的时空信息,替代图像和视频,将从根本上重塑计算机视觉和视觉信息处理技术和产业.创立了脉冲视觉开源算法体系SpikeCV,用较低计算复杂度实现了速度比人眼快千倍的超高速目标检测跟踪识别系统.作为脉冲连续摄影的对称过程,脉冲连续显示把超高速脉冲流调制为极高速光子流,能够实现类似自然光的高速变化,解决了传统显示系统因为低帧率带来的运动模糊和视觉疲劳眩晕等问题.连续摄影和连续显示相结合,可实现近似玻璃的单向透明显示和超高速无介质光通信. |
|
其他语种文摘
|
Traditional camera accumulates photons during an exposure time window to generate a still image or a video in the form of image sequences, which lose the temporal process of the photons flow, leads to an irreconcilable dilemma between high-dynamic and high-speed imaging. Based on the fact that the pixels of the photoelectric sensor are independent, a new continuous photographing principle is proposed: each pixel converts the received photon flow into an electronic flow independently, continuously measures the photoelectric current and converts it into a digital flow, and represents continuously at the pixel level. Then the sequence array ranked according to the pixel layout is a continuous representation of the photons shotting on the sensing plane. The image of any moment can be obtained by intercepting the state of the sequence array, so as to realize continuous imaging. Furthermore, the spiking continuous photographing principle that modulates the photon flow into a spike sequence is proposed: for each pixel, accumulate charge from the reset state, generate a spike as a flag once the specified threshold is reached, reset and repeat. The duration that a spike takes to be fired is called its spiking width, which is inversely proportional to the light intensity during this period. Based on this, the light intensity during this period can be estimated. The spikes sequence with natural temporal order is a digital representation of the photon flow process. The array of the spike sequences according to the spatial layout of pixels is called" viform", which contains rich spatial and temporal information of the light process. An image at any moment can be calculated from viform, thereby ultrahigh-speed, high-dynamic and non-blurred continuous imaging is achieved, and the dilemma between high-dynamic and high-speed imaging of exposure imaging is solved completely. The unique parameter of the spiking continuous photographing principle is the accumulation threshold Q, which corresponds to the number of photons/electrons required to fire a spike. The unique variable is the spike accumulation time τ, which is the easiest to measure accurately and can be concisely represented by natural temporal order. The light intensity at any moment is Q/τ, which is determined by the spike width τ, which range is (0, ∞). Thus, theoretically, arbitrary light intensity could by be represented, and lead to infinite dynamic range imaging. In the physical implementation, the strongest light could be represented depending on the shortest readout time τ̂ of the circuit, and the weakest light is the dark current, which can be accumulated as a spike within duration τ̂. τ̂/τ̌ makes up the imaging dynamic range. With conventional photoelectric devices and circuits, ultrahigh-dynamic imaging with 160 dB or even 180 dB can be achieved. When the imaging time sensitivity is required to be less than τ̂, ultrahigh-dynamic imaging can be achieved by sub-threshold quantization of the accumulated voltage. Photon and photoelectron flow are discrete statistical process that follows a Poisson distribution and is often a" segmental linear" process light intensity changes suddenly while the intensity is stable between successive mutations. A method so-called first spike encoding is proposed: only the moment of the first spike firing and its spike width are output when the intensity changes, and no spike is output thereafter to indicate repetition. This is the optimal lossless compression method of the spike sequence, where the dynamic range and time sensitivity can be significantly improved by improving the time measurement accuracy with almost no increase in representation data volume. Using mature CMOS photonic devices and standard processes, two spike continuous photographing chips and spike cameras with spatial resolutions of 0.1 million and 1 million pixels have been developed, respectively. |
|
来源
|
电子学报
,2022,50(12):2919-2927 【核心库】
|
|
DOI
|
10.12263/DZXB.20221075
|
|
关键词
|
光电传感器
;
脉冲序列调制
;
连续摄影
;
高动态成像
;
超高速成像
;
相机
;
视象
|
|
地址
|
北京大学计算机学院, 多媒体信息处理全国重点实验室, 北京, 100871
|
|
语种
|
中文 |
|
文献类型
|
研究性论文 |
|
ISSN
|
0372-2112 |
|
学科
|
一般工业技术;自动化技术、计算机技术 |
|
基金
|
国家自然科学基金
|
|
文献收藏号
|
CSCD:7415469
|
参考文献 共
14
共1页
|
|
1.
Rodney L.
The Quantum Theory of Light. 3rd Edition,2000
|
CSCD被引
1
次
|
|
|
|
|
2.
Philip N.
From Photon to Neuron: Light, Imaging, Vision,2017
|
CSCD被引
1
次
|
|
|
|
|
3.
Weckler G P. Operation of p-n junction photodetectors in a photon flux integration mode.
IEEE Journal of Solid-State Circuits,1967,2(3):65-73
|
CSCD被引
7
次
|
|
|
|
|
4.
Jun O.
Smart CMOS Image Sensors and its Applications. 2nd Edition,2020
|
CSCD被引
1
次
|
|
|
|
|
5.
Muller R. I2/L timing circuit for the 1ms-10s range.
IEEE Journal of Solid-State Circuits,1977,12(2):139-143
|
CSCD被引
1
次
|
|
|
|
|
6.
Brajovic V. A sorting image sensor: An example of massively parallel intensity-to-time processing for low-latency computational sensors.
Proceedings of IEEE International Conference on Robotics and Automation,1996:1638-1643
|
CSCD被引
1
次
|
|
|
|
|
7.
Frohmader K P. A novel MOS compatible light intensity-to-frequency converter suited for monolithic integration.
IEEE Journal of Solid-State Circuits,1982,17(3):588-591
|
CSCD被引
1
次
|
|
|
|
|
8.
Tanaka K. Novel digital photosensor cell in GaAs IC using conversion of light intensity to pulse frequency.
Japan Journal of Applied Physics,1993,32(11A):5002-5007
|
CSCD被引
2
次
|
|
|
|
|
9.
黄铁军.
对时空信号进行编码的方法和装置: CN105681787B,2019
|
CSCD被引
1
次
|
|
|
|
|
10.
Huang Tiejun.
Method and device for encoding spacetime signals: US10523972B2,2019
|
CSCD被引
1
次
|
|
|
|
|
11.
黄铁军. 超速电眼:全时成像芯片重塑机器视觉.
前沿科学,2019,13(2):16-21
|
CSCD被引
1
次
|
|
|
|
|
12.
黄铁军. 脉冲视觉:冲破图像的幻相.
中国图象图形学会通讯,2022,1(1):8-13
|
CSCD被引
1
次
|
|
|
|
|
13.
黄铁军. 脉冲视觉研究进展.
中国图象图形学报,2022,27(6):1823-1839
|
CSCD被引
5
次
|
|
|
|
|
14.
Huang T. 1000× faster camera and machine vision with ordinary devices.
Engineering,2022,15(11):1-10
|
CSCD被引
1
次
|
|
|
|
|
了解气象卫星更多信息，请访问