Principle and system of T-CUP

To enable real-time, ultrafast, passive imaging of temporal focusing, here, we have developed single-shot trillion-frame-per-second compressed ultrafast photography (T-CUP), which can image non-repeatable transient events at a frame rate of up to 10 Tfps in a receive-only fashion. The operation of T-CUP consists of data acquisition and image reconstruction (Fig. 1). For the data acquisition, the intensity distribution of a 3D spatiotemporal scene, I[m,n,k], is first imaged with a beam splitter to form two images. The first image is directly recorded by a 2D imaging sensor via spatiotemporal integration (defined as spatial integration over each pixel and temporal integration over the entire exposure time). This process, which forms a time-unsheared view with an optical energy distribution of E u [m,n], can be expressed by

$$E_{\rm{u}}\left[ {m,n} \right] = \eta \mathop {\sum }\limits_k \left( {h_{\rm{u}} \ast I} \right)\left[ {m,n,k} \right]$$ (1)

where η is a constant, h u represents spatial low-pass filtering imposed by optics in the time-unsheared view, and * denotes the discrete 2D spatial convolution operation. Equation 1 can be regarded as a single-angle Radon transformation operated on I[m, n, k] (detailed in Supplementary Note 1).

Fig. 1: Principle of operation for T-CUP. The beam paths for time-unsheared and time-sheared views are illustrated using magenta and green colors, respectively Full size image

The second image is spatially encoded by a pseudo-random binary pattern. Then the spatially encoded scene is relayed to a femtosecond shearing unit, where temporal frames are sheared on one spatial axis. Finally, the spatially encoded, temporally sheared frames are recorded by another 2D imaging sensor via spatiotemporal integration to form a time-sheared view with an optical energy distribution of E s [m,n]. This process can be described by

$$E_{\rm{s}}\left[ {m,n} \right] = \eta \mathop {\sum }\limits_k \left( {h_{\rm{s}} \ast I_{\rm{C}}} \right)\left[ {f_{\rm{D}},g_{\rm{D}} + k,k} \right]$$ (2)

where h s represents spatial low-pass filtering in the time-sheared view. I C [f D ,g D +k,k] is the spatially encoded scene. f D and g D are the discrete coordinates transformed from m and n, according to the distortion in the time-sheared view24. Equation 2 can be regarded as the Radon transformation of the spatiotemporal datacube from an oblique angle determined by the shearing speed of the streak camera and pixel size of the sensor (detailed in Supplementary Note 1).

Combining the two views, the data acquisition of T-CUP can be expressed by a linear equation,

$$\left[ {E_{\rm{u}},\alpha E_{\rm{s}}} \right]^T = \left[ {{\boldsymbol{O}}_{\rm{u}},\alpha {\boldsymbol{O}}_{\rm{s}}} \right]^TI$$ (3)

where α is a scalar factor introduced to balance the energy ratio between the two views during measurement, and O u and O s are the measurement operators for the two views (see Materials and methods and Supplementary Fig. S1). Thus T-CUP records a 3D dynamic scene into two 2D projections in a single exposure.

Image reconstruction of the scene can be done by solving the minimization problem of \(\min _I\left\{ {\frac{1}{2}\Vert \left[ {E_{\rm{u}},\alpha E_{\rm{s}}} \right]^T - \left[ {{\boldsymbol{O}}_{\rm{u}},\alpha {\boldsymbol{O}}_{\rm{s}}} \right]^TI \Vert_2^2 + \rho {\it{\Phi }}\left( I \right)} \right\}\), where \(\left\| \cdot \right\|\) denotes the l2 norm, Ф(I) is a regularization function that promotes sparsity in the dynamic scene, and ρ is the regularization parameter (detailed in Supplementary Notes 2). The solution to this minimization problem can be stably and accurately recovered, even with a highly compressed measurement26.

The integration of compressed sensing into the Radon transformation drastically reduces the required number of projections to two. The time-unsheared view, in which the projection is parallel to the time axis, losslessly retains spatial information while discarding all temporal information. The time-sheared view, on the other hand, preserves temporal information by projecting the spatiotemporal datacube from an oblique angle. As a result, these two views, as an optimal combination, enable one to record an optimum amount of information with the minimum number of measurements. However, a direct inversion of the Radon transform is not possible in this case due to the small number of projections and the fact that the linear system (Eq. 3) that needs to be inverted is under-determined. To solve this problem, compressed sensing is used. Leveraging the sparsity of the scene, as well as the random encoding in the time-sheared view as prior information, the compressed-sensing-based reconstruction algorithm uses the regularization-function-guided search to find a unique solution. Our simulation has demonstrated that this compressed-sensing-augmented two-view projection can retrieve a dynamic scene with a high reconstruction quality (Supplementary Fig. S2 and detailed in Supplementary Note 3).

In practice, T-CUP is embodied in an imaging system (Fig. 2 and detailed in Materials and methods) that uses several key devices to realize specific operations. Specifically, a charge-coupled device (CCD) camera performs spatiotemporal integration, a digital micromirror device (DMD) performs spatial encoding, and the time-varying voltage applied to the sweep electrodes in a femtosecond streak camera accomplishes femtosecond shearing. In addition, a compressed-sensing-based two-view reconstruction algorithm recovers the dynamic scene. The T-CUP system can capture a dynamic scene with spatial dimensions of 450 × 150 pixels and a sequence depth (i.e., number of frames per movie) of 350 frames in a single camera exposure. The frame rate of the reconstructed video is determined by v/d, where v is the temporal shearing velocity of the streak camera, and d is the pixel size of the internal CCD along the temporal shearing direction. By varying v, the frame rate can be widely adjusted from 0.5 to 10 Tfps. Thus, with single-shot data capture, a tunable ultrahigh frame rate, and an appreciable sequence depth, the T-CUP system is well suited for imaging single-event ultrafast transient phenomena occurring over a wide range of timescales (the characterization of the spatial and temporal resolutions of T-CUP is detailed in Supplementary Fig. S3 and Supplementary Note 4). The T-CUP temporal resolutions for 0.5, 1, 2.5, and 10 Tfps frame rates have been quantified to be 6.34, 4.53, 1.81, and 0.58 ps, respectively.

Fig. 2: Schematic of the T-CUP system. Inset (black dashed box): detailed illustration of the streak tube. CCD charge-coupled device, DMD digital micromirror device, MCP micro-channel plate Full size image

Imaging temporal focusing of a single femtosecond laser pulse using the T-CUP system

A typical temporal focusing setup consists of a diffraction grating and a 4f imaging system (Fig. 3a). The incident laser pulse is first spatially dispersed by the grating and then collected by a collimation lens. Finally, a focusing lens recombines all the frequencies at the focal plane of the lens (Supplementary Fig. S4 and detailed in Supplementary Note 5). Temporal focusing has two major features: first, the shortest pulse width is at the focal plane of the focusing lens4; second, the angular dispersion of the grating creates a pulse front tilt so that the recombined pulse scans across the focal plane5. The pulse front tilt angle can be expressed by \(\gamma = \tan ^{ - 1}\left( {\lambda _{\rm{c}}/Md_{\rm{g}}} \right)\) (refs. 27,28), where M is the overall magnification ratio, λ c is the central wavelength of the ultrashort pulse, and d g is the grating period. The femtosecond pulse that undergoes temporal focusing presents a complex spatiotemporal profile (Supplementary Fig. S4) that can be revealed only in the captured instantaneous light patterns. Even a picosecond-level exposure time would erase these spatiotemporal details via significant temporal blurring. This speed requirement excludes previous CUP systems23,24,25 from visualizing this ultrafast optical phenomenon. In contrast, T-CUP can achieve unprecedented real-time visualization with a single camera exposure.

Fig. 3: T-CUP for temporal focusing. a Experimental setup. CL collimating lens, FL focusing lens, TFP temporal focusing plane, f 1 and f 2 focal lengths. For the side view, a small amount of water vapor was used to scatter photons into the T-CUP system. b Representative frames from the front view (see Supplementary Movie S1 for the full evolution), showing the laser pulse sweeping along the y axis of the temporal focusing plane. The black circles denote the z axis. c Representative frames from the side view (see Supplementary Movies S1 and S2 for the full evolution), showing a single ultrashort laser beam propagating through the temporal focusing plane. The temporal focusing plane is marked by the dashed line. d Light intensity projected along the z axis, averaged and normalized individually, for each frame in c. The red dots are the data measured from the images (to avoid cluttering, only one data point is shown for every five data points measured and used for fitting), and the blue curves show the Gaussian fittings to the measured data. Full widths at half maxima (FWHMs) are computed for each fitted curve. e Surface plot of the normalized intensity along the primary optical axis as a function of t and z near the temporal focusing plane. f Measured pulse widths (FWHMs) as a function of position on the z axis near the temporal focusing plane. The location of the temporal focusing plane is set to be zero Full size image

We imaged the temporal focusing from both the front and the side (Fig. 3a) at 2.5 Tfps. A collimated femtosecond laser pulse (800 nm central wavelength, 50 fs pulse duration, 1 × 3 mm2 spatial beam size) was used to illuminate a 1200 line mm−1 grating. The 4f imaging system had a magnification ratio of M=1/4. In theory, the tilt angle for the pulse front at the temporal focusing plane was 75.4°.

For front-view detection, T-CUP captured the impingement of the tilted laser pulse front sweeping along the y axis of the temporal focusing plane (Fig. 3b and Supplementary Movie S1). The pulse swept a distance of ~0.75 mm over 10 ps, corresponding to a pulse front tilt of ~76°, which closely matches the theoretical prediction.

For side-view detection, weak water vapor was spread as a dynamic scattering medium. T-CUP revealed the full evolution of the pulse propagation across the temporal focusing plane (Fig. 3c, d, Supplementary Fig. S5, and Supplementary Movies S1 and S2): a tilted pulse propagates toward the right. As it approaches the temporal focusing plane, the pulse width continuously reduces, manifesting as an increasing intensity. At the temporal focusing plane, the focus of the pulse sweeps along the y axis at its peak intensity. The evolution after the temporal focusing plane mirrors the preceding process: the pulse width is elongated, and the intensity is continuously weakened. We then quantitatively analyzed the pulse compression effect of temporal focusing. Figure 3e shows the temporal profiles of the laser pulse on the z axis near the temporal focusing plane, demonstrating the sharp temporal focusing of the laser pulse. Figure 3f shows the pulse duration along the z axis near the temporal focusing plane. The full width at half maximum of the temporal profile is reduced from 10.4 ps to 1.9 ps—compressed by a factor of 5.5. It is notable that the measured pulse width is wider than the incident pulse, which is likely due to dispersion by optical elements and scattering, as well as to the temporal broadening caused by the finite temporal resolution of the T-CUP system.

T-CUP is currently the only technology capable of observing temporal focusing in real time. First, the entire process of the imaged temporal focusing event occurred in ~10 ps, which equals the previous state-of-the-art exposure time for a single frame23; hence, it could not be resolved previously. In contrast, T-CUP, using a frame interval of 0.4 ps, clearly resolved the intensity fluctuation, width compression, and structural change of the temporal focusing process. Second, the dynamic scattering induced by the water vapor makes the scattered temporal focusing pulse non-repeatable. In different measurements, the reconstructed results show a difference in spatial shape, compression ratio, and intensity fluctuation. To demonstrate the non-repeatability, another dataset for the sideways detection of temporal focusing is shown in Supplementary Fig. S6.

Although the ultrashort laser pulse was dispersed and converged in space by the 4f imaging system, it is worth noting that the effect of spatial focusing is limited. As the pulse approached the temporal focusing plane, the beam size fluctuated with a normalized standard deviation of 5.6% over a duration of 4.8 ps (Fig. 3d), while the peak on-axis intensity of the pulse increased approximately five-fold (Fig. 3e). Thus the intensity increase is caused dominantly by the temporal focusing.

Imaging light-speed phenomena in real time in both the visible and near-infrared spectral ranges

Four fundamental optical phenomena, namely, a beam sweeping across a surface, spatial focusing, splitting, and reflection, were imaged by the T-CUP system in real time (Fig. 4). In the beam sweeping experiment, a collimated near-infrared ultrashort laser pulse (800 nm wavelength, 50 fs pulse duration) obliquely impinged on a scattering bar pattern. The T-CUP system was placed perpendicular to the target to collect the scattered photons (Fig. 4a). Imaging at 10 Tfps, the T-CUP system clearly reveals how the pulse front of the ultrashort laser pulse swept across the bar pattern (Fig. 4b and Supplementary Movie S3).

Fig. 4: T-CUP for laser pulse sweeping, spatial focusing, reflection, and splitting. a Experimental setup for laser pulse sweeping through a scattering bar-pattern target. b Representative frames showing a single laser pulse obliquely impinging upon the bar pattern, imaged by T-CUP at 10 Tfps (see Supplementary Movie S3 for the full evolution). In the top left panel, the dashed line indicates the light pulse front, and the arrow denotes the in-plane light propagation direction (k xy ). c Top left panel: Experimental setup for the spatial focusing of a single laser pulse (532 nm wavelength, 7 ps pulse width) in a weakly scattering aqueous suspension. The 10× objective has a 0.3 NA and a 15 mm focal length. The field of view is indicated by the yellow dashed box. Remaining panels: Representative frames of T-CUP for spatial focusing of a single laser pulse, imaged by T-CUP at 2.5 Tfps (see Supplementary Movie S4 for the full evolution). d Time-lapse normalized intensity of the focus with a Gaussian fit. e Representative frames showing a single laser pulse (532 nm wavelength, 7 ps pulse duration) split by a 50:50 beam splitter, imaged by T-CUP at 2.5 Tfps (see Supplementary Movie S5 for the full evolution). A small amount of water vapor was sprayed into the path in the air to scatter the light from the scene into the T-CUP system. f Time courses of the average normalized intensities on both sides of the beam splitter (the blue and red dashed boxes in the top left panel of e). Both time courses were fitted by a Gaussian profile. g Representative frames showing a single laser pulse being bounced by two mirrors, imaged by T-CUP at 1 Tfps (see Supplementary Movie S6 for the full evolution). The field of view for T-CUP is indicated by the white dashed box Full size image

In addition, T-CUP enables real-time video recording of spatial focusing of a single picosecond pulse. This phenomenon has been previously documented by phase-contrast microscopy29 and interferometry30 using conventional pump–probe schemes. In contrast, here, T-CUP was used to capture the scattered light intensity in a single measurement. In the setup, a single laser pulse (532 nm wavelength, 7 ps pulse width) was focused by a 10× objective lens into a weakly scattering aqueous suspension. T-CUP imaged this phenomenon at 2.5 Tfps (Fig. 4c and Supplementary Movie S4). We analyzed the time course of the light intensity at the spatial focus. After normalization, the intensity profile (Fig. 4d) was fitted by a Gaussian function,\(\hat I\left( t \right) = {\rm{exp}}\left[ { - 2\left( {t - t_0} \right)^2/\tau _{\rm{g}}^2} \right]\), where t 0 = 24.76 ps, and τ g = 4.94 ps. The fitted result yields a 1/e width of 6.99 ps, closely matching the experimental specifications.

Imaging at 2.5 Tfps, T-CUP also revealed the spatiotemporal details of the beam splitting process of a single laser pulse (Fig. 4e and Supplementary Movie S5). Impinging on a beam splitter, part of the laser pulse was reflected immediately, while the transmitted portion propagated into the beam splitter and appeared on the other side of the beam splitter after a finite time. To quantitatively analyze the time course of the incident and transmitted pulses, we calculated the average light intensities in the two dashed boxes on both sides of the beam splitter (Fig. 4f). The measured temporal separation between the incident and transmitted pulses was 9.6 ps. Given the 2-mm thickness of this float glass beam splitter (refractive index n=1.52 at 532 nm) and the incident angle of ~25°, in theory, the light pulse needs approximately 10 ps to pass through the beam splitter. Thus our measured result agrees well with the theoretical value. It is also noteworthy that the time latency for the reflected and transmitted pulse (9.6 ps) is beyond the imaging capability of previous techniques23. T-CUP’s unprecedented frame rate reveals for the first time the spatiotemporal details of this transient event.

Finally, imaging at 1 Tfps, T-CUP was used to capture the reflection of a laser pulse by two mirrors over a sufficiently long time window (Supplementary Movie S6). In Fig. 4g, the first frame shows that the laser pulse has just entered the field of view (FOV). Subsequent frames show the propagating pulse being reflected by the two mirrors before finally traveling out of the FOV. It is noted that an inhomogeneous distribution of scatterers in the aqueous suspension led to increased scattered light intensity in the frames after 74 ps. For this reason, the pulse visually appears to be larger. However, the pulse width, when quantitatively measured via the cross-sectional full width at half maximum, was comparable to that in the rest of the frames.