Simultaneous optical and electrical recording by QPM and MEA

The setup for quantitative phase imaging was adapted from diffraction phase microscopy27,28 and is shown in Fig. 4. Recordings were initially performed at a 1 kHz frame rate using a fiber-coupled SLD (SLD830S-A20, Thorlabs, NJ) for illumination. For faster imaging (50 kHz), a supercontinuum laser (Fianium SC-400-4, NKT Photonics, Birkerød, Denmark) was used. In both cases, light from the fiber was collimated (SLD: F220FC-780, Thorlabs, NJ; the supercontinuum laser had a built-in collimator), and the spectral components of interest were reflected towards the sample arm with a dichroic mirror (FF980-Di01-t1-25×36, Semrock, Rochester, NY). The wavelength range was further restricted to 797–841 nm by an optical bandpass filter (FF01-819/44-25, Semrock, Rochester, NY). Images formed by a 10× objective (CFI Plan Fluor 10×, NA 0.3, WD 16.0 mm, Nikon, Tokyo, Japan) and a 200 mm tube lens (Nikon, Tokyo, Japan) were projected onto a transmission grating (46-074, 110 grooves/mm, Edmund Optics, Barrington, NJ). The first diffraction order passed through unobstructed, while the 0th order was filtered with a 150 μm pinhole mask placed in the Fourier plane of a 4-f optical system, consisting of a 50 mm lens (AF Nikkor 50 mm f/1.8D, Nikon, Tokyo, Japan) and a 250 mm biconvex lens (LB1889-B, Thorlabs, NJ). Interferograms were formed on the camera sensor (Phantom v641, Vision Research, Wayne, NJ), which has a full well capacity of 11,000 electrons (digitized to 12 bit). At up to 1000 fps, the camera can operate at a field size of up to 2560 × 1600 pixels, while at 50,000 fps, the FOV is reduced to 256 × 128 pixels. To decrease the memory storage requirements, we used a FOV of 768 × 480 pixels at 1000 fps. The external clock signal (Model 2100 Isolated Pulse Stimulator, A-M Systems, Sequim, WA) provided to the camera’s F-Sync input was triggered by the falling edge of a TTL trigger generated by the MEA. This trigger was also delivered to the camera via a digital delay generator (DG535, Stanford Research Systems, Sunnyvale, CA) to start the image acquisition in synchrony with the MEA recordings. The signal from the camera indicating that it is ready for a trigger, which was high during both image acquisition (10 s for 768 × 480 pixels at 1000 fps) and data transfer to the nonvolatile memory of the camera (~8 s for 768 × 480 pixels at 1000 fps), was recorded by the data acquisition card (DAQ) of the MEA system to mark the start of each movie.

Fig. 4: System layout. a Ultrafast QPM synchronized with the MEA recording system. Light from a supercontinuum laser is collimated (C1) and filtered by a dichroic mirror and a bandpass filter (F1). An optical phase image of the sample is obtained from the off-axis interferogram captured by the high-speed camera. b A transparent MEA plated with spiking HEK cells allows simultaneous near-infrared (NIR) optical recording and extracellular electrical recording. c Electrical and optical measurements are synchronized by recording the camera “ready” signal on one of the MEA channels. Trigger signals from the MEA and an external clock control the timing of the captured frames (see Materials and methods) Full size image

To retrieve the phase image, the Fourier transform of each interferogram was first calculated. The first diffraction order was then centered and isolated with a low-pass Gaussian filter. To monitor the changes in the phase image, the first interferogram in each movie sequence served as a reference. The phase difference between the reference and subsequent interferograms in the sequence was calculated by taking the argument of the pointwise complex division of the inverse Fourier transform of each filtered interferogram in the series and the filtered reference interferogram46. Fluctuations resulting from 1/f noise of the illumination source were eliminated by subtracting the average of each phase image to achieve zero mean over the FOV. Highly noisy pixels, typically corresponding to a region obstructed by the electrodes of the MEA, were excluded from the analysis. The phase retrieval process was accelerated using a graphics processing unit (Tesla K40c, Nvidia, Santa Clara, CA).

Electrical signals were recorded using a custom 61-channel MEA system built on a transparent substrate with ITO leads47,48. The recording electrodes were 10 μm in diameter and laid out in a hexagonal lattice with 30 µm spacing between the neighboring electrodes and 30 µm spacing between the rows. Platinum black was electrodeposited on the electrodes prior to every recording. The signals were amplified with a gain of 840 and filtered with a 43–2000 Hz bandpass filter. Signals were sampled at 20 kHz using a National Instruments DAQ (NI PCI-6110, National Instruments, Austin, TX). The “ready” signal from the high-speed camera (Phantom v641, Vision Research, Wayne, NJ), marking the start of an image sequence acquisition, was used to synchronize the electrical and optical recordings.

SNR optimization for spatial averaging

Since the pixels display both positive and negative phase shifts during the action potential, proper spatial averaging should take into account the polarity of the phase shift in each area:

$$\bar \varphi \left( t \right) = \frac{1}{N}\mathop {\sum }\limits_{ij} \varphi _{ij}(t) \cdot g_{ij}$$ (1)

where N is the total number of pixels, φ ij (t) is the phase shift at time t in pixel (i,j) and g ij is the sign of the overall phase shift in that area. Note that this is different from averaging the absolute value of the phase, since g ij remains constant for each given pixel, while the sign of the noise changes over time. Averaging of the absolute values would not reduce the noise. A subset of the FOV can be selected to optimize the SNR of the spatially averaged phase signal based on the knowledge of the maximum phase change during the action potential (signal amplitude Φ ij ) and noise level (Δ ij ) in each pixel. Both Φ ij and Δ ij are sorted across all pixels according to decreasing SNR into arrays Φ k and Δ k . Then, the maximum SNR for spatial averaging can be calculated as follows:

$$\overline {{\mathrm{SNR}}} = \mathop {{\max }}\limits_M \frac{{\mathop {\sum }

olimits_{k = 1}^M {\mathrm{\Phi }}_k}}{{\sqrt {\mathop {\sum }

olimits_{k = 1}^M {\mathrm{\Delta }}_k^2} }},M \in \left[ {1,N} \right] \cap {\Bbb Z}^ + $$ (2)

The first M pixels of the sorted arrays are then selected as the optimal subset for spatial averaging in that area.

Self-reinforcing lock-in spike detection

The SNR of individual pixels in the phase image is too low to reliably detect an action potential. However, since the spiking in a confluent culture of HEK cells is synchronized, spatial averaging can improve the SNR of the collective measurement. This spatial averaging must be applied taking into account the positive and negative phase shifts across the cell, as described previously. However, since neither the distribution of the phase shift across the FOV nor the spike timing in the optical recordings are known a priori, an iterative lock-in spike detection algorithm was developed to detect spikes in the noisy raw recording (see Fig. 5 and Supplementary Fig. S4).

Fig. 5: Block diagram of QPM data processing and the self-reinforcing lock-in algorithm for all-optical spike detection. After QPM processing, frame binning, and background removal, a random region of interest (ROI) is selected for the first iteration of the lock-in detection loop. The phase is spatially averaged across the new spiking ROI, band-stop filtered and correlated with the spike template. This correlation output is used to detect an optical spike trigger, which is applied to the original frames of the phase movie to produce a spike triggered average (STA). Noisy frames are detected and discarded at this step. The STA is then used to threshold a new spiking ROI and the loop repeats. With each iteration, the estimate of the ROI and the resulting STA are improved, and the loop exits when the ROI converges to a stable result Full size image

Since the SNR of individual pixels is insufficient for reliably determining the sign of the phase shift at that location, initial spiking ROI are chosen randomly. Phase changes are spatially averaged over the ROIs with a positive and negative sign randomly assigned to each pixel, yielding a single trace showing the displacement of the whole ROI. The randomly spatially averaged phase signal has a slightly improved SNR compared to single pixels. The resulting phase signal is filtered to remove mechanical vibrations and then cross-correlated with a characteristic template of the displacement during the action potential, which was obtained from a separate experiment using the reference MEA electrical recording to perform spatiotemporal spike-triggered averaging. The resulting cross-correlogram gives an estimate of the spike timing in the spatially averaged phase signal, with spike times corresponding to peaks above a set prominence threshold. The detected spikes are used to create a STA from the original movie, corresponding to a single action potential seen across the entire FOV. Each pixel in the STA movie is then correlated with the spike displacement template again to measure the similarity between that location’s displacement and the template, which we summarize as the lock-in image L defined as

$$L\left( {x,y} \right) = \mathop {\sum }\limits_t \phi (x,y,t) \cdot T(t)$$ (3)

Here, ϕ(x, y, t) is the phase movie and T(t) is the displacement template each pixel is expected to follow. The result provides an improved estimate of which parts of the FOV move together, and this ROI is used to start a new iteration of the loop. The process is repeated until the fraction of updated pixels between the new and old ROI decreases below a set threshold (200 pixels), and a final SNR optimization step reduces the size of the ROI. A step-by-step diagram of how the data evolves throughout the lock-in algorithm is shown in Supplementary Fig. S4.

Each iteration of the lock-in detection algorithm improves the estimate of the ROI and thus the quality of the optically detected spike train and the STA movie. Figure 3f shows that convergence occurs in four iterations. The standard deviation of the delay between the detected optical and electrical spikes decreases with subsequent iterations (Fig. 3e), converging into a narrower distribution around zero delay (perfect detection). The timing of the spikes detected in the initial iteration is nearly random and contains a large number of false positives and false negatives, but the final distributions are narrowly confined with an 11.6 ms standard deviation.

Sample preparation

Spontaneously spiking HEK cells expressing the Na v 1.3 ion channel were originally developed by Adam Cohen’s group at Harvard University35. The cells were grown in a 1:1 mixture of Dulbecco’s modified Eagle medium and F-12 supplement (DMEM/F12). The medium contained 10% fetal bovine serum, 1% penicillin (100 U/mL), streptomycin (100 µg/mL), geneticin (500 µg/mL), and puromycin (2 µg/mL). To spontaneously spike, HEK cells need to express not only Na v 1.3 but also the K ir 2.1 ion channel. Hence, they were transfected with the plasmid pIRES-hyg-K ir 2.1 AMP Resistance using CalFectin as the transfection reagent. Thirty minutes after a medium change, 3 µL of 2.2 µg/µL of the plasmid was mixed with 1 mL DMEM. Then, 3 µL of CalFectin was added to the solution, and 10–15 min later, the mixture was added to the cell culture. Approximately 6 h later, the cell culture was replaced, and the cells were used in experiments 24 h later.

Spiking HEK-293 cells were plated on the MEA coated with poly-d-lysine (P6407, Sigma-Aldrich) at a density of 2000 cells/mm2 one day before the recording. Culture medium (DMEM + 10% FBS) filtered with a sterile vacuum filter (SCGP00525, EMD Millipore, Darmstadt, Germany) was used to wash away any floating particles in the MEA chamber, using two applications of 800 μL each time. Two hours prior to the recording, 7/8 of the culture medium was replaced with the recording medium (Tyrode’s solution, in mM: 137 NaCl, 2.7 KCl, 1 MgCl 2 , 1.8 CaCl 2 , 0.2 Na 2 HPO 4 , 12 NaHCO 3 , 5.5 d-glucose). 1/8 of the culture medium was kept to avoid osmotic shock. Extra recording medium was then aspirated until the fluid just filled the 1.5 mm gap between the coverslip and the MEA (see the diagram in Fig. 4b and the actual bright-field image of the confluent HEK cells on MEA in Supplementary Fig. S2). The temperature was maintained at 29 °C during the recordings.

Whole-cell patch clamp