Amyloid beta biomarker for dementia detection by hyperspectral ophthalmoscope images

Hyperspectral fundus camera system

The hyperspectral system used in this study consists of a TRC NW8/8F fundus system combined with a DSLR camera, supporting Color, Red-Free, and Fluorescein Angiography imaging. It features a 16.2-megapixel camera, providing high-resolution images with a 45° field of view. The system includes nine internal fixation points, enabling the capture of wide-angle retinal views. Spectral data were collected using a spectrometer (Ocean Optics, QE65000) from a 24-color reference chart, specifically a 24-color checker. Additionally, the spectrum of the light source used in the fundus camera was recorded (Figure 2).

Hyperspectral fundus camera system

Hyperspectral fundus imaging algorithm was employed using an ophthalmoscope (Kowa Nonmyd 7) and a spectrometer (Ocean Optics, QE65000) to establish the correlation between the two devices by capturing a shared target. The standard 24-color card (X-Rite Classic, 24 Color Checkers) served as the common target. The calibration between fundus camera and spectrometer is performed by correcting the signal received from those devices through the 24-color chart. The resultant image was then converted into a set of sRGB channel values. From these datasets, a transformation matrix was derived. The entire modeling process is depicted in Figure 3.

Under identical lighting conditions, a digital camera is utilized to capture images of color checkers. The captured images are saved in the sRGB format, specifically as JPEG files. This process involves normalizing the values of the three primary color channels—Red (R), Green (G), and Blue (B)—from their original range of approximately 0–255 to a normalized range of approximately 0–1. Subsequently, the CIE XYZ tristimulus values are derived by converting these normalized RGB values into XYZ values as XYZ_camera. This conversion is facilitated through the application of a specific transformation formula, detailed below:

$$\left[\begin{array}{c}X\\ Y\\ Z\end{array}\right]=[T]\left[\begin{array}{c}f(R_{sRGB})\\ f(G_{sRGB})\\ f(B_{sRGB})\end{array}\right]\times 100,0\le \begin{array}{c}{R}_{sRGB}\\ G_{sRGB}\\ B_{sRGB}\end{array}\le 1(1)$$

where 0≤ R_sRGB, G_sRGB, B_sRGB ≤1. Here, R_sRGB, G_sRGB, and B_sRGB denote the three color channels of the sRGB image. These channel values are linearized through a function defined by gamma correction. For sRGB images, this correction can be approximated by a power function with an exponent of 2.2, incorporating a linear segment for lower values to account for the non-linear sensitivity of human vision at these intensities. The formula for removing gamma correction, or linearization, is defined as follows:

$$f(n)=\left\{\begin{array}{c}{\left(\frac{n+0.055}{1.055}\right)}^{2.4},n>0.04045\\ \left(\frac{n}{12.92}\right),otherwise\end{array}\right. (2)$$

and the transformation matrix T is defined by:

$$[T]=\left[\begin{array}{ccc}0.4104& 0.3576& 0.1805\\ 0.2126& 0.7152& 0.0722\\ 0.0193& 0.1192& 0.9505\end{array}\right](3)$$

Converting the obtained spectral data R(λ) (380 nm ~ 780 nm, 1 nm) into the XYZ color gamut space requires the light source spectrum S(λ) of the ophthalmoscope hyperspectral system, and the XYZ color matching function. Through equations 4, 5, and 6, the spectral data are converted into the XYZ values as XYZ_spectrometer.

$$X=k{\displaystyle {\int }_{380nm}^{780nm}S(\lambda )}R(\lambda )\overline{x}(\lambda )d\lambda (4)$$

$$Y=k{\displaystyle {\int }_{380nm}^{780nm}S(\lambda )}R(\lambda )\overline{y}(\lambda )d\lambda (5)$$

$$Z=k{\displaystyle {\int }_{380nm}^{780nm}S(\lambda )}R(\lambda )\overline{z}(\lambda )d\lambda (6)$$

The brightness ratio k, as shown in Equation 7:

$$k=100/{\displaystyle {\int }_{380nm}^{780nm}S(\lambda )}\overline{y}(\lambda )d\lambda (7)$$

where S(λ) is the light source spectrum in XYZ color gamut space, $\overline{x}(\lambda )$, $\overline{y}(\lambda )$, and $\overline{z}(\lambda )$ are the XYZ values of color matching function. The spectral data signal R(λ) from the spectrometer are converted to the XYZ color space.

The color correlation between the spectrophotometer and camera was obtained; the regression matrix (C) is defined as follows:

$$[C]=\left[XY{Z}_{spectrometer}\right]\times pinv([V])(8)$$

where V is defined as:

$$\begin{array}{c}V=[X^3{Y}^3{Z}^3{X}^{2}Y{X}^{2}Z{Y}^{2}ZX{Y}^{2}X{Z}^{2}Y{Z}^2\\ XYZ{X}^2{Y}^2{Z}^{2}XYXZYZXYZ1{]}^T\end{array}(9)$$

with X, Y, and Z components derived from XYZ_camera, and (XYZ_spectrometer) is matrix constituted by X, Y, and Z components obtained from the spectrometer.

Equation 9 is also employed to extend the (XYZ_camera) as the (XYZ_{camera_extend}) resulting in the corrected values between the camera and spectrometer, denoted as XYZ_correction, as delineated in Equation 10:

$$\left[XY{Z}_{correction}\right]=\left[C\right]\times \left[XY{Z}_{camera\_extend}\right](10)$$

where (XYZ_{camera_extend}) is defined as Equation 9 with X, Y, and Z components derived from XYZ_camera, and (C) is correction matrix obtained from Equation 8.

The spectra were organized into a matrix, denoted as (R(λ))_{401 × 24}, where the columns represent the number of color checkers, and the rows correspond to the intensities of the wavelengths at 1 nm intervals. 12 eigenvectors, identified as having the most significant contribution, were established as the basis for spectral calculation and organized into a matrix (E)_{12 × 401} by calculating the eigen system and utilizing principal component analysis (PCA). The corresponding eigenvalues of these six eigenvectors, denoted as (α)_{12 × 24}, were determined as follows:

$${[\alpha ]}^T={[R(\lambda )]}^{T}pinv[E](11)$$

Eventually, a transformation matrix (M) among the camera and spectrophotometer was determined by the following equation:

$$[M]=[\alpha ]pinv[V_{colour}](12)$$

where V_color is defined as Equation 13:

$$V_{color}={[XYZXYYZXZXYZ]}^T(13)$$

with X, Y, and Z components derived from XYZ_spectrometer; (α) represents principal component scores obtained from 12 sets of principal components for dimensionality reduction of R(λ).

The calculated spectra in the visible light range (380–780 nm) were determined according to the following equation:

$${[spectra]}_{380-780nm}=[E][M][XY{Z}_{camera}](14)$$