Deep longitudinal phenotyping of wearable sensor data reveals independent markers of longevity, stress, and resilience

Biological age predicts morbidity and mortality

We trained the GeroSense system, a deep artificial neural network (Figure 1) to extract health-associated features from the physical activity recordings. The system included the encoder part, which took the input in the form of a series of step count per minute measurements for at least as long as one week and compressed the signal into 4-dimensional representations (embeddings). During the training and test procedure, we used one week-long samples of steps per minute recordings for 97,320 UK Biobank and 6,510 NHANES participants along with recordings samples from longitudinal data obtained for 1,876 smartphone and 723 smartwatch users. The embedding vectors were further fed into the domain-adaptation network, trained to reduce the difference between the feature sets distribution in samples originating from different devices. In such a way, we were able to produce the most common features present in the motion data.

Figure 1. Architecture of the neural network predicting biological age acceleration (BAA). GeroSense model predicts BAA once per day based on step counts recorded by wearable or mobile device sensors using each individual’s week-long physical activity tracks. The network components responsible for the feature extraction and BAA output are shown in green. BAA can be predicted for any sample of arbitrary length exceeding one week. For example, BAA on day 10 is predicted using the step counts data coming from day 4 through day 10, and so forth. Shown in red are the network components used only during the training procedure. One is the discriminator responsible for domain adaptation between e.g. smartphones and smartwatches. The other is the class predictor based on the log-odds ratio trained to predict morbidity binary status for UK Biobank and NHANES.

At the top layer, log-linear proportional hazards models of all-cause mortality are natural tools to build the biological age acceleration models, see, e.g., the PhenoAge model [2, 17]. If, however, the number of observed events is small, a simple logistic regression model provides an excellent approximation to the solution of the corresponding proportional hazards [18, 19]. Therefore, in the present study, we trained the neural network using cross-entropy loss to predict binary labels: the prevalence of at least one chronic disease. Overall, we labeled events for 23% and 29% samples in NHANES and UKB, respectively (see Materials and Methods section “Morbidity status” for the precise definition).

The model’s output was the Biological Age Acceleration (BAA), estimated once per each seven days and calculated as the linear combination of the physical activity signal embeddings and biological sex label. During the training procedure, BAA was added to the chronological age of each participant to produce biological age followed by sigmoid activation layer and cross-entropy loss on the prediction of morbidity status.

To control for over-fitting, we split all data into training and test subsets. The quality of GeroSense BAA for predicting the morbidity status was similar in training and test subsets in both NHANES (Figure 2A) and UK Biobank (Figure 2C) with ROC AUC 0.60−0.61 in test subsets.

Figure 2. Biological age acceleration (BAA) ranks mortality and morbidity events. BAA estimated from patterns of intraday changes in physical activity level is associated with morbidity and mortality in NHANES (A, B) and UK Biobank (C, D) datasets. The performance was tested in participants aged 45−75 y.o. and was similar in training and test subsets.

We also expected the high concordance between the mortality and morbidity predictors [20]. Accordingly, we tested the ability of the model to predict future mortality events (see Figure 2B, 2D for the summary of the GeroSense BAA model performance in NHANES and UKB datasets, respectively). The scoring performance was similar to that of morbidity status and yielded ROC AUC 0.60−0.62 in test subsets.

BAA and the life expectancy in professional occupation groups

BAA from the network was superior to average daily physical activity-based BAA in scoring life expectancy in various professional occupations. The number of steps per day averaged over a sufficiently long period is an easy-to-understand and adjustable parameter that predicts mortality and morbidity [20]. This can be readily seen in Figure 2, where the negative logarithm of the number of daily steps (nloga) has all the properties required of BAA. However, the average physical activity obviously cannot be a good biological age measure. It is strongly affected by social factors and working schedule and therefore has a poor correlation with life expectancy across countries [8] and between groups of different professional occupations (Figure 3A).

Figure 3. Biological age acceleration (BAA) correctly ranks life expectancy. (A) Assuming the negative logarithm of average daily steps is a proxy for bioage, the observed positive correlation (Pearson’s r = 0.81 for males) with life expectancy is an incorrect prediction. (B) The negative correlation (Pearson’s r=–0.27 fro males) of GeroSense BAA with life expectancy is correct. Similar results were observed for females with Pearson’s r=0.19 and r=–0.55, respectively (data not shown). The calculations were performed in NHANES 2005−2006 cohort aged 30−60 y.o.

Notably, the GeroSense system produced BAA from wearable sensors data, which properly ranked professional occupation groups in NHANES according to both genders’ empirical life expectancy (Figure 3B). We did not have access to and hence could not test the association of physical activity and lifespan data across countries. Therefore, Gerosense BAA’s ability to score the life-expectancy of populations of different countries remains an open issue.

Cross-platform transferability of BAA and seasonal variations

The embeddings of physical activity tracks depend on the signal source, whether it is a smartphone or a smartwatch. Deep Neural networks are powerful feature-extraction tools and a proper choice to address this issue. We employed the domain adaptation network minimizing the feature-wise Kullback-Leibler divergence loss between samples originating from different devices during the training procedure. The problem is akin to batch removal. The proposed procedure helped the GeroSense network to learn the most common features between UKB, NHANES, and samples obtained from iPhone and Apple Watch.

Seasonal changes affect blood parameters [21], and physical activity patterns recorded by wearables [12]. The seasonal variations of the activity patterns may be an additional source of unwarranted fluctuations of the biological age estimates. We applied another Kullback-Leibler divergence minimization to penalize pair-wise differences in distributions of features for UK Biobank samples collected in the summer and winter.

The domain adaptation worked well: BAA level distributions were almost indistinguishable between the samples originating from smartphones and smartwatches (p=2E−5). In contrast, the levels of negative logarithm of average physical activity were much more different (p=2.7E−80). The difference was expected but is still striking since we analyzed the smartphone and smartwatch data from the same users.

The results of the statistical testing (p-values) strongly depend on the sample size. That is why, here and in all the following examples, we report p-values obtained for the same maximum size of 500 in each group. The p-values themselves are calculated using Fisher’s combined probability test (see details in Materials and Methods section).

Notably, there was a very significant drop in the physical activity levels during the COVID-19 pandemic lockdown in March through May 2020 as compared to the same period in 2019 (p<1E−30 for nloga). This was consistent with what was reported earlier [9]. In contrast, the increase in BAA was much less significant (p>1E−10). This may indicate that BAA responds weaker to the lockdown than the expected decrease in physical activity, see Figure 4F. Moreover, this was in contrast to the improved ability of BAA to predict future risks of COVID-19 incidence and mortality rates in UKB as compared to nloga.

Figure 4. Effect of lockdown and the future risks of COVID-19 ranked by wearable BAA and blood-based bioage models. Association of BAA with the future incidence of COVID-19: (A) BAA in the form of the negative logarithm of daily step counts, (B) GeroSense BAA, (C) CBC-based DOSI [4], (D) CBC and Blood biochemistry hazards model, and (E) Blood-based PhenoAge [2] (all data are given for UK Biobank participants aged 45−75 y.o.). (F) Longitudinal data obtained by smartwatch show that while GeroSense is more sensitive to future risks it is also more selective and does not change immediately (green line) due to merely walking less during March to May 2020 lockdown while the average activity level does (red line). The values in panel (F) were scaled to zero mean and unit variance for comparison; shaded areas show one standard deviation range at each time point.

The decreased average level of physical activity (nloga) was associated with the increased COVID-19 risk in UKB [11], although it was not clear if this was not an effect of chronic disease burden (also known for its association with increased BAA). In Figure 4 we report that the excess BAA predicted the increased risk of COVID-19 incidence (for example, HR=2.4, p=4E−2 for 16 of UKB subjects died from the disease) in the subset of randomly sampled 500 UK Biobank participants free of chronic diseases at the time of measurements (2013−2015).

Side-by-side comparison of motion data- and blood-based aging clocks

We compared the performance of different BAA models for stratification of cohorts of NHANES participants of various lifestyles and health status. We have already seen in physical activity data [22] that the disease and smoking labels are associated with elevated BAA among individuals without chronic diseases. In our tests, the sensitivity of the BAA derived from blood markers was comparable to that of the self-reported questionnaire. GeroSense BAA performed consistently well in the same set of tests and conditions, see Figures 5, 6.

Figure 5. Morbidity status scored by wearable BAA and blood-based bioage models. BAA and chronic diseases: (A) the negative logarithm of daily step counts, (B) GeroSense BAA (C) questionnaire [22], (D) CBC-based DOSI [4], (E) log-mortality risk trained using combined CBC and Blood biochemistry variables, and (F) Blood-based PhenoAge [2]. The plots are produced for NAHNES participants aged 45−75 y.o.

Figure 6. Smoking status representing an unhealthy lifestyle ranked by wearable BAA and blood-based bioage models. BAA and smoking: (A) BAA in the form of the negative logarithm of daily step counts, (B) GeroSense BAA, (C) questionnaire [22], (D) CBC-based DOSI [4], (E) log-mortality risk trained using combined CBC and Blood biochemistry variables (F) PhenoAge [2]. The plots are produced for NAHNES participants aged 45−75 y.o.

Estimation of the BAA from wearable sensors has an advantage over blood-based models. It arises from its ability to further improve the signal-to-noise ratio by averaging over sufficiently long motion data streams. We demonstrated this with self-reported morbidity and smoking status provided by a smartphone app and wristband tracker users.

Averaging of GeroSense BAA predictions over a few weeks-long tracks led to a dramatic improvement of association between the BAA and morbidity/smoking status (Figure 7A). As expected, the sensitivity of the model was comparatively lower once we used smartphones instead of wristbands as the source of the data (Figure 7B) but also improved upon averaging over several weeks.

Figure 7. Accuracy of BAA grows with longer data collection intervals. Significance of association of GeroSense BAA with morbidity and current smoking status is improved when BAA is averaged over several weeks of data obtained from sensors of smartwatch (A) and smartphone (B).

Longitudinal analysis of BAA fluctuations reveals age-dependent loss of resilience

BAA reversibly depends on lifestyles. Hence, BAA is a dynamic variable more characteristic of stress rather than aging and responding to random organism state perturbations in a stochastic manner. We used longitudinal tracks of step counts from Fitbit devices and calculated the autocorrelation function for every user. The autocorrelation function decayed exponentially. Accordingly, we carried out the exponential fit to infer the autocorrelation time as a measure of recovery rate or resilience. This quantity is a natural quantitative measure of an organism’s ability to recover its equilibrium state after stress.

The characteristic decay time was typically in the range of a few weeks and increased with age. Figure 8A shows the dependence of the recovery rate (the inverse auto-correlation time) on chronological age. The graph was produced by averaging over age-stratified cohorts and resembles much what we have previously reported for blood-based marker DOSI [4]. The recovery rate decreased approximately linearly with age, indicating the effective loss of resilience at some age exceeding 100 y.o. The same extrapolation would suggest that the recovery time increases approximately hyperbolically and would diverge at the same age, indicating the complete loss of resilience and the dynamic stability of the organism state.

Figure 8. Resilience and its age-related degradation can be measured using longitudinal motion sensor data. (A) The relaxation rate (or the inverse characteristic recovery time) computed for sequential age-matched cohorts of Fitbit users decreased approximately linearly with age. The recovery rate could be extrapolated to zero in the age exceeding ~110 y.o. (at this point, we may expect the complete loss of resilience and, hence, loss of stability of the organism state). The shaded area shows the 95% confidence interval of fit using GeroSense BAA. (B) The fraction of individuals suffering from the lack of resilience (defined as BAA’s autocorrelation time exceeding 3 weeks; the vertical axis) as the function of chronological age (the horizontal axis). The autocorrelation time was computed from longitudinal tracks of GeroSense BAA predicted for Fitbit wristband users.

To further investigate the relationship between resilience and aging, we identified individuals, which failed to recover quickly under stress. We established a somewhat arbitrary resilience cutoff corresponding to the recovery time exceeding 3 weeks. The fraction of such “non-resilient” individuals increased exponentially as a function of age (see Figure 8B). Moreover, this growth demonstrated the characteristic doubling rate of 0.087 per year, which was close to the mortality rate doubling rate according to the Gompertz mortality law.

Datasets

UK Biobank

Physical activity for UK Biobank participants aged 40−80 y.o. (54,777 female and 42,543 male) was measured by Axivity AX3 tri-axial accelerometers worn on the wrist for one week. We converted 100Hz raw acceleration measurements to step counts per minute to fit the format of data in other datasets used in this study. The number of steps during each consecutive minute was counted as the number of peaks of the absolute value of acceleration exceeding 1.3g. To ensure the local noise does not affect the result, only one peak (the highest) was counted in each 480ms sliding window with a step of 160ms resulting in at most 3 step counts during each 960ms. Steps closer to each other than 90s were combined into walking bouts and bouts with less than 5 steps in total were discarded.

NHANES

Physical activity for NHANES 2005−2006 participants was used in the form of step counts per minute collected by ActiGraph AM-7164 single-axis accelerometer worn on hip. Data were retrieved from the file “Physical Activity Monitor” of the “Examination data” category. Samples for 3,362 female and 3,148 male participants aged 6−85 y.o. were used.

Healthkit

Physical activity for users of Gero app aged 45−75 y.o. (464 female and 1,412 male users of smartphone, 125 female and 598 male users of smartwatch) was obtained from Healthkit. Raw activity data comprised the number of steps recorded by either smartphone or smartwatch during a time period with start and end timestamps and was resampled to equispaced time series of steps per minute.

Morbidity status

Binary morbidity status for the Healthkit dataset was assigned according to response to the survey question "Have you ever been told you have one of the following: diabetes, hypertension, cancer, coronary heart disease, heart failure, heart attack, or stroke?" Binary morbidity status of NHANES and UK Biobank participants was assigned according to the presence of at least one of those diagnoses. We used NHANES data on health condition and age at diagnosis available in the questionnaire category “Medical Conditions” (MCQ). Data on diabetes and hypertension was retrieved additionally from questionnaire categories “Diabetes” (DIQ) and “Blood Pressure and Cholesterol” (BPQ), respectively. For UK Biobank we aggregated ICD10 (block level) data to match that of NHANES and used the following ICD10 codes to cover the health conditions in UK Biobank: diabetes (E10-E14), hypertension (I10-I15), cancer (C00-C99), coronary heart disease (I20-I25), congestive heart failure (I50), myocardial infarction (I21, I22), and stroke (I60-I64).

Life expectancy

Empirical life expectancy from birth was determined for professional occupation groups using linked death register follow-up data for NHANES 2005−2015 surveys. To do that we fitted parameters of Gompertz likelihood adopted from [41]:

$-\ln LH=\sum _{all}\frac{M_0}{\text{Γ}}{e}^{\text{Γ}{t}_n}\left(1-e^{-\text{ΓΔ}{t}_n}\right)-{\displaystyle \sum }_{dead}\left(\ln M_0+\text{Γ}{t}_n\right),$

where M₀ and Γ are the initial mortality rate and mortality doubling rate of the Gompertz mortality law, t_n is the age of n-th participant at the end of followup, and Δt_n is the follow-up time since enrollment in NHANES survey.

Once M₀ and Γ were obtained by the fit for each professional occupation group, the life expectancy $\overline{t}$ in the group was calculated as:

$\overline{t}=\frac{1}{\text{Γ}}\ln \left(\frac{\text{Γ}}{M_0}\right)-\frac{\gamma }{\text{Γ}},$

Where γ≈0.58 is the Euler-Mascheroni constant. The expression for life expectancy is asymptotically correct whenever $M_0/\text{Γ}\ll 1$, which is definitely true in human cohorts.

Statistical analysis

Statistical analysis of the association of various Biological Age measures with morbidity/smoking status was performed using two-sided Mann-Whitney test. To ensure the reported p-values are comparable between tests we used the same cutoff of maximum of 500 samples in each test with 100 random samplings followed by combining p-value according to Fisher’s combined probability method [42]. All statistical tests were carried out using the python package SciPy (version 1.5.2).

Blood tests-based biological age models

In this work, we used blood tests-based biological age models trained using Cox proportional hazards approach in NHANES mortality follow-up data and reported elsewhere earlier. The Blood CBC (DOSI) model was trained using log-scaled values of hemoglobin, mean corpuscular volume, mean corpuscular hemoglobin concentration, red blood cell distribution width, red blood cell, platelet, neutrophil, lymphocyte, monocyte, and eosinophil counts as well as biological sex label [4]. The Blood Biochemistry model additionally included age, and log-scaled values of C-reactive protein, albumin, alkaline phosphatase, gamma-glutamyl transferase, globulin, and serum glucose [22]. The Blood PhenoAge model was based on age, albumin, creatinine, serum glucose, log-scaled C-reactive protein, lymphocyte percent, mean cell volume, red cell distribution width, alkaline phosphatase, and white blood cell count [2].

Neural network architecture

Deep neural network architecture is schematically shown in Figure 1. Wearable data is input in the form of a continuous array of steps per minute. The input is immediately converted to a one-hot embedding representation, where each bin corresponds to an increment of 4 steps per minute. Next, the encoded data is processed by a block of 16 1D-convolutional layers, each having 16 filters with a kernel size of 3 and “elu”-activation. One in two convolutional layers is followed by a local max-pooling with stride 2, 3 or 5, and each layer is followed by batch-normalization. The output of the convolutional block was 4 features per every 1440 points in the input array, which corresponds to the number of minutes in one day. Finally, the features were subject to a 7 day-long average pooling and linearly combined with binary biological sex label so that the deep neural network was capable of outputting a prediction once per day based on 7 previous days.

The output of the deep neural network was interpreted as the Biological Age Acceleration (BAA) expressed in years of healthy life expectancy gained or lost. To guarantee this, during the supervised training of class label predictor we obtained the value of the Biological age of each NHANES and UK Biobank user by adding the network output (BAA) to the chronological age. The Biological age was then subject to sigmoid activation and fitted to binary morbidity status label, assuming that such procedure is an approximation to fitting proportional hazards model [18, 19].

The Domain adaptation networks were employed in the form of pairwise Kullback-Leibler divergence loss functions applied to enforce similar feature distributions for samples from UK Biobank on one side and NHANES, HealthKit smartphones, and smartwatches on the other side. Additionally, a domain adaptation was applied to UK Biobank samples collected during summer and winter as well as to samples with up to 3 zero-imputed (missing) days.

The training procedure was run for 2000 iterations, each batch comprising 256 samples. The class predictor was trained on each iteration for UK Biobank samples and only on one in five iterations for NHANES to avoid potential overfitting since the number of NHANES samples was small. All domain adaptation networks were trained on each iteration. Each network was trained using Adam optimizer as implemented in python package tensorflow-gpu (version 2.3.1) with a learning rate of 1E−3.

Data and code availability

Data and code are available at https://gerosense.ai from the corresponding authors upon reasonable request.