Varying coefficient frailty models with applications in single molecular experiments.

Motivated by an analysis of single molecular experiments in the study of T-cell signaling, a new model called varying coefficient frailty model with local linear estimation is proposed. Frailty models have been extensively studied, but extensions to nonconstant coefficients are limited to spline-based methods that tend to produce estimation bias near the boundary. To address this problem, we introduce a local polynomial kernel smoothing technique with a modified expectation-maximization algorithm to estimate the unknown parameters. Theoretical properties of the estimators, including their unbiased property near the boundary, are derived along with discussions on the asymptotic bias-variance trade-off. The finite sample performance is examined by simulation studies, and comparisons with existing spline-based approaches are conducted to show the potential advantages of the proposed approach. The proposed method is implemented for the analysis of T-cell signaling. The fitted varying coefficient model provides a rigorous quantification of an early and rapid impact on T-cell signaling from the accumulation of bond lifetime, which can shed new light on the fundamental understanding of how T cells initiate immune responses.