Inactive high area-to-mass ratio (A/m) resident space objects (RSOs) in the geosynchronous orbit (GEO) regime pose a hazard to active GEO RSOs. This attribute results in their increased sensitivity to non-conservative force effects manifested as perturbations of mean motion, inclination and eccentricity. This work examines the sensitivity of the trajectory prediction accuracies to various fidelities of complexity in the modeling of the SRP acceleration contributions to the overall dynamics. A physics-based solar radiation pressure model which includes the effects of refraction and absorption from the Earth's atmosphere during penumbral transitions is implemented. Additionally, variations in the area with respect to the sun are examined using representative orbits with associated eclipsing cycles. The trajectory prediction errors from combined modeling errors show significant growth consistent with loss of tracking. The errors are, in general, non normally distributed given their rejection of the null hypothesis to a standard normal distribution in various normality tests. This contributes to the prediction errors through errors in the orbit determination assumptions.