Soil acidification is a major global issue of sustainable development for ecosystems. The increasing soil acidity induced by excessive nitrogen (N) fertilization in farmlands has profoundly impacted the soil carbon dynamics. However, the way in which changes in soil pH regulating the soil carbon dynamics in a deep soil profile is still not well elucidated. In this study, through a 12-year field N fertilization experiment with three N fertilizer treatments (0, 120, and 240 kg N/(hm²·a)) in a dryland agroecosystem of China, we explored the soil pH changes over a soil profile up to a depth of 200 cm and determined the responses of soil organic carbon (SOC) and soil inorganic carbon (SIC) to the changed soil pH. Using a generalized additive model, we identified the soil depth intervals with the most powerful statistical relationships between changes in soil pH and soil carbon dynamics. Hierarchical responses of SOC and SIC dynamics to soil acidification were found. The results indicate that the changes in soil pH explained the SOC dynamics well by using a non-linear relationship at the soil depth of 0-80 cm (P=0.006), whereas the changes in soil pH were significantly linearly correlated with SIC dynamics at the 100-180 cm soil depth (P=0.015). After a long-term N fertilization in the experimental field, the soil pH value decreased in all three N fertilizer treatments. Furthermore, the declines in soil pH in the deep soil layer (100-200 cm) were significantly greater (P=0.035) than those in the upper soil layer (0-80 cm). These results indicate that soil acidification in the upper soil layer can transfer excess protons to the deep soil layer, and subsequently, the structural heterogeneous responses of SOC and SIC to soil acidification were identified because of different buffer capacities for the SOC and SIC. To better estimate the effects of soil acidification on soil carbon dynamics, we suggest that future investigations for soil acidification should be extended to a deeper soil depth, e.g., 200 cm.