Critical to the reliable application of gridded soil moisture products is a thorough assessment of their quality at spatially compatible scales. While previous studies have attempted to evaluate different soil moisture products, comprehensive assessments at the appropriate scale remain challenging and rare. This study explores the potential of the Cosmic-Ray Soil Moisture Observation System (COSMOS) in Germany for effectively mitigating the scale mismatch between soil moisture products and reference measurements. A newly released extensive COSMOS data set provides time series of hectare-scale soil moisture of the main root zone at different locations in Germany and offers a unique opportunity for a comprehensive quality assessment of 15 commonly-used coarse-scale soil moisture products. Those are either satellite-based (AMSR2 LPRM, ASCAT H115/H116, Sentinel-1 SSM, SMAP L3E, SMOS L3, ASCAT/Sentinel-1 SWI, SMAP/Sentinel-1 L2, CCI Combined, and NOAA SMOPS) or model-based (ERA5-Land, GLDAS-Noah, ASCAT H141/H142, GLEAM, SMAP L4, and SMOS L4). We compared the temporal dynamics of the soil moisture products against that of the COSMOS soil moisture estimates at 21 sites of different land cover types over six years (2015–2020), including the drought of 2018. We found that the model-based products generally yield a higher correlation (0.74) and lower unbiased root-mean-square differences (0.05m^3m^-3) than the satellite-based products (0.60 and 0.07 m^3m^-3, respectively) against the COSMOS data in Germany. Notably, the application of the exponential filter significantly improves the performance of the products. Conversely, deseasonalized time series of all selected products demonstrate lower performances across all COSMOS sites. Most products show a considerable positive bias, which limits their usability for the assessment of absolute soil water storage. We also found that the land cover type, mean annual soil moisture, and vertical support have notable influences on the performance of the soil moisture products. Additionally, the performances of the soil moisture products show seasonal variations, such that both correlation and bias are highest during the summer season. This study highlights the strengths of COSMOS data as a robust reference for evaluating soil moisture products. Additionally, it provides insights on how to assess, interpret, and improve large-scale soil moisture products.