Over the past decades, solar panels have been widely used to harvest solar energy owing to the decreased cost of silicon-based photovoltaic (PV) modules, and therefore it is essential to remotely map and monitor the presence of solar PV modules. Many studies have explored on PV module detection based on color aerial photography and manual photo interpretation. Imaging spectroscopy data are capable of providing detailed spectral information to identify the spectral features of PV, and thus potentially become a promising resource for automated and operational PV detection. However, PV detection with imaging spectroscopy data must cope with the vast spectral diversity of surface materials, which is commonly divided into spectral intra-class variability and inter-class similarity. We have developed an approach to detect PV modules based on their physical absorption and reflection characteristics using airborne imaging spectroscopy data. A large database was implemented for training and validating the approach, including spectra-goniometric measurements of PV modules and other materials, a HyMap image spectral library containing 31 materials with 5627 spectra, and HySpex imaging spectroscopy data sets covering Oldenburg, Germany. By normalizing the widely used Hydrocarbon Index (HI), we solved the intra-class variability caused by different detection angles, and validated it against the spectra-goniometric measurements. Knowing that PV modules are composed of materials with different transparencies, we used a group of spectral indices and investigated their interdependencies for PV detection with implementing the image spectral library. Finally, six well-trained spectral indices were applied to HySpex data acquired in Oldenburg, Germany, yielding an overall PV map. Four subsets were selected for validation and achieved overall accuracies, producer’s accuracies and user’s accuracies, respectively. This physics-based approach was validated against a large database collected from multiple platforms (laboratory measurements, airborne imaging spectroscopy data), thus providing a robust, transferable and applicable way to detect PV modules using imaging spectroscopy data. We aim to create greater awareness of the potential importance and applicability of airborne and spaceborne imaging spectroscopy data for PV modules identification.