We present a hybrid method combining both weather data and ground-based photography to generate physically-based full hemisphere cloudy sky illumination. We validate our illumination model using precise spatial and spectral radiance and total irradiance measurements of cloudy skies gathered over the timespan of several months using custom-built hardware. Our system reconstructs cloud volumes from fish-eye photographs utilizing satellite data to bound the vertical and horizontal extent of the cloud geometry. We estimate optical parameters for the cloud volumes from measured cloud micro-physical properties gathered from weather data. When combined with a physically-based light transport algorithm, our cloud illumination model accurately reproduces the highly varying anisotropic structure of a cloudy sky. In contrast to purely image-based techniques which produce static RGB illumination maps, by generating an actual cloud volume our method efficiently handles changes in sun direction, viewing angle, and cloud geometry. Our approach can accurately simulate a wide range of cloud and weather conditions and has applications for physically-based rendering, daylighting studies, and scene illumination.