Deep learning has made great advances in the domain of medical image and digital pathology analysis, specifically in: (1) diagnosis and detection, (2) prognosis and subtyping. Training models to predict established clinical markers (area 1) helps pathologists in clinical practice. However, many of these baseline clinical markers are developed manually and themselves are subject to inter- and intra-observer variance. A greater challenge is developing completely new clinical markers (area 2) and using them to understand cancer prognosis and the biology of cancer. This dissertation highlights the power of area 1 by showing how deep learning can be used to predict early glaucoma and predict established renal cell carcinoma subtypes. Further, it explores the potential of deep learning in area 2 by developing an unsupervised model to discover human interpretable histomorphological subtypes in intrahepatic cholangiocarcinoma and testing their prognostic power. Finally, an end-to-end prognostic model is developed to produce a rich feature embedding which is directly correlated to survival data, optimized with a new survival-stratification loss.