There has been much work done in developing semantic structures for interpreting probabilistic programs. In particular, there have been many models based either on Markov kernels or linear operators, each with their own set of strengths and weaknesses. Concurrently, mathematicians have been working on categorical semantics for probability theory with the goal of obtaining a more abstract understanding of the field. This has led to the definition of Markov categories, an abstraction of Markov kernels. However, a similar treatment to the linear operator approach to probability is currently eluded by existing methods. This thesis sits at the intersection of probabilistic semantics and categorical probability theory. We propose a new categorical semantics and core calculus that extends Markov categories with linear operators, we justify its viability by showing how many useful categories used in probabilistic semantics are instances of our framework and, furthermore, we define a new model inspired by a functional-analytic treatment of measure theory. We conclude by showing how this formalism can be used to reason about a generalized notion of probabilistic independence via a substructural type system.