The Stochastic-Calculus Approach to Selected Topics in Information Theory

Abstract

We study the following three information-theoretic problems using tools derived from stochastic calculus: the multi-receiver Poisson channel, lossy compression of point-processes, and the second-order coding rate in discrete memoryless channels (DMCs) with feedback. We obtain a general formula for the mutual information involving the point processes that allows for conditioning and the use of auxiliary random variables. We then use this formula to compute necessary and sufficient conditions under which one Poisson channel is less noisy and/or more capable than another, which turn out to be distinct from the conditions under which this ordering holds for the discretized versions of the channels. We also use the general formula to determine the capacity region of various multi-receiver Poisson channel. We introduce a new distortion measure for point processes called functional covering distortion. We obtain the distortion-rate function with feedforward under this distortion measure for a large class of point processes. For Poisson processes, stronger results are obtained by constraining the reconstruction. We derive the rate-distortion function for this constrained functional-covering and show that feedforward does not improve it. Moreover, we characterize the rate-distortion region for a two-encoder CEO problem for Poisson process and show that feedforward does not improve this region. As a corollary, we obtain the rate-distortion region of remote Poisson source. A strong data processing inequality for Poisson processes under superposition is derived to prove the converse of the CEO problem. For DMCs, we show that feedback does not improve the second-order coding rate for a class of DMCs which complements the class of channels for which feedback is known to improve the second-order coding rate. We derive an upper bound on the achievable rate with feedback utilizing a novel proof technique for general DMCs.