We develop parallel heuristic algorithms for partitioning the vertices of a graph into many groups of roughly equal size so that few edges connect vertices in different groups. The algorithms are intended for a message passing multiprocessor such as a hypercube, but only require the processors to be connected as a ring. They are based on the Kernighan-Lin algorithm, which finds a small edge separator that divides a graph into two pieces of equal size. Each of the algorithms features a parameter that allows for a trade-off between the potentially conflicting goals of reducing the number of edges between different groups and keeping the sizes of the groups roughly the same. These algorithms are applied to the problem of parallel block oriented Cholesky factorization. For an efficient factorization, we need a graph partition in which few pairs of vertex groups are interconnected; that is, we desire the quotient graph induced by the partition to be sparse. We discuss how standard partitioning heuristics may fail to give sparse quotient graphs and how they can be modified to correct this.