Though Belief Propagation (BP) algorithms generate high quality results for a wide range of Markov Random Field (MRF) formulated energy minimization problems, they require large memory bandwidths and could not achieve real-time performance when applied to many real-life inference tasks. There is an increasing demand for efficient parallel inference algorithms as the size of problems increase and computer architectures move towards multi-core. In this work, we proposed a new high speed parallel computational structure for hierarchical Belief Propagation on shared memory architecture, which is based on a modification and generalization of the hierarchical BP algorithm presented by Felzenszwalb and Huttenlocher. Our parallel hierarchical belief propagation (PHBP) computational structure supports arbitrary grouping of nodes in multiscale computation and works for graphs in general topologies (including non grid structure graphs). Secondly, a fully parallel framework of hierarchical BP using sequential asynchronous message updating scheme (accelerated message updating) is developed. We achieved parallelization of both pre-computation portion and computational intense message passing portion. Lastly, we empirically evaluated the performance of algorithm on several computer vision tasks where we achieved nearly linear parallel scaling and outperform other alternative algorithms. Specifically, for the task of restoring a 608*456 noisy image with 16 gray levels, our PHBP takes around 100ms while a comparable result needs around 30s using Parallel Splash on a same 8 core shared memory system.