In this paper, we study the complexity of computing better solutions to optimization problems given other solutions. This is done in the context of the counterexample computation model introduced in [KPS90]. Assuming $PH \neq \sum^{P}_{3}$, we prove that PTIME transducers cannot compute optimal solutions for many problems, even given $O(n^{1-\epsilon})$ non-trivial solutions. These results are used to establish sharp lower bounds for several problems in the counterexample model. We extend the model by defining probabilistic counterexample computations and show that our results hold even in the presence of randomness.