Substantial misgivings have been voiced over the years about the methodologies used to evaluate information retrieval procedures, and about the credibility of many of the available test results. In this note, an attempt is made to review the state of retrieval evaluation and to separate certain misgivings about the design of retrieval test from conclusions that can legitimately be drawn from the evaluation results.