A Note on Inverse Document Frequency Weighting Scheme
Collections
Author
Wong, S. K. M.
Yao, Y. Y.
Abstract
Based on the Shannon information theory, a measure for term value is introduced. This study is an attempt to provide a theoretical justification for the inverse document frequency (IDF) weighting scheme. The argument presented in this paper is somewhat different from those suggested earlier. It is shown that IDF weights can be derived from the proposed approach by assuming that each index term has an even distribution within a subset of documents. A critical comment on the signal-noise ratio (S/N) weighting method is also included.
Date Issued
1989-04
Publisher
Cornell University
Keywords
Previously Published as
http://techreports.library.cornell.edu:8081/Dienst/UI/1.0/Display/cul.cs/TR89-990
Type
technical report