solr / lucene idf score
lucene, scoring, solr
Solution
Looks like the lucene site has a typo.
http://lucene.apache.org/core/3_6_0/api/core/org/apache/lucene/search/Similarity.html
states 1 + log(numDocs/(docFreq+1))
but it is actually 1 + ln(numDocs/(docFreq+1))
Problem
I'm trying to get a better understanding of how lucene scored my search so that I can make necessary tweaks to my search configuration or the document content. The below is a part of the score breakdown. product of: ``` 0.34472802 = queryWeight, product of: 2.2 = boost 7.880174 = idf(docFreq=48, maxDocs=47667) 0.019884655 = queryNorm 1.9700435 = fieldWeight in 14363, product of: 1.0 = tf(freq=1.0), with freq of: 1.0 = termFreq=1.0 7.880174 = idf(docFreq=48, maxDocs=47667) 0.25 = fieldNorm(doc=14363) 0.26806915 = (MATCH) max of: 0.07832639 = (MATCH) weight(shortDescription:tires^1.1 in 14363) [DefaultSimilarity], result of: 0.07832639 = score(doc=14363,freq=1.0 = termFreq=1.0 ``` I understand how the boost is calculated as that is my configuration value But how was idf calculated (7.880174 = idf value). According to the lucene, the idf formula is: idf(t) = 1 + log(numDocs/(docFreq+1)) I checked the core admin console and found that my docFreq = maxDocs = 47667. Using the formula from lucene, I was not able to calculate expected 7.880174. Instead I get: idf = 3.988 = 1 + log(47667/(48+1)). Is there something I am missing in my formula.