Combining TF-IDF (cosine similarity) with pagerank?
cosine-similarity, search, search-engine, tf-idf
Solution
The weighted sum is probably better as a ranking rule.
It helps to break the problem up into a retrieval/ filtering step and a ranking step. The problem outlined with the weighted sum approach then no longer holds.
The process outlined in this paper by Sergey Brin and Lawrence Page uses a variant of the vector/ cosine model for retrieval and it seems some kind of weighted sum for the ranking where the weights are determined by user activity (see section 4.5.1). Using this approach a document with zero cosine would not get pass the retrieval/ filtering step and thus would not be considered for ranking.
Problem
Given a query I have a cosine score for a document. I also have the documents pagerank. Is there a standard good way of combining the two? I was thinking of multiply them ``` Total_Score = cosine-score * pagerank ``` Because if you get to low on either pagerank or the cosine-score, the document is not interesting. Or is it preferable to have a weighted sum? ``` Total_Score = weight1 * cosine-score + weight2 * pagerank ``` Is this better? Then you might have zero cosine score, but a high pagerank, and the page will show up among the results.