Evaluation of lists: AvgP@K and R@K are they same?

algorithm, machine-learning, precision-recall, python

Solution

I guess, you've already checked wiki. Based on its formula, the 3rd and the biggest one (after the words 'This finite sum is equivalent to:'), let's see at your example for each iteration:

- i=1 p = 1

- i=2 rel = 0

- i=3 p = 2/3

- i=4 p = 3/4

- i=5 rel = 0

So, avp@4 = avp@5 = (1 + 0.66 + 0.75) / 3 = 0.805; avp@3 = (1 + 0.66) / 3 and so on.

Recall@5 = Recall@4 = 3/3 = 1; Recall@3 = 2/3; Recall@2 =Recall@1 = 1/3

Below is the code for precision@k and recall@k. I kept your notation, while it seems to be more common to use `actual` for observed/returned value and `expected` for ground truth (see for example JUnit defaults).

def precision(actual, predicted, k):
    act_set = set(actual)
    pred_set = set(predicted[:k])
    result = len(act_set & pred_set) / float(k)
    return result

def recall(actual, predicted, k):
    act_set = set(actual)
    pred_set = set(predicted[:k])
    result = len(act_set & pred_set) / float(len(act_set))
    return result

Problem

My goal is to understand Average `Precision at K`, and `Recall at K`. I have two lists, one is predicted and other is actual (ground truth) lets call these two lists as predicted and actual. Now I want to do `precision@k` and `recall@k`. Using python I implemented Avg precision at K as follows: ``` def apk(actual, predicted, k=10): """ Computes the average precision at k. This function computes the average precision at k between two lists of items. Parameters ---------- actual: list A list of elements that are to be predicted (order doesn't matter) predicted : list A list of predicted elements (order does matter) k: int, optional Returns ------- score : double The average precision at k over the input lists """ if len(predicted) > k: predicted = predicted[:k] score = 0.0 num_hits = 0.0 for i,p in enumerate(predicted): if p in actual and p not in predicted[:i]: num_hits += 1.0 score += num_hits / (i + 1.0) if not actual: return 1.0 if min(len(actual), k) == 0: return 0.0 else: return score / min(len(actual), k) ``` lets assume that our predicted has 5 strings in following order: `predicted = ['b','c','a','e','d'] and`actual = ['a','b','e']`since we are doing @k would the precision@k is same as`recall@k`? If not how would I do`recall@k` If I want to do `f-measure (f-score)` what would be the best route to do for above mention list?

Original source