What does "word count" refer to when calculating unigram probabilities in an unigram language model?

nlp

Solution

Divide by the total number of tokens, i.e. word occurrences, in the training set. The reason is quite easy to see: if you divide by the number of distinct words, the probabilities for all words will not necessarily sum to one so they won't form a probability distribution.

Problem

I'm using an unigram language model. I want to calculate the probability of each unigram. Should I divide the number of occurrences of an unigram with the number of distinct unigrams, or by the count of all unigrams?

Original source