Autocomplete Style Prefix Lookup
algorithm, prefix-tree, python, trie
Solution
Yes, we can use a trie. The most frequent names for a trie node are either (1) the name at that trie node or (2) a most frequent name for a child of the trie node. Here's some Python code to play with.
from collections import defaultdict
class trie:
__slots__ = ('children', 'freq', 'name', 'top5')
def __init__(self):
self.children = defaultdict(trie)
self.freq = 0
self.name = None
self.top5 = []
def __getitem__(self, suffix):
node = self
for letter in suffix:
node = node.children[letter]
return node
def computetop5(self):
candidates = []
for letter, child in self.children.items():
child.computetop5()
candidates.extend(child.top5)
if self.name is not None:
candidates.append((self.freq, self.name))
candidates.sort(reverse=True)
self.top5 = candidates[:5]
def insert(self, freq, name):
node = self[name]
node.freq += freq
node.name = name
root = trie()
with open('letter_s.txt') as f:
for line in f:
freq, name = line.split(None, 1)
root.insert(int(freq.strip()), name.strip())
root.computetop5()
print(root['St'].top5)
Problem
Making a specific example: - You have a list of every first name in the USA. - You want to autosuggest completions in a GUI. The obvious thing is to do is use a radix tree to get a list of names for the given prefix. However, this doesn't take into account the frequency information. So, instead of just having the top 5 results be the first lexical results I would like the most common 5 names: e.g. For the prefix `dan` ``` (5913, 'Daniel') (889, 'Danny') (820, 'Dana') (272, 'Dan') (60, 'Dane') ``` Is there a trie tree algorithm that I've missed? Of course the ideal implementation (if one exists) is in python in my mind. UPDATE: Generally happy with what Paddy3113 has proposed, though I will say that it blows up completely when I feed it the 2.6GB file which is one of the files I'm reducing. Looking into the details the output gives some insight: ``` samz;Samzetta|Samzara|Samzie samza;Samzara samzar;Samzara samzara;Samzara samze;Samzetta samzet;Samzetta samzett;Samzetta samzetta;Samzetta samzi;Samzie samzie;Samzie # Format - PREFIX;"|".join(CHOICES). ``` We've got a few more days on the bounty side of things, so I'm still looking for the killer solution. Since it's not just about the reduction but also about the lookup side of things.