In Python what type of data structure would allow fast searches and be most efficient?
data-structures, python
Solution
Store a tuple of viewCount and timesPreferred in a dict using the videoID as the key. Updating each entry will run in constant time.
For the extra data wait until you're putting the data into the database before getting it. No point in it cluttering stuff up while you're counting.
Problem
I have a text file that lists 10,000,000 YouTube video IDs, like this: ``` 9bZkp7q19f0 t4H_Zoh7G5A 9bZkp7q19f0 etc... ``` I open the file, take the YouTube video ID, and look it up to see what its statistics are: https://www.googleapis.com/youtube/v3/videos?part=topicDetails,statistics&id=9bZkp7q19f0&key={API_KEY} For the first video (Psy Gangnam Style), the API call returned: ``` "viewCount": "1895378471", "likeCount": "8110831", "dislikeCount": "976065", "favoriteCount": "0", "commentCount": "5100187" ``` I also calculate custom values such as how many times each video was contained in the file, incrementing by one each time. I need to record all this information into some type of Python data structure, which would look like this: ``` videoID , viewCount, count, etc 9bZkp7q19f0, 1895378471, 10000 t4H_Zoh7G5A, 512345678, 10000 ``` Since videoIDs often repeat in the input file, I would not just append new rows to the data structure, but need to be able to just find the existing row, and increment the value of count. I think that text like "videoID", "count", etc do not really have to be in the data structure, a two dimensional type of array is fine, as long as I know what each column represents. The point of this question is I am trying to decide what type of data structure would be best. Performance is critical. I must be able to quickly determine by the videoID, the key, if that row already exists in the data structure, so if I could index the first column, that would be ideal. What type of Python data structure could accomplish this?