MySQL: low cardinality/selectivity columns = how to index?
database, mysql
Solution
The index that you describe is pretty much pointless. An index is best used when you need to select a small number of rows in comparison to the total rows.
The reason for this is related to how a database accesses a table. Tables can be assessed either by a full table scan, where each block is read and processed in turn. Or by a rowid or key lookup, where the database has a key/rowid and reads the exact row it requires.
In the case where you use a where clause based on the primary key or another unique index, eg. `where id = 1`, the database can use the index to get an exact reference to where the row's data is stored. This is clearly more efficient than doing a full table scan and processing every block.
Now back to your example, you have a where clause of `where status = 'enabled'`, the index will return 150m rows and the database will have to read each row in turn using separate small reads. Whereas accessing the table with a full table scan allows the database to make use of more efficient larger reads.
There is a point at which it is better to just do a full table scan rather than use the index. With mysql you can use `FORCE INDEX (idx_name)` as part of your query to allow comparisons between each table access method.
Reference: http://dev.mysql.com/doc/refman/5.5/en/how-to-avoid-table-scan.html
Problem
I need to add indexes to my table (columns) and stumbled across this post: How many database indexes is too many? Quote: “Having said that, you can clearly add a lot of pointless indexes to a table that won't do anything. Adding B-Tree indexes to a column with 2 distinct values will be pointless since it doesn't add anything in terms of looking the data up. The more unique the values in a column, the more it will benefit from an index.” Is an Index really pointless if there are only two distinct values? Given a table as follows (MySQL Database, InnoDB) ``` Id (BIGINT) fullname (VARCHAR) address (VARCHAR) status (VARCHAR) ``` Further conditions: - The Database contains 300 Million records - Status can only be “enabled” and “disabled” - 150 Million records have status= enabled and 150 Million records have stauts= disabled My understanding is, without having an index on status, a select with `where status=’enabled’` would result in a full tablescan with 300 Million Records to process? How efficient is the lookup when I use a BTREE index on status? Should I index this column or not? What alternatives (maybe any other indexes) does MySQL InnoDB provide to efficiently look records up by the "where status="enabled" clause in the given example with a very low cardinality/selectivity of the values?