Just How Difficult *IS* Index Tuning?

OK, so here’s the answer to the quiz questions, folks, with some dialog about being practical in all this.
Good job by those who contributed on this thread, and those who contacted me directly. We’ll send some Toad goodies to y’all shortly.

1. What is the total number of possible indexes on just one table? How about on all five tables?
   Answer:
   Assuming you can define one-, two-, or three-column indexes, then 15 indexes per table.
   5 x 15 = 75 total possible indexes on the five tables.

   Let’s be practical:
   Indexes that include the third (last) column of the table offer no performance advantage over its two-column index. There are 6 such 3-column indexes. So for practical purposes, we can safely ignore those six for each table. That brings us to 9 practical indexes per table.
   9 x 5 = 45 total possible practical indexes.

2. What is the total number of different combinations of these indexes, choosing up to, say, three indexes at a time?
   Answer:
   If we’re considering 75 total indexes on the five tables, then there are 70,375 such index combination sets.
   (Original 75 indexes, plus the combination of those 75 indexes taken 2 at a time, plus the number of indexes taken 3 at a time.)

   Let’s be practical:
   If we’re considering 45 practice indexes, then there are 15,225 such index sets.
   (45 indexes, plus the combination of those 45 indexes taken 2 at a time, plus the number of indexes taken 3 at a time.)

3. How about the total number of index sets choosing up to, say, 10 indexes at a time?
   Answer:
   If we’re considering 75 possible indexes on the five tables, then there are just under one trillion such index sets (973,602,516,870 to be exact).

   Let’s be practical:
   If we’re considering 45 useful indexes in practice, then there are over four billion such index sets (4,337,283,236 to be exact).

4. (Extra Credit) What is the total number of index sets, regardless of the number of indexes in each set?
   Answer:
   2 to the power 75, if we’re considering 75 total indexes for all 5 tables.
   2 to the power 45, if we’re considering the 45 practical indexes for all 5 tables.

These numbers, by the way, are enormous. Consider the “smaller” of these two (2 to the 45th power). This number is so large that it would take your fastest computer over one year of no-downtime continuous operation to simply identify all the index sets here, assuming it could generate them at one million sets per second! And that’s not counting the time it would take to test the performance of any of these index combinations!

But really, let’s be practical:
While these numbers are mathematically interesting, an overwhelming percentage of these index combinations are definitely not feasible or practical. In fact, any performance consultant would argue (and rightfully so) that one should never (ever) over index database tables.

Ok, time for a different tact. Let’s say I only want to consider 2 indexes per table. That sounds much more reasonable, right? But even here, we’re running into largeness. Nine possible practical indexes choosing two at a time for each of the 5 tables gives us 36 to the power 5, or over 60 million possible combinations.

Alright. Let's be frugal and choose just one of the nine possible indexes per table. Even here, we would need to look at 9 to the power 5 = 59,049 feasible sets of indexes for just those five tables.

Lastly, I would be remiss if I didn’t talk about other factors that determine which index sets are feasible, practical, optimal, etc. Factors like

• The volume of data in the tables
  (Do we really need to index small tables ?)

• The referential integrity between the tables
  (Might want to index those foreign key columns, etc.)

• The SQL Workload of your applications
  (What is the mix of SQL? Are some queries more executed more than others?)

• The datatypes of the columns to be indexed
  (Are bitmap- or function-based indexes in order? How about clustered indexes? Index-organized tables?)

• How often columns are cited, and in which DML filters
  (Some columns are cited more frequently in WHERE clauses than others, etc.)

• .............................................................
  (Fill in the blank here with other factors… there’s lots more)

Is your head spinning? Mine is. This is tough stuff to do. Is it any wonder why vendors of CRM, ERP, and other enterprise-level solutions---which have thousands of tables and columns, by the way---can’t quite seem to nail down an optimal set of indexes for their solutions that satisfy their customers’ performance needs? Now you know. It’s near impossible.

Read full topic