Why does the query run faster after I force full table scan

oracle-11g-r2performancequery-performance

I have a MERGE statement that took forever to execute the select as the table it is running from (SUB_DATA) is rather big (read 1.4 Tb, 700 million rows). My SELECT looks like this.

SELECT MIN(TRUNC(DATETIME,'DD'))    AS FIRST_ACTIVITY,
       MAX(TRUNC(DATETIME,'DD'))    AS LAST_ACTIVITY,
       NVL(PHONE_NUMBER, '-1')      AS PHONE_NUMBER,
       MAX(NVL(PHONE_ID, '-1')) KEEP (DENSE_RANK LAST ORDER BY DATETIME) AS LAST_PHONE_ID,
       MAX(TYPE_ID) KEEP (DENSE_RANK LAST ORDER BY DATETIME)             AS TYPE_ID
FROM SUB_DATA   AD,
     SUB_PROF   ICP
WHERE DATETIME BETWEEN minDate AND maxDate
AND   PROFILE_ID = ICP.ICP_SEQID
AND   REF_RT_ID in (13, 63) 
AND   DURATION > 3 
AND   PHONE_NUMBER like '123%' 
GROUP BY NVL(PHONE_NUMBER, '-1')

There are two INDEXES on the table, one on PHONE_NUMBER and one on another column not used here.

I struggled to get the query to run faster and one of the more senior guys at work suggested I force a FULL TABLE SCAN. I did this and it worked wonders.

Even though it worked, it goes against the general concept that full table scans are bad.

Could someone please explain why this is the case?

Best Answer

[...] it goes against the general concept that full table scans are bad.

This has exceptions, like all general concepts. A full table scan can be less expensive than an index scan followed by table access by rowid - sometimes much less expensive.

Remember that:

  • Index blocks need to be read too, in addition to the table's blocks. If the index is large (a sizable percentage of the table's size), the pressure on the SGA (and associates latches) might be noticeable
  • If the sort order of the index doesn't match the way the data is stored in the actual table, the number of logical I/Os necessary to fulfill the query could be (potentially much) more than the number of LIOs to do the full table scan. (The index's clustering factor is one of the indicators you can look at to estimate this.)

I find the figures in this article Oracle clustering_factor tips explain the issue pretty well.

Essentially, if scanning via your index makes the data block I/O requests jump all over your table, the overall cost is going to be higher than if you could do large sequential reads. An FTS is more likely to use multi-block direct-path reads, bypassing the SGA entirely, which is also potentially good - no "cache thrashing", less latching.

If you have a covering index, chances are that's going to beat a full table scan all the time. If not, it's going to depend on what percentage of actual blocks (data + index) will need to be processed (the index's selectivity for that query), and how well they're "physically sorted" with respect to each-other.

As to why the optimizer is picking the "wrong" path for you here: hard to tell. Stale statistics on the index or table could be an issue like always, the estimate calculated base on the LIKE might be off for certain patterns, instance parameters could favor indexes a bit too much, ... If this is the only query that's misbehaving, and your stats are up to date, using a /*+ full */ hint doesn't sound too bad.

Related Question