Sql-server – Truncating TEMP table before insert leads to big amount of logical reads on TEMP TABLE

sql serversql-server-2016temporary-tablestruncate

Setup:
We have a complex set of queries, that runs quite often (about 3000 times per hour) for different users.
At the start of the process, .NET application creates a set of temp tables
during the process, .NET application calls a set of SQL queries, they look like this:

truncate #QuerySpecificTable

insert into #QuerySpecificTable
select PrimaryKey
  from Production.QuerySpecificTable
  join ...
 where ...

Issue i came across while performance tuning one specific query. ~139000 records get inserted into the temp table. This causes ~300k logical reads on the #QuerySpecificTable.

When i remove the truncate, it drops to ~300 reads on #QuerySpecificTable.
CPU statistics show drop in CPU time (~900->300ms so not sure about precision and reliability.)

The issue is, when trying to do this in SSMS, i cannot 100% reproduce the behaviour and make sure and demonstrate removing the truncate (even through it wouldnt be a problem) is a good thing to do. The biggest issue is NOT KNOWING why and what is happening here

WITH TRUNCATE (NOT ALWAYS, thats the weird part)

truncate table #test
insert into #test select top 150000 PrimKey from dbo.test

Table '#test_______________________________________________________________________ ________________________________________000000000141'. Scan count 0, logical reads 319955, physical reads 0, read-ahead reads 217, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table 'Test'. Scan count 1, logical reads 246, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.

(150000 rows affected)

DROP/CREATE

drop table if exists #test
create table #test (PrimKey integer primary key) 
truncate table #test
insert into #test select top 150000 PrimKey from dbo.test

Table 'Test'. Scan count 1, logical reads 246, physical reads 0, read-ahead reads 232, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.

(150000 rows affected)

Best Answer

The data in there is not really dropped, first of all. It has to do with how Temdb is implemented

Also, Microsoft has a great blog on TempDB and contention that addresses the following points:

Tempdb - Files, Trace flags, updates - Microsoft Blogs

We are continuing to work on improving tempdb performance and metadata contention, but in the meantime, there are some best practices you can employ in your code that might help avoid the contention:

Do not explicitly drop temp tables at the end of a stored procedure, they will get cleaned up when the session that created them ends.

Do not alter temp tables after they have been created.

Do not truncate temp tables

Move index creation statements on temp tables to the new inline index creation syntax that was introduced in SQL Server 2014.

Related Solutions

Sql-server – Which of these queries is best for performance

Sometimes I wonder if SHORT scripts really is the best thing to focus on.

The size of a script has little to do with how efficiently the query will execute. A more compact statement will likely consume fewer resources in terms of compilation, but (re)compilation is usually a rare occurrence in a live system.

Fewer table accesses is usually desirable, though, and this does lead to more compact code.

Very generally speaking, a smaller execution plan will yield better results, and a lower estimated cost will yield better results. Again, though, it's highly situational. Cost estimates in particular can be way off in some cases. It's important to measure the actual execution time, because at the end of the day, that's what matters.

With left joins i can achieve what i want with just a few lines. But then I tried with a longer script, using unions. Which is the best method?

First of all, we need to know how much data will be in these tables in a real system. Right now there's so little it will be difficult to use the STATISTICS TIME performance metrics to figure out a winner -- the results that come back will be dominated by factors other than the query execution. With more data, it's likely the plans will change, thus rendering the comparison here moot.

Having said that, by looking at the query plans as they are now from a logical point of view, the first one is the winner.

You can see that the Clustered Index Scan of quantities appears once in the first plan, while it appears four times in the second one. The second plan also contains an expensive Distinct Sort as a result of using UNIONs (this operator could be eliminated by using UNION ALLs instead, which won't change the results).

The first query could also probably be improved, by getting index seeks on the colors and sizes tables, instead of table scans. It might be worth trying a hash match plan as well (which is what you'll probably see when quantities and products are larger), but for tables this small, the startup cost may be too much overhead to be of benefit.

What I would suggest you do is run each of the statements you want to test 10,000+ times in a loop, figure out the average execution time, and then compare.

Multi-Statement TVF vs Inline TVF Performance in SQL Server

Your numbers table is a heap and is potentially being fully scanned each time.

Add a clustered primary key on Number and try the following with a forceseek hint to get the desired seek.

As far as I can tell this hint is needed as SQL Server just estimates that 27% of the table will match the predicate (30% for the <= and reduced down to 27% by the <>). And therefore that it will only have to read 3-4 rows before finding one that matches and it can exit the semi join. So the scan option is costed very cheaply. But in fact if any palindromes do exist then it will have to read the whole table so this is not a good plan.

CREATE FUNCTION dbo.InlineIsPalindrome
(
    @Word NVARCHAR(500)
)
RETURNS TABLE
WITH SCHEMABINDING
AS RETURN (
    WITH Nums AS
    (
      SELECT
        N = number
      FROM
        dbo.Numbers WITH(FORCESEEK)
    )
    SELECT
      IsPalindrome =
        CASE
          WHEN EXISTS
          (
            SELECT N
            FROM Nums
            WHERE N <= L / 2
              AND SUBSTRING(S, N, 1) <> SUBSTRING(S, 1 + L - N, 1)
          )
          THEN 0
          ELSE 1
        END
    FROM
      (SELECT LTRIM(RTRIM(@Word)), LEN(@Word)) AS v (S, L)
);
GO

With those changes in place it flies for me (takes 228ms)

Best Answer

Related Solutions

Sql-server – Which of these queries is best for performance

Multi-Statement TVF vs Inline TVF Performance in SQL Server

Related Question