Sql-server – How to convert memory optimized table into disk based table in sql server

To start with I must say you have set max server memory to 6 GB and total memory is 8 GB so you have just left 2 GB for the OS, which in many cases, even if nothing is installed apart from SQL Server on a Windows machine, is too little memory provided to OS. To function properly, on a system with antivirus installed, OS must be given 4 GB at least. I leave 2GB for OS straight away and 1.5 G for AV.

The Target Server Memory is sometimes 6GB and then drops back to Total Server Memory (approx 5.3GB, never reaches 6GB).

Target server memory signifies how much memory is required by SQL Server to function properly in the ideal case. Target server memory is trying to be 6 GB because you have set the max server memory value to 6 GB. It's trying to consume all the memory it is allowed to.

Total server memory is what SQL Server is actually able to consume right now. This is committed memory and backed by physical RAM. This is 5.5 GB max in your case.

SQL Server is trying to grow its memory consumption but after reaching 5.3 or 5.5 GB, the OS is asking SQL Server to not grow its memory consumption further and might be actually flagging low memory notification. This is happening because OS might be facing low memory as already said above. SQLOS responds if Windows OS faces memory pressure by asking its caches its trim down their consumption. You can Query the Ring Buffer to check if there was low memory notification signaled. I must add DMV sys.dm_os_ring_buffer is undocumented but safe.

I see that pages are dropped from the cache - but there is still 700MB of memory left. If nothing needed the memory, how would you explain the fact that pages are removed from cache? I would expect that SQL Server only removes pages when it needs memory.

If you are looking for free memory, I would not suggest you to look at the DMV sys.dm_os_buffer_descriptors. The OS counter Available Mbytes will tell you the amount of physical memory, in bytes, available to processes running on the computer. I suggest you to also see What is a deterministic method for evaluating a sensible buffer pool size? and also read Does SQL Server need more RAM to find out how much RAM SQL Server needs and if SQL Server is facing memory pressure. From what you mentioned, if you are sure pages are getting removed from buffer pool then yes SQL Server feels that pages have to be moved because it needs space to accommodate new pages. I am not sure how you calculated the 700 MB free.

One other thing, please don't look at Task Manager for SQL Server memory consumption. It does not always give you the correct value especially when the SQL Server service account has the lock pages in memory privilege. In your case, even if SQL Server has max server memory of 6 GB, the OS being given just 2 GB, which is forcing SQL Server to not grow its consumption because 2 GB is low for SQL Server. Is there anything apart from SQL Server running on the system?

If you want to calculate SQL Server memory consumption please use:

select
(physical_memory_in_use_kb/1024) Physical_Memory_usedby_Sqlserver_MB,
(locked_page_allocations_kb/1024 ) Locked_pages_used_Sqlserver_MB,
(virtual_address_space_committed_kb/1024 ) Total_Memory_in_MB,--RAM+ Pagefile
process_physical_memory_low,
process_virtual_memory_low
from sys.dm_os_process_memory

What I don't understand is why Total_Memory_in_MB isn't equal to 6144 (max memory).

The column Total_Memory_in_MB signifies total memory used by SQL Server (RAM+page file). The RAM is actually physical memory used or committed memory. Some part of SQL Server process is also paged to disk and that constitutes as as virtual memory or page file and so if you are going to see TOTAL memory consumed by SQL Server it would be sum of physical memory and the Page file.

While the column Physical_Memory_usedby_Sqlserver_MB is just the physical memory (memory backed by physical RAM or committed memory) used. This is the reason why both are different. If you see the real column first one is Physical memory used and other one is Virtual memory committed.

If you want to see paged memory that would be the difference between Total_Memory_in_MB and Physical_Memory_usedby_Sqlserver_MB.

NOTE: Total memory used would be greater than physical memory used.

I would like to close the loop on my own question. We worked with Microsoft support on this and the bottom line is : "it's by design". I strongly believe that this is a flawed design and should be changed. The Microsoft support engineer opened a feature change request for this. In addition I created a "MS Connect" item about this. Please up-vote it if you agree with the request: https://connect.microsoft.com/SQLServer/feedback/details/3116296/enabling-in-memory-feature-for-databases-takes-up-huge-amount-of-disk-space-for-checkpoint-file-pairs-cfps-in-cases-when-there-are-many-databases-on-a-sql-server-instance