Sql-server – Operation System error 665 with VLDB backup (The requested operation could not be completed due to a file system limitation)

backupsql server

Has anyone seen the following error while doing backups? We have tried different LUNS from different storage pools and are seeing the same result.

Here is the error message:
Operating system error 665 (The requested operation could not be completed due to a file system limitation).

NTFS Compression is not enabled on the drives. This failure usually comes when the backup is about half completed. I believe when the backup file size is extended again. We are striping against eight files. The database is 60+ TB. I also saw some references to file fragmentation and I see that these drives show 0% percent.

I also saw several references to DBCC CHECKDB and sparse files. We are just doing a full backup and not any DBCC CHECKDBs. The goal would be to do this on a secondary server until any errors are recorded.

Best Answer

This error usually means that NTFS cannot add more fragments to the file being written to. Normally, this happens for CHECKDB sparse files but it could happen to any file. It is not sparse file specific.

I suspect your NTFS volume is already heavily fragmented. If there are many free space holes NTFS is prone to create many small fragments for new files. I have just seen a 16GB paging file with 120,000 fragments that was created on such a volume.

There is no way to tell NTFS to allocate more smartly. (Subjective note: NTFS is really stupid when it comes to allocating files. Many simple improvements come to mind but no investment is being made.)

Use a defrag tool that compacts the drive and removes holes. O&O Defrag can do that for example. Of course you can manually compact by recreating the partition. But I don't know how realistic that is at your storage sizes.

I also saw some references to file fragmentation and I see that these drives show 0% percent.

But the free space could be fragmented. That's where new data goes.