No, in-memory really is this unpolished. If you are familiar with Agile you will know the concept of "minimal shippable product"; in-memory is that. I get the feeling that MS needed a response to SAP's Hana and its ilk. This is what they could get debugged in the timeframe for a 2014 release.

As with anything else in-memory has costs and benefits associated with it. The major benefit is the throughput that can be achieved. One of the costs is the overhead for change management, as you mentioned. This doesn't make it a useless product, in my opinion, it just reduces the number of cases where it will provide net benefit. Just as columnstore indexes are now updatable and indexes can be filtered I have no doubt that the functionality of in-memory will improve over coming releases.

SQL Server 2016 is now generally available. Just as I supposed, In-Memory OLTP has received a number of enhancements. Most of the changes implement functionality that traditional tables have enjoyed for some time. My guess is that future features will be released at the same time for both in-memory and traditional tables. Temporal tables is a case-in-point. New in this version it is supported by both In-Memory and disk-based tables.

Is this a bug?

No it is not a bug. Its by design. They are kept for troubleshooting and supportability purposes.

From the SQL_Server_2014_In-Memory_OLTP White_Paper

Database administrators do not need to maintain the files that are generated by native compilation. SQL Server automatically removes generated files that are no longer needed, for example on table and stored procedure deletion, and on drop database, but also on server or database restart.

I tried to repro your scenario on SQL Server 2014 + RTM + (Build12.0.2000.8) - Dev Edition server by creating a test memory optimized table and checking the dll loaded using

     SELECT name, description FROM sys.dm_os_loaded_modules
WHERE description = 'XTP Native DLL'

After I dropped my table, the dll still appears in the output of above select statement and the files are still in the folder and after restart they are still there too.

From Books Online -

No user interaction is needed to manage these files (.c, .obj, .xml, .pdb., .dll). SQL Server will create and remove the files as necessary.

SO I guess, we just have to follow what Microsoft says - SQL server will manage them for us :-)

ONLY FOR EDUCATIONAL PURPOSE :

I managed to clean up the old files by

• Issuing a manual CHECKPOINT on the database.
• Taking database offline and then bringing it online.

Ideally, you should not be restarting the server instance, just manual checkpoint and offline/online of the database will clear the files.

e.g. Repro :

USE master
GO
create database db1
GO
ALTER DATABASE db1 ADD FILEGROUP db1_mod CONTAINS memory_optimized_data
GO
-- adapt filename as needed
ALTER DATABASE db1 ADD FILE (name='db1_mod', filename='D:\SQLServer2014\MSSQL12.SQL2014\MSSQL\DATA\db1_mod') -- change here as per your need !!
 TO FILEGROUP db1_mod
GO
USE db1
GO
CREATE TABLE dbo.t1
(c1 int not null primary key nonclustered,
c2 int)
WITH (MEMORY_OPTIMIZED=ON)
GO

--- now check if the dll is loaded or not

SELECT name, description FROM sys.dm_os_loaded_modules WHERE description = 'XTP Native DLL'

--- now drop table and do a manual checkpoint

use db1;
drop table dbo.t1;
checkpoint

Still the module is loaded in memory (even server restart will load the module sometimes)

The (.c, .obj, .xml, .pdb., .dll) are still present in the folder :

Now take the database offline and then bring it online - the (.c, .obj, .xml, .pdb., .dll) are all gone ...