MacBook – Diagnosing Kernel Panic “IONVMeFamily”

kernel-panicmacbook promojave

My 15" Macbook Pro (mid-2015) starts to thrash despite plenty of disk space after sleeping until it becomes unusably slow with a lagging mouse pointer. After a restart, it shows the dreaded "your computer restarted due to a problem" screen.

I'm running a few kernel extensions, most notably Karabiner-Elements v10.22.

What could the cause be? Is there a corrupt kernel file somewhere? Memory problem? Other hardware issue?

I'm running macOS Mojave 10.14.5, but this has happened with prior versions.

Here is the panic report:

Anonymous UUID:       32E22B79-B363-6DD0-8A47-EF0AA229B07B

Mon Mar 30 09:44:30 2020

*** Panic Report ***
panic(cpu 2 caller 0xffffff7f88992c0c): nvme: "Fatal error occurred. CSTS=0xffffffff US[1]=0x0 US[0]=0x16 VID/DID=0x2260126f
. FW Revision=C2.3.21\n"@/BuildRoot/Library/Caches/com.apple.xbs/Sources/IONVMeFamily/IONVMeFamily-387.260.15/IONVMeController.cpp:5334
Backtrace (CPU 2), Frame : Return Address
0xffffff91f4fabae0 : 0xffffff80057aea2d 
0xffffff91f4fabb30 : 0xffffff80058e9e95 
0xffffff91f4fabb70 : 0xffffff80058db70a 
0xffffff91f4fabbe0 : 0xffffff800575bb40 
0xffffff91f4fabc00 : 0xffffff80057ae447 
0xffffff91f4fabd20 : 0xffffff80057ae293 
0xffffff91f4fabd90 : 0xffffff7f88992c0c 
0xffffff91f4fabef0 : 0xffffff7f88992e93 
0xffffff91f4fabf30 : 0xffffff8005ebc6ae 
0xffffff91f4fabf70 : 0xffffff8005ebc428 
0xffffff91f4fabfa0 : 0xffffff800575b0ce 
      Kernel Extensions in backtrace:
         com.apple.iokit.IONVMeFamily(2.1)[FAFF8EA6-B8EF-314D-886A-DEAB39FF3EA9]@0xffffff7f88985000->0xffffff7f889c4fff
            dependency: com.apple.driver.AppleMobileFileIntegrity(1.0.5)[A367CF6B-B3EA-36FE-86E6-03B771CA8A69]@0xffffff7f866e3000
            dependency: com.apple.iokit.IOPCIFamily(2.9)[CC6A465F-5A24-304D-B9DF-8C27819CC214]@0xffffff7f86095000
            dependency: com.apple.driver.AppleEFINVRAM(2.1)[C3C74D36-E06B-3753-A373-5BC9D486DD9C]@0xffffff7f868b9000
            dependency: com.apple.iokit.IOStorageFamily(2.1)[D43332F2-2D40-3DF1-8F77-D633BCDAE660]@0xffffff7f864d9000
            dependency: com.apple.iokit.IOReportFamily(47)[5772C11C-ED28-322A-910F-874583B23101]@0xffffff7f86a08000

BSD process name corresponding to current thread: kernel_task

Mac OS version:
18F132

Kernel version:
Darwin Kernel Version 18.6.0: Thu Apr 25 23:16:27 PDT 2019; root:xnu-4903.261.4~2/RELEASE_X86_64
Kernel UUID: 7C8BB636-E593-3CE4-8528-9BD24A688851
Kernel slide:     0x0000000005400000
Kernel text base: 0xffffff8005600000
__HIB  text base: 0xffffff8005500000
System model name: MacBookPro11,4 (Mac-06F11FD93F0323C5)

System uptime in nanoseconds: 433481437913449
last loaded kext at 386888820549295: com.apple.filesystems.ntfs 3.13 (addr 0xffffff7f88f12000, size 389120)
last unloaded kext at 430964519803754: com.apple.driver.usb.cdc 5.0.0 (addr 0xffffff7f88efc000, size 28672)
loaded kexts:
com.zerotier.tap    1.0
org.pqrs.driver.Karabiner.VirtualHIDDevice.v061000  6.10.0
com.intel.kext.intelhaxm    6.2.1
org.pqrs.driver.Karabiner   10.22.0
com.Cycling74.driver.Soundflower    2
com.apple.filesystems.smbfs 3.3.2
com.apple.fileutil  20.036.15
com.apple.filesystems.autofs    3.0
com.apple.iokit.IOBluetoothSerialManager    6.0.12f1
com.apple.driver.AppleUpstreamUserClient    3.6.5
com.apple.driver.AppleHDA   282.54
com.apple.driver.AGPM   110.25.10
com.apple.driver.ApplePlatformEnabler   2.7.0d0
com.apple.driver.X86PlatformShim    1.0.0
com.apple.driver.AppleGraphicsDevicePolicy  3.40.16
com.apple.AGDCPluginDisplayMetrics  3.40.16
com.apple.driver.AppleHV    1
com.apple.iokit.IOUserEthernet  1.0.1
com.apple.driver.AppleIntelHD5000Graphics   12.0.9
com.apple.driver.pmtelemetry    1
com.apple.driver.AppleBacklight 170.12.11
com.apple.Dont_Steal_Mac_OS_X   7.0.0
com.apple.driver.AppleIntelFramebufferAzul  12.0.9
com.apple.driver.eficheck   1
com.apple.driver.AppleThunderboltIP 3.1.2
com.apple.driver.AudioAUUC  1.70
com.apple.driver.AppleOSXWatchdog   1
com.apple.driver.AppleIntelSlowAdaptiveClocking 4.0.0
com.apple.iokit.BroadcomBluetooth20703USBTransport  6.0.12f1
com.apple.driver.AppleLPC   3.1
com.apple.driver.AppleCameraInterface   6.8.0
com.apple.driver.AppleSMCLMU    212
com.apple.driver.AppleMCCSControl   1.5.9
com.apple.driver.AppleTopCaseHIDEventDriver 138.3
com.apple.driver.AppleUSBTopCaseDriver  138.3
com.apple.driver.AppleUSBCardReader 456.260.3
com.apple.filesystems.apfs  945.260.7
com.apple.driver.AirPort.BrcmNIC    1400.1.1
com.apple.driver.AppleVirtIO    2.1.3
com.apple.filesystems.hfs.kext  407.200.4
com.apple.AppleFSCompression.AppleFSCompressionTypeDataless 1.0.0d1
com.apple.BootCache 40
com.apple.AppleFSCompression.AppleFSCompressionTypeZlib 1.0.0
com.apple.AppleSystemPolicy 1.0
com.apple.private.KextAudit 1.0
com.apple.driver.AppleSmartBatteryManager   161.0.0
com.apple.driver.AppleACPIButtons   6.1
com.apple.driver.AppleRTC   2.0
com.apple.driver.AppleHPET  1.8
com.apple.driver.AppleSMBIOS    2.1
com.apple.driver.AppleACPIEC    6.1
com.apple.driver.AppleAPIC  1.7
com.apple.nke.applicationfirewall   201
com.apple.security.TMSafetyNet  8
com.apple.kext.triggers 1.0
com.apple.driver.DspFuncLib 282.54
com.apple.kext.OSvKernDSPLib    528
com.apple.iokit.IOAVBFamily 740.1
com.apple.plugin.IOgPTPPlugin   740.2
com.apple.iokit.IOEthernetAVBController 1.1.0
com.apple.iokit.IOSkywalkFamily 1
com.apple.driver.AppleSSE   1.0
com.apple.driver.AppleBacklightExpert   1.1.0
com.apple.iokit.IONDRVSupport   530.51
com.apple.iokit.IOAcceleratorFamily2    404.9
com.apple.iokit.IOSurface   255.5.4
com.apple.driver.X86PlatformPlugin  1.0.0
com.apple.AppleGPUWrangler  3.40.16
com.apple.iokit.IOSlowAdaptiveClockingFamily    1.0.0
com.apple.iokit.BroadcomBluetoothHostControllerUSBTransport 6.0.12f1
com.apple.iokit.IOBluetoothHostControllerUSBTransport   6.0.12f1
com.apple.iokit.IOBluetoothHostControllerTransport  6.0.12f1
com.apple.driver.AppleHDAController 282.54
com.apple.iokit.IOHDAFamily 282.54
com.apple.driver.IOPlatformPluginFamily 6.0.0d8
com.apple.AppleGraphicsDeviceControl    3.40.16
com.apple.driver.AppleGraphicsControl   3.40.16
com.apple.driver.AppleSMBusController   1.0.18d1
com.apple.iokit.IOGraphicsFamily    530.66
com.apple.driver.AppleActuatorDriver    2450.1
com.apple.driver.AppleHSBluetoothDriver 138.3
com.apple.driver.IOBluetoothHIDDriver   6.0.12f1
com.apple.iokit.IOBluetoothFamily   6.0.12f1
com.apple.driver.AppleHIDKeyboard   208
com.apple.driver.AppleMultitouchDriver  2450.1
com.apple.driver.AppleInputDeviceSupport    2440.2
com.apple.driver.AppleUSBAudio  315.6
com.apple.driver.usb.IOUSBHostHIDDevice 1.2
com.apple.iokit.IOAudioFamily   206.5
com.apple.vecLib.kext   1.2.0
com.apple.driver.usb.networking 5.0.0
com.apple.driver.usb.AppleUSBHostCompositeDevice    1.2
com.apple.driver.usb.AppleUSBHub    1.2
com.apple.driver.AppleThunderboltDPInAdapter    5.6.4
com.apple.driver.AppleThunderboltDPAdapterFamily    5.6.4
com.apple.driver.AppleThunderboltPCIDownAdapter 2.1.4
com.apple.driver.AppleThunderboltNHI    4.7.6
com.apple.iokit.IOThunderboltFamily 6.8.4
com.apple.iokit.IO80211Family   1200.12.2
com.apple.driver.mDNSOffloadUserClient  1.0.1b8
com.apple.driver.corecapture    1.0.4
com.apple.iokit.IONVMeFamily    2.1.0
com.apple.driver.usb.AppleUSBXHCIPCI    1.2
com.apple.driver.usb.AppleUSBXHCI   1.2
com.apple.iokit.IOSerialFamily  11
com.apple.filesystems.hfs.encodings.kext    1
com.apple.driver.usb.AppleUSBHostPacketFilter   1.0
com.apple.iokit.IOUSBFamily 900.4.2
com.apple.driver.AppleUSBHostMergeProperties    1.2
com.apple.driver.AppleEFINVRAM  2.1
com.apple.driver.AppleEFIRuntime    2.1
com.apple.iokit.IOSMBusFamily   1.1
com.apple.iokit.IOHIDFamily 2.0.0
com.apple.security.quarantine   3
com.apple.security.sandbox  300.0
com.apple.kext.AppleMatch   1.0.0d1
com.apple.driver.DiskImages 493.0.0
com.apple.driver.AppleFDEKeyStore   28.30
com.apple.driver.AppleEffaceableStorage 1.0
com.apple.driver.AppleKeyStore  2
com.apple.driver.AppleUSBTDM    456.260.3
com.apple.driver.AppleMobileFileIntegrity   1.0.5
com.apple.iokit.IOUSBMassStorageDriver  145.200.2
com.apple.iokit.IOSCSIBlockCommandsDevice   408.250.3
com.apple.iokit.IOSCSIArchitectureModelFamily   408.250.3
com.apple.iokit.IOStorageFamily 2.1
com.apple.kext.CoreTrust    1
com.apple.driver.AppleCredentialManager 1.0
com.apple.driver.KernelRelayHost    1
com.apple.iokit.IOUSBHostFamily 1.2
com.apple.driver.usb.AppleUSBCommon 1.0
com.apple.driver.AppleBusPowerController    1.0
com.apple.driver.AppleSEPManager    1.0.1
com.apple.driver.IOSlaveProcessor   1
com.apple.iokit.IOReportFamily  47
com.apple.iokit.IOTimeSyncFamily    740.2
com.apple.iokit.IONetworkingFamily  3.4
com.apple.driver.AppleACPIPlatform  6.1
com.apple.driver.AppleSMC   3.1.9
com.apple.iokit.IOPCIFamily 2.9
com.apple.iokit.IOACPIFamily    1.4
com.apple.kec.pthread   1
com.apple.kec.corecrypto    1.0
com.apple.kec.Libm  1

EOF
Model: MacBookPro11,4, BootROM 189.0.0.0.0, 4 processors, Intel Core i7, 2.2 GHz, 16 GB, SMC 2.29f24
Graphics: kHW_IntelIrisProItem, Intel Iris Pro, spdisplays_builtin
Memory Module: BANK 0/DIMM0, 8 GB, DDR3, 1600 MHz, 0x802C, 0x31364B544631473634485A2D314736453120
Memory Module: BANK 1/DIMM0, 8 GB, DDR3, 1600 MHz, 0x802C, 0x31364B544631473634485A2D314736453120
AirPort: spairport_wireless_card_type_airport_extreme (0x14E4, 0x152), Broadcom BCM43xx 1.0 (7.77.61.2 AirPortDriverBrcmNIC-1305.8)
Bluetooth: Version 6.0.12f1, 3 services, 27 devices, 1 incoming serial ports
Network Service: Wi-Fi, AirPort, en0
USB Device: USB 3.0 Bus
USB Device: AS2105
USB Device: Apple Internal Keyboard / Trackpad
USB Device: Bluetooth USB Host Controller
USB Device: Hub
USB Device: Scarlett 2i4 USB
USB Device: USB Optical Mouse
USB Device: Kinesis Keyboard Hub
USB Device: Composite Device
Thunderbolt Bus: MacBook Pro, Apple Inc., 27.1

Best Answer

It’s going to be a bit challenging to put that panic and then get a core dump and put it in Inspector or a debugger to determine the actual code running.

I would get stats - can you figure out how to trigger this? How long does it take to get 10 panics?

Then pull all third party KEXT and wait the time it would have panicked 10 times. You’ll have a pretty good idea if it’s hardware or software at that point.

Since you’re likely out of warranty, getting Apple to analyze this will be challenging. Memory can cause panics anywhere, but since you’re seeing storage specific, I’d consider swapping that or upgrading if you can’t get diagnostics run from Apple directly or AASP.

tldr - you can spend effort to symbolicate / trace exactly what happened here, but sometimes doing a little process of elimination and logging / looking at the frequency of panics can get you the same answer with much less time invested.

Related Question