MacOS – Mac Pro (Early 2008) periodically crashes in unusual way, requires reinstall

crashmac promacos

Update 07/10/13: I have traced this problem down to symbolic link corruption. Eventually a symlink in one of the system components (frameworks, etc.) gets corrupted, which could explain the unusual (and in each case different) system-wide failures. This see to affect a fair number of people. Current working theory is that there is a bizarre bug introduced around 10.7 Lion that affects volumes greater than 2 TB in size, that eventually causes symbolic links associated with most frequently used software to corrupt themselves. Third party software does not seem to affect this, people have seen this problem even on "vanilla" systems. The only commonality appears to be volumes >= 2 TB in size. Repartitioning to volumes < 2 TB seems to do the trick. Here are two threads discussing the issue at the Apple Forums.

Mac Pro (Early 2008), 2x Intel Quad-core Xeon 3.2 GHz (8 cores), 14 GB RAM, 3 TB hard drive, NVIDIA GeForce GT 120 and/or ATI Radeon HD 4870), OS X Lion 10.7.4

OK, I've got a weird one here. Every few days (could be as short as two or three days, could be as long as up to several weeks), my machine crashes in an unusual manner. Examples:

  • The first time this happened, any attempt to play a video file (either from within iTunes or externally using Quicktime) would result in a blank, black video window, but I could still hear the sound from the video. If I moved my mouse within the black window I'd get playback controls and when pressed, I could hear the audio rewind/fast forward/etc. (but of course the video window would remain black) Also, any attempt to use screen sharing would result in a completely white window where the remote screen should be.
  • Another time this happened, I would constantly get the error message that VTDecoderXPCService quit unexpectedly, even if I wasn't playing a video. Trying the suggested fixes from https://discussions.apple.com/thread/3193904?start=15&tstart=0 did not fix the issue.
  • Another time this happened, the machine appeared to work fine, except any attempt to access System Preferences resulted in the spinning beach ball of death and would completely lock up the machine.
  • Another time this happened, the machine still appeared to work OK, but graphic updates were REALLY REALLY SLOW (noticeably jerky animation when, e.g. moving between desktops or jumping to Mission Control or Dashboard)

In all cases, rebooting the machine does not result in the problem being fixed — in fact, it gets worse. After I reboot, I am unable to log in to my machine at all. Either the desktop window flashes briefly but then it goes back to the login screen, or the login screen never appears. I have tried both the "guest" account as well as a standard "test" user I set up when I first installed OS X (i.e. a "clean" user), and none of those help.

What does help, however, is doing a reinstall of OS X Lion (including running the combo updater to get back to 10.7.4). Once I do this, the machine becomes usable again, and in fact runs perfectly, and I can use it error-free for a few more days to a few more weeks – after which it fails again in some strange manner.

At first I thought my hard drive might be failing; however replacing the hard drive did not help. I then thought that I might have a glitchy graphics card (the black video windows and the VTDecoderXPCService crashing suggested this to me). However replacing the video card didn't help either. (I tried another GeForce GT 120 as well as an ATI Radeon HD 4870, both of which were known to be good in other machines). I have also tried countless times fixing permissions, clearing cache files, etc. and none of that helps. The only thing that brings this machine back from the dead is, like I said before, reinstalling OS X – but only for a little while.

I didn't see anything out of the ordinary when checking the system log files.

In case there was some sort of hardware problem going on, I tried running Apple Hardware Test overnight in loop mode. The hardware passed all checks. I also tried running Memtest86+ overnight, again with no failures or errors.

The machine doesn't seem to me to run particularly hot – typically temps hover in the 40-45 C range, and can jump to 55-60 when doing heavy activity (mostly video encoding). Voltage levels look good as well. (I'm using iStat Menus for both the temperature and voltage readings)

I posted this message in the Apple support communities, and someone suggested I run the following commands and post their output:

kextstat -kl | awk '!/com\.apple/{printf "%s %s\n", $6, $7}'  
sudo launchctl list | sed 1d | awk '!/0x|com\.(apple|openssh|vix)|edu\.mit|org\.(amavis|apache|cups|isc|ntp|postfix|x)/{print $3}'  
launchctl list | sed 1d | awk '!/0x|com\.apple|edu\.mit|org\.(x|openbsd)/{print $3}'  
ls -A /e*/mach*  {,/}L*/{Ad,Compon,Ex,Fram,In,Keyb,La,Mail/Bu,P*P,Priv,Qu,Scripti,Servi,Spo,Sta}* L*/Fonts 2> /dev/null  
osascript -e 'tell application "System Events" to get name of every login item' 2> /dev/null

When he looked at the results, he noted that I was running SIMBL, and suggested that may be the problem. However, removing SIMBL had no effect.

I'm about at my wit's end here. Unfortunately the machine is out of warranty (AppleCare, too) and I don't really have the money to take this into a shop and get it diagnosed. So does anyone have any ideas?

Below I've attached the output of the commands I mentioned earlier (just ran it a few minutes ago).

Output of: kextstat -kl | awk '!/com\.apple/{printf "%s %s\n", $6, $7}'

com.rogueamoeba.InstantOn (5.0.2)
com.makemkv.kext.daspi (1)
com.blackmagic-design.iokit.DeckLinkFrameBuffer (9.5.2)
com.blackmagic-design.iokit.DeckLinkHD_Driver (1.0.0)
com.blackmagic-design.iokit.DeckLinkDriver (9.5.2)
com.squirrels.driver.AirParrotSpeakers (1.7)
com.squirrels.airparrot.framebuffer (3)
com.attotech.driver.ATTOiSCSI (3.4.1b1)
com.avatron.AVExVideo (1.6.1)
com.Cycling74.driver.Soundflower (1.5.3)
net.telestream.driver.TelestreamAudio (1.0.5)
com.avatron.AVExFramebuffer (1.6.1)
com.parallels.kext.prl_usb_connect (7.0
foo.tun (1.0)
foo.tap (1.0)
com.parallels.kext.prl_hypervisor (7.0
com.parallels.kext.prl_hid_hook (7.0
com.parallels.kext.prl_netbridge (7.0
com.parallels.kext.prl_vnic (7.0

Output of: sudo launchctl list | sed 1d | awk '!/0x|com\.(apple|openssh|vix)|edu\.mit|org\.(amavis|apache|cups|isc|ntp|postfix|x)/{print $3}'

com.agilebits.onepassword-osx-thumbs
com.parallels.vm.prl_naptd
de.bernhard-baehr.sleepwatcher
com.squirrels.kextinstaller
com.parallels.desktop.launchdaemon
com.microsoft.office.licensing.helper
com.datarobotics.ddserviced
com.datarobotics.ddservice64d
com.blackmagic-design.streaming.BMDStreamingServer
com.bjango.istatlocaldaemon
com.adobe.fpsaud

Output of: launchctl list | sed 1d | awk '!/0x|com\.apple|edu\.mit|org\.(x|openbsd)/{print $3}'

com.agilebits.onepassword-osx-helper
com.growl.GrowlLauncher
com.parallels.vm.prl_pcproxy
com.parallels.DesktopControlAgent
com.parallels.desktop.client.launch
com.delicioussafari.DeliciousSafariAgent
com.blackmagic-design.DesktopVideoFirmwareUpdater
com.bjango.istatlocal
com.google.keystone.user.agent
com.adobe.AAM.Scheduler-1.0

Output of: ls -A /etc/mach_init.d /etc/mach_init_per_login_session.d /etc/mach_init_per_user.d 'L*/Ad*' 'L*/Compon*' 'L*/Ex*' 'L*/Fram*' 'L*/In*' 'L*/Keyb*' 'L*/La*' 'L*/Mail/Bu*' 'L*/P*P*' 'L*/Priv*' 'L*/Qu*' 'L*/Scripti*' 'L*/Servi*' 'L*/Spo*' 'L*/Sta*' '/L*/Ad*' /Library/Components /Library/Extensions /Library/Frameworks '/Library/Input Methods' /Library/InputManagers '/Library/Internet Plug-Ins' '/Library/Keyboard Layouts' /Library/LaunchAgents /Library/LaunchDaemons /Library/Mail/Bundles /Library/PreferencePanes /Library/PrivilegedHelperTools /Library/QuickLook /Library/QuickTime /Library/ScriptingAdditions '/L*/Servi*' /Library/Spotlight /Library/StartupItems 'L*/Fonts'

/Library/Components:

/Library/Extensions:
fuse4x.kext
tap.kext
tun.kext

/Library/Frameworks:
AEProfiling.framework
AERegistration.framework
Adobe AIR.framework
AudioMixEngine.framework
NyxAudioAnalysis.framework
PluginManager.framework
Turbo264.framework
iLifeFaceRecognition.framework
iLifeSlideshow.framework
iTunesLibrary.framework

/Library/Input Methods:

/Library/InputManagers:
DeliciousSafari
Ecamm

/Library/Internet Plug-Ins:
AdobePDFViewer.plugin
Flash Player.plugin
JavaAppletPlugin.plugin
Quartz Composer.webplugin
QuickTime Plugin.plugin
SharePointBrowserPlugin.plugin
SharePointWebKitPlugin.webplugin
Silverlight.plugin
flashplayer.xpt
npContributeMac.bundle
nsIQTScriptablePlugin.xpt

/Library/Keyboard Layouts:

/Library/LaunchAgents:
com.adobe.AAM.Updater-1.0.plist
com.bjango.istatlocal.plist
com.blackmagic-design.DesktopVideoFirmwareUpdater.plist
com.delicioussafari.DeliciousSafariAgent.plist
com.parallels.DesktopControlAgent.plist
com.parallels.desktop.launch.plist
com.parallels.vm.prl_pcproxy.plist

/Library/LaunchDaemons:
com.adobe.SwitchBoard.plist
com.adobe.fpsaud.plist
com.apple.remotepairtool.plist
com.attotech.iscsid.plist
com.bjango.istatlocaldaemon.plist
com.blackmagic-design.streaming.BMDStreamingServer.plist
com.datarobotics.ddservice64d.plist
com.datarobotics.ddserviced.plist
com.econtechnologies.ChronoAgentRemote.plist
com.microsoft.office.licensing.helper.plist
com.parallels.desktop.launchdaemon.plist
com.squirrels.kextinstaller.plist
de.bernhard-baehr.sleepwatcher-20compatibility.plist
org.macports.rsyncd.plist
org.macports.slapd.plist

/Library/Mail/Bundles:
GPGMail.mailbundle
OMiC.mailbundle

/Library/PreferencePanes:
Air Display Preferences.prefPane
Choosy.prefPane
ChronoAgent.prefPane
Cursorcerer.prefPane
DesktopVideoPrefsPanel.prefPane
Flash Player.prefPane
Hazel.prefPane
Perian.prefPane
TrackballWorks.prefPane
Tuxera NTFS.prefPane

/Library/PrivilegedHelperTools:
com.microsoft.office.licensing.helper
com.squirrels.kextinstaller

/Library/QuickLook:
QLMarkdown.qlgenerator
iWork.qlgenerator

/Library/QuickTime:
AC3MovieImport.component
AppleAVCIntraCodec.component
AppleHDVCodec.component
AppleIntermediateCodec.component
AppleMPEG2Codec.component
AppleProResCodec.component
Blackmagic Codec.component
DVCPROHDCodec.component
Elgato Turbo.component
FCP Uncompressed 422.component
IMXCodec.component
Perian.component

/Library/ScriptingAdditions:
Adobe Unit Types.osax
DeliciousSafari Addition.osax
TotalFinder.osax

/Library/Spotlight:
Microsoft Office.mdimporter
iWork.mdimporter

/Library/StartupItems:
tap
tun

/etc/mach_init.d:

/etc/mach_init_per_login_session.d:

/etc/mach_init_per_user.d:

Output of: osascript -e 'tell application "System Events" to get name of every login item'

Cobook, ScreenSharingMenulet, PhotoSync, PinPoint, EvernoteHelper, Caffeine, TrackballWorksHelper, Alfred, DropCopy Pro, Fantastical, Moom, TextExpander, ChronoSyncBackgrounder, TotalFinder, Yoink, AirDisplayStatusItem, Choosy, AirParrot, Melo, Synergy, Cursorcerer, ScreenCapture, HazelHelper, PleaseSleep, SMARTReporter, iTunesHelper, DDAssist, Dropbox

Best Answer

Thanks all for your suggestions. Turns out that my crashes and instability were due to symbolic links getting corrupted. There are a LOT of symlinks in OS X, especially within frameworks; and when a framework gets screwed up, really odd things can happen. Reinstalling the OS effectively reloads the system from a known good state, so fixes all the corrupted symlinks, which is why reinstalling made things work again, for a while at least.

Turns out I am not the only person to experience this, although it seems to be very rare, it has happened to some others.

Now I'm not going to say that the problem has gone away for sure lest I jinx myself, but I haven't seen this happen in a while now. Not since 10.8.2, possibly 10.8.1. So it is possible that Apple may have silently fixed something. keeps fingers and toes crossed