Mojave 10.14.4 here — mds_worker
processes constantly crashing and re-spawning as per the logs. The 'indexing' spotlight bar stops at various different points, always getting stuck, though in different places apparently. I've rebuilt the database many times (using mdutil -i on/off, -a -E, etc., as well as the GUI method). The database is clearly not complete – searching filenames that I can literally see turns up nothing in some cases and finds it in others. Mail isn't returning good search results either.
In-place OS reinstall has been done; didn't clear up the issue.
Apple is now recommending I install Mojave on a separate partition, boot into it and see if it has the same issue – don't see how that's going to help (yet), but I have to do it in order to move the process forward. I'll update with any findings once I've done that.
It seems like it's choking on some piece of data I have, and I need to figure out what that is so I can remove it.
It seems like many md_worker
processes get spawned from a parent — and the process that crashes does so, so quickly that I can't don't have time to do an lsof -p
on it.
Is there another way to do this, or to get more information about the file(s) it's choking on from log
?
Edit1: Ok, I figured out that I can get significantly more information about what mds* is doing with the following commands. I have yet to run this on the affected machine.
sudo log config --mode "private_data:on"
log stream --style compact --predicate '(process contains[c] "mdworker") && (subsystem == [c] "com.apple.spotlightserver")' --info --debug
The first command turns off obfuscation, otherwise file paths are obscured (and replaced with "<private>") – I suppose to make it safe to share logs. See here: https://superuser.com/questions/1311578/in-console-app-how-can-i-reveal-to-what-private-tags-are-actually-referring
Best Answer
Okay, I think the issue is resolved.
The culprit turned out to be an old piece of software from 2011 - Tags, for Mail (http://www.caseapps.com/tags/).
It installed an mdimporter into /Library/Spotlight/.
After reading up on the new logging system and figuring out how to disable obfuscation in order to not get those
<private>
entries in the logs since those don't help me see which files are being accessed by the md processes,I ran
log stream --style compact --predicate '(process contains[c] "mdworker") && (subsystem ==[c] "com.apple.spotlightserver")' --info --debug
and watched it.It kept hanging up in a particular directory, in ~/Mail/V6. I went looking through the directory tree looking for permissions/uid issues -- nothing.
I re-ran the command with
tee
, so that I couldtail
it andgrep
out interesting parts to investigate.Then I noticed that it seemed like
_TagsMail.mdimporter
was being mentioned in each line wheremdworker
was being called to import an email.An excerpt:
2019-05-08 21:36:07.002 Db mdworker[53808:1463e7e] [com.apple.spotlightserver:Server] sandbox profile activated time:0.01667 uid:501 profile:mdworker cache:/var/folders/xb/25w333f48xl1ns0059bd0kxh0000gn/C/ temp:/var/folders/xb/25w333f48xl1ns0059bd0kxh0000gn/T/ home:/Users/B**** lib:/Users/B****/Library M:(null) MV2:(null) typeExtension:0 bundleId:com.gravityapps.TagsMail
I looked up
tagsmail.mdimporter
and the first web page hit is "_TagsMail.mdimporter prevents Spotlight in Mojave to finish": http://www.mnott.de/_tagsmail-mdimporter-prevents-spotlight-in-mojave-to-finish/Interesting. I moved the mdimporter plugin out and immediately saw progress being made, rather than the same roughly 10x files being attempted to index continually.
Rebooted the machine, created a test file on the desktop, searched for it and saw it pop up almost immediately. Good sign that the indexer is working properly.