Spotlight Indexing Failure

3 minute read

While using the preview feature of the Hazel app, I noticed that files were greyed out in search results, but results would be grayed out. At first, I thought it was a bug in Hazel. When I did the same search search in Finder, it also reported the wrong location for the file.

Not being able to find file is a pretty big deal for me, since the document index huge part of my knowledge management system(KMS). If I need find information, I look for it in my Joplin notes or electronic document library.

My Setup

  • 2019 Macbook Pro 16” 32GB RAM 1TB SSD
  • Big Sur 11.2.3

The Symptoms

  • Finder search returns the wrong results for files.
  • I get the mds related error messages every few seconds in /var/log/system.log.

      > tail -F /var/log/system.log
    
      com.apple.xpc.launchd[1] (com.apple.mdworker.shared.0D000000-0700-0000-0000-000000000000[18816]): Service exited due to SIGKILL | sent by mds[98]
    
  • If I filter out the mdwork messages, I can see there is a memory allocation error ( root cause?) .

      > tail -F /var/log/system.log | grep -v com.apple.mdworker
    
      mdworker_shared[88110]: mdworker_shared(88110,0x700000e3a000) malloc: malloc_memory_event_handler: approaching memory limit. Starting stack-logging.
    
      mdworker_shared[88110]: mdworker_shared(88110,0x700000e3a000) malloc: recording malloc (and VM allocation) stacks using lite mode
    
      mdworker_shared[88110]: mdworker_shared(88110,0x700000e3a000) malloc: malloc_memory_event_handler: stopping stack-logging
    
      mdworker_shared[88110]: mdworker_shared(88110,0x700000e3a000) malloc: turning off recording malloc and VM allocation stacks using lite mode
    
      mdworker_shared[88110]: mdworker_shared(88110,0x700000fc0000) malloc: MallocStackLogging: stack id is invalid. Turning off stack logging
    
      mdworker_shared[88110]: mdworker_shared(88110,0x700000fc0000) malloc: malloc stack logging not enabled.
    

Troubleshooting

  • Run disk repair on all container in recovery mode ( cmd+r on startup ).
  • Rebuild Spotlight Index as per Apple support article.
  • Disconnect network and USB drives.
  • Check indexing settings on the command line.

Run Disk Repair

I booted my system into recovery mode by holding down cmd+r keys, powering on my Mac, and waiting for the Apple logo to appear. I ran the disk repair utility on the boot and data containers ( I only have one of each ). It didn’t find any problems.

Rebuild Spotlight Indexes

This Apple support article explains how to delete and rebuild indexes. I followed the instructions. It did delete the system indexes, but the previous error returned when I tried to rebuild them. I tried excluding folders at various levels, but only excluding my entire Internal Drive stopped the errors.

Disconnect network and USB drives

To isolate the internal drive as the location of the issues, I disconnected both network and USB drives from my Mac.

I use an application called AutoMounter to manage network drives. I “paused” all of the drives and then disconnected them from the command line.

sudo su #open shell session as root user.
mount -t smb  # list smb mounted drives.
umount /path/to/drive/mount  # do this for each drive.
mount -t smb  # check to make sure that you got them all.

I used Finder to eject my USB drive.

Check Indexing on the Command Line

You can view Spotlight indexing settings using the mdutil utlity. Some Mac users have reported issues where Spotlight was indexing time machine volumes. This should tell you if that is happening.

Here is my output

sh-3.2# mdutil -Ea

/:

    Indexing enabled.

/System/Volumes/Data:

    Indexing enabled.

/Volumes/DATA:

2021-04-18 11:42:43.584 mdutil[53796:224046] mdutil disabling Spotlight: /Volumes/DATA -> kMDConfigSearchLevelOff

    Indexing and searching disabled.

Nothing interesting here. It matches what System Preferences->Spotlight->Privacy settings shows me.

Next Steps

While I didn’t fix the issue, I became more familiar with how Spotlight works and the mdutil utility. I researched the memory allocation error but didn’t find any useful information on how to resolve that error. I’ll look into the indexing issue some more via posting on forums. However, I will report it to Apple support if a solution doesn’t turn up soon.

Take Aways

The system log at /var/log/system.log provides useful insight into what is going on under the hood of your Mac. I found some additional issues such as missing system extensions

    VTDecoderXPCService[26428]: getattrlist failed for /Library/GPUBundles/AMDRadeonVADriver2.bundle/Contents/MacOS/AMDRadeonVADriver2: #2: No such file or directory
    VTDecoderXPCService[26428]: getattrlist failed for /System/Library/Extensions/AMDRadeonVADriver2.bundle/Contents/MacOS/AMDRadeonVADriver2: #2: No such file or directory

And this mysterious error message

     mdsync[28161]: objc[28161]: Class MDSPathFilter is implemented in both /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Metadata.framework/Versions/A/Metadata and /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Metadata.framework/Versions/A/Support/mdsync. One of the two will be used. Which one is undefined.

It isn’t a bad idea to take a peek at the log on occasion. Tailing the log is a good way to do so.

tail -F /var/log/system.log

It might give you good information on issues such as why your backups are taking a long time to run, why you cannot find files in Finder, or other odd behaviors.

Leave a comment