OS X El Capitan: Displace ~/Library/Caches to Another Disk

Prologue

So, to save money and real estate, here in NYC I have been using a quad-core Late-2012 Mac Mini, which was the cheapest (and reasonably new) quad-core Mac still floating around last year, with aftermarket 16GB RAM upgrades, and the change from the pitiful 5400RPM hard disk to a Samsung 1TB SSHD with 8GB NAND cache. I didn’t want to place all my data on an external disk — especially when it’s connected through USB, so I wouldn’t want to only have one SSD inside the Mac — even though I knew my usage pattern, along with OS X’s memory management strategy, would incur lots of swap usage, I couldn’t afford a 128GB SSD + 1TB HDD setup. So I gave a shot with the SSHD.

Well, time went by and it turned out that while the machine did a wonderful job when I was editing my dozens-layers HeHuan Mountain star trails photo, or transcoding videos, it actually performed like hell during day-to-day usage, when I switched programs (do I need to say “apps” here?) frequently and, taking browsers as example, when data (transient or permanent) was read from and written to disk in super small chunks. And I always recorded a swap usage at around 8GB to 12GB, and the SSHD was falling short of tempering my impatience.

So earlier this year I began to ponder on getting an SSD for special purposes. Unexpectedly, after a year, SSD price has dropped significantly, and I could now get a 250GB Samsung SSD for less than $80 before tax (it’s way cheaper than the aftermarket vendor’s offer). Though, that’s a TLC one! But I knew what my special purpose was going to be: virtual machines (only OS and programs, not data), browser caches for Firefox and Chrome, and Adobe Camera Raw caches, and currently-being-processed photos. These were more or less not mission-critical, and I still employed daily rsync backups to my NAS, so any data loss due to it being a TLC SSD and USB-connected would be bearable.

Oh and, while I could have bought an iFixit toolkit to hack the SSD into the Mac (which also gave me TRIM support), I forsook that and only used a USB to SATA converter. While the latter did have drawbacks like USB protocol overheads and the CPU processing power needed to handle the USB stack, the overall access latency (and even bandwidth) was still significantly improved.

The Actual Thing (of this post’s title)

But that’s actually quite not enough. Even though I could put Firefox and Chrome cache to the SSD (the latter was actually being done through symlinks, and the former using about:config), there were still tons of programs and user-related system services that accessed ~/Library/Caches which were not benefiting from the SSD. While I thought it could easily be mitigated by symlinking the whole ~/Library/Caches off-disk, this soon proved to be wrong.

While the system didn’t crash (even after reboots) and programs didn’t behave obviously erratically, I soon discovered that my Notes.app (okay, “app”) was not syncing with iCloud. Further investigations in Console.app showed that… (only showing relevant stack frames here for brevity)

May 15 16:16:19 mnjulnimi kernel[0]: Sandbox: cloudd(10110) deny(1) file-write-create /Volumes/Cache_VM/MnjulLibraryCaches/CloudKit
May 15 16:16:19 mnjulnimi cloudd[10110]: *** Terminating app due to uncaught exception 'NSGenericException', reason: 'Error opening db at /Users/USER/Library/Caches/*/CloudKitMetadata: <CKError 0x7f9602433850: "Internal Error" (1); "Error creating db dir at /Users/USER/Library/Caches/CloudKit: Error Domain=NSCocoaErrorDomain Code=513 "You don't have permission to save the file "CloudKit" in the folder "Caches"." UserInfo={NSFilePath=/Users/mnjul/Library/Caches/CloudKit, NSUnderlyingError=0x7f960270b180 {Error Domain=NSPOSIXErrorDomain Code=1 "Operation not permitted"}}">'
        *** First throw call stack:
        (
                2   CoreFoundation                      0x00007fff947a84bd +[NSException raise:format:] + 205
                3   CloudKit                            0x00007fff8d902d8c +[CKSQLiteError raise:code:extended:] + 1660
                4   CloudKit                            0x00007fff8d8ff148 -[CKSQLite raise:] + 1055
                5   CloudKit                            0x00007fff8d8fe82b -[CKSQLite open] + 145
                6   CloudKitDaemon                      0x00007fff932f41cf -[CKDMetadataCache init] + 262
        )
May 15 16:16:19 mnjulnimi com.apple.xpc.launchd[1] (com.apple.cloudd[10110]): Service exited due to signal: Abort trap: 6

Yikes. cloudd’s CloudKit library’s SQLite had no permission to create a file (or directory for that matter) in my symlinked-to caches folder. At first sight I thought it was the usual file mode/owner/ACL issues, but after some mangling around with chmod/chown, I found everything was actually already good. It’s at this moment I saw cloudd was access-denied from sandboxd. Okay. So, OS X was trying to restrict placement of (potentially sensitive) user data into external file systems. This actually made sense, but, well, I knew what I was doing. Googling around using a mix of keywords brought me to an answer from Apple Support Communities that instructed whitelisting the symlinked-to directory in sandboxd profile.

Here’s the detour: OS X El Capitan has System Integrity Protection, where even root user cannot modify the said application.sb file when the system is booted normally, and for that, I had to reboot into Recovery to modify the file. And a double detour: vim in that Recovery was somehow… the plain vi. Yes. I needed to use hjkl for navigation, and x for deleting chars. Man I was proud for being old school.

Well, it turned out the detours weren’t worthwhile, as the whitelisting seemed to have no effect (according to the same discussion thread, it actually apparently ceased to have any effect since Mavericks). So it appeared that I had to ditch symlink and go down a more convoluted/low-level way.

From my OS knowledge, I couldn’t use hard links since I was crossing file system boundaries. This left me only one choice to experiment with: split out a new partition inside my SSD that would only be used for the ~/Library/Caches, and mount it directly on the directory.

It sounds nasty but it’s actually not as hard to do. The SSD (i.e. the original single partition that’s as large as the disk) was not really full and was already on GPT, with the existing partition on HFS+, so resizing/appending partitions without data loss wasn’t hard (but better do that in Recovery’s Disk Utilities though, not on a live OS). And then, since this mounting had to be ready before I logged in, I needed (only) to modify /etc/fstab — never thought I’d touch this file on OS X, but it’s good enough that BSD fstabs were pretty much in the same format as Linux ones, which I’d be more familiar with. Upon some extra googling, using UUID to specify the device was said commendable over using file paths, so I needed to dig out the UUID using diskutil info. From my experience, I needed to use Volumn UUID, instead of Disk / Partition UUID. Reboot — Voila, Notes.app was syncing, no issue in Console.app. (I didn’t need to copy the old cache files into the new partition, since they were, well, caches.)

Epilogue

In the end, the configuration was good (as far as performance is concerned, I didn’t do a formal benchmark so I couldn’t tell if it’s effect-wise a placebo) until I was updating my iPhone’s iOS from 9.3.1 to 9.3.2 using iTunes: during the process, iTunes needed to use the caches directory to store something that was nearly 2GB large. But I had allocated only 2GB to the cache partition (I had never observed it exceeding 1GB, but guess this kind of usage spikes happen), so it failed due to insufficient disk space. So I went to Recovery again, and adjusted the partition to be 10GB large. (I also updated OS X to 10.11.5 afterwards, which went without a hiccup). But that also means I have almost 8~9GB of precious SSD space wasted. Uhm. I’ll probably convince myself that it allows the SSD to do GC and wear-leveling more nicely, since I don’t have TRIM over USB. But that anyway depends on how Samsung’s MGX controller is doing things.

For sure, this is still far from hacking for optimal situations: I can even try to hack things to put swap files in SSD (probably following a detailed step-by-step posted in StackExchange), or put /Library/Caches or /System/Library/Caches there, but that’s gonna need some careful planning (the latter two directories don’t have large volumes of file as far as I can see, though). Additionally, I’m also wondering if I can put the whole Spotlight (mds) database of my SSHD into the SSD with the same trick, or whatsoever.

發表迴響

你的電子郵件位址並不會被公開。 必要欄位標記為 *