Safari 13 update sets negative com.apple.quarantine -1 making multiple directories inaccessible

Updated to Safari 13 today on Mojave 10.14.6 and suddenly a bunch of my ~/Library/ directories are inaccessible via terminal and other apps (even with sudo). Checking the xattr I see:

com.apple.quarantine -1

Directories affected:

~/Library/Calendars (previously 28)
~/Library/Cookies (previously 15)
~/Library/Mail (previously 19)
~/Library/Safari (previously 15)

Two other changes in ~/Library/ I found are with:

com.apple.metadata:com_apple_backup_excludeItem -1

Directories affected:

~/Library/PersonalizationPortrait (previously 61)
~/Library/Saved Application State (previously 61)

Now I can't access these directories via terminal or some other apps. Interestingly I can navigate to the directory with Finder. e.g. Navigate to ~/Library/Safari/Extensions/ But when I click on Extensions.plist, Xcode does not have permission to open it.

I can't seem to find documentation on what the negative values mean and how to change it because sudo doesn't even have permission to change it.

Something more weird is my High Sierra machine doesn't have these com.apple.quarantine changes.

Update: It's very possible that multiple directories outside ~/Library/ are also affected because my next Time Machine backup was over 1 GB and the only thing that changed is the install of Safari 13.

Update 2: My High Sierra machine's next Time Machine was over 11 GB after this one Safari 13 install. Something weird is definitely happening with this update.