Safari history disappears despite settings

backupbrowser-historyhistorysafariweb-browser

I like to have my browser history span years back so I can look at things which may have interested me a long time ago. To pursue this desire, I have set Safari to remove history items "manually" under the General tab in the Preferences. I have done the same thing for download list items. My intent was (and is) to not remove history items ever and to keep Safari from doing it automatically.

This seemed to be working fine for a while, and I still have history which goes back more than a year. However, I have recently noticed that the oldest days in my history are disappearing. At first I wasn't sure, but after conducting some tests I am sure. It seems to happen randomly and infrequently. I think that the Safari window also loses focus when it happens.

I am not sure how or why this is happening. The only thing I can think of is that there may be some hard-coded limit in Safari itself. Or perhaps it purges old history items when it reaches a memory limit.

I see that my History.db file is 80MiB in ls output:

  -rw-r--r--    1 my_username staff  80M Dec 29 13:02 History.db

Is 80MiB some sort of implicit limit for Safari?

Another troubling facet of this issue is that I cannot export my Safari history to another browser. Both Chrome and Firefox refuse to import anything from Safari except bookmarks. Moreover, I do not have many backups of my Safari history. Although, backups would not help too much anyway since a backup would lack my recent history.

So how can I get Safari to stop losing my oldest history items?

Some environment info:

My Safari version is currently 11.0.2 on macOS 10.13.2. Also, I am not using iCloud integration with Safari.

EDIT:

At the time of the bounty on this question, I will also consider accepting answers which offer a way to export my history to another browser (preferably Firefox or Chrome).

Best Answer

This is size limit imposed by Safari.

A sqlite database can obviously be larger than 80M; it is only limited by the amount of available storage space, but technically it has an upper file size limit of about 140TB

Other developers have noted that in (older versions of) Safari, the limit is capped at 50GB. It's reasonable to assume that this limit has been increased in subsequent releases of macOS/iOS/Safari.

It also stands to reason that this size limit is imposed by (Apple) Safari because, by default, it is configured to synchronize with iCloud to other Apple devices many of which are constrained by storage availability (iPhones and iPads, for example).

We also know that iCloud's basic (free) storage allotment is 5GB and this must account for much more than browsing history. Things like files, settings, device backups (where applicable) keychains, notes, etc. must be allocated space. Synchronization of these items is set by default by Apple.

It wouldn't make much sense to allow an unlimited history where History.dbcould outstrip the capacity of iCloud/iPad/iPhone.

Even though you're not using iCloud integration, the average user does and Apple will architect it's products around this premise and design their products accordingly.

Extracting Your History

You can pull your history items out of the History.db file with this (very) simple sqlite3 script

Create a "commands.txt" file and populate it with the following commands:

.mode csv
.once history.txt
select visit_time, title, url from history_visits inner join history_items on history_items.id = history_visits.history_item;

Then issue the command 

sqlite3 History.db < commands.txt

This will create a CSV file called history.txt containing the time, title and URL of the history items. This assumes you are in the ~/Library/Safari directory. Keep in mind that the date/time of your history item will be in epoch time (although the epoch is different in this case) and will need to be ultimately converted.

Related Question