Linux – kernel eats memory, but never release

I know, this is the 1000th question about the same topic. I've spent days reading so many threads, but I still could not find an answer to my weird situation.

When I did have 16GB of RAM on my machine, everything was fine. I did upgrade to 32GB, and the kernel did start eating all the RAM it could (~25GB), giving no space left for applications. free and atop reports this memory usage as buffer/cache.
Cleaning the cache gives me few GB back, but only for few minutes. I did try to close all applications, except 2 SSH sessions. Same result. With less than 10 apps running, I still have about 20GB of buffer/cache eaten by the kernel. The only way to get rid of it is to reboot.

I'm running a Linux Mint 18 with kernel 4.4.0-79-generic. And for information, my system starts with less than 2GB of used RAM (no hungry app running in background).

Does anybody have an idea how to identify the leak ?

[SOLVED] – the cause was identified, it was because of bootchart (my comment about it is in my own answer below)