dmesg – Custom Time Format

datedmesg

dmesg has the option -T to print human-readable timestamps, and new versions have the option --time-format, where the time format can be changed, for instance:

--time-format iso

Unfortunately, the iso time format is not much readable either. Can I define my own time format using the standard date syntax? ie:

date +%F-%T

Best Answer

Yes, you can, e.g. by re-formatting the time with gawk (GNU awk) like this:

dmesg --time-format iso | gawk '{ printf (strftime("+%F-%T",mktime(gensub("[-+T:,]"," ","g",$1)))) ; $1 = ""; print $0}'

dmesg prints iso dates here like this:

2019-09-02T06:10:30,708864+02:00 this is an important message

We strip the first string of special characters then use the mktime function in (g)awk to parse the string into a date, then strftime to format - use your favourite parameters.

This gives something like

+2019-09-02-06:10:30 this is an important message

Small notes:

We ignore the time zone here assuming this is obvious since we are running the command locally.
Please note that the time given by dmesg can be inaccurate (in fact, sometimes way off) if you suspend and then wake the computer due to the way timestamps are stored, see dmesg(1):
```
man 1 dmesg
```

The Solution:

In a nutshell,

KDE just uses QT’s QLocale. Which itself uses hard-coded data inside qlocale_data_p.h, deep in QT’s core library code.

This data is apparently manually generated from the Unicode Consortium’s “Common Locale Data Repository”, using several Python scripts inside util/local_database/ of the QT source code package. Which itself is not even part of the source code, I am certain. Let alone, using any kind of configuration or data files on your computer.

So the only way to do this, is to …

Download the CLDR files, either as a package, at http://unicode.org/Public/cldr/latest/core.zip (or cldr-common-*.zip, they both seem to contain the same), or at https://www.unicode.org/repos/cldr/trunk/, and extract the XML files under common/main/.
Gentoo also has a package called app-i18n/unicode-cldr btw.
Acquire the QT core source code, e.g. from your distribution’s repository, and unpack it.
Use the scripts in util/local_database/ (like cldr2qlocalexml.py and qlocalexml2cpp.py), with your language of choice’s XML file, to re-generate the static qlocale_data_p.h.
Then compile and install the package as normal.

And don’t forget to recompile all other packages that include qlocale_data_p.h.

Or, on Gentoo, make a lasting patch with:

ebuild $(equery w dev-qt/qtcore:5) prepare
pushd /var/tmp/portage/dev-qt/qtcore-5*/work/ # assuming default Portage build directory
mv qtbase-opensource-src-5.9.3 a
cp -a a b

NOW make the above changes in b, and not in a.

mkdir -p /etc/portage/patches/dev-qt/$(basename $(dirname $(pwd))) # only for this version
#mkdir -p /etc/portage/patches/dev-qt/qtcore # for all versions
diff -ur a b > /etc/portage/patches/dev-qt/qtcore*/my-locale.patch
rm -rf a b
popd
ebuild $(equery w dev-qt/qtcore:5) clean
# Rebuild all packages that depend on it, just to be sure. May be optional.
emerge -1 dev-qt/qtcore:5 $(equery d dev-qt/qtcore:5 | sed 's/^/=/')

Final thoughts

To be perfectly frank, I didn’t check if those Python scripts actually worked by just passing an XML file. As, at that point, I just stopped bothering, and wanted to kill it with fire. :)

If anyone wants to actually do it in practice, please report, and I will update this answer. (Or do it yourself, if you can)

Here’s how I got there:

I opened KDE’s time formats setting dialog, via right-click on the task bar clock.
Then, with htop I found the most recently started processes, which included the obvious kcmshell5 formats.
Using htop’s list open files feature on it, I filtered for “format”, and found /usr/lib64/qt5/plugins/kcm_formats.so.
With the help of Gentoo’s equery belongs /usr/lib64/qt5/plugins/kcm_formats.so I could identify the package kde-plasma/plasma-desktop.
After unpacking its source file, I quickly found kcms/formats/kcmformats.cpp, whose addLocaleToCombo used the QLocale, and included <QLocale> too.
This QLocale could then be found with locate -i QLocale to reside at /usr/include/qt[5]/QtCore/qlocale.h. It included some generated enumerations of the languages, scripts, etc.
Another equery belongs /usr/include/qt[5]/QtCore/qlocale.h pointing me to dev-qt/qtcore/qtcore, and unpacking the source file later…
… I already had a gut feeling, that it was all generated with some tool. So I looked into util/ and found util/local_database/README stating “local_database is used to generate qlocale data from the Common Locale Data Repository (The database for localized names (like date formats, country names etc)).”.
Having never heard of that, I looked it up, and it’s apparently by the Unicode Consortium. The code of cldr2qlocalexml.py was not very useful though, as it is called with the path to a directory containing the CLDR locales. Its result is used for qlocalexml2cpp.py though, which generates the file src/corelib/tools/qlocale_data_p.h, which includes huge hard-coded tables of all the locale data. And that file already existed in the source. So… yep, it’s (partially?) hard-coded.
But the CLDR XML files were nowhere to be found. So I assume it just uses the previously prepared qlocale_data_p.h. Which is not very in the spirit of open source. But at least you can do it yourself, as described above.

Linux – How to convert a date with a named month to a unix timestamp with only Busybox tools

The busybox date² is fully capable of parsing the date in the given string with some help¹ (except for the GMT time zone).

$ gdate='Mon Jan 1 23:59:59 2018 GMT'
$ TZ=GMT0 busybox date -d "$gdate" -D '%a %b %d %T %Y %Z'
Mon Jan  1 23:59:59 GMT 2018

The help is given with the -D option: a description of the source format.

To get a UNIX timestamp, just add the output format expected +'%s':

$  TZ=GMT0 busybox date -d "$gdate" -D '%a %b %d %T %Y %Z' +'%s'
1514851199

¹
The busybox date has most of the GNU date command's capabilities and one that the GNU date command doesn't: the -D option. Get the busybox help as follows:

$ busybox date --help

BusyBox v1.27.2 (Debian 1:1.27.2-2) multi-call binary.

Usage: date [OPTIONS] [+FMT] [TIME]

Display time (using +FMT), or set time

    [-s,--set] TIME Set time to TIME
    -u,--utc        Work in UTC (don't convert to local time)
    -R,--rfc-2822   Output RFC-2822 compliant date string
    -I[SPEC]        Output ISO-8601 compliant date string
                    SPEC='date' (default) for date only,
                    'hours', 'minutes', or 'seconds' for date and
                    time to the indicated precision
    -r,--reference FILE     Display last modification time of FILE
    -d,--date TIME  Display TIME, not 'now'
    -D FMT          Use FMT for -d TIME conversion

Note the -D FMT option.

²
Note that you may be able to call busybox date in two ways:

$ busybox date

Or, if a link to busybox with the name date has been installed in the correct PATH directory:

$ date

To verify, just ask for --version or --help to find out which date you have installed.

With GNU date:

$ date --version
date (GNU coreutils) 8.28

Or (busybox date):

$ date --help
BusyBox v1.27.2 (Debian 1:1.27.2-2) multi-call binary.
…
…