Subtract time using date and bash

dategnu

All the other questions on the SE network deal with scenarios where either the date is assumed to be now (Q) or where only a date is specified (Q).

What I want to do is supply a date and time, and then subtract a time from that.
Here is what I tried first:

date -d "2018-12-10 00:00:00 - 5 hours - 20 minutes - 5 seconds"

This results in 2018-12-10 06:39:55 – It added 7 hours. Then subtracted 20:05 minutes.

After reading the man and info page of date, I thought I have it fixed with this:

date -d "2018-12-10T00:00:00 - 5 hours - 20 minutes - 5 seconds"

But, same result. Where does it even get the 7 hours from?

I tried other dates as well because I thought maybe we had 7200 leap seconds on that day, who knows lol. But same results.

A few more examples:

$ date -d "2018-12-16T00:00:00 - 24 hours" +%Y-%m-%d_%H:%M:%S
2018-12-17_02:00:00

$ date -d "2019-01-19T05:00:00 - 2 hours - 5 minutes" +%Y-%m-%d_%H:%M:%S
2019-01-19_08:55:00

But here it becomes interesting. If I omit the time on input, it works fine:

$ date -d "2018-12-16 - 24 hours" +%Y-%m-%d_%H:%M:%S
2018-12-15_00:00:00

$ date -d "2019-01-19 - 2 hours - 5 minutes" +%Y-%m-%d_%H:%M:%S
2019-01-18_21:55:00

$ date --version
date (GNU coreutils) 8.30

What am I missing?

Update: I've added a Z at the end, and it changed the behaviour:

$ date -d "2019-01-19T05:00:00Z - 2 hours" +%Y-%m-%d_%H:%M:%S
2019-01-19_04:00:00

I'm still confused though. There is not much about this in the GNU info page about date.

I'm guessing this is a timezone issue, but quoting The Calendar Wiki on ISO 8601:

If no UTC relation information is given with a time representation,
the time is assumed to be in local time.

Which is what I want. My local time is set correctly too. I'm not sure why date would mess with the timezone at all in this simple case of me supplying a datetime and wanting to subtract something off of it. Shouldn't it subtract the hours from the date string first? Even if it does convert it to a date first and then does the subtraction, if I leave out any subtractions I get exactly what I want:

$ date -d "2019-01-19T05:00:00" +%Y-%m-%d_%H:%M:%S
2019-01-19_05:00:00

So IF this truly is a timezone issue, where does that madness come from?

Best Answer

That last example should have clarified things for you: timezones.

$ TZ=UTC date -d "2019-01-19T05:00:00Z - 2 hours" +%Y-%m-%d_%H:%M:%S
2019-01-19_03:00:00
$ TZ=Asia/Colombo date -d "2019-01-19T05:00:00Z - 2 hours" +%Y-%m-%d_%H:%M:%S 
2019-01-19_08:30:00

As the output clearly varies by the timezone, I'd suspect some non-obvious default taken for a time string without a timezone specified. Testing a couple of values, it seems to be UTC-05:00, though I'm not sure what that is.

$ TZ=UTC date -d "2019-01-19T05:00:00 - 2 hours" +%Y-%m-%d_%H:%M:%S%Z
2019-01-19_08:00:00UTC
$ TZ=UTC date -d "2019-01-19T05:00:00Z - 2 hours" +%Y-%m-%d_%H:%M:%S%Z
2019-01-19_03:00:00UTC
$ TZ=UTC date -d "2019-01-19T05:00:00" +%Y-%m-%d_%H:%M:%S%Z           
2019-01-19_05:00:00UTC

It's only used when performing date arithmetic.

It seems the issue here is that - 2 hours is not taken as arithmetic, but as a timezone specifier:

# TZ=UTC date -d "2019-01-19T05:00:00 - 2 hours" +%Y-%m-%d_%H:%M:%S%Z --debug
date: parsed datetime part: (Y-M-D) 2019-01-19 05:00:00 UTC-02
date: parsed relative part: +1 hour(s)
date: input timezone: parsed date/time string (-02)
date: using specified time as starting value: '05:00:00'
date: starting date/time: '(Y-M-D) 2019-01-19 05:00:00 TZ=-02'
date: '(Y-M-D) 2019-01-19 05:00:00 TZ=-02' = 1547881200 epoch-seconds
date: after time adjustment (+1 hours, +0 minutes, +0 seconds, +0 ns),
date:     new time = 1547884800 epoch-seconds
date: timezone: TZ="UTC" environment value
date: final: 1547884800.000000000 (epoch-seconds)
date: final: (Y-M-D) 2019-01-19 08:00:00 (UTC)
date: final: (Y-M-D) 2019-01-19 08:00:00 (UTC+00)
2019-01-19_08:00:00UTC

So, not only is no arithmetic being done, there seems to be a daylight savings 1 hour adjustment on the time, leading to a somewhat nonsensical time for us.

This also holds for addition:

# TZ=UTC date -d "2019-01-19T05:00:00 + 5:30 hours" +%Y-%m-%d_%H:%M:%S%Z --debug
date: parsed datetime part: (Y-M-D) 2019-01-19 05:00:00 UTC+05:30
date: parsed relative part: +1 hour(s)
date: input timezone: parsed date/time string (+05:30)
date: using specified time as starting value: '05:00:00'
date: starting date/time: '(Y-M-D) 2019-01-19 05:00:00 TZ=+05:30'
date: '(Y-M-D) 2019-01-19 05:00:00 TZ=+05:30' = 1547854200 epoch-seconds
date: after time adjustment (+1 hours, +0 minutes, +0 seconds, +0 ns),
date:     new time = 1547857800 epoch-seconds
date: timezone: TZ="UTC" environment value
date: final: 1547857800.000000000 (epoch-seconds)
date: final: (Y-M-D) 2019-01-19 00:30:00 (UTC)
date: final: (Y-M-D) 2019-01-19 00:30:00 (UTC+00)
2019-01-19_00:30:00UTC

Debugging a bit more, the parsing seems to be: 2019-01-19T05:00:00 - 2 (-2 being the timezone), and hours (= 1 hour), with an implied addition. It becomes easier to see if you use minutes instead:

# TZ=UTC date -d "2019-01-19T05:00:00 - 2 minutes" +%Y-%m-%d_%H:%M:%S%Z --debug
date: parsed datetime part: (Y-M-D) 2019-01-19 05:00:00 UTC-02
date: parsed relative part: +1 minutes
date: input timezone: parsed date/time string (-02)
date: using specified time as starting value: '05:00:00'
date: starting date/time: '(Y-M-D) 2019-01-19 05:00:00 TZ=-02'
date: '(Y-M-D) 2019-01-19 05:00:00 TZ=-02' = 1547881200 epoch-seconds
date: after time adjustment (+0 hours, +1 minutes, +0 seconds, +0 ns),
date:     new time = 1547881260 epoch-seconds
date: timezone: TZ="UTC" environment value
date: final: 1547881260.000000000 (epoch-seconds)
date: final: (Y-M-D) 2019-01-19 07:01:00 (UTC)
date: final: (Y-M-D) 2019-01-19 07:01:00 (UTC+00)
2019-01-19_07:01:00UTC

So, well, date arithmetic is being done, just not the one that we asked for. ¯\(ツ)/¯

Related Question