Ubuntu – way to make ntpd -gq run faster

This question is related to this question and this earlier question I asked on serverfault.

Per my previous question, I'm interested in synchronizing the time on my machine in a fast (ideally less than a second), synchronous (if the process completes with 0 status, the time should be synced), and robust way ("robust" here meaning that as long we receive a response from the ntp server, the time should be set according to the response). I will run ntp in the background after this fast initial sync.

Using ntpdate seems to be too flaky, and isn't recommended anyway (I get sporadic "no server available errors", I think mostly stemming from discrepancies with the ntp server, not from a failure to reach the ntp server). However, one good thing about ntpdate is that it has an option -p that lets me shorten the time it takes to set my time to mere milliseconds (when it works).

Using sudo ntpd -gq, as suggested in the previous questions seems to make clock syncing more reliable, but it takes at least 7-8 seconds on my machine, presumably because ntpd is taking several samples with intervening sleep time. Worse, ntpd often hangs for minutes before exiting successfully. I'm not sure why this happens because the ntp server is reachable and responding to requests all throughout this timespan.

Thus, I'm looking for a way hopefully to speed up the time it takes to synchronize a machines clock using ntpd -gq, and I'm willing to sacrifice a few milliseconds of initial clock accuracy in order to have faster clock synchronization. The reason I'm willing to make this sacrifice is that synchronization is part of a bootstrap process that I'm running many times per day and waiting 8 seconds to several minutes for ntp to run each time is slowing the process down (to the point that I have no choice but to accept ntpdate's fast but deprecated and unreliable behavior).

Is there a way to achieve reliable fast performance with ntpd?

EDIT: Here is my ntp.conf (if you're wondering about the location of server, it was programmatically appended to the file after stripping out all existing servers):

# /etc/ntp.conf, configuration for ntpd; see ntp.conf(5) for help

driftfile /var/lib/ntp/ntp.drift


# Enable this if you want statistics to be logged.
#statsdir /var/log/ntpstats/

statistics loopstats peerstats clockstats
filegen loopstats file loopstats type day enable
filegen peerstats file peerstats type day enable
filegen clockstats file clockstats type day enable

# Specify one or more NTP servers.

# Use servers from the NTP Pool Project. Approved by Ubuntu Technical Board
# on 2011-02-08 (LP: #104525). See http://www.pool.ntp.org/join.html for
# more information.

# Use Ubuntu's ntp server as a fallback.

# Access control configuration; see /usr/share/doc/ntp-doc/html/accopt.html for
# details.  The web page <http://support.ntp.org/bin/view/Support/AccessRestrictions>
# might also be helpful.
#
# Note that "restrict" applies to both servers and clients, so a configuration
# that might be intended to block requests from certain clients could also end
# up blocking replies from your own upstream servers.

# By default, exchange time with everybody, but don't allow configuration.
restrict -4 default kod notrap nomodify nopeer noquery
restrict -6 default kod notrap nomodify nopeer noquery

# Local users may interrogate the ntp server more closely.
restrict 127.0.0.1
restrict ::1

# Clients from this (example!) subnet have unlimited access, but only if
# cryptographically authenticated.
#restrict 192.168.123.0 mask 255.255.255.0 notrust


# If you want to provide time to your local subnet, change the next line.
# (Again, the address is an example only.)
#broadcast 192.168.123.255

# If you want to listen to time broadcasts on your local subnet, de-comment the
# next lines.  Please do this only if you trust everybody on the network!
#disable auth
#broadcastclient
server pool.ntp.org