0

I have a Debian Squeeze VM that has suddenly chosen to jump it's date forwards just over a month, which seems to confuse it no end and cause it to require a hard reset (yikes!).

There is nothing unusual in the logs, except that the datestamp suddenly jumps (today back to 2005).

It's happened three times, so I don't think it's to do with the leap second issue as the last one of those was in July.

When it happened once I spent ages checking stuff but could not find anything, decided to forget it. But three times is becoming an issue on a production server.

Edits providing information requested in comments (thanks!):

I do not have control over the hypervisor, it is a hired VM.

# cat /sys/devices/system/clocksource/clocksource0/current_clocksource
kvm-clock

# ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
+grendel.exizten 130.149.17.8     2 u   29   64   77   14.811    1.778   1.744
*panoramix.linoc 193.67.79.202    2 u   32   64   77   19.729   -0.419   1.691
+robert.elnounch 213.251.128.249  2 u   27   64   77   17.762    0.600   1.722
-janetzki.eu     83.169.43.165    3 u   31   64   77   27.214    3.575   1.638
Community
  • 1
artfulrobot
  • 3,119
  • Please tell more about the virtualization technology you are using and what clocksource you are using (cat /sys/devices/system/clocksource/clocksource0/current_clocksource). – gertvdijk Nov 23 '12 at 09:42
  • Thanks @gertvdijk, output of that is kvm-clock – artfulrobot Nov 23 '12 at 09:46
  • Is a Vmware VM ? – Sacx Nov 23 '12 at 09:48
  • Do you have control over the hypervisor? Are you running NTPd in the machine? If so, which servers is it listening to (ntpq -p)? @Sacx no, it probably is KVM regarding the clocksource. – gertvdijk Nov 23 '12 at 09:51
  • Does dmesg show something about changing the clock? Is there a difference with date and hwclock? – ott-- Nov 23 '12 at 10:03
  • 1
    I think you should contact your hosting provider. Maybe a live migration is causing this or it is hitting a bug in KVM somewhere. – gertvdijk Nov 23 '12 at 10:04
  • @ott-- nothing in /var/log/messages. # hwclock -r;date gives Fri 23 Nov 2012 10:38:17 GMT -0.997829 seconds followed by Fri Nov 23 10:38:18 GMT 2012 But note that the machine is up again now - can't test this when it's down for obvious reasons. – artfulrobot Nov 23 '12 at 10:39
  • @gertvdijk have done, will report back if they find anything (they couldn't the first time) – artfulrobot Nov 23 '12 at 10:40
  • It's definitely the host. If they change the date on the host, it passes through to the VM and causes your clock to go nuts. Since they should be running ntp on the host, this is inexcusable. Find a better host. – Michael Hampton Feb 28 '13 at 08:55

1 Answers1

1

I've seen several situations where NTP really doesn't like running in a VM. One provider advised me to stop running ntpd, and instead have something like this in /etc/cron.d/ntpdate:

* * * * * root /usr/bin/ntpdate your.ntp.server another.ntp.server > /var/log/ntpdate.log 2>&1

This make the clock on my VMs much more stable, and stopped ntpd from doing the sync-unsync-sync dance.

If you do this, remember to put something appropriate in /etc/logrotate.d to stop the logfile growing indefinitely.

Flup
  • 8,118