Ubuntu – How to Fix apt-get Update ‘Hash Sum Mismatch’

aptUbuntu

I have an Ubuntu 12.04 virtual box vm that I instantiate using Vagrant. 

git clone https://github.com/spuder/puppet-gitlab
vagrant up

As soon as the vagrant box runs apt-get update, I get the following error. 

...
W: Failed to fetch gzip:/var/lib/apt/lists/partial/apt.puppetlabs.com_dists_precise_main_binary-amd64_Packages  Hash Sum mismatch
W: Failed to fetch gzip:/var/lib/apt/lists/partial/apt.puppetlabs.com_dists_precise_main_binary-i386_Packages  Hash Sum mismatch
W: Failed to fetch gzip:/var/lib/apt/lists/partial/apt.puppetlabs.com_dists_precise_dependencies_binary-i386_Packages  Hash Sum mismatch
W: Failed to fetch http://br.archive.ubuntu.com/ubuntu/dists/precise-updates/restricted/binary-i386/Packages  404  Not Found

Things I've tried to work around this error.

  • Used 3 different ubuntu 12.04 boxes from 'http://www.vagrantbox.es'

  • solution suggested here:

    sudo rm -rf /var/lib/apt/lists/*
    sudo apt-get update
    sudo apt-get clean

  • Removed and readded the puppet labs packages

I've also tried similar suggestions that I've found in the first few pages of google. I've even tried multiple computers, and multiple internet connections.

The fact that this has affected multiple ubuntu vm's on multiple internet connections makes me think there is something wrong with the ubuntu repo.

How else can I try to fix this issue?

Update

I tried cleaning out '/var/lib/apt/lists/partial' and running apt-get clean then replaced the sources in /etc/sources/list by using the amazon mirrors suggested here:

I still get a similar error

Fetched 18.9 MB in 10s (1,865 kB/s)                                                                                                                                                                            
W: Failed to fetch bzip2:/var/lib/apt/lists/partial/us-west-1.ec2.archive.ubuntu.com_ubuntu_dists_precise_main_binary-amd64_Packages  Hash Sum mismatch
W: Failed to fetch bzip2:/var/lib/apt/lists/partial/us-west-1.ec2.archive.ubuntu.com_ubuntu_dists_precise_universe_binary-amd64_Packages  Hash Sum mismatch
W: Failed to fetch bzip2:/var/lib/apt/lists/partial/us-west-1.ec2.archive.ubuntu.com_ubuntu_dists_precise_multiverse_binary-amd64_Packages  Hash Sum mismatch
W: Failed to fetch bzip2:/var/lib/apt/lists/partial/us-west-1.ec2.archive.ubuntu.com_ubuntu_dists_precise_main_binary-i386_Packages  Hash Sum mismatch
E: Some index files failed to download. They have been ignored, or old ones used instead.

Update2

I have 2 internet connections at home, both of them gave the same error. As soon as I took my laptop to my work internet connection, the problem went away.

I'm still curious to understand why my internet connection would make any difference.

Update3

See my answer below for an explanation. My internet filter was corrupting the download.

I'll rephrase the question since there are no answers yet.

Is there a way to override Hash Sum mismatches in apt-get?

Best Answer

First, you should understand why hash sum mismatch errors occur. In general, there are 2 reasons:

Firstly, Some apt repositories use LZMA (.xz) compressed metadata. apt before version 1.0 fails to decompressed LZMA archives correctly (sometimes) giving the hash sum mismatch error.

There's two work arounds for this:

  1. Tell your apt client not to use XZ compressed metadata
  2. Upgrade apt on your system to a version newer than 1.0

Check out this blog post I wrote about this issue which explains both work arounds in greater detail.

Secondly, APT repositories are inherently racy. The actual APT metadata is buggy and the design of it makes it impossible for apt clients to download the repository metadata in a consistent way if the apt-get update happens while the repository is being updated.

There's two work arounds for this:

  1. Upgrade to a newer version of APT and ensure that the repository you create (or want to use) supports the Acquire-by-hash feature. This fixes the issue at its core, but not be possible in some cases if you don't control the repository.
  2. You can delete the cached metadata on your system and try again. To do this, first run apt-get clean followed by rm -rf /var/lib/apt/lists/*. Next, run apt-get update. This will re-download all the metadata. If the repository you are trying to connect to doesn't update itself while you are running apt-get update, you will be fine. Otherwise, you'll have to do this again.
Related Question