This answer solved my problem. While I initially thought that that question dealt with an unrelated issue, it turns out that it actually is the same issue.

To summarize here, the solution is to choose the desired layout on the login screen. Then, the old layout will stop showing up and the keyboard layout indicator will be gone for good.

1. Click the Settings button in the upper-right corner of the screen
2. Select "System Settings".
3. Click "Text Entry". It may also be labeled "Language & Text" or "Keyboard Layout"
4. Click the "+" button beneath the list of installed keyboard layouts.
5. Choose the layout that you want to add to Ubuntu.
6. Click [Add] to add it to your list of layouts.
7. Note the keyboard shortcuts. The "Switch to next source" and "Switch to previous source" shortcuts allow you to quickly cycle through your enabled layouts. You can select each one and set a new shortcut if you'd like.
8. Change your layout. After enabling the keyboard shortcuts, you have two ways to change the keyboard layout: Click the Language button in the Ubuntu menu bar and select the layout you want from the list. OR Press the shortcut you assigned to cycle through layouts.

Source: http://www.wikihow.com/Change-Keyboard-Layout-in-Ubuntu