Command Line Utility + System Service

This is a two-part solution: first a means of changing the keyboard layout, and then a means to activate that by a keyboard shortcut. But it's fairly easily accomplished.

1. Changing the Layout

To change the layout programmatically, there's a handy (and free) little utility called changeInput, which you can download here.

Download and unzip it, and place the changeInput executable somewhere. Be sure to note the path for the second section.

2. Creating Keyboard Shortcuts

There several ways to do this, but to make things accessible for anyone, we'll do it with built-in tools by making a System Service with Automator.

1. Open Automator, and select Service as the document type.
2. Change the drop-down menus in the upper right so that it reads Service recieves no input in any application.
3. Add the Run Shell Script action to the workflow by double-clicking it or dragging it to the right pane.
4. In the text field, enter /path/to/changeInput "Name of Layout Here". Be sure to enter the exact path to changeInput as you saved it (tip: you can drag the executable right into the text field to get the proper path). Replace "Name of Layout Here" with the exact text that's displayed in the Input Sources section of System Preferences or the Input Menu — be sure to keep the quotes.
5. Save your service. It will prompt you for a name, make sure to note it for step 7 (something descriptive is best). The service file gets saved to ~/Library/Services/.
6. Open the Keyboard panel of System Preferences. Click the Keyboard Shortcuts tab, and select the Services item in the left pane.
7. Scroll through the list until you find the name of the service as you saved it in step 5 (it should be in the General section, near the bottom. Make sure the box is checked, then select it and click add shortcut, and set whatever shortcut you desire.
8. Repeat steps 1-7 for each layout you want a shortcut for.
9. Test out your shortcuts. You may need to open the services menu once (click the application menu for the current app and reveal the Services menu) before the shortcuts get activated.

As an alternative to using the Automator/Services combo to create the keyboard shortcuts, you can use FastScripts, the free version of which allows you to quickly set keyboard shortcuts for 10 scripts, or $14.95 for an unlimited number. Keyboard Maestro can also easily run a script with a keyboard shortcut, but it's a bit overkill if this is your only need (it is worth checking out for lots of other scripty/powerusery type things).

Hold the shift key then the number corresponding. It works just fine. I think you are forgetting to let go of the shift before hitting the number. Hence the unexpected output.

Hold shift+U then, after popup let go of the shift key and hit the corresponding number.
Ü ← result