UPDATE AUGUST 2018: Microsoft keeps changing things a bit, but you can still "Disable" the Edge browser by doing the following:

1. Hold down the Ctrl and Alt keys and tap the delete key, then click on Task Manager.
2. If it says "More details" at the bottom of the Task Manager window, click on it to show more details.
3. Scroll down the list and look for "Microsoft Edge". If you find it in the list, right click on it and select "End task".
4. Now, open File Explorer and find the folder "C:\Windows\SystemApps\Microsoft.MicrosoftEdge_8wekyb3d8bbwe" and rename that folder to anything you wish. (I normally just append some text to it like "_remove" so I can easily re-enable Edge in the future if I desire.)

Edge should now be "disabled".

Keep in mind that Microsoft may try to re-enable Edge during a future update...

THE ANSWERS BELOW ARE OLDER ANSWERS...

UPDATE: In January 2017 I disabled Edge on a new Windows 10 install by simply renaming the "C:\Windows\SystemApps\Microsoft.MicrosoftEdge_8wekyb3d8bbwe" folder. (If you are asked to allow administrator privileges be sure to answer in the affirmative.) Renaming this folder was not possible in the past without following the more detailed steps below. This much simpler approach achieves the exact same results as the more complex instructions below! However, for older installations and/or Windows upgrades the below instructions may still be the only answer.

THE ANSWER BELOW THIS LINE IS THE OLDER ANSWER...

As already stated, the other answers are no longer valid due to changes in Windows 10 -- Microsoft has removed all supported methods for disabling Edge. As of today, this answer still works. Although parts of this answer were given, not all of it was given together in a step by step manner.

• Go to: C:\Windows\SystemApps\
• Open folder Microsoft.MicrosoftEdge_8wekyb3d8bbwe
• Right click on MicrosoftEdge file then Properties / Security / Advanced / Click Change (next to owner) and change to your username, then click OK
• Then Click Edit, then add your username and give full permissions
• Rename MicrosoftEdge file to MicrosoftEdge_remove or a name of your choice
• Repeat all above steps for file MicrosoftEdgeCP also
• Done! Edge will not run!

Been running Windows 10 Pro like this on 100+ PCs for several months without issues.

A NOTE REGARDING WINDOWS ANNIVERSARY UPDATE: The Anniversary update appears to undo the above changes and re-enable Edge. In my case I just ran the above steps again, and Edge is once again disabled.