The "official" way to fix, per Apple, it is to email Apple's iTunes support (iTunesStoreSupport@apple.com), inform them of the problem, and then wait for them to fix it for you on their end. The good news about this solution is that my email was answered very quickly, the support person knew what the issue was and how to fix it, and within a couple days of telling them about the problem they had fixed it and I am up and working again.
After they told me it was fixed, it did take almost exactly 24 hours to take effect, and in the email they told me "wait about 24 hours and try purchasing an app again."
At the time I was having this problem (while traveling abroad) I was in no mood to "wait it out" as other answers have suggested, but since it was handled so capably and promptly by iTunes Support, I wouldn't suggest that as a course of action. It is a known issue for them.
After spending over an hour and a half to Apple UK's support, and eventually getting transferred to a senior advisor from their US-based support, it turns out that I may have uncovered a bug in iCloud that has since been forwarded to the iCloud Development Team.
The old email address had been used as my Apple ID for around 3 years before I changed it. Rather than add an additional one (as I was deleting the old email completely), I change the primary Apple ID; however I neglected to change it for iCloud (I wasn't actually using iCloud for anything, so I didn't think about it). As a consequence, iCloud is still associated with the old Apple ID email, and it wouldn't let me sign out of iCloud unless I signed in with the old Apple ID so I could disable Find my iPad so that I could sign out.
Now, here's the rub: for whatever reason, Apple have no record of the original Apple ID email. None at all. Normally, as @Tetsujin pointed out in his comment, I should be able to sign in to my Apple ID with my old email even though it's no longer associated with my account, because there should still be a record of it on the system. However, because (for whatever reason) it seems to have been completely purged from Apple's records, there is effectively no way for me to actually sign out of iCloud.
Apple have admitted that this never should have happened, and they have passed the details of the case on to the iCloud Development Team... so I may be indirectly responsible for another iOS9 update. Sorry.
In the end, my 'solution' was to abandon the original iPad and set my iPad Mini 4 up as new, manually transferring my apps and settings across, using the old one as a reference.
Best Answer
The real answer is because Apple Pay stores the card info locally on your phone/watch. That's why when you get a new device, you need to re-sync and re-setup all your cards.
In contrast, iCloud subscription is billed directly from Apple's servers with whatever payment method they have on file. They don't have access to the card in your Apple Pay which is stored securely on your individual devices.
In the article about Comcast posted by oa- in the other answer, the actual thing Comcast is allowing users to do is to open their app and make a bill payment from the phone using the Apple Pay card stored on that phone. It still doesn't allow reoccurring automatic payments, because it can't.