Resolution – macOS Resolution Can’t Scale Up in VMware

I had exactly the same problem, with the same monitor, Dell U2713 / U2713H / U2713HM / U2713HMt. I had two of them plugged in to my late-2014 Retina Macbook Pro 15" and only one was experiencing the problem. I tried 100s of different solutions (I can say for sure it's definitely not the cable!) and nothing worked. Just like you, I could get landscape 2560x1440 but not portrait. And I was getting the same error WindowServer: description_for_mode_number: IOFBGetPixelInformation failed (-536870206) when I tried to switch to portrait. That is, until I came across this: https://discussions.apple.com/thread/5485930?start=0&tstart=0. I regenerated the EDID override for the screen using the Ruby script, copied it into the correct location, rebooted and now it works again. I think I caused the problem by playing a game (Trine 2) in vertical (720x1280) resolution - after that, I tried everything to get the screen back to displaying portrait 1440x2560 but to no avail. Until I got the Ruby script from the thread above which fixed it.

Just to note, in case the linked thread disappears... the override file is in /System/Library/Displays/Overrides/ under the folder DisplayVendorID-10ac which is for Dell displays. The file for the U2713-series of monitors (and I think the U2711-series too) is DisplayProductID-4080. The contents of the file should be:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>DisplayProductName</key>
  <string>Dell U2713HMt</string>
  <key>IODisplayEDID</key>
  <data>AP///////wAQrIBATDA1NB4WAQSlPCJ4Iku1p1ZLoyUKUFSlSwCBALMA0QBx
T6lAgYABAQEBVl4AoKCgKVAwIDUAVVAhAAAaAAAA/wBHSzBLRDI3TjQ1MEwK
AAAA/ABERUxMIFUyNzEzSE0KAAAA/QAxVh1xHgEKICAgICAgAA0=
</data>
  <key>DisplayVendorID</key>
  <integer>4268</integer>
  <key>DisplayProductID</key>
  <integer>16512</integer>
</dict>
</plist>

The minimum resolution supported is 640x480. To get to this resolution (at least on my monitors on Sierra), when I'm in display preferences, I have to Control + click on "Scaled". This gives every resolution option, not just the main ones Apple thinks I want to use.