If you look in the iPhone user manual - in the section on Voice Memos (Chapter 28, page 105) - it walks you through the intended process for syncing with your computer.
http://manuals.info.apple.com/en_US/iphone_user_guide.pdf
Sharing voice memos with your computer
You can sync voice memos with
the primary iTunes library on your computer, then listen to memos on
your computer or sync them with another iPhone or iPod touch.
When you
delete a synced memo from iTunes, it stays on the device where it was
recorded, but is deleted from any other iPhone or iPod touch you
synced. If you delete a synced memo on iPhone, it's copied back to
iPhone the next time you sync with iTunes, but you can’t sync that
copy back to iTunes a second time.
Sync voice memos with iTunes:
Connect iPhone to your computer, then in iTunes select iPhone. Select
Music at the top of the screen (between Apps and Movies), select Sync
Music, select “Include voice memos,” and click Apply.
Voice memos
synced from iPhone to your computer appear in the Music list and in
the Voice Memos playlist in iTunes. Memos synced from your computer
appear in the Voice Memos app on iPhone, but not in the Music app
iTunes should actually sync Voice Memos if you have the option 'Sync Voice Memos' checked in the 'Music' tab when the iOS device is connected (either through USB or Wifi).
I actually forgot I had this item checked in iTunes, and noticed I had a new playlist named 'Voice Memos' in iTunes. These were rather short (only a minute to 5 long I think), but synced ok.
If you delete a Voice Memo from the iTunes Library, it will not be deleted from the device on which it was recorded, as stated by the Manual, the next time that device is synced. In fact when you re-sync that device again, it is possible to have that Memo (which had been deleted from the iTunes Library on the computer) be copied back again into the iTunes Library on the computer, by checking the Include Voice Memos on the Device/Settings/Music page (as of iTunes 12.11.0.26).
Additionally, if the iTunes Library on the computer is empty at the time you connect a device on which some memos have been recorded, these memos will not be erased on the device when you sync; on the contrary, they will be copied back to the iTunes Library on the computer, no matter if Include Voice Memos is checked or not on the Device/Settings/Music page. Do not be deterred by the warning message in iTunes :
This iTunes library is empty. If you link the iPhone [device name] to
this library, all songs on the iPhone will be erased. Are you sure you
want to continue?
as this will not apply to the Voice Memos which were recorded on the device (as of iTunes 12.11.0.26).
You can download a trial of ecamm's PhoneView from their site -- and perhaps buy it, I've found it very handy. This will allow you to see everything on your phone, copy photos back and forth, and with Admin mode, get into the works.
You should be able to easily delete Voice Memos, those show up on the opening menu.
Best Answer
Best way to do is, is using an cloudsharing app, like Instashare (There are a bunch of them, just search the App store and you'll find a lot of them).
Concidering you'll use Instashare, download the app and set it up. Open your voice memo app and click on the share button. Your familiar share screen will popup. If it isn't listed yet, click on the more button next to Messages & Mail. Enable instashare or any app you want to use, close it by clicking done. Then click yours app its icon. Depending on the app, you have to choose where to place it, but after that it's stored for you. Just now download on you're pc and you're done.
For this method you'll need IOS 8, which is supported for the 4S. Otherwise it'll become really complicated to share it, then you'd better use mail or USB.