Win10: VS Code integrated bash not loading ~/.bash_profile

On a modern system it is not especially common to run into the cases where it matters, but it does happen. (In particular, if you use shell operations in vim such as :r !command or the in-line !<motion>command form.)

What would you put under ~/.bashrc? only aliases?

You put things in ~/.bashrc that would not be inherited by subshells automatically; this means aliases and functions, mostly, although sometimes you have variable settings that you don't want visible outside the shell (this is very rare). It could be argued that those should be exported somehow, but various experimental attempts have run into compatibility issues with trying to hide them within the environment and have mostly been abandoned.

If I want to set an environment variable and add it to the PATH (for example JAVA_HOME) where it would be the best place to put the export entry? in ~/.bash_profile or ~/.bashrc?

You put environment settings in ~/.bash_profile so that they are given sane initial settings. Sometimes you will want to override these (often this is done by complex environments such as Matlab or Cadence); if you put the environment settings in ~/.bashrc then shells run from within those environments will lose the environments' customizations, and things may not work properly as a result. This also applies if you use a package like modules, virtualenv, rvm, etc. to manage multiple development environments; putting your settings in ~/.bashrc means you can't run the environment you want from within your editor, but instead will be forced into the system default.

In a non-login shell, I believe the ~/.bash_profile is not being "picked up".

This is correct; you normally want the initial shell to be a login shell and any shells started under that one to not be login shells. If the initial shell is not a login shell, you won't have a default PATH or various other settings (including your JAVA_HOME example).

Most desktop environments launched from display managers (which is to say, the vast majority of graphical logins) do not set up a login environment for the entire desktop, so you are forced to run the initial shell in terminals as a login shell. This causes a number of problems (notably that the PATH and such available to programs run from e.g. panels is not set up properly, because the panel is not a terminal and has not run ~/.bash_profile), but is a reasonable compromise given that it is not always possible to sanely run ~/.bash_profile in the non-interactive environment at the beginning of a session started by a display manager, depending on its contents. It is sometimes suggested to place environment settings in ~/.bashrc instead of configuring a login shell instead; as discussed above, this works as long as you do not need to override that environment, and causes odd breakages once you do need to do so.

I recently helped diagnose an issue like this on OS X where a user who had placed settings in ~/.bashrc then later started using rvm and perlbrew saw odd behavior, because the environments set up by the two were "undone" by ~/.bashrc inside editors and sudo (which on OS X, unlike Linux, propagates the user's $HOME so that their ~/.bashrc was run by the root shell). Before trying to use those environments, there was no problem; on starting to use them, they were bewildered by the unexpected loss of their settings.

Just putting all my comments together for an answer:

First thing you should do is change the shell, this way you can set a shell that will not load the bash init-scripts (.bashrc, .bash_profile) - how to do this for the Mac OS X terminal app can be seen here: Apple Support

Now you should be able to open a terminal again and use your favourite command-line editor to open the .bash_profile file (e.g. nano or vi):

In this file you have to reset your PATH variable that is used by the terminal to find the programs it can execute.

This can be done with the following two lines:

PATH="/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:$PATH" # Make sure to use double quotes not single quotes And on a new line 
export PATH

This will first set the directories that you want to have on your PATH and the export this PATH to make it available to all programs started from this shell (via export).