That's correct: PS1 is only meaningful in interactive instances of bash, so it should be set in ~/.bashrc and should not be exported. PS1 is also meaningful in other shells, but it has a different meaning, because prompt expansions differ between shells. In fact, even between instances of bash, PS1 can have different meanings, since the meaning depends on shell options (at least promptvars).
Exporting PS1 to the environment from .profile is a throwback to the 1970s, when there was only one shell that used it (the Bourne shell) and it didn't have a configuration file. It still works today if you always use the same shell and never configure it differently. But all modern shells that aren't designed purely for scripting (csh, ksh, bash, zsh, …) read a configuration file when started interactively (.cshrc, .kshrc, .bashrc, .zshrc, …), so the 1970s method is no longer necessary. Setting PS1 and other shell-specific settings in a shell-specific file, and not exporting it to the environment, avoids breaking things when you use a different shell configuration or a different shell or a different terminal that isn't capable of showing your usual prompt fanciness. Setting PS1 in a shell-specific file works all the time, whereas setting it in .profile and exporting it only works in “simple” cases, so there's no reason not to do it the right way, but there are plenty of bad tutorials around the web and even bad default configurations in distributions. C'est la vie.
Best Answer
PS1 stands for "Prompt String One" or "Prompt Statement One", the first prompt string (that you see at a command line).
Yes, there is a PS2 and more! Please read this article and the Arch wiki and of course The Bash Reference Manual.