Your trip the Genius bar was too focused in my opinion. I suggest you make another appointment and lay out your case above, but this time, don't mention the memory tests, but that the problems showed up when you upgraded the RAM. Refer them to your old case number (tack this visit on as a follow-up).
Tell the Genius on hand that the problems still exist and that they need to do further testing. As it stands, your Mac is unusable. Additionally, tell them to make note that you have tried a fresh re-installation and it did nothing to help (don't mention that things ran fine for a day or too, stick to the current state of the system), which will rule out any software problems.
Ask them on what they suggest and follow it (no matter how annoying it may be). If it doesn't fix it, call them and do another follow-up. If they tested the RAM and it passes, then it might be the logicboard.
In any case, do not visit them with "the solution." Let them find it through there own testing. Clearly, the logs containing kernel panics and all the problems you list will support your case. Moreover, their internal testing tools are much more sophisticated than what you'll scrounge up on the net through 3rd party apps (they actually have in-house diagnostic utilities for these occasions).
I would highly recommend that you stop spending your time diagnosing the problem and let them do it for you. That's what they're there for.
If after everything is said and done, and they still can't pinpoint a problem, you will likely walk away with a new Mac. Just focus on the problems you are suffering from and leave the diagnostics to the Geniuses and technicians.
You may want to downgrade back to the 4 GB for now. And it's of note that replacing the RAM and the hard drive yourself do not void your warranty for that particular Mac model.
PS: You don't specify if you took your notebook to an Apple Store and spoke with a Genius or some Apple authorized repair centre. I would strongly encourage you to deal with Apple directly. They'll not only treat you better, but likely solve the problem with less headache. My advice is tailored to them; you are totally on your own should you choose a 3rd party repair shop. To galvanize the need to deal with Apple, let me put forth the following. I had a friend who purchased a Mac mini. There were strange problems during boot up where it would occasionally load the keyboard set-up assistant. The machine was also sluggish and exhibited other unusual anomalies in its day to day. We took it into an Apple Store and the Genius tried to reproduce the errors. He couldn't. He also expressly stated that minis never required repair or servicing. But with all that, he elected to replace the entire unit and gave my friend a new mini. Took it back to his place and it hummed, no more issues. I doubt he would have received the same treatment from a computer store authorized to service Macs. Deal with Apple direct; I cannot stress that enough.
Not sure if this is too late of an answer. I know you've already attempted to disable the password, but try these steps exactly:
You can remove the firmware password by changing the RAM configuration, booting up to the startup menu, shutting down and changing it back. Basically, if you have two 1GB chips of RAM in there now, you can remove one of the chips, start up holding alt/option, shut down and put the other chip back in. This will remove the firmware password.
Now, you can follow these instructions for doing the erase and install.
Good luck!
Best Answer
Did you migrate an account from an old machine? It doesn't look like a hardware problem, but it could be corrupt preferences. Bad video cards will corrupt the whole screen, not have it confined to a single window like your screen shot shows.
Try moving the preferences for Messages.app from your ~/Library/Preferences folder to the desktop. You will have to re-configure messages, but you can always put your old prefs file back. If that works then you probably have a corrupt Messages prefs file. Try that for each app to see if it helps.