Should I Denormalize Tables To Avoid Joins

I'm in the process of developing a web application, and need some help with some basic db design. Currently, I am unsure of the following three tables:

### User ###
 - Id
 - DetailId
 - etc... (About 6 more columns)

### Contact ###
 - Id
 - UserId
 - DetailId
 - etc... (About 6 more columns)

### Detail ###
 - Id
 - FirstName
 - LastName
 - etc... (About 10 more columns)

I am starting to wonder if it may be best to just eliminate the Details table and simply denormalize those columns back into the User and Contact tables. Being that these tables will be read much more than they will be edited or updated, this denormalization should improve read performance by avoiding a table join. Is this a sound judgement or am I just falling for the root of all evil (Premature Optimization). Either way, which would be considered a better practice?

Edit/Update

Thanks for the great feedback! I realize I should have been a bit more descriptive, so here is some additional information on the use case for these tables.

• The Details table contains mainly display information, like name, address, phone number, email, but this is NOT account information. Instead it is more like print information, that will be used when the user prints quotes, invoices, etc…

• Details table contains only columns that are duplicated in both a user and contact in terms of printing data. So if I denormalize those columns back to the User and Contact table, any edits in column definition (or adding/removing columns) will have to be made twice.

• The Users table may only have 100 records, but the Contacts and Details tables could potentially hold hundred of thousands, because each user can have hundreds of contacts.
  But to be clear, a Detail "should" only belong to one Contact or User. However, a User can have many Contacts. So If there are 100 Users and each User has 100 Contacts, that's 10,000 Contacts, and 10,100 Details.