I used Active Sync since I got my very first iPhone, and switched to CardDAV the day Google announced support for it. I can say without a doubt CardDAV is the way to go
With CardDAV, you will get you much more accurate synchronization than with Active Sync. First, as has been mentioned, you can synchronize more than 3 e-mail addresses. In fact, I have 31 e-mail addresses myself, and they finally all sync up with my iPhone. Even more importantly, two-way sync works now. If you make an edit on your iPhone, it won't clobber the field in your Google Contacts. For instance, editing e-mail addresses on your iOS device won't change their labels in Google Contacts to Other instead of Home, Work, and whatever custom fields you might have had.
In particular, custom fields are preserved in both directions. This is especially important if you want to use Siri to "Send an e-mail to my brother". The Brother field in Google Contacts was not correctly imported into iOS through Active Sync, and if you allowed Siri to save who your brother was in your contacts, the import back into Google Contacts mangled everything up, including all other fields that you did not edit.
Finally, phone numbers also all sync in both directions without trouble using CardDAV.
To me, two-way editing (or the lack thereof) was the biggest problem with Active Sync. I never, ever edited any contacts on my iPhone using Active Sync because I didn't want to have to fix all the fields on a desktop later on. Now I can do it freely and haven't noticed any data corruption.
The main gripe people have concerning CardDAV is the lack of push. However, for me Active Sync push was so slow anyway that the slight delay of Fetch barely makes a difference. Plus, my experience with the reliability of Push has been less than stellar (connections that die and are never re-established, primarily). I've found the polling model to be far more robust.
What you may want to avoid while using CardDAV is updating a contact in Gmail or on your iPhone, then immediately updating the same contact in the other. This will likely cause one of your modifications to get overwritten by the modification made in the other direction.
Update 1: The one oddity I've found in two-way syncing relates to the Google Voice phone number type. For some reason it doesn't sync correctly in either direction (it becomes "Phone" on your iOS device when entered in Google Contacts, and Other in Google Contacts when entered from an iOS device). My only guess can be that the field has a special meaning to Google.
Update 2: There is another strangeness with the iPhone phone number type. It will make it to your iOS devices correctly if entered as such from Google Contacts. However, if you specify iPhone as a phone number type from an iOS device, Google Contacts will change that to Mobile. The iOS device that originally specified iPhone will retain it as its phone number type until that contact is updated again from either Google Contacts (without first changing the phone number type to iPhone) or from another iOS device. I recommend only entering iPhone as a phone number type from Google Contacts. Contacts with that phone number type should also only be edited from Google Contacts if you care about listing iPhone as their mobile phone number.
Best Answer
These additional contacts may be included in the SIM card you got from your provider. To easily find them, turn off Google Contacts in Settings->Password&Accounts->"your Google account" and select "delete synced contacts from iPhone" (don't worry, they still we be stored on Google itself). Then open the Contacts app, you now should see only these three contacts.