Problem with iCal invitations

I have noticed a problem with iCal invitations.

When I recieve an invitation (via email) it pops up in my calendar. I then click it to say accept. When I do my calendar shows that I have accepted, but the person who sent me the invitation does not see my status.

In addition I get an error Mail with the following error.

<com.apple.calendarserver+2594a782-bbe9-4385-8ea7-239172296549@MYSERVERNAME.com>:
XX.XX.XX.XX does not like recipient.
Remote host said: 554 5.7.1 <com.apple.calendarserver+2594a782-bbe9-4385-8ea7-239172296549@MYSERVERNAME.com>: Relay access denied
Giving up on XX.XX.XX.XX.

Both sender and recipient are using OSX Lion Server as Mail and Calendar servers and the standard calendar/mail app. With static IP-Adresses. Does iCal need a special port to be forwarded? Or do I need to set some flag in postfix's configuration so that iCal mails are relayed?

The servers relay outgoing mail through a separate host which requires authentication. Could it be that iCal does not use the authentication when it tries to relay the message?

Thanks