Several weeks ago an issue was brought to my attention where a user had stopped receiving mail and those sending to this user were reporting delivery failures. The delivery failures stated that mail could not be delivered to ” AT @DOMAIN”. There is a space before and after ‘AT’ and ‘DOMAIN’ is instead the name of the Domino domain.
A quick look at the Person document revealed that the Forwarding Address field had been populated with ” AT “. The document had last been modified late the previous evening by the AdminP server. A look into admin4.nsf revealed that at the time the document had been modified AdminP had updated the client information. So, obviously, AdminP had something to do with the issue, right? Not exactly.
After working with IBM for weeks on this issue, and after seeing it happen in another Domino domain in a completely separate company, the issue turned out to be related to both a new change and an old bug.
In pubnames.ntf, in the $PersonInheritableSchema subform, the MailAddress field (which is the Forwarding Address field) contains this formula:
MailType=”1″;ccMailUserName+” AT “+ ccMailPO;
Do you see it? Look at the second line. Yep. ” AT “. But what is the MailType field? Well, there isn’t one which makes this is the old bug portion of the issue. Depending on how much familiarity you have with the Person form you may realize that this formula should be referencing the MailSystem field. This bug exists today in 8.5.1 and goes back as far as R3, possibly further (I do not have an older pubnames template to verify). We’re talking about a 17 year old bug if not older. All of the affected person documents in these two companies have MailType fields set to a value of “1″, even though MailType is not a valid Person document field. The fields have been in the person documents for 1-2 years. So, why are they there and why are they a problem now?
These two companies are experiencing another issue with what IBM is referring to as a cross-pollination of personal contact documents and server-based person documents. Even though I can find no evidence of the existence of a MailType field in contact documents, IBM assures me that it is a valid field and that some how the contact document and the person document are cross-pollinating causing the MailType field to be saved into the Person document. This piece is still iffy to me and because a root cause has not been found all I can say is that these two companies have, for whatever reason, MailType fields in the Person documents. And, as you can see in the second line of the formula, the existence of a MailType field with a value of “1″ will cause ” AT ” to be added to the MailAddress field if ccMailUserName and ccMailPO fields do not exist, which they do not. But, still, why is this a problem now?
Apparently AdminP is using the new directory API when it makes modifications to Person documents. The portion of the API that is saving the document is re-computing the formulas. This did not happen previous to the new directory API so the existence of this 17+ year old bug and the MailType fields were never an issue before.
For now the best workaround to this issue is to modify the Person form and simply remove the entire formula because it is not doing anything anyway and, considering that this formula has been incorrect for 17 or more years it is obviously pointless.
Of course, the drama doesn’t end there because several users ended up with recent contact documents for the affected users and now whenever they attempt to send mail to one of these people it always goes to ” AT @DOMAIN”. Because there is a lack of fine grained controls for recent contacts and no ability to simply purge all recent contacts, we are resorting to sending out mass mail distributions with PostOpen code to purge the recent contacts.
I certainly hope that this turns out to be one of those exceptionally rare situations and that none of you ever see this happen, but you still may want to consider removing that formula just in case.