I think, regarding D_SYNC, that perhaps you mean O_DSYNC? Don't know how that plays out with NFS, as (for version 2) NFS operations are synchronous anyway (but the Filer's NVRAM allows it to return completion before data is resident on disk.) NetApp's Tech Note document specifically uses Version 2, though not apparently for reasons of data integrity (Can the author of that Tech Note, or anyone else, comment on this?)
This is a good point, but I didn't want to comment on it before because Mr. Hashemi says he eventually got Linux configured in a way that corresponds to the tech note, and I didn't want to question that. He he said he rebooted the client and that this corrupted his db over NFS ("filesystem") but apparrently not local disk, I concluded that it must be an issue particular to how Sybase handles now-raw partitions that was inconsistent. But if he was using NFS3 without synchronous writes, then that too could have caused the corruption. Keep in mind that this is entirely a client side issue, and nothing to do with Netapp (in that case).
The solution is clear, of course, that either Sybase supports synchronous writes for use with NFSv3, or NFSv2 should be used. I know personally that Sybase worked fine over NFS v2 without this sort of cuorruption. Then again, there could be something funky with Linux, which has had a notoriously bad NFS history.
Bruce