The real secret to this is re-laying out the filesystem data and altering and relaying out the metadata, which is why AFAIK, only VxFS can do this at the moment - it's perceived to be both hard and uncommon (despite all sysadmin's experiences to the contrary, and the manifest existence of fs-reorganisers) so most fs authors don't bother.
Shrinking volumes, by comparison, is just moving a fencepost. It's what they contain that needs the work.