What switch are you using (mfg/model)? There are many differences in how load balancing is implemented. I have even found differences in different model switches from the same manufacturer. All that I have seen are based on MAC addresses - Either source, destination or a combination. Some mfg's let you choose which to use. If your client base is reasonably large with random MAC address and equal loads, then the traffic will be balanced. Small client pools or most traffic being generated/requested by a few clients can skew the load. Some implementations have standby ports in the trunk, which seems like what you are seeing. I believe Extreme allocates one port out of the trunk as a standby. Perhaps this is configurable.
---
Its a FEC blade in a Cisco 5505.