TCP usually implements a variety of congestion control and avoidance algorithms that are transparent to the TCP protocol itself. That is, using them doesn't require any collaboration from the remote end. Many of these implement a ramping feature. While your numbers seem to ramp fairly slowly based on my familiarity with those algorithms, I'd be surprised if they weren't the culprits.