but dont all of them use the same exchange (the BT one)? Or do different suppliers get affected to different degrees?
Just thinking about ADSL in the UK (forgetting cable, wimax, sat). Many will use BT's exchanges, and BT backhauls. Some will use their own equipment (in BT exchanges) and their own backhauls.
Those using BT backhauls connect themselves via connections called Centrals, often 622Mbps connections.
Once on the ISP network, they have to provide connectivity across their network and transit onto Internet.
So, bottlenecks can exist at any point, though tends to be in 3 places - the backhauls (either BT's or ISP if ISP have their own), the Centrals, or the ISP network.
The Centrals and the ISP network are specific to the ISP, the backhauls generally not (unless ISP owned).
The biggest cost for the ISP are the Centrals (in the £m per year) and their own networks/connectivity, so this is where they are most likely to skimp. Easiest way is to modify traffic throughput - 'shaping'. Virtually all ISPs shape, whether they admit it or not. p2p is normally the first, rightly, to get shaped. The ISP decide how much shaping goes on.
Thats a very simplistic, and general view, but you can see that ISP won't affect sync rate (what your computer/router reports), but will dramatically affect real world throughput. Congestion on the VPs (backhauls) will affect groups of ISPs.