When should a Jitter Buffer be added

Can anyone provide a reasonable answer here?
If both endpoints of a media stream have their own jitter buffers, then Asterisk is purely a passthrough, whatever type of bridging it is, and any jitter buffering should be accommodated by the endpoints involved!

Mike