Asterisk will do this fine, without being in the media path, as long as the devices can directly reach each other. In other words, if it's possible to do, Asterisk will let it happen. As a matter of fact, you'd have to go out of your way to disable end-to-end media by doing something like listening for DTMF or recording the call or setting "canreinvite=no" in sip.conf.
Many hundreds of shops are using Asterisk as a hosted VoIP platform and reselling to SOHO/small business customers - take a look around the voip-info.org areas, and read some of the older messages on the asterisk-* mailing lists. There are even people who will sell you software to do it.