Seems like a good idea.
combined with a lightweight ordinary phone sounds like what you are describing.
Not exactly what you descirbed, but close:
This device sounds close: voicestick
, although the hardware seems to be tied to their service. There are some additional reviews here DSL Reports Voicestick reviews
Looks like Vonage has something similar too: Vonage V-Phone