Because that removes the need of server to know your contact list?

Telegram knows who you are and who you talk to, they store your contact list. Signal knows who you are, but they don't store your contact list.

Also, Signal is already working on usernames. Your criticism is just silly.