Just ban caller id spoofing as a whole and white list a few folks in a highly regulated list.
How?
Edited: added explanation of why this is more difficult now.
VoIP a computer tells another computer information.
Example: # Call making the call: Information here; Caller ID: Information here; Call to #: Information here.
They don't even require 10 digit numbers to call.
No number making the call,
None for caller ID name, only filling out Call to #: information inputted.
What you are asking for is a series of keys or passwords. Rhe problem with keys or passwords is, one side needs to know the key/password, and the other side must also know the key/password. So the elimination of weak points is at every transaction of widely known keys/passwords.