what kind of phone do you have? On the iPhone you can set "do not disturb" hours. You can also allow calls from "Favorites" during do not disturb hours, such as family etc.

Oops sorry just reread your post. This won't fix your problem with voicemail, but at least you won't be woken up in the middle of the night.

Be aware caller id is easily spoofed, so there's a good chance that the real caller is just using a fake caller id number, especially if you call the number and it's somebody else.