With bind (dns daemon for *nix) the hosts file is always checked locally before it attempts to resolve the hostname. If it's the hosts file on the dns server (and not hosted on the box in question), your machine will end up sending out a dns query like it normally does. it's up to the server to resolve it. i'm pretty sure win32 operates this way too. hell, win32's network stack was bsd up until not too long ago. maybe there's a way to rehash the service w/out rebooting?
If you're doing this for a network of people, i'd suggest a firewall rather than blackholing the offending hostnames' IPs to something non-routable. Or a nice caching squid proxy..
i know it doesnt solve the problem, but hopefully sheds light on it as to why