Okay, I'm trying to block ad banners on my system.

I do this successfully on my home PC by using the HOSTS file supplied here. For the uninitiated, this is simply a huge list of known ad-banner servers, and it forces them to all resolve to 127.0.0.1. That way, the browser returns page-not-found for every ad that attempts to load.

This works like a charm on my home PC because it is running Win98. My work PC is running Win2k, and I of course run into the "slow system" problem described here. When the hosts file is big, Windows 2000 slows down.

Well, I just saw that they updated their page recently, and the new page says that in order to get Windows2000 working normally with a big hosts file, you can disable the "DNS Client" service and all will be well again. (Note that I am unable to disable the "DHCP Client" as I need that because my PC is on a DHCP network, but disabling DNS client seems to be enough for solving the speed problem anyway.)

My initial experiment seems to bear this out. Disabling the DNS client service makes Win2k work normally (I can still resolve addresses and everything, it seems). However, the ad-blocking features don't seem to be working for me. All the ads still come up. Why?

I've already done the following troubleshooting steps:

- I have verified that the ad sites in question are in fact in the HOSTS file. For instance, when I open up www.bluesnews.com and the ad banner appears on the right side of the screen, I can look at the page source and see what server the ad image is coming from. And then when I look in the hosts file, that server is there, and it should be pointing to 127.0.0.1.

- I have verified that the HOSTS file is in the correct place. On my system, this place happens to be c:\win2k\system32\drivers\etc.

- I have verified that other programs which use the HOSTS file are working correctly. For example, FINDEMPEG properly writes the address of my empeg to the end of the hosts file.

- I have verified that the OS does in fact recognize the HOSTS file and uses it. For instance, I can type PING EMPEG at the DOS prompt, and my car player responds. So if the last line of the hosts file is working, why don't the lines above it work?

Any ideas?

Greetings!

Caching? DNS or page...

Perhaps...

Can you temporarily force your DNS servers to 127.0.0.1? This way, your DNS lookups will always fail. See if you can still get through / resolve those addresses, and to where. This will check to see if the DNS query is still occuring somewhere (will eventually come back with a lookup failure), you have something cached (immediate success) or a resolution (to your file contents) but failure.

My DNS servers are not under my control. When I attempt to enter 127.0.0.1 into the DNS box in Windows 2000, it complains and tells me that 127.x is an invalid address for a DNS server.

I realize this isn't your question, but have you tried any ad blocking software? The only one i've used so far is called "Proxomitron" and it works surprisingly well at killing just about anything that might be an add or pop up. You can also bypass it with two clicks if you find yourself in a situation where you want to see something it's blocked by accident.

Matthew

continuing in the software vein, how well does webwasher work on w2k? I'm exploring the migratory thang myself.
Webwasher seems to be pretty good in w98se.

I have messed with Proxomitron before, but the HOSTS file thing is much simpler. Well, at least when it works (as it does on my 98 box).

Are you on W2K server or workstation?

Did you copy or rename it? : http://support.microsoft.com/default.aspx?scid=kb;EN-US;q283092

Tony, do you know how to disable this fix for the banners by any chance?
I have this fix on my system (XP Pro) but it screws up some other sites as well.

Did you copy or rename it?

I did not copy or rename it myself, but I have no idea what the FINDEMPEG program does to the file "under the hood". Findempeg may very well do a rename as part of its process, which would account for this behavior (since I have also put Findempeg in my startup group which would explain why even a reboot did not fix the problem). I will see whether or not that is the root of the problem by messing with it some more.

Thanks for that very informative link.

Tony, do you know how to disable this fix for the banners by any chance?

Assuming that you have done exactly the same thing that I did, you simply put the HOSTS file back the way it was. A default HOSTS file has simply one entry:

127.0.0.1 localhost

Okay, the rename-of-the-hosts file thing doesn't seem to be the problem. I made sure nothing was running at startup and rebooted the PC. Ads are still coming up, even with a proper hosts file.

Could it be a proxy thing? Maybe because I'm going through a proxy server, it gives a second try to DNS queries for those addresses?

Right, Tony.

With the use of a proxy server, a local host file does absolutely nothing regarding the elimination of ads. If you use a proxy (and didn't specifically disable it for the advertising hosts), your webbrowser just asks the proxy for the complete URL, and the proxy does all the DNS lookup etc.
I haven't tried the following (so it might just as well not work), but assuming that you are using IE as your browser, go to IE's proxy settings (for LAN connection, I suppose) and enable the checkbox "disable proxy for local adresses" (might be named different, but you get the idea). That _might_ disable the proxy for any hosts that resolves to 127.0.0.1 if it doesn't, you might try to go into advanced proxy options and add 127.0.0.1 to the list of hosts that you don't want IE to use the proxy for (disable proxy for hosts starting with... setting).

cu,
sven

Greetings!

The question is, is your proxy doing your lookup for you? Do you have a sniffer (or snoop) to see what the call looks like? Can you put the host list on your proxy?

The checkbox to bypass the proxy for local addresses is already checked. Adding 127.* and 127.0.0.1 to its specific list of proxy exclusions did not help.

Our system here is an interesting one: We get our DNS information from a completely different place than we get our actual web traffic from. Sounds like it would be a problem when I say it, but it's always worked great.

Here's how it works: We connect to our local LAN as well as communicating to our corporate headquarers via a frame relay connection. The corporate servers dole out our DHCP addresses and DNS/WINS queries across this frame relay link. That allows us to all work on the same network. Everyone here simply sets their system to "automatic" (the default Windows settings) and the Cisco router between us and HQ properly forwards all DHCP and name server requests and replies.

For our web traffic, we have a local DSL line running off of a proxy server at a fixed IP address. We manually configure our web browser to point to the address of this local proxy. But this proxy does not handle our name requests, it just handles all the actual web data transfers. Clear as mud?

I can't tell this DSL proxy to do my name requests, or else I wouldn't be able to communicate on our local LAN, or be able to connect the servers at HQ.

Anyway, do you think this sytem might explain it?

Greetings!

In any case, the fact that you are making a DNS query at all implies that your host file is not working. Otherwise your client would not be looking, from whatever source.

Are you certain the host file is working? Can you put a bogus entry (something you know does not resolve) in it, and verify that it is coming back with the host file entry?

In any case, the fact that you are making a DNS query at all implies that your host file is not working.

I didn't say I was making a DNS query, I was describing the way DNS normally works on this PC and in our company.

As I understand it, it should work like this:

1) Request for an address is made.
2) HOSTS file is checked first to see if the address is there.
3) If so, the address from HOSTS is used.
4) If not, then a DNS query is made.

That's the way it works on my machine at home anyway.

Are you certain the host file is working? Can you put a bogus entry (something you know does not resolve) in it, and verify that it is coming back with the host file entry?

Yes. The hosts file DOES seem to be working.

My car player is entered into the hosts file, like so:

10.134.75.122 empeg

When I do a ping, it works:

c:\temp>ping empeg
Pinging empeg [10.134.75.122] with 32 bytes of data:
Reply from 10.134.75.122: bytes=32 time<10ms TTL=255

When I change the hosts file to a bogus address:

233.134.75.122 empeg

Then the ping fails:

c:\temp>ping empeg
Pinging empeg [233.134.75.122] with 32 bytes of data:
Request timed out.

So I know the hosts file is working, for sure. But the ads are still appearing in the web browser.

Oh, and before the next obvious question is asked: The empeg entry is at the bottom of my hosts file, so unless Windows reads it backwards, it should be getting all of the prior entries.

Also, I have tried it with both a 127.0.0.1 entry for all of the ad sites, and also as a 0.0.0.0 entry. Both produce the same result.

Hmmm... okay...

Dumb question, you did clear your browser cache...

How about this: take a real address on the internet, put a fake entry to a live machine (your empeg) in your hosts file, see where it resolves. Pick a site you have not visited, so will not be cached anywhere. If you still get out there, your hosts file was not read (DNS query occurred). If you get a the fake entry's site, you know the hosts file was read correctly, and the problem would have to be outside your hosts file.

Yes, I cleared my browser cache.

take a real address on the internet, put a fake entry to a live machine (your empeg) in your hosts file, see where it resolves.

Sunuvabitch. I changed my empeg entry to this:

10.134.75.122 www.bellatlantic.net

And when I type "www.bellatlantic.net" the goddamn verizon page comes up. But when I do:

10.134.75.122 empeg

Then my empeg comes up because it's considered a local address by the browser.

If I try to force a bogus non-local DNS address:

10.134.75.122 empeg.foobarasdfadesfas.com

Then when I paste that address into the URL box, it tries to do that microsoft-search-the-web-thingy and reports that they couldn't find that web site.

Crap, how do I tell my system that it's OK to use the hosts file even if the address is non-local?

I think I have something... The last bullet looks interesting...

When multiple names or IP addresses are used in the (hosts) file, the DNS Client service must be
running for all entries to be returned or used in answering queries. If the DNS Client service is
not running, only the first entry in the file is used to resolve the query.

Try giving your test case a unique IP address, or move your test case to the first line in the file, or turn the DNS Client on again...

Useful commands from MS:

ipconfig /flushdns - flushes your DNS cache
ipconfig /displaydns - shows your DNS cache
nbtstat -r : shows a count of all names resolved by broadcast or WINS

Interesting.

I agree that this could all be due to the fact that I turned off the DNS client. But unless I do that, the system is unusable. And certainly (as proven above) parts of the system are working.

I will investigate this more, thanks.

I can't tell this DSL proxy to do my name requests, or else I wouldn't be able to communicate on our local LAN, or be able to connect the servers at HQ.

I don't think you quite understood the previous point about proxies and DNS. When you use an HTTP proxy, it doesn't use that proxy as a DNS server, which you seem to follow, but your web browser makes a connection to the proxy and makes a request that looks something like ``GET http://ads.doubleclick.com/annoyingads/004.html HTTP/1.1''. Note that it's including the full URL, not just the pathname in the GET request. That's because the proxy server has to connect to that remote host. The problem likely lies here, in that your proxy host is now making a DNS request for ads.doubleclick.com. And since it's a remote host, it has no way of seeing the hosts file you've installed on your local machine. (As far as the ``ignore proxy for local hosts'' option that was mentioned, I don't know. It may ignore the 127/8 network for some reason. (Or are you using the 0.0.0.0 form of the hosts file?) Microsoft products have had weirder problems.)

As I see it, your solution for this problem lies in changing the hosts file on the proxy, which you may not want to do since it will affect everyone using that proxy. You might also try pointing them to an IP address you have on your local network and set that ``ignore proxy for local hosts'' option. That might work better. Then again, it might not.

As a test, you could change your routing manually temporarily and have it make requests directly out your DSL line, but you might have other routing or addressing problems there that I don't know about that make that unfeasible.

Edit: You might also want to try pinging ads.doubleclick.net or some other hosts listed in your hosts file from your workstation, just to make sure that it's getting that IP address from your hosts file. (I know you said that your empeg entry works fine, but check one of the failing ones, just to make sure that it's not the exception.)

Bitt, you might be on the right track.

Change the hosts file on the proxy, hmm? It -is- my server, and I -can- do whatever I want with it. Hmmm.....

/me goes to try it...

Hmm. Didn't work. Changed the HOSTS file on the proxy server (NT running MS proxy server 2.0) and nothing happened.

Hell, I should probably just tell the proxy server to filter all these sites and be done with it, right? Jeez I'm stupid.

It's also possible (if unlikely) that the proxy server software you're using is bypassing hosts file usage and doing direct DNS lookups and/or caching hostname-to-IP mappings (I can see how there would be a performance benefit to both of those). You could try rebooting the proxy after changing the hosts file....

(I hate MS software. You never have any idea what it's actually doing.)

Bitt, You da man.

Rebooting the proxy server worked.

I discovered quickly that I needed to use the "0.0.0.0" form of the new hosts file instead of the 127.0.0.1 form. If I used the latter, then the end-user is prompted to authenticate with the proxy server.

But after changing it to 0.0.0.0 and rebooting the server, all is well. Each ad box comes up as a broken link now.

And, this happens to benefit the users on the rest of my network, too. Rock.

Out of curiosity, what proxy software are you using? Just so I know that that sort of thing might be an issue if I encounter it in the future.

(Solaris has a similar annoying problem. It has a process called nscd that is the name service cache daemon, and it likes to cache way more than it should. It crops up often when changing IP addresses on a web server. Used to be that you could disable it temporarily using ``nscd -e hosts,no'', and they claim that you still can, but now there's a bug where you actually have to kill the process. Very annoying. But at least you don't have to reboot.)

Microsoft Proxy Server 2.0 on Windows NT 4.0.

It's not our only firewall, it's actually a second stage firewall behind our Linksys NAT router/firewall. I wouldn't expose a Windows box directly on the net unless I had to.

(Note that I -DO- have one windows box running as a test/development web server which is exposed directly to the internet, but it's physically separated from the rest of our network and is on its own independent DSL line.)

Dose this help explain what is happening.

Somewhere in all that is says that the hosts file is loaded when the DNS Client starts. Tells me that no DNS Client = no hosts file. Not that that will fix the problem but it clears up one thing.

There is a lot more in there but some of it is in geek. Heck its all in geek. What really bothers me is that I'm beginning to understand a limited amount of it.

Glenn

edit: PS. Is there any way to load the hosts file into the resolver cache outside of starting the DNS Client? Seems to me there should be some kind of command that can be invoked on startup to do this.

I would have assumed that, but for some reaons the hosts file still worked even with the DNS Client service disabled. I was still able to resolve "empeg" from that file.

The problem was that I was going through a proxy server.

Was the w98 machine going thru the same proxy?

Oh, and I forgot:

Bitt, You da man.

Woo-hoo!

Was the w98 machine going thru the same proxy?

Negative, the Win98 box is my home machine, and it does not point to a proxy server. So I can do the hostsfile trick locally on that box.

Microsoft Proxy Server 2.0 on Windows NT 4.0.
...It's not our only firewall, it's actually a second stage firewall...

A proxy/junk tangent here....

I had a blocking 127.0.0.1 host file for a while, but didn't like the ungraceful "X not found" side effects. Maybe there was a way to fix that, but instead I took another whack at setting up Squid and the "Internet Junkbuster" (recently renamed on Sourceforge to Privoxy ).

The combination seems to work pretty well. I have it set up as:

Browser-->Squid-->Privoxy-->Internet

That's what the Privoxy folks recommend and it keeps Squid from caching junk (see the sample in the Privoxy config file).

I have both of these running on my Linux file server so that they can serve multiple clients, but it looks like it would be possible to compile both for Windows and run them locally (allowing enough resources/memory). It also occured to me (and the reason I post) that it would be possible to run Privoxy on the same box with MS Proxy Server in the same way as you'd run it with Squid -- set up a few simple forwarding configurations.

Privoxy just substitutes generic checkerboard placeholder graphics where all those Doubleclick banners would be, and Squid really speeds up repeat loads of my frequently visited site (like empeg.comms.net).

For anyone who has run Junkbuster off of distributions like Redhat, it's likely that those versions are way out of date (Redhat's certainly is).