In theory, yes, all you need at the client end is to set up a VPN connection through Windows, as long as the router at the head-end is configured properly (and it's of a type that really can accept a standard VPN connection instead of something proprietary).

Note that I tried to do that very thing two years ago with a cheap Linksys router that was, in theory, supposed to be able to work as a VPN endpoint. I couldn't get it to work. I gave up and went with SonicWall, which was more expensive, but which Just Worked.

My recommendation is to look online at the user manuals for the router you're thinking of getting (or the user manuals for the replacement flash software, if you're going that route), and make sure the instructions explicitly show how to set up such a situation, indicating that it's a supported feature. If you can find that info, then you're probably OK to buy that router.