You can't turn off both the remote and the face. If you
turn them both off, then one of them will reappear. If
you want to make them both hidden, minimize it to the
system tray.

It would seem to be more logical to have the face always present unless minimized. Then allow the remote to show or hide. I can't imagine too much demand to be able to use the remote without seeing what's going on with the display.

EDIT: unless the hardware display is within easy sight. nevermind.