One of Microsoft's many audio APIs (MME, AKA "waveOut") has a bug in Windows 7.
I am using the library PortAudio, so my solution is to compile it without MME support, forcing it to use DirectSound or WASAPI instead.
Could be a few things
You may be borderline clipping. Your sine wave has an amplitude of 1, which is just at the edge of clipping (depending on how its rendered). Try it with an amplitude of 0.5
Your hardware is sloppy. For example, cheap laptop sound cards often cut corners in the anti aliasing filters and or clipping management
Your operating system is ...
That's is most probably because the filtering post sampling is not adequate. After sampling you would repetitions of the frequency spectrum of the sinusoid, which need to be filtered out.
Or there are non linearities being produced in the acquisition.
The reason you only hear it with certain sampling rates is the filtering (bandwidth) in Audacity amplifies ...