Adjust Tolerance

Drag the Tolerance Level higher, and experiment until the setting seems about right. The Tolerance Level represents the tonal range of colors that are selected when you click one pixel with the Magic Wand. If you enter 0 and click a pixel, only adjacent pixels of exactly the same tone are selected. If you enter a higher number, a wider range of similar tones is selected.

Results are starting to look a little better: