can't deselect after Select All in Graph panel

I loaded 12 graphs and then used the Select All button in the Graph tab to select all the graphs and set them to a common scale.

However, when I tried to deselect the graphs by clicking empty space in the map, I couldn't do it. The graphs all stayed selected.