Location block with map does not follow field privacy

Location block with map  does not follow field privacy. You can enable location itself for info block and it will show/hide there depending on the field privacy. The actual location block/map will still show though. Has anyone found a workaround. I may just disable it for now.