Looks like Lichen and its harmless ! Lichens do no harm to the trees but they can indicate a problem with the tree itself !
So even if you were to remove it this time of year its bad because damaged bark in the warmth of summer invites fungus and insects with their nasty bacteria and viruses to stop by for a sap snack. There are sprays used in winter dormancy like lime sulfur and dormant oils and even soap but its getting late for that as the sap has begun to run.
No point in using any sprays or anything else ! Lichen and fungus it has feeds off itself,, not the tree its on!
