The deer in my area do not eat rosemary, lavendar, oleander, hyrdragea, black eye susan, calla lilies and peonies. However, you might want to call your local extension office to see what they recommend, they usually have a list. Deer in different regions enjoy different things, and it has also been my experience that they can change their "minds" about something too....and suddenly start eating a certain genus of flower they had not previously enjoyed.
Good luck!