You should always ask. Even if the flowers grew out side of the proptery line the owner most likely wants to keep that look.

I planted some plants on the other side of my fence to hide the fence some. I think it looks better that way and I know I don't want anyone coming along picking them. I worked to hard on it. If someone asked me for one...I say yes. To not ask though, NO.