**Optional Step:** Combine half the lemon zest with milk. Heat on medium until warmed, then cover and let cool for 20 minutes. Place in the fridge until ready to use and strain before using.
In a mixer, cream together butter and sugar on medium-high until light and fluffy (about 3 minutes). Add the egg, lemon juice, and lemon zest, beating for 1 more minute.
Add the milk and stir to combine.
Mix in the dry ingredients until just combined.
Pour the mixture into a greased 8.5 x 4.5 x 2.5-inch loaf pan.
Bake at 350°F for about 1 hour or until a toothpick comes out clean. Let cool for 15 minutes and remove from the pan.