Preheat the oven to 350°F. Grease a 9 x 13-inch baking dish with nonstick spray and line it with parchment paper.
In a medium bowl, sift together flour, almond meal, baking powder, baking soda, and salt.
In a separate bowl or large liquid measuring cup, whisk together buttermilk, vanilla, and lemon zest.
In the bowl of an electric mixer, beat egg yolks, egg, and sugar on medium speed until light and fluffy (about 5 minutes). Slowly add olive oil and continue beating for another 5 minutes.
Add half of the dry ingredients, mixing just until combined, then add half of the buttermilk mixture. Repeat, scraping down the sides of the bowl as needed. Stir the batter by hand to incorporate any ingredients at the bottom.
Pour the batter into the prepared pan and bake for 25 minutes. Let the cake cool completely in the pan on a wire rack.