Cream together the softened butter and granulated sugar in a large bowl until it's light and fluffy, about 2-3 minutes. Add your eggs one at a time, beating well after each addition. Stir in the vanilla bean paste and milk until everything's combined. Don't worry if it looks a little thick—that's exactly what you want before adding flour.
In a separate bowl, whisk together the flour, baking powder, and salt. Gradually add the dry ingredients to your wet mixture, stirring until just combined. The dough should come together without any streaks of flour showing. Don't overmix or your 4th of july sugar cookies will turn out tough instead of tender.
Divide the dough in half and wrap each piece in plastic wrap. Pop both into the fridge for at least 30 minutes—I usually do mine for a full hour because I'm impatient and want them extra cold. Chilled dough is way easier to roll out and holds its shape better when baking.
Preheat your oven to 350°F and line your baking sheets with parchment paper. Roll out your dough between two pieces of parchment to about 1/4 inch thickness. Use your favorite cookie cutters to make stars, hearts, or whatever patriotic shapes you're going for.
Place cutouts on your prepared baking sheets, leaving about 1 inch between each cookie. Bake for 10-12 minutes until the edges are just barely golden but the centers still look slightly soft. They'll firm up as they cool, so don't overbake—I learned that the hard way my first batch.
Let cookies cool on the baking sheet for 5 minutes, then transfer to a wire rack to cool completely. This is super important because the royal icing won't stick properly to warm cookies. I usually make my icing while they're cooling so everything's ready to go.
For the royal icing, whisk together powdered sugar, water, and lemon juice until smooth. Divide into three bowls and leave one white, tint one red, and one blue using your gel food coloring. Start with just a tiny bit of coloring and add more until you get the shade you want—gel is strong stuff and goes a long way.