Preheat your oven to 350°F (175°C). Line a muffin tin with cupcake liners.
In a medium bowl, whisk together the flour, baking powder, and salt. Set aside.
In a separate large bowl, cream together the softened butter and granulated sugar until light and fluffy.
Add the eggs one at a time, beating well after each addition. Stir in the vanilla extract.
Gradually add the dry ingredient mixture to the butter-sugar mixture, alternating with the eggnog and milk. Begin and end with the dry ingredients, mixing until just combined.
Divide the batter evenly among the cupcake liners, filling each about two-thirds full and bake for about 18 minutes.
While the cupcakes are baking, prepare the eggnog filling. In a small saucepan, whisk together the eggnog, cornstarch, sugar, and a pinch of nutmeg. Cook over medium heat, stirring constantly, until the mixture thickens and comes to a gentle boil. Remove from heat and let it cool.
Once the cupcakes have cooled, use a knife or cupcake corer to remove a small portion from the center of each cupcake. Fill the cavities with the cooled eggnog filling.
Now, make the buttercream:
In a stand mixer, beat the softened butter until creamy.
Sift in the powdered sugar and mix until well combined and peaks form.
Gradually add the warm milk, one tablespoon at a time, until the desired consistency is reached.
Pipe or spread the frosting onto the cupcakes, covering the filled centers.
Optional: Dust the cupcakes with ground nutmeg or cinnamon for garnish.
Chill the cupcakes in the refrigerator for at least 30 minutes before serving to allow the flavors to meld together.