Preheat your oven to 375°F (190°C). Line a standard 12-cup muffin tin with paper liners or grease thoroughly with cooking spray.
In a large bowl, whisk together the flour, sugar, baking powder, and salt until well combined.
In a separate bowl, whisk together the milk, melted butter, eggs, and vanilla extract until smooth.
Pour the wet ingredients over the dry ingredients and stir just until combined, being careful not to overmix (aim for 12-15 mixing motions).
In a small bowl, mix together the melted butter, brown sugar, cinnamon, and nutmeg (if using) for the cinnamon swirl. The mixture should be the consistency of wet sand.
Fill each muffin cup halfway with batter. Add a spoonful of the cinnamon swirl mixture, then top with the remaining batter.
Use a toothpick or knife to create swirls by gently drawing figure-eights through each muffin.
Bake for 18-20 minutes, or until a toothpick inserted in the center comes out clean with just a few moist crumbs. The tops should be golden and spring back when lightly touched.
While the muffins cool, beat the softened cream cheese until smooth, then gradually add powdered sugar, milk, and vanilla, adjusting consistency as needed.
Once the muffins have cooled for 5-10 minutes, drizzle with the cream cheese glaze. For a bakery-style finish, add the glaze while muffins are still slightly warm.