Place marshmallows in the freezer for 10 minutes while preparing the batter. This prevents them from melting too quickly during frying.
In a medium bowl, whisk together flour, sugar, baking powder, and salt.
Make a well in the center of the dry ingredients and add cold water, beaten egg, and vanilla extract. Whisk until just combined. The batter should coat the back of a spoon but still drip off easily.
Fill a medium heavy-bottomed pot with about 2 inches of oil. Heat to 350°F (175°C) using a candy thermometer for accuracy.
Remove marshmallows from the freezer. Using a fork or skewer, dip each marshmallow into the batter, ensuring complete coverage while allowing excess to drip off.
Carefully drop 3-4 coated marshmallows into the hot oil. Fry for 30-45 seconds, turning occasionally, until golden brown on all sides.
Remove fried marshmallows with a slotted spoon and place on prepared paper towels to drain excess oil.
While still warm, dust with powdered sugar or roll in cinnamon sugar, if desired.
Serve immediately with chocolate or caramel sauce for dipping, if using.