Preheat the oven to 350 degrees.

In a large bowl, cream together the butter, sugar, and vanilla with an electric mixer.

Whisk together flour, cocoa powder, baking powder, baking soda, and salt in a medium-size bowl.

Gradually add the milk and dry ingredients to the creamed butter and sugar until well incorporated. This is a fairly dry dough but if it is too crumbly add in more milk a teaspoon at a time until a dough forms.

Use a dough scoop or spoon to scoop out dough and roll it into balls. 

Place the balls about 2 inches apart on a cookie sheet and flatten them to about ¼ of an inch with the bottom of a glass (You can dip it in sugar if it sticks). For cookies with smoother edges, you can flatten them by hand and then place them on the cookie sheet. 

Bake for 11-13 minutes until the tops appear dry.

Allow to cool for 3 minutes on the baking sheet and then transfer to a wire rack to cool completely. 

Allow the cookies to cool completely before frosting them.

To make the frosting, add the butter, cocoa powder, and milk to a saucepan and heat over medium heat.

Stir and heat until the butter is melted. Mix in the powdered sugar, and continue stirring until well incorporated and smooth. Remove from the heat.

One cookie at a time spoon frosting on and quickly top with marshmallow bits.

The frosting sets quickly so working this way allows the marshmallows to stick. If the frosting thickens too much while frosting the cookies just reheat it until it softens again. 

Store the cookies in an airtight container.