Preheat your oven to 350°F (175°C) and line a baking sheet with parchment paper.
In a medium bowl, whisk together the flour, baking powder, and salt. Set aside.
In a large mixing bowl, cream the softened butter and brown sugar together until light and fluffy.
Add the egg, vanilla extract, and cinnamon to the butter mixture, mixing until well combined.
Gradually add the dry ingredients to the wet ingredients, alternating with the milk, and mix until a dough forms.
Divide the dough in half and roll each half out on a lightly floured surface to about 1/4 inch thick.
Cut the rolled dough into rectangles, about 3x4 inches in size.
Place half of the rectangles on the prepared baking sheet. Spoon about 1 tablespoon of brown sugar filling onto the center of each rectangle.
Top each filled rectangle with another rectangle of dough, pressing the edges to seal. Use a fork to crimp the edges for a decorative touch.
Bake in the preheated oven for 12-15 minutes, or until the edges are lightly golden.
While the cookies are baking, prepare the icing by mixing the powdered sugar and milk in a small bowl until smooth.
Once the cookies are out of the oven and cooled slightly, drizzle the icing over the top and add sprinkles if desired.