Preheat the oven to 350°F/180°C. Line three baking sheets with parchment paper and set aside. In a medium bowl, stir together the flour, baking powder, and salt; set aside.
In the bowl of a stand mixer fitted with the paddle attachment, cream together the peanut butter, butter, brown sugar, and granulated sugar until light and fluffy. Beat in the egg and vanilla until smooth.
Divide each dough ball in half, and roll into balls. Place two dough balls side by side on prepared baking sheet and flatten each slightly to join them together, making them into a peanut shape.
Use a fork to press the criss cross shape on the cookies, pinch the middle to ensure the peanut shape stays. Repeat until all dough has been made into peanut shapes, making sure each cookie is at least 2 inches apart.
Add half the powdered sugar and mix until smooth, add 2 tablespoons of milk and mix well. Add the remaining powdered sugar and beat until fully mixed. Mix in the vanilla extract, and add additional milk a teaspoon at a time until desired consistency is reached.
To assemble, flip half the baked cookies so that the bottom is facing up and pipe or spread a generous amount of peanut butter frosting over the surface. Place a second cookie on top and gently press down to create cookie sandwiches.