First we will begin making the dough. Heat up milk in a small sauce pan until it begins to simmer. Remove from heat and add sugar and salt and stir. Let it cool down to lukewarm to the touch. While milk is cooling, proof the yeast by adding the warm water and yeast in a small bowl and stirring together. Let it activate by setting it aside for 10 minutes.
Once milk is cool, add to a large bowl, along with 3 ½ cups of the flour, 2 tablespoons of oil, eggs, and yeast mixture. Stir together with a wooden spoon until combined. The dough will be very thick but still pretty wet and sticky.
Place ¼ cup of flour on the surface of your counter. Place dough on top of it and begin kneading the dough with the flour until all the flour is mixed into the dough. Keep kneading for about 4-5 minutes until the dough is smooth and elastic. If it starts to stick to the counter, add a little more flour. Once you finish kneading, shape into a ball. Lightly grease another large bowl and place the ball of dough into bowl. Cover with a kitchen towel and let it rise for 1 ½ hours, where it will be double the size. I like to place mine in the microwave to keep it away from any cool draft from the A/C.
Prepare a 12-cup muffin pan by spraying with non-stick spray. Once dough has risen, punch it down with your fist and remove from the bowl. Lightly dust your counter with flour and place the dough on top. Roll out dough into a large rectangle, 16 inches x 24 inches. Spread Nutella over the entire rectangle of dough into an even layer. You don't want the Nutella layer to be too thick so remove any excess, if necessary.
Cut rectangle into 6 strips lengthwise and then 8 strips widthwise. You should have 48 small rectangles now. Stack 4 rectangles together place into greased muffin cup standing up. Repeat with the rest of the dough. Set muffins aside to let dough rise for another 30 minutes. Preheat oven to 350 degrees.
Once muffins have risen and oven is preheated, bake until tops begin to turn golden brown, about 15 minutes. Remove from oven and let them cool for 10 minutes before adding icing.
Mix icing ingredients together. Pour over the muffins. Serve immediately.