Grease 9x13" baking dish. Set aside.
Cut bread into cubes and set aside.
Melt butter over medium high heat in large skillet until melted.
Add 3/4 cup brown sugar, 1 teaspoon cinnamon and 1 teaspoon vanilla.
Stir until mixture thickens a little, then add bananas and stir until coated.
Remove from heat and set aside.
In a large bowl, beat eggs.
Beat in remaining brown sugar, cinnamon and vanilla.
Add nutmeg, half and half and milk.
Whisk until blended.
Add bread cubes and gently stir until bread has soaked up liquid.
Pour bread mixture into prepared baking dish.
Pour banana mixture over bread mixture.
Cover with plastic wrap and refrigerate overnight.
Preheat oven to 350.
Remove plastic wrap and bake for about 50 minutes until golden brown.