Spray a 13x9 baking dish with nonstick cooking spray.
In a small saucepan, add butter and melt completely. Add in brown sugar, heavy cream, cinnamon, cloves, nutmeg, and walnuts. Stir to combine. Add bananas and toss gently to coat.
Pour into prepared baking dish and spread evenly.
In a medium bowl, add milk, eggs, sugar, and vanilla extract. Mix together using an electric mixer until combined.
Top the Bananas Foster mixture with a layer of Texas toast, then pour half of the egg mixture over the bread and add another layer of bread and pour the remaining egg mixture over the top.
Cover with aluminum foil and place in refrigerator for 4 hours, or overnight if desired.
Preheat oven to 375 degrees. Bake, uncovered, until a toothpick inserted into the middle comes out clean, about 40-45 minutes. Let stand for 10 minutes before inverting to serve.