Preheat your oven to 350 degrees F. Grease and flour a standard 9×5 inch loaf pan.
In a medium bowl, whisk together the flour, baking soda, 1 teaspoon cinnamon, salt, and nutmeg. Set aside.
In a large bowl, cream the softened butter and granulated sugar together until light and fluffy. Beat in the eggs one at a time, then mix in the vanilla extract.
Alternate adding the dry ingredients and the buttermilk to the wet mixture, beginning and ending with the dry ingredients. Mix until just combined; do not overmix. Fold in the diced apples.
Prepare the swirl: In a small bowl, mix the brown sugar, 1 teaspoon cinnamon, and 1 tablespoon melted butter until crumbly.
Pour half of the batter into the prepared loaf pan. Sprinkle half of the brown sugar swirl mixture evenly over the batter. Top with the remaining batter, then sprinkle the rest of the swirl mixture on top. Use a knife to gently swirl the top layer.
Bake for 55 to 65 minutes, or until a wooden skewer inserted into the center comes out clean.
Let the bread cool in the pan for 10 minutes before turning it out onto a wire rack to cool completely.
If using the glaze, whisk the powdered sugar, milk, and vanilla until smooth. Drizzle over the cooled bread.
Notes
For the moistest bread, use slightly tart apples like Granny Smith or Honeycrisp.
If you do not have buttermilk, mix 1/2 cup of regular milk with 1/2 teaspoon of white vinegar or lemon juice and let it sit for 5 minutes before using.
This bread freezes well; wrap tightly once completely cooled.