Heat your oven to 375°F and grab a 9x13 baking dish — butter it lightly so nothing sticks later. Slice your 4 large potatoes thin (about 1/8 inch) using a mandoline or sharp knife. Pat them dry with paper towels because moisture is the enemy of crispy edges here.
Combine milk, heavy cream, minced garlic, thyme, pepper, and nutmeg in a medium saucepan over medium heat. Warm the mixture for about 3-4 minutes — you want it steamy but not boiling. Stir in 1 cup of cheddar and the parmesan until completely melted and smooth.
Layer one-third of your potato slices on the bottom of the buttered dish, slightly overlapping them. Pour one-third of the cheese sauce over the potatoes, making sure it gets between the layers. Repeat with another third of potatoes and another third of sauce.
Add your final layer of potatoes on top and pour the remaining cheese sauce over everything. Make sure the potatoes are mostly covered — this keeps them from drying out during baking. Reserve the remaining 1 cup of cheddar for later.
Cover your dish tightly with foil and slide it into the oven. Bake covered for 45 minutes, then remove the foil carefully (watch out for the steam). The potatoes should be mostly tender when you pierce them with a fork.
Mix panko breadcrumbs with 2 tbsp olive oil and the reserved cheddar cheese in a small bowl. Sprinkle this mixture evenly over the top of your scalloped potato recipe. Return it to the oven uncovered for the final 20-25 minutes until the topping turns golden brown and the edges bubble.
Let it rest for 5 minutes before serving — I know this is hard because it smells incredible, but this helps it set. A sharp knife dipped in hot water helps you get clean squares when serving. If you're making this ahead, just reheat it gently covered with foil until warmed through.