Place the almonds for the crisp topping on a parchment lined rimmed baking sheet. Lightly toast the almonds in the oven for about 10 minutes. Set aside to cool
Mix the pears, rhubarb, and vanilla together in a 10-inch cast iron skillet.
Combine the brown sugar, flour, lemon zest, and sea salt for the filling together in a small bowl. Stir into the rhubarb and pear mixture until they are well-coated.
Mix together all the ingredients for the almond crisp , including the cooled almonds in a bowl. Use your hands or a pastry cutter to cut the butter into the mix until it starts to look like crumbs in varying sizes.
Place the crisp topping over the filling in an even layer and bake in the oven for about 40 - 50 minutes, or until the filling is bubbling and the pears are tender. You may need to cover the top with aluminum foil if it starts to brown too fast.