Beat the butter, sugar, vanilla extract, orange zest, vanilla bean seeds, and ginger together until light and fluffy. Add in the egg yolk and beat well.
In another bowl mix together the flour. almond flour, salt, and salt.
Slowly add the flour mixture into the butter mixture and mix until just combined.
Turn out the dough onto a lightly floured surface. Divide the dough into two and form into discs. Wrap tightly in plastic wrap and chill for at least 2 hours, preferably overnight to further develop the flavors.
Preheat the oven to 350 degrees F. Line two baking sheets with a silicone baking mat (you can use parchment, but the baking mat is preferable).
Take out one of the discs and lightly flour the dough and your surface. Roll out to about ¼ inch thick. Use a linzer cookie cutter to cut out the cookies. Make sure you have enough plain cookies and cookies with the cut outs to sandwich together.
Bake for about 6 - 8 minutes. Keep a close eye on the cookies so as not to get them get too much color. You should remove them as soon as you see the edges just start to turn a light golden brown.
Allow to cool on the baking sheets before transferring to a wire rack.
Once the cookies are completely cool, brush the plain cookies with about a teaspoon of plum jam, then top with the cut out cookies.