Today I went to an event happening at the building I went to elementary school in decades ago. I was worried the directions weren't clear enough and that I might get lost, but when I got there everything felt immediately familiar and I could still walk on autopilot exactly where I needed to go.
There's parts of our mind that encode information like about places that aren't part of explicit memory. You may therefore "remember" something that you don't recall knowing. What if rather than being my elementary school, this was a building I'd been to once a long time ago but forgotten? Or maybe a building that I've not been to, but unbeknownst to me was designed with a unique style by the same architect as a building I was more familiar with? It might also seem oddly familiar.
Reincarnation by nature is hard to define, let alone prove true or false. So I couldn't really rule it out entirely. But given all the other explanations, I'd lean against it.