Linguistics

Esperanto is perhaps the most successful constructed language of the "batch" that popped up between the 1860s and the 1940s. The text mentions Volapük, but was also Universalglot, Latino sine flexione, Idiom Neutral, plus a bunch of conlangs derived from Esperanto.

It's easy to look at those projects nowadays and say "nope! [feature] is the wrong way to go!"; for Esperanto this would be probably

• rather convoluted phonotactics
• large consonant set
• a "masculine is unmarked" approach to derivation (NB: the feminine -in- was there since the start, the masculine -iĉ- is a recent development)
• the small case system being a bit of the worst of both worlds (less syntactic freedom than a full-fledged case system, still added complexity that one needs to learn)
• the vowel alternations not working so well in practice

But Linguistics back then was barely a science, and those guys like Zamenhof were doing things by gut instinct.

And, more importantly, those conlangs were part of a historical context, where you got a bunch of factors making the intellectuals believe that one language was the solution for everything:

• Nationalism was already well established as a political meme, creating conflicts; with linguistic identity being often seen as synonymous for national identity.
• Increased communication across speakers of different languages. Steam locomotives would "kick in" around 1830, but their social impact would be felt later on.
• War. I don't think that it's a coincidence that the "batch" of conlangs that I mentioned popped up between the Franco-Prussian War and the Second World War.

Once you "glue" those factors together, the idea of a language not tied to any national identity, for the sake of peace, pops up naturally.

Specifically in the case of Esperanto, there's also the fact that Zamenhof was ethnically Jewish. That would make him a direct target of nationalism, and perhaps give him the "insight" to split apart ethnic identity and language (as you have the ethnic identity being associated with Hebrew, not with Zamenhof's native Yiddish).