https://www.quora.com/Is-there-an-open-source-alternative-to-Blackboard

The above link looks like a pretty reasonable answer to this question to me. In short, Moodle, Canvas, Sakai, and OpenEdX are all open-source e-classroom solutions. They are not really targeted toward casual self-hosters though. These packages are typically run by full-time engineering staff on multiple beefy servers at schools where the setup serves thousands of students. If you're quite strong technically, it sounds like Moodle might be at the easier end of things, but I don't commonly see individual teachers standing up their own Moodle servers, but I also don't hang out with technically oriented teachers so maybe I'm not in the right crowd.

There may be something oriented toward more casual self-hosting, but these are what I'm aware of. I haven't used them though.