Plex is excellent, and even if you prefer the features or interface of Jellyfin, you should never expose any application (Plex, Jellyfin, or otherwise) directly to the Internet. This should be non-negotiable. Plex solves for external access with the mobile/desktop apps and app.plex.tv by brokering client connections into your network without a NAT/PAT on your router or firewall.

For a music library, even a small one, tracks should have proper metadata applied to them and be stored in directories. Plex provides guidance on this here: https://support.plex.tv/articles/200265296-adding-music-media-from-folders/

My own strategy: I deviate slightly from Plex's file and directory naming strategy, but it works perfectly. I start with high quality music, mostly from Bandcamp and process it through Musicbrainz Picard into ALBUMARTIST\YYYY - ALBUMNAME\01 - TRACKNAME.FLAC. Picard sets the metadata and ensures that there is an album cover image also.

Before moving the organized files to my Plex server, I run them through MP3Tag and overwrite any mismatched artist names with the album artist (getting rid of artist fields with 'feat xxxx artist's). This is important for when I sync files in Media Monkey to my iPod, since the iPod would break apart albums with multiple artists. My preference is to keep them grouped together.

Hope this helps good luck 👍. Let me know if you want to know a decent strategy on movie backups also.