I don’t think you’re too paranoid, but it seems like this idea is kinda unexamined and needs to be bounced off someone else first:
Wrapping your phone up and putting it in a box won’t be nearly effective enough to prevent audio recording. If you want to try this yourself, start your voice recorder app, wrap up your phone and set it in the box, say some stuff at a normal volume then play it back. It’s been a while since I used that function on android, but a long time ago ios had variable gain automatically applied so in quiet situations (like being wrapped up in a box, or night time in the woods) recordings would contain the information you’re trying to capture.
If you do this (or have already done it), and feel like it’s good enough for your needs, export the audio to a program like audacity and run some of the voice filters there on it. Even in situations where your voice is, to human ears, completely covered up in background and room tone often these free, open source tools can automatically pull them up out of the noise floor.
Imagine what a professional using purpose built software is capable of.
But even if you had a perfect towel and box: your computer has a microphone and camera on it.
Now you might be able to comfortably disconnect both of those and only connect them when someone calls, but if you’re forwarding the data stream through the device you want to treat as compromised there is a good chance that your communication data will have to be decrypted on the device before retransmission.
But if somehow your preferred platform can maintain perfect forward secrecy while handing off between clients (it shouldn’t, because this is a feature used by surveillance organizations), going through voip is a security downgrade because the encryption used from your pots ata (the box that goes Ethernet to phone) to the pc running the pbx software is less strong than that used by your communication platform.
In addition, surfacing your communications to the whole network like this would do opens you up to attacks on your ata and the ones for soho that you’d use are incredibly insecure to the network they are on. They’re worse than those consumer routers you always see with internet facing management pages.
So the next logical step, assuming you have the aforementioned perfect towel and box, is to just use the native pc programs for the communications software you want to make and receive calls through.
Of course, theres nothing preventing your assigned agent from compromising your pc, and in some ways thats an easier job than with a phone.
So I want to ask this as a person who has been surveilled: what kind of eavesdropping are you trying to avoid?