Local, privacy respecting, open source LLM that can transcribe phone calls and summarize the call? (lemmy.ml)

submitted 3 months ago by makingStuffForFun@lemmy.ml to c/fosai@lemmy.world

8 comments fedilink hide all child comments

Hi everybody, I find a huge part of my job is talking to colleagues and clients and at the end of those phone calls, I have to write a summary of what happened, plus any key points that I need to focus on followup.

I figured it would be an excellent task for a LLM.

It would need intercept the phone call dialogue, and transcribe the dialogue.

Then afterwards I would want to summarize it.

I'm not talking about teams meetings or anything like that, I'm talking a traditional phone call, via a mobile phone to another phone.

I understand that that could be two different pieces of software, and that would be fine, but I am wondering if there is any such tool out there, or a tool in the making?

If you have any leads, I'd love to hear them.

Thank you so much

you are viewing a single comment's thread
view the rest of the comments

[-] Audalin@lemmy.world 13 points 3 months ago

Haven't heard of all-in-one solutions, but once you have a recording, whisper.cpp can do the transcription:

https://github.com/ggerganov/whisper.cpp

The underlying Whisper models are MIT.

Then you can use any LLM inference engine, e.g. llama.cpp, and ask the model of your choice to summarise the transcript:

https://github.com/ggerganov/llama.cpp

You can also write a small bash/python script to make the process a bit more automatic.

[-] makingStuffForFun@lemmy.ml 5 points 3 months ago

Okay, the idea is excellent, but I've just spent the last hour trying to get any app out there to record my calls.

I've tried the open source one on f droid, and it almost works. I can get it to record my side, but that's it.

I tried commercial ones. I tried commercial ones with horrendous privacy policies. Nothing seems to work.

I've used the accessibility options. I've gone deep down into the rabbit hole, so it looks like Android is fully cutting off the ability to record calls. In Australia at least.

What a shame.

These apps all seem to have the same ability of dropping the recording into a folder, so I could synchronize that across my network, have my server check for new files that appear into that folder, and then the LLM could convert that into a text file and send it straight back to me.

Living the dream! But... Not

[-] Audalin@lemmy.world 4 points 3 months ago* (last edited 3 months ago)

I expected that recording would be the hard part.

I think some of the open-source ones should work if your phone is rooted?

I've heard that Google's phone app can record calls (though it says it aloud when starting the recording). Of course, it wouldn't work if Google thinks it shouldn't in your region.

By the way, Bluetooth headphones can have both speakers and a microphone. And Android can't tell a peripheral device what it should or shouldn't do with audio streams. Sounds like a fun DIY project if you're into it, or maybe somebody sells these already.