008 Open Source WebRTC softphone with AI

DavidGOrtega · March 12, 2024, 2:34pm

Hi there!

I would like you to introduce our Open source webRTC softphone with AI included https://008agent.ai/

Features:

Open source
Event based. We deliver the events and recordings to the desired web-hook
AI. We offer right now STT and summarisation with our small embedded LLM trained in call-center conversations.
HTML and electron

It’s totally free and we hope the community would benefit from it.
We are eager to hear feedback from you guys and help you as much as you may need.

Thanks

penguinpbx · March 12, 2024, 4:00pm

Looks interesting!

Questions about this 92.3MB Web Assembler binary (?), which looks like it is used as the language model locally on the client (?):

https://kunziteq.s3.gra.perf.cloud.ovh.net/ttsb.bin

…which appears to be downloaded by:

github.com

kunzite-app/008/blob/master/packages/008Q/src/Q.js

import * as whisper from "whisper-webgpu";
import toWav from "audiobuffer-to-wav";

import { InferenceSession, Tensor } from "onnxruntime-web/webgpu";
import * as webllm from "@mlc-ai/web-llm";

const S3Q = "https://kunziteq.s3.gra.perf.cloud.ovh.net";

export const fetchBytes = async ({ url, cache = true, onProgress }) => {
  const cache_ = await caches.open("008Q");
  const response = await cache_.match(url);

  if (cache && response) {
    const buffer = await response.arrayBuffer();
    return new Uint8Array(buffer);
  }

  return new Promise((resolve, reject) => {
    const request = new XMLHttpRequest();
    request.open("GET", url, true);

This file has been truncated. show original

…is there source code available for it ?

Tried a few inspection tools (wasm-decompile, wasm-validate, wasm2c) and kept getting:

0000004: error: bad magic value

DavidGOrtega · March 12, 2024, 4:41pm

@penguinpbx thanks for your interest!

The language models runs within the softphone.
What its being downloaded are the weights of the AI models.
We use Whisper for TTS and Phi-2 for our LLM.

The model that you are asking for is the TTS Whisper. The format is GGUF. We are moving them to Huggingface soon.

Our LLM has been also specially trained for call-center conversations summarization. We are releasing paper and code soon.

system · April 11, 2024, 4:41pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.