HopPT offers a complete chatbot experience for any LLM API provider. It is also great to access your local models.
The web search functionality uses serper.dev for searching, but the whole pipeline runs on your device. Your conversations are just between you and your LLM.
We are very proud of the web search performance.
Instructions:
-Add an endpoint in settings (with API Key if required).
-Press Manage Models, select the models from that endpoint that you want to be able to see in the model selector.
-If you want to use web search, add your serper.dev API Key.
-For a very slight improvement in web search answer quality (but a slight hit on context window and speed) you can add a jina.ai API Key to outsource the url scraping to Jina Reader. We recommend using the local WebKit solution.
For an optimal web search experience we advise not to use thinking models when you enable the web search functionality. We also advise to use web search with models that are faster than 20 tokens per second in token generation and 200 token per second in prompt processing speed (web search context is on average 8k tokens, but it can reach 30/40k for complex searches).
The app also has a fully local whisper-v3-large-turbo model that the user has to download and compile in settings. It will stay compiled throughout sessions without occupying RAM space. The transcription functionality is a super cool thing by itself. It's 15x faster than real life speed and is fully local.
You can try a voice-only bonus functionality (for better voice quality download premium voices on your iPhone's settings)
The web search functionality uses serper.dev for searching, but the whole pipeline runs on your device. Your conversations are just between you and your LLM.
We are very proud of the web search performance.
Instructions:
-Add an endpoint in settings (with API Key if required).
-Press Manage Models, select the models from that endpoint that you want to be able to see in the model selector.
-If you want to use web search, add your serper.dev API Key.
-For a very slight improvement in web search answer quality (but a slight hit on context window and speed) you can add a jina.ai API Key to outsource the url scraping to Jina Reader. We recommend using the local WebKit solution.
For an optimal web search experience we advise not to use thinking models when you enable the web search functionality. We also advise to use web search with models that are faster than 20 tokens per second in token generation and 200 token per second in prompt processing speed (web search context is on average 8k tokens, but it can reach 30/40k for complex searches).
The app also has a fully local whisper-v3-large-turbo model that the user has to download and compile in settings. It will stay compiled throughout sessions without occupying RAM space. The transcription functionality is a super cool thing by itself. It's 15x faster than real life speed and is fully local.
You can try a voice-only bonus functionality (for better voice quality download premium voices on your iPhone's settings)
Show More
