Another AI for help

[maxkarpyza54]

You can use Mistral or Local AI for help

[Gintaras]

When implementing the AI-based documentation search in LibreAutomate, achieving good-quality results was challenging. Were tested many AI embedding and reranking models, but only a few provided sufficiently good results. LibreAutomate currently uses those models. Support for local models was considered too, but not implemented because the available local models did not provide the required combination of quality and simplicity.

[maxkarpyza54]

К моему сожалению эти 2 сервиса у меня не работают. Вот и ищу альтернативу.

[Gintaras]

These non-local AI embedding API were tested and the results are not bad, although not as good as Voyage:
- Mistral. Almost as good. But no reranking API.
- OpenAI. No reranking API.
- Cohere. Has reranking API too. But with LA docs the test results were surprisingly poor.

They can be supported in LA with a simple LA extension. If you can use some of these, I'll show how.

[maxkarpyza54]

Расскажите, пожалуйста.

[Gintaras]

Create this script. Run once. Later, if everything OK, add its name in Options > Workspace > Startup scripts.
 

Code:

Copy      Help

// script "init AI.cs"
/*/ role editorExtension; testInternal Au.Editor; r Au.Editor.dll; /*/
using System.Text.Json.Nodes;
using AI;

AiModel.Models.Add(new ModelMistralEmbed());

record class ModelMistralEmbed : AiEmbeddingModel {
    public ModelMistralEmbed() : base("Mistral", "https://api.mistral.ai/v1/embeddings", "codestral-embed", 1024, "float", new(32000, 256, requestPeriod: 1100)) { }
    //"mistral-embed" (only 1024 dim float), "codestral-embed"
    
    public override object GetPostData(IList<EmInput> input, bool isQuery)
        => new { model, input, output_dimension = dimensions, output_dtype = emType };
    
    public override IEnumerable<JsonNode> GetVectors(JsonNode j)
        => j["data"].AsArray().Select(o => o["embedding"]);
    
    public override int GetTokens(JsonNode j)
        => (int)j["usage"]["total_tokens"];
}

Then in Options > AI you'll see Mistral in the Embedding and API keys combo boxes. Select them, enter your Mistral API key, OK.

Then try to search. The first time it will create AI embedding vectors using the Mistral API. If it is very slow and hangs, probably you use Mistral without subscription. It's free but limited. A workaround of using it without subscription exists.