Re-embed the knowledge base when the embedding model changes

Vectors from two different embedding models are not comparable — Titan v2 has
1024 dimensions where nomic-embed-text has 768 — and _cosine_similarity zips
them, so a mismatch produced quietly meaningless scores rather than an error.
Nothing forced a re-embed either: sync_local_kb only looked at the content
hash, so switching models left every cached vector in place.

local_knowledge_docs now records the model each vector came from, and a row is
re-embedded when its content hash OR its model differs. Saving a new
LOCAL_KB_EMBEDDING_MODEL from the settings panel starts a background pass over
every file; the knowledge base panel also has a button for it. While that pass
runs, the per-request sync steps aside so no file is embedded twice.

The panel now shows what the embedding model is doing: model name, vector
size, how many files are embedded with the current model, how many are still
on an older one, a progress bar while a pass runs, and the last error. A
"接続テスト" button embeds one short string so a broken or unreachable model
can be told apart from one that is merely slow to warm up.

Also read TAVILY_API_KEY from os.environ at call time. It was imported as a
config constant, which is evaluated before load_into_env() runs, so a key
saved from the settings panel was stored but never used — not even after a
restart.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-23 22:56:12 +09:00
co-authored by Claude Opus 5
parent 52dfc0c349
commit 97568ba90e
14 changed files with 550 additions and 41 deletions
+14 -4
View File
@@ -1,13 +1,22 @@
import logging
import os
import httpx
from app.core.config import TAVILY_API_KEY
logger = logging.getLogger(__name__)
TAVILY_SEARCH_URL = "https://api.tavily.com/search"
def _api_key() -> str | None:
"""GUIの設定画面から実行時に変更できるよう、起動時定数ではなく都度 os.environ を見に行く。
app.core.config の定数は import 時に評価されるため、DBに保存した値を
settings_store.load_into_env() が os.environ に入れても反映されない。
"""
return os.environ.get("TAVILY_API_KEY") or None
WEB_SEARCH_TOOL = {
"type": "function",
"function": {
@@ -27,7 +36,8 @@ WEB_SEARCH_TOOL = {
async def search_web(query: str, max_results: int = 5) -> list[dict]:
"""Tavily APIでWeb検索する。結果は [{title, url, content}, ...]。失敗時は空リスト。"""
if not TAVILY_API_KEY:
api_key = _api_key()
if not api_key:
logger.warning("TAVILY_API_KEY is not set; skipping web search")
return []
@@ -36,7 +46,7 @@ async def search_web(query: str, max_results: int = 5) -> list[dict]:
response = await client.post(
TAVILY_SEARCH_URL,
json={
"api_key": TAVILY_API_KEY,
"api_key": api_key,
"query": query,
"max_results": max_results,
},