colcaroliandClaude Opus 5
97568ba90e
Vectors from two different embedding models are not comparable — Titan v2 has 1024 dimensions where nomic-embed-text has 768 — and _cosine_similarity zips them, so a mismatch produced quietly meaningless scores rather than an error. Nothing forced a re-embed either: sync_local_kb only looked at the content hash, so switching models left every cached vector in place. local_knowledge_docs now records the model each vector came from, and a row is re-embedded when its content hash OR its model differs. Saving a new LOCAL_KB_EMBEDDING_MODEL from the settings panel starts a background pass over every file; the knowledge base panel also has a button for it. While that pass runs, the per-request sync steps aside so no file is embedded twice. The panel now shows what the embedding model is doing: model name, vector size, how many files are embedded with the current model, how many are still on an older one, a progress bar while a pass runs, and the last error. A "接続テスト" button embeds one short string so a broken or unreachable model can be told apart from one that is merely slow to warm up. Also read TAVILY_API_KEY from os.environ at call time. It was imported as a config constant, which is evaluated before load_into_env() runs, so a key saved from the settings panel was stored but never used — not even after a restart. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
46 lines
1.5 KiB
Python
46 lines
1.5 KiB
Python
from sqlalchemy import create_engine, text
|
|
from sqlalchemy.orm import sessionmaker, DeclarativeBase
|
|
from app.core.config import DATABASE_URL
|
|
|
|
engine = create_engine(DATABASE_URL, connect_args={"check_same_thread": False})
|
|
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
|
|
|
|
|
|
class Base(DeclarativeBase):
|
|
pass
|
|
|
|
|
|
def get_db():
|
|
db = SessionLocal()
|
|
try:
|
|
yield db
|
|
finally:
|
|
db.close()
|
|
|
|
|
|
# 既存DBに後から追加されたカラム: (テーブル名, カラム名, 型)
|
|
_ADDED_COLUMNS = [
|
|
("note_pages", "section", "VARCHAR"),
|
|
("knowledge_entries", "headline", "VARCHAR"),
|
|
("messages", "archived_at", "DATETIME"),
|
|
("messages", "memory_id", "INTEGER"),
|
|
("local_knowledge_docs", "embedding_model", "VARCHAR"),
|
|
]
|
|
|
|
|
|
def _add_missing_columns() -> None:
|
|
"""マイグレーションフレームワークを使っていないため、既存DBに後から追加されたカラムを
|
|
ここで簡易的に補う。create_all() は既存テーブルへのカラム追加は行わないので必要。"""
|
|
with engine.connect() as conn:
|
|
for table, column, column_type in _ADDED_COLUMNS:
|
|
existing = {row[1] for row in conn.execute(text(f"PRAGMA table_info({table})"))}
|
|
if existing and column not in existing:
|
|
conn.execute(text(f"ALTER TABLE {table} ADD COLUMN {column} {column_type}"))
|
|
conn.commit()
|
|
|
|
|
|
def init_db():
|
|
from app.db import models # noqa: F401
|
|
Base.metadata.create_all(bind=engine)
|
|
_add_missing_columns()
|