Commit inicial - upload de todos os arquivos da pasta

openspec/changes/categoria/.openspec.yaml

@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-07-24

openspec/changes/categoria/design.md

@@ -0,0 +1,81 @@
+## Context
+
+LerNotaFiscal is a FastAPI app that ingests Brazilian invoices (PDF/image), extracts `supplier_name`, `purchase_date`, `total_paid` via `extract_with_ai` (`app/ai_extraction.py`, OpenAI `chat.completions.create` with vision, JSON mode), with an OCR/heuristic fallback in `ingestion.py` when AI is disabled or fails. Persistence is raw `sqlite3` (`app/database.py`): no ORM, no migrations framework — the `SCHEMA` string is executed via `executescript()` on every connection open, so new tables/columns must be added additively (`CREATE TABLE IF NOT EXISTS`, `ALTER TABLE ... ADD COLUMN` guarded by a `PRAGMA table_info` check, since SQLite's `ADD COLUMN IF NOT EXISTS` isn't available in older syntax). The Dashboard (`app/routes/dashboard_routes.py` + `app/templates/dashboard.html`) currently shows KPIs, a monthly bar chart, and a top-8 supplier bar chart, with no filter controls; filtering by supplier/date exists only on the documents list page (`documents_routes.py`, via `list_fiscal(conn, start, end, supplier)`). There is currently no rate-limiting, retry, or caching layer anywhere in the codebase — AI safety today is a bare `try/except` around the extraction call.
+
+This change adds category classification to fiscal documents, with AI as the primary classifier and a user-maintained keyword table as deterministic fallback, plus safety limits so the new AI call path cannot loop or blow up token spend.
+
+## Goals / Non-Goals
+
+**Goals:**
+- Persist a category per fiscal document, derived automatically (AI first, keyword fallback second, reserved "Não Encontrado" category otherwise — never left unclassified for documents that go through the flow).
+- Let the categorization result be visible and filterable on the Dashboard.
+- Guarantee the AI categorization call can run at most once per document per ingestion, with a supplier-level cache and a per-batch call ceiling, so it can never loop or run away on cost.
+- Keep the `categoria` table simple and editable (id, categoria, palavra_chave) so non-technical users can extend keyword matching without code changes.
+
+**Non-Goals:**
+- No multi-category-per-document (one category per fiscal document in this change).
+- No retraining/fine-tuning of the AI model; categorization uses the existing OpenAI client with a prompt, not a separate ML pipeline.
+- No UI for bulk re-categorization of historical documents (existing documents predating this change simply show "Sem categoria" (`categoria_id` still `NULL`) until reprocessed; a manual "re-run categorization" action is out of scope unless trivial to add in tasks).
+- No category hierarchy/subcategories.
+
+## Decisions
+
+**1. Categorization is a separate step after extraction, not baked into `extract_with_ai`'s prompt.**
+Rationale: `extract_with_ai` already has a fixed JSON schema for fornecedor/data/valor and is also exercised by the OCR-fallback path (where there is no AI call at all). Categorization needs its own safety limits (cache, per-batch ceiling) that are orthogonal to extraction, so it's cleaner as a new module `app/categorization.py` with a function like `categorize_supplier(supplier_name: str, conn, batch_id) -> int`, called from `ingestion.py` right before/at the same time `fiscal_documents` is created. The function always returns a valid `categoria.id` — a real category, or `1` ("Não Encontrado") when AI and keyword matching both fail — never `None`. Alternative considered: extend the extraction prompt to also return a category — rejected because it would apply AI categorization even when the caller only wants OCR fallback, and would entangle unrelated retry/cache logic with extraction.
+
+**2. `categoria_id` lives on `fiscal_documents` (nullable FK), not a separate join table.**
+Rationale: one category per document is sufficient (see Non-Goals); a nullable integer FK matches the existing schema style (`INTEGER PRIMARY KEY AUTOINCREMENT`, snake_case). Added via `ALTER TABLE fiscal_documents ADD COLUMN categoria_id INTEGER REFERENCES categoria(id)`, guarded by a `PRAGMA table_info(fiscal_documents)` check before running the ALTER, consistent with the additive-schema pattern already used in `database.py`. `detected_documents` is left unchanged since categorization only needs to run once a document is confirmed into `fiscal_documents`. The column stays nullable at the schema level purely so pre-existing rows (created before this migration) don't need a backfill — the categorization flow itself never writes `NULL`; it always writes either a real category id or the reserved `1` (see Decision 3).
+
+**3. Reserved row `categoria.id = 1` = "Não Encontrado", auto-ensured, undeletable.**
+Rationale: the user will populate the `categoria` table with their own categories/keywords, but the categorization flow needs a guaranteed, stable target to write to when AI and keyword matching both fail — it cannot depend on the user having remembered to seed anything. So the schema initialization step (the same `executescript`/setup path that creates the `categoria` table) also runs an idempotent `INSERT OR IGNORE INTO categoria (id, categoria, palavra_chave) VALUES (1, 'Não Encontrado', '')`, guaranteeing the row exists on first run without ever overwriting it if the user has since edited its `categoria`/`palavra_chave` values. `delete_categoria` rejects deletion when `categoria_id == 1` (checked before running the delete). Alternative considered: have the app raise/500 if row 1 is missing and require the user to seed it manually first — rejected as too fragile; a one-line idempotent insert removes an entire class of "why isn't categorization working" support issues at negligible cost.
+
+**4. AI categorization is a lightweight, separate OpenAI call scoped to just the supplier name.**
+Rationale: sending only `supplier_name` (plus the list of known category names as allowed options, sourced from distinct `categoria.categoria` values already in the table) keeps the prompt tiny and cheap compared to the vision-based extraction call, and lets us validate the AI's answer against a closed set of categories. If the AI returns a value outside that set (or the call fails), it's treated as "no usable category" and the flow falls to keyword fallback. Alternative considered: let the AI invent free-text categories — rejected, since ungoverned category proliferation would break the keyword table concept and the Dashboard grouping.
+
+**5. Keyword fallback matching is case-insensitive substring match of `palavra_chave` in `supplier_name`, first match wins, excluding the reserved row.**
+Rationale: simplest possible rule that a non-technical user can reason about when populating the `categoria` table; matches the existing `LIKE`-based supplier filter style already in `list_fiscal`. Row order (`id ASC`, `id != 1`) determines precedence when multiple keywords could match — documented so admins can order entries if needed. **Important**: `find_categoria_by_keyword` must exclude `id = 1` from its search. Since the reserved row's `palavra_chave` is normally an empty string, and an empty string is a substring of every value, including it in the ordered scan would make it match first (lowest id) on every call and defeat all real keyword matching. `id = 1` is only ever reached as the final, explicit fallback in `categorize_supplier` when `find_categoria_by_keyword` (over `id != 1`) returns no match — never as a keyword-table hit itself. Alternative considered: token-based/fuzzy matching — deferred as unnecessary complexity for v1.
+
+**6. Safety limits are implemented as a small in-process module, not an external rate-limiter.**
+Rationale: the app has no existing rate-limiting infrastructure and volume is modest (single-user/small-team invoice processing), so a simple in-memory cache dict (`supplier_name -> categoria_id`) plus a per-import-batch counter (reset at the start of each `import_batches` row) is sufficient and avoids new dependencies. Concretely:
+   - **No retry**: the categorization call wraps a single `try/except`, mirroring `extract_with_ai`'s existing pattern — on any exception, treat as "no category from AI" and continue.
+   - **Supplier cache**: keyed by normalized (`strip().upper()`) supplier name, populated the first time a supplier is categorized (whether by AI, keyword, or the `1`/"Não Encontrado" fallback) within the process lifetime; on cache hit, skip the AI call entirely and reuse the cached `categoria_id`. This is an in-memory dict for this change (module-level), acceptable since it degrades gracefully (worst case: re-categorize on process restart, still bounded by the per-batch limit).
+   - **Per-batch ceiling**: a configurable `CATEGORIZATION_MAX_AI_CALLS_PER_BATCH` (default e.g. 50) in `app/config.py`; a counter tied to the current `import_batches.id` is incremented per AI call and checked before each call; once reached, remaining documents in that batch use keyword fallback (then `1`/"Não Encontrado" if no keyword matches) with no further AI calls.
+   - **Logging**: each skip (cache hit or limit reached) logs at INFO/DEBUG with supplier name and reason, using the existing logging setup.
+   Alternative considered: persisting the cache in a DB table — deferred; in-memory is enough to satisfy "no infinite loop / no runaway cost" and keeps the change additive and low-risk.
+
+**7. Dashboard category breakdown reuses the existing chart/query pattern (`supplier_totals`-style function), plus a new `category` query param alongside `start`/`end`.**
+Rationale: `dashboard_routes.py` already computes `monthly_totals` and `supplier_totals` from `fiscal_documents`; a new `category_totals(conn, start, end)` follows the same shape, and threading an optional `category` filter through the existing query functions (`list_fiscal`, `monthly_totals`, `supplier_totals`, new `category_totals`) is consistent with how `supplier` filtering already works. The category filter control is added to the Dashboard template, populated from `list_categorias` (which includes "Não Encontrado") plus a separate "Sem categoria" option for any legacy `categoria_id IS NULL` rows.
+
+**8. Categoria CRUD admin UI mirrors the existing "Documentos" list/form pattern exactly — new files, same conventions, no new UI framework.**
+Rationale: the project already has an established, working pattern for list + create/edit + delete pages (`documents_list.html` + `document_form.html`, driven by `documents_routes.py`), all server-rendered Jinja2, plain HTML forms (POST-only, no JS/AJAX), CSRF via a hidden `csrf_token` field checked with `auth.check_csrf`, and a shared stylesheet (`app/static/styles.css`, classes like `card`, `table`, `table-scroll`, `page-head`, `btn btn-primary/ghost/sm`, `linkbtn danger`, `field`, `filters`). Reusing it exactly (rather than introducing a component library, modal dialogs, or AJAX) keeps the new admin screens visually and behaviorally indistinguishable from the rest of the app. Concretely:
+   - New router `app/routes/categoria_routes.py` (`router = APIRouter()`), imported and registered in `app/main.py` alongside the existing routers, following the same `GET/POST` route-pair convention used for documents:
+     - `GET /categorias` — list (reuses `card`/`table`/`table-scroll` markup)
+     - `GET /categorias/new`, `POST /categorias/new` — create form + handler
+     - `GET /categorias/{categoria_id}/edit`, `POST /categorias/{categoria_id}/edit` — edit form + handler
+     - `POST /categorias/{categoria_id}/delete` — delete (inline per-row form with `onsubmit="return confirm(...)"` and hidden `csrf_token`, exactly like the documents list's delete action)
+   - New templates `app/templates/categorias_list.html` and `app/templates/categoria_form.html`, both `{% extends "base.html" %}`, the form template reused for both create and edit via a `mode` variable (`"new"`/`"edit"`) exactly as `document_form.html` does.
+   - `app/database.py` gains `create_categoria(conn, categoria, palavra_chave)`, `get_categoria(conn, categoria_id)`, `update_categoria(conn, categoria_id, categoria, palavra_chave)`, `delete_categoria(conn, categoria_id)`, alongside the already-planned `list_categorias`/`find_categoria_by_keyword`, matching the naming style of `create_fiscal`/`update_fiscal`.
+   - A new "Categorias" link is added to the nav bar in `base.html` (`<nav class="nav">`), alongside "Dashboard"/"Documentos", using the same active-link `request.url.path.startswith(...)` pattern.
+   - `delete_categoria` first checks `categoria_id == 1` and rejects the delete (flash error, no DB change) per Decision 3's reserved-row protection. Otherwise, deleting a `categoria` that is referenced by existing `fiscal_documents.categoria_id` must not orphan those rows or raise a foreign-key error: `delete_categoria` explicitly runs `UPDATE fiscal_documents SET categoria_id = 1 WHERE categoria_id = ?` (reassigning affected documents to "Não Encontrado") before deleting the `categoria` row, in the same transaction, rather than relying on `ON DELETE SET NULL` — SQLite foreign key enforcement (`PRAGMA foreign_keys`) is off by default and this codebase does not currently enable it, so the app must enforce this itself.
+   Alternative considered: leave orphaned documents' `categoria_id` as `NULL` on delete instead of reassigning to `1` — rejected for consistency: `1` ("Não Encontrado") is now the single canonical "not properly classified" bucket for anything the categorization flow processes or re-processes, while `NULL` is reserved strictly for legacy pre-migration rows never touched by this flow.
+
+## Risks / Trade-offs
+
+- **[Risk]** In-memory supplier cache and per-batch counter are lost on process restart (e.g., app redeploy mid-batch) → could allow a few extra AI calls right after restart. **Mitigation**: the per-batch ceiling is intentionally conservative (default well below any reasonable per-import volume), and the ceiling still applies from the moment the process restarts, bounding worst-case cost; a persistent cache can be added later if needed.
+- **[Risk]** AI-suggested categories drifting from the closed set (typos, casing) could be silently rejected, causing over-reliance on keyword fallback. **Mitigation**: normalize AI output (trim/case-fold) before validating against known categories; log rejected AI suggestions so gaps in the `categoria` table are visible for the admin to fix.
+- **[Risk]** Existing fiscal documents (created before this change) will show "Sem categoria" (`categoria_id` `NULL`) until reprocessed, distinct from "Não Encontrado" (`categoria_id = 1`) used for documents the flow actively tried and failed to classify. **Mitigation**: acceptable for this change (Non-Goal: no bulk re-categorization); documented as an intentional distinction so it isn't mistaken for a bug; can be revisited if users need a backfill.
+- **[Risk]** If the operator edits the reserved row's `categoria`/`palavra_chave` values (e.g. changes `palavra_chave` to something non-empty), it could start matching real suppliers via the keyword fallback path, diluting its meaning as a pure "nothing matched" bucket. **Mitigation**: document that `id = 1` is reserved for "no match" semantics and its `palavra_chave` should normally stay empty; the app does not enforce this beyond the initial seed since the user is expected to manage the table's content.
+- **[Trade-off]** Choosing "first keyword match wins" instead of "most specific match wins" is simpler but could misclassify if keywords overlap (e.g., "MERCADO" matching both a generic and specific entry). Documented as a known limitation; admins should keep keywords reasonably distinct.
+
+## Migration Plan
+
+1. Add `categoria` table (with the idempotent `id = 1` "Não Encontrado" seed insert) and `fiscal_documents.categoria_id` column via additive `executescript`/`ALTER TABLE` in `app/database.py` (guarded by existence checks so it's safe to run against existing databases).
+2. Ship `app/categorization.py` with the AI-then-keyword-then-none flow and safety limits, wired into `ingestion.py` at document confirmation time.
+3. Update `dashboard_routes.py`/`dashboard.html` to add category totals + filter.
+4. Ship `app/routes/categoria_routes.py` + `categorias_list.html`/`categoria_form.html` for CRUD admin management of the `categoria` table, registered in `app/main.py` and linked from `base.html`'s nav.
+5. Deploy is a normal code + schema update; no data backfill required (existing rows simply have `categoria_id = NULL`).
+6. Rollback: revert code; the added column/table can remain harmless if rolled back (nullable, unused), or be dropped manually if desired.
+
+## Open Questions
+
+- Should users be able to manually override/edit a document's category from the documents list UI in this change, or is that a follow-up? (Assumed follow-up unless trivial.)

openspec/changes/categoria/proposal.md

@@ -0,0 +1,37 @@
+## Why
+
+Hoje o LerNotaFiscal extrai fornecedor, data e valor de cada nota fiscal, mas não classifica a despesa por categoria (ex.: Alimentação, Transporte, Saúde). Sem categoria, o usuário não consegue entender para onde o dinheiro está indo pelo Dashboard, apenas por fornecedor ou período. Precisamos categorizar automaticamente cada nota no momento da extração, com uma regra de negócio clara (AI primeiro, palavra-chave como fallback) e sem risco de gerar custo descontrolado de tokens de IA.
+
+## What Changes
+
+- Criar tabela `categoria` (`id`, `categoria`, `palavra_chave`) para mapear palavras-chave a categorias, usada como fallback e administrável pelo usuário. O registro `id = 1` é reservado pelo sistema para a categoria "Não Encontrado" e é garantido automaticamente na inicialização do schema; o usuário é responsável por cadastrar as demais categorias/palavras-chave pela tela de CRUD.
+- Adicionar coluna `categoria_id` (FK para `categoria`) em `fiscal_documents` para persistir a classificação. Documentos processados pelo novo fluxo sempre recebem um `categoria_id` válido (nunca ficam nulos); apenas documentos já existentes antes desta mudança (não reprocessados) permanecem com `categoria_id` nulo ("Sem categoria").
+- Implementar fluxo de categorização automática:
+  1. Ao processar uma nota, pedir à IA (mesma chamada de extração ou chamada dedicada) para sugerir a categoria a partir do nome do fornecedor.
+  2. Se a IA não retornar uma categoria válida/reconhecida, buscar na tabela `categoria` por correspondência de `palavra_chave` no `supplier_name` e usar a `categoria` encontrada.
+  3. Se a IA não conseguir categorizar **e** nenhuma `palavra_chave` corresponder, gravar `categoria_id = 1` ("Não Encontrado") sem bloquear o processamento.
+- Ajustar o Dashboard (`app/templates/dashboard.html` + `dashboard_routes.py`) para exibir gastos agrupados por categoria (gráfico/lista) e permitir filtrar os dados exibidos por categoria, na mesma linha dos filtros de fornecedor/período já existentes na tela de documentos.
+- Criar tela administrativa de CRUD (Create, Read, Update, Delete) para a tabela `categoria`, seguindo exatamente o mesmo layout/padrão já usado em "Documentos" (`documents_list.html` + `document_form.html`, mesmo `base.html`, mesmas classes CSS de `app/static/styles.css`, mesmo padrão de rotas `GET/POST /categorias`, `/categorias/new`, `/categorias/{id}/edit`, `POST /categorias/{id}/delete` com CSRF), para que o usuário possa cadastrar/editar/remover categorias e palavras-chave sem precisar de acesso direto ao banco.
+- Criar mecanismo de segurança contra loop infinito / consumo excessivo de tokens ao acionar a IA para categorização:
+  - Limite de tentativas de chamada de IA por nota (ex.: no máximo 1 tentativa de categorização por nota, sem retry automático).
+  - Cache/memória de categorização por fornecedor (se já categorizamos "Fornecedor X" antes, não chamar a IA de novo — reusar resultado ou usar a tabela `categoria`).
+  - Circuit breaker/limite global (ex.: máximo de N chamadas de categorização por lote de importação ou por janela de tempo), com log e interrupção segura (fallback para palavra-chave) ao atingir o limite.
+
+## Capabilities
+
+### New Capabilities
+- `expense-categorization`: Classificação automática de notas fiscais por categoria, com IA como fonte primária e a tabela `categoria` (palavra-chave) como fallback determinístico.
+- `categorization-safety-limits`: Limites e proteções (retries, cache, circuit breaker) para chamadas de IA usadas na categorização, evitando loops e consumo excessivo de tokens.
+
+### Modified Capabilities
+- (nenhuma capability existente com spec.md hoje — projeto ainda não possui specs em `openspec/specs/`)
+
+## Impact
+
+- **Banco de dados** (`app/database.py`): nova tabela `categoria`; nova coluna `categoria_id` em `fiscal_documents` (e possivelmente `detected_documents`); nova migração aditiva no `SCHEMA`/`executescript`.
+- **IA** (`app/ai_extraction.py` ou novo módulo `app/categorization.py`): nova função/chamada para sugerir categoria a partir do `supplier_name`; ajuste no fluxo de `extract_with_ai` ou chamada adicional pós-extração.
+- **Ingestão** (`ingestion.py`): aplicar a lógica de categorização (IA → palavra-chave → `categoria_id = 1` "Não Encontrado") ao confirmar/criar `fiscal_documents`.
+- **Rotas/Dashboard** (`app/routes/dashboard_routes.py`, `app/templates/dashboard.html`): novo agrupamento e filtro por categoria.
+- **Nova rota de administração** (`app/routes/categoria_routes.py`, registrada em `app/main.py`; novos templates `app/templates/categorias_list.html` e `app/templates/categoria_form.html`, reaproveitando `base.html`): CRUD completo da tabela `categoria`, no mesmo padrão das rotas/telas de "Documentos".
+- **Rotas de documentos** (`app/routes/documents_routes.py`): opcionalmente permitir filtrar/editar categoria manualmente por nota.
+- **Configuração** (`app/config.py`): novos parâmetros para limites de segurança (ex.: `CATEGORIZATION_MAX_CALLS_PER_BATCH`, cache TTL).

openspec/changes/categoria/specs/categorization-safety-limits/spec.md

@@ -0,0 +1,40 @@
+## ADDED Requirements
+
+### Requirement: Single AI Attempt Per Document
+The system SHALL make at most one AI categorization call per fiscal document per ingestion attempt and SHALL NOT automatically retry a failed AI categorization call.
+
+#### Scenario: AI categorization call fails
+- **WHEN** the AI categorization call for a document errors or times out
+- **THEN** the system does not retry the AI call for that document and proceeds directly to keyword fallback
+
+### Requirement: Supplier Categorization Cache
+The system SHALL cache the category determined for a given supplier name (in-memory or persisted) and SHALL reuse the cached category for subsequent documents from the same supplier instead of calling the AI again.
+
+#### Scenario: Second document from a known supplier
+- **WHEN** a fiscal document is processed for a supplier name that already has a cached category from a prior categorization
+- **THEN** the system uses the cached category and does not issue a new AI categorization call
+
+#### Scenario: Cache miss for a new supplier
+- **WHEN** a fiscal document is processed for a supplier name with no cached category
+- **THEN** the system proceeds with the normal AI-then-keyword categorization flow and stores the result in the cache
+
+### Requirement: Per-Batch AI Call Limit
+The system SHALL enforce a configurable maximum number of AI categorization calls within a single import batch (or time window). Once the limit is reached, remaining documents in that batch SHALL be categorized using only the keyword fallback, with no further AI calls, until the batch/window resets.
+
+#### Scenario: Limit reached mid-batch
+- **WHEN** the number of AI categorization calls in the current batch reaches the configured maximum
+- **THEN** subsequent documents in the same batch skip the AI call and go directly to keyword fallback (or "Sem categoria" if no keyword matches)
+
+### Requirement: Configurable Safety Limits
+The maximum AI calls per batch and cache behavior SHALL be configurable via application configuration/environment variables, not hardcoded in the categorization logic.
+
+#### Scenario: Operator changes the configured limit
+- **WHEN** the configured maximum AI calls per batch is changed
+- **THEN** the categorization flow honors the new limit on the next run without code changes
+
+### Requirement: Observability of Skipped AI Calls
+When an AI categorization call is skipped due to the cache or the per-batch limit, the system SHALL log the reason (cache hit or limit reached) so the behavior is observable and auditable.
+
+#### Scenario: Skip logged
+- **WHEN** the system skips an AI categorization call because of a cache hit or because the batch limit was reached
+- **THEN** a log entry is recorded indicating which reason caused the skip and for which document/supplier

openspec/changes/categoria/specs/expense-categorization/spec.md

@@ -0,0 +1,113 @@
+## ADDED Requirements
+
+### Requirement: Categoria Table Schema
+The system SHALL provide a `categoria` table with fields `id`, `categoria`, and `palavra_chave`, used to map keywords to category names as a fallback for automatic classification. The row with `id = 1` is reserved by the system for the "Não Encontrado" category and SHALL always exist; the system SHALL ensure this row is present (creating it if missing) during schema initialization, independent of any other categories the user registers.
+
+#### Scenario: Table created on schema initialization
+- **WHEN** the application initializes or migrates the database schema
+- **THEN** the `categoria` table exists with columns `id`, `categoria`, `palavra_chave`, and a row with `id = 1` and `categoria = "Não Encontrado"` is present
+
+#### Scenario: Reserved row already present
+- **WHEN** the application initializes and a `categoria` row with `id = 1` already exists (e.g., the user has customized its `categoria`/`palavra_chave` values)
+- **THEN** the system does not overwrite or duplicate that row
+
+### Requirement: Automatic Categorization on Ingestion
+The system SHALL attempt to determine the category of a fiscal document automatically at the moment it is confirmed/created, without requiring manual user input.
+
+#### Scenario: New fiscal document confirmed triggers categorization
+- **WHEN** a detected document is confirmed and a `fiscal_documents` row is created
+- **THEN** the system runs the categorization flow (AI, then keyword fallback) before finishing the request
+
+### Requirement: AI-Based Categorization by Supplier Name
+The system SHALL first attempt to categorize a fiscal document by sending the supplier name to the AI service and requesting a category classification.
+
+#### Scenario: AI returns a valid category
+- **WHEN** the AI service returns a recognized, non-empty category for the supplier name
+- **THEN** the system stores that category on the fiscal document and does not consult the `categoria` keyword table
+
+#### Scenario: AI call fails or returns no usable category
+- **WHEN** the AI call raises an error, times out, or returns an empty/unrecognized value
+- **THEN** the system proceeds to keyword fallback categorization instead of failing the request
+
+### Requirement: Keyword Fallback Categorization
+When AI categorization does not produce a valid category, the system SHALL search the `categoria` table for a `palavra_chave` that matches (case-insensitive, substring) the supplier name, and use the corresponding `categoria` value.
+
+#### Scenario: Keyword match found
+- **WHEN** AI categorization did not yield a category and a `categoria.palavra_chave` is found as a substring of the supplier name (case-insensitive)
+- **THEN** the system assigns the matching `categoria.categoria` value to the fiscal document
+
+#### Scenario: No keyword match found
+- **WHEN** AI categorization did not yield a category and no `palavra_chave` matches the supplier name
+- **THEN** the system assigns `categoria_id = 1` ("Não Encontrado") to the fiscal document instead of guessing
+
+### Requirement: Fallback to Reserved "Não Encontrado" Category
+When neither AI nor keyword matching determines a category for a fiscal document being processed by the categorization flow, the system SHALL persist that document with `categoria_id = 1` (the reserved "Não Encontrado" category) and SHALL NOT block or fail document processing.
+
+#### Scenario: No category determined by any method
+- **WHEN** AI categorization fails/is inconclusive and no keyword match exists for a document going through the categorization flow
+- **THEN** the fiscal document is saved successfully with `categoria_id = 1` and shown as "Não Encontrado" in the UI
+
+#### Scenario: Legacy documents predating this change remain distinct from "Não Encontrado"
+- **WHEN** a `fiscal_documents` row was created before this change and has never been run through the categorization flow
+- **THEN** its `categoria_id` remains `NULL` and it is shown as "Sem categoria" (distinct from the explicit "Não Encontrado" outcome), until it is reprocessed
+
+### Requirement: Dashboard Category Breakdown
+The Dashboard SHALL display expenses aggregated by category (e.g., totals per category) alongside existing period and supplier breakdowns.
+
+#### Scenario: Dashboard shows category totals
+- **WHEN** a user opens the Dashboard
+- **THEN** the page displays total spend grouped by category, including the "Não Encontrado" group (`categoria_id = 1`) for documents the categorization flow could not classify, and a separate "Sem categoria" group for any legacy documents with `categoria_id` still `NULL`
+
+### Requirement: Dashboard Category Filter
+The Dashboard SHALL allow the user to filter the displayed expenses by a single selected category.
+
+#### Scenario: User filters by category
+- **WHEN** the user selects a category from the Dashboard's category filter
+- **THEN** all Dashboard figures (KPIs, charts, recent documents) update to reflect only fiscal documents in that category
+
+#### Scenario: User clears the category filter
+- **WHEN** the user clears the selected category filter
+- **THEN** the Dashboard returns to showing all fiscal documents regardless of category
+
+### Requirement: Categoria Administration List
+The system SHALL provide an authenticated admin page listing all rows of the `categoria` table (`id`, `categoria`, `palavra_chave`), following the same page layout, table markup, and CSS classes already used by the existing "Documentos" list page.
+
+#### Scenario: User views the categoria list
+- **WHEN** an authenticated user navigates to the categorias admin page
+- **THEN** the system displays all `categoria` rows in a table matching the existing list-page layout (`card`/`table`/`table-scroll` styling), with actions to create, edit, and delete a row
+
+### Requirement: Categoria Creation
+The system SHALL allow an authenticated user to create a new `categoria` row (`categoria`, `palavra_chave`) via a form that follows the same structure, validation, and CSRF protection as the existing document create/edit form.
+
+#### Scenario: User creates a new categoria
+- **WHEN** an authenticated user submits the "new categoria" form with a non-empty `categoria` and `palavra_chave`
+- **THEN** the system inserts a new row into the `categoria` table and redirects to the categoria list showing the new entry
+
+#### Scenario: User submits an invalid categoria form
+- **WHEN** an authenticated user submits the "new categoria" form with a missing `categoria` or `palavra_chave`
+- **THEN** the system re-displays the form with a validation error and does not create a row
+
+### Requirement: Categoria Update
+The system SHALL allow an authenticated user to edit an existing `categoria` row's `categoria` and `palavra_chave` values, reusing the same form template pattern used for creation (single form, `mode` toggling between new/edit).
+
+#### Scenario: User edits an existing categoria
+- **WHEN** an authenticated user submits the edit form for an existing `categoria` row with valid values
+- **THEN** the system updates that row and redirects to the categoria list reflecting the new values
+
+### Requirement: Categoria Deletion
+The system SHALL allow an authenticated user to delete an existing `categoria` row (other than the reserved `id = 1` row) via a CSRF-protected POST action, following the same inline delete-form-with-confirmation pattern used on the existing "Documentos" list page.
+
+#### Scenario: User deletes a categoria
+- **WHEN** an authenticated user confirms deletion of a `categoria` row with `id != 1`
+- **THEN** the system removes that row from the `categoria` table and redirects to the categoria list without it
+
+#### Scenario: Deleting a categoria referenced by fiscal documents
+- **WHEN** an authenticated user deletes a `categoria` row that is currently referenced by one or more `fiscal_documents.categoria_id`
+- **THEN** the system completes the deletion and reassigns those fiscal documents' `categoria_id` to `1` ("Não Encontrado"), without errors or orphaned references
+
+### Requirement: Reserved Categoria Cannot Be Deleted
+The system SHALL prevent deletion of the `categoria` row with `id = 1` ("Não Encontrado"), since it is the required fallback target for automatic categorization.
+
+#### Scenario: User attempts to delete the reserved categoria
+- **WHEN** an authenticated user attempts to delete the `categoria` row with `id = 1`
+- **THEN** the system rejects the deletion, shows an error message, and the row remains unchanged

openspec/changes/categoria/tasks.md

@@ -0,0 +1,44 @@
+## 1. Schema: tabela `categoria` e coluna `categoria_id`
+
+- [x] 1.1 Adicionar `categoria` (`id INTEGER PRIMARY KEY AUTOINCREMENT`, `categoria TEXT NOT NULL`, `palavra_chave TEXT NOT NULL`) ao `SCHEMA`/`executescript` em `app/database.py` (`CREATE TABLE IF NOT EXISTS`)
+- [x] 1.2 Adicionar coluna `categoria_id INTEGER REFERENCES categoria(id)` (nullable) em `fiscal_documents`, com checagem via `PRAGMA table_info(fiscal_documents)` antes do `ALTER TABLE` para não falhar em bancos já existentes
+- [x] 1.3 Adicionar insert idempotente `INSERT OR IGNORE INTO categoria (id, categoria, palavra_chave) VALUES (1, 'Não Encontrado', '')` no mesmo passo de inicialização do schema, garantindo que a linha reservada `id = 1` sempre exista sem nunca sobrescrever edições feitas pelo usuário
+- [x] 1.4 Adicionar helpers em `app/database.py`: `create_categoria(conn, categoria, palavra_chave)`, `get_categoria(conn, categoria_id)`, `list_categorias(conn)`, `update_categoria(conn, categoria_id, categoria, palavra_chave)`, `delete_categoria(conn, categoria_id)` (deve rejeitar `categoria_id == 1` sem alterar nada; caso contrário, fazer `UPDATE fiscal_documents SET categoria_id = 1 WHERE categoria_id = ?` antes de excluir a linha, na mesma transação), `find_categoria_by_keyword(conn, supplier_name)` (busca case-insensitive de `palavra_chave` como substring de `supplier_name`, **excluindo `id = 1`**, ordenado por `id ASC`, retorna a primeira correspondência — importante: `palavra_chave` vazia da linha reservada é substring de qualquer texto, então incluí-la quebraria o casamento de palavras-chave reais)
+
+## 2. Módulo de categorização com limites de segurança
+
+- [x] 2.1 Criar `app/categorization.py` com função `categorize_supplier(supplier_name: str, conn, batch_id) -> int` (sempre retorna um `categoria.id` válido, nunca `None`) implementando a ordem: cache em memória → limite por lote → IA → palavra-chave → `1` ("Não Encontrado")
+- [x] 2.2 Implementar cache em memória por fornecedor normalizado (`strip().upper()`), populado após qualquer categorização (IA, palavra-chave, ou `1` quando nada é encontrado), consultado antes de qualquer chamada de IA
+- [x] 2.3 Implementar contador de chamadas de IA por `import_batches.id`, comparado a `settings.CATEGORIZATION_MAX_AI_CALLS_PER_BATCH`; ao atingir o limite, pular direto para o fallback de palavra-chave (e depois `1` se nada corresponder)
+- [x] 2.4 Implementar chamada de IA dedicada (OpenAI `chat.completions.create`, prompt curto com `supplier_name` + lista de categorias já existentes na tabela `categoria`), validando que a resposta pertence ao conjunto conhecido; qualquer exceção ou valor fora do conjunto é tratado como "sem categoria da IA" (sem retry)
+- [x] 2.5 Adicionar `CATEGORIZATION_MAX_AI_CALLS_PER_BATCH` (default configurável) em `app/config.py`
+- [x] 2.6 Adicionar logs (INFO/DEBUG) quando uma chamada de IA é pulada por cache hit ou por limite de lote atingido, incluindo fornecedor e motivo
+
+## 3. Integração no fluxo de ingestão
+
+- [x] 3.1 Chamar `categorize_supplier` no momento em que o `fiscal_documents` é criado/confirmado, persistindo o `categoria_id` retornado (sempre um valor válido, no mínimo `1`) — implementado em `database.py::confirm_batch` (não em `ingestion.py`: a criação/confirmação de `fiscal_documents` acontece em `database.py`, `ingestion.py` só extrai)
+- [x] 3.2 Garantir que falha na categorização (qualquer exceção não tratada dentro do módulo) não impede a criação do `fiscal_documents` — a nota deve ser salva com `categoria_id = 1` ("Não Encontrado") nesse caso
+
+## 4. Dashboard: exibição e filtro por categoria
+
+- [x] 4.1 Criar função `category_totals(conn, start, end)` em `app/database.py` (mesmo padrão de `monthly_totals`/`supplier_totals`), agrupando por `categoria.categoria` (via LEFT JOIN, incluindo a linha `id = 1` "Não Encontrado") e um grupo separado "Sem categoria" apenas para `categoria_id IS NULL` (documentos legados não reprocessados)
+- [x] 4.2 Adicionar parâmetro opcional `category` em `dashboard_routes.py`, propagando o filtro para `monthly_totals`, `supplier_totals`, `category_totals` e demais consultas da página
+- [x] 4.3 Atualizar `app/templates/dashboard.html` com um gráfico/lista de gastos por categoria e um controle de filtro (select) populado a partir de `list_categorias` (inclui "Não Encontrado") + opção "Sem categoria" para os registros legados
+- [x] 4.4 Validado (sem extensão de navegador disponível na sessão: verificado via requisições HTTP autenticadas + inspeção do HTML renderizado, e via chamadas diretas às funções de banco simulando um lote confirmado) — Dashboard exibe totais por categoria e o filtro reduz corretamente KPIs/gráficos/lista de documentos recentes
+
+## 5. CRUD administrativo da tabela `categoria`
+
+- [x] 5.1 Criar `app/routes/categoria_routes.py` com `router = APIRouter()` e as rotas `GET /categorias`, `GET/POST /categorias/new`, `GET/POST /categorias/{categoria_id}/edit`, `POST /categorias/{categoria_id}/delete`, seguindo exatamente o padrão de `documents_routes.py` (uso de `render()`, `flash()`, `auth.check_csrf`)
+- [x] 5.2 Registrar `categoria_routes` em `app/main.py` (import + `app.include_router(categoria_routes.router)`), no mesmo bloco onde os demais routers são registrados
+- [x] 5.3 Criar `app/templates/categorias_list.html` (`{% extends "base.html" %}`) reaproveitando o layout de `documents_list.html`: `.page-head` com botão "+ Nova", tabela (`card`/`table`/`table-scroll`) listando `id`, `categoria`, `palavra_chave`, e ações de editar/excluir por linha (form inline com `onsubmit="return confirm(...)"` e `csrf_token` oculto, igual ao padrão de exclusão de documentos); a linha `id = 1` ("Não Encontrado") mostra a ação de excluir desabilitada/oculta, já que a exclusão é sempre rejeitada
+- [x] 5.4 Criar `app/templates/categoria_form.html` (`{% extends "base.html" %}`) reaproveitando o layout de `document_form.html`, com variável `mode` (`"new"`/`"edit"`) controlando a `action` do form entre `/categorias/new` e `/categorias/{id}/edit`
+- [x] 5.5 Adicionar link "Categorias" na `<nav class="nav">` de `app/templates/base.html`, com a mesma lógica de classe `active` (`request.url.path.startswith('/categorias')`) usada pelos demais links
+- [x] 5.6 Validado (sem extensão de navegador disponível na sessão: driver via requisições HTTP autenticadas contra o servidor real, cookies de sessão + CSRF) o fluxo completo: criar, listar, editar e excluir uma categoria; confirmado que excluir a categoria `id = 1` é rejeitado, e que excluir uma categoria em uso reatribui os documentos relacionados para "Não Encontrado" sem erro
+
+## 6. Testes e validação
+
+- [x] 6.1 Testes unitários para `find_categoria_by_keyword` (match, no match, case-insensitive, múltiplos candidatos → primeiro por `id`, e confirmando que `id = 1` nunca é retornado por essa função mesmo com `palavra_chave` vazia)
+- [x] 6.2 Testes unitários para `categorize_supplier` cobrindo: cache hit (sem chamada de IA), limite de lote atingido (sem chamada de IA), IA retorna categoria válida, IA falha/retorna inválido → fallback por palavra-chave, nenhum método encontra categoria → retorna `1`
+- [x] 6.3 Teste de integração do fluxo de ingestão ponta a ponta: nota processada resulta em `fiscal_documents.categoria_id` correto nos cenários de IA, fallback por palavra-chave e "Não Encontrado" (`1`)
+- [x] 6.4 Testes unitários/integração para o CRUD de `categoria`: criar, ler, atualizar, excluir (incluindo excluir categoria referenciada por `fiscal_documents` → reatribuição para `1`, e tentar excluir `id = 1` → rejeitado)
+- [x] 6.5 Automatizado como equivalente ao teste manual (`test_repeated_suppliers_call_ai_once_each_and_respect_batch_limit` em `tests/test_categorization.py`, usando mock + `assertLogs`): lote com fornecedores repetidos confirma que a IA é chamada apenas uma vez por fornecedor distinto e que o limite por lote é respeitado