Màj du mode d'emploi de Tableau #42

- Amélioration des permormances de l'observatoire
- Possibilité dans observatoire (champ objet) et tableau (tous champs texte) de soit chercher des mots présents, soit une suite de mot précise (voir mode d'emploi dans Tableau)
- Ajout d'une animation pendant le chargement de la prévisualisation des données de l'observatoire

.github/workflows/deploy.yaml

@@ -20,7 +20,7 @@ jobs:
     environment: ${{ github.ref_name }}
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
 
       - name: Set up SSH key
         run: |

.template.env

@@ -1,4 +1,5 @@
 DATA_FILE_PARQUET_PATH=https://www.data.gouv.fr/fr/datasets/r/11cea8e8-df3e-4ed1-932b-781e2635e432
+DUCKDB_PATH=./decp.duckdb
 PORT=8050
 DEVELOPMENT=True
 SOURCE_STATS_CSV_PATH="https://www.data.gouv.fr/api/1/datasets/r/8ded94de-3b80-4840-a5bb-7faad1c9c234"

CHANGELOG.md

@@ -1,3 +1,13 @@
+##### 2.7.5 (24 avril 2026)
+
+- Amélioration des permormances de l'observatoire
+- Possibilité dans observatoire (champ objet) et tableau (tous champs texte) de soit chercher des mots présents, soit une suite de mot précise (voir mode d'emploi dans Tableau)
+- Ajout d'une animation pendant le chargement de la prévisualisation des données de l'observatoire
+
+##### 2.7.4 (22 avril 2026)
+
+- Utilisation élargie de DuckDB au détriment de Polars => bien meilleure perf ([#72](https://github.com/ColinMaudry/decp.info/issues/72)
+
 ##### 2.7.3 (20 avril 2026)
 
 - Mise en cache des vues tableau par ensemble de filtres et de tris

CLAUDE.md

@@ -10,27 +10,38 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 ### Setup
 
+Setting up the virtual environment:
+
 ```bash
-cp template.env .env   # then customize .env
+python -m venv .venv # s'il n'existe pas déjà
+source .venv/bin/activate
+rtk pip install -U pip > /dev/null 2>&1
+rtk pip install -e . --group=dev
+```
+
+Environment variables:
+
+```bash
+cp .template.env .env   # then customize .env
 ```
 
 ### Development
 
 ```bash
-uv run run.py         # starts Dash with debug=True and hot reload
+python run.py         # starts Dash app
 ```
 
 ### Production
 
 ```bash
-uv run gunicorn app:server
+gunicorn app:server
 ```
 
 ### Tests
 
 ```bash
-rtk uv run pytest                  # run all tests (Selenium-based integration tests)
-rtk uv run pytest tests/test_main.py::test_001_logo_and_search   # run a single test
+rtk pytest                  # run all tests (some are Selenium-based integration tests)
+rtk pytest tests/test_main.py::test_001_logo_and_search   # run a single test
 ```
 
 Tests require a running Chrome/Chromium browser. They use `DashComposite` from `dash[testing]` with Selenium WebDriver.
@@ -40,12 +51,12 @@ Tests require a running Chrome/Chromium browser. They use `DashComposite` from `
 ### Multi-page Dash app
 
 - `src/app.py` — creates the Dash app instance, navbar, SEO endpoints (robots.txt, sitemap.xml), Matomo analytics
-- `src/pages/*.py` — each page registers itself with `@register_page()` and owns its own layout and callbacks
+- `src/pages/*.py` — each page registers itself with `@register_page()` and o.wns its own layout and callbacks
 - `run.py` — dev entry point; exports `server` (Flask) for gunicorn
 
 ### Module imports
 
-- always import modules from the app starting with `src.` (e.g. `src.utils.`, `src.pages.recherche`, etc.)
+- always import modules from the app starting with `src.` (e.g. `src.utils.`, `src.pages.recherche`, etc.), NOT `utils.cache` or `pages.observatoire`.
 
 ### Key pages
 
@@ -60,9 +71,9 @@ Tests require a running Chrome/Chromium browser. They use `DashComposite` from `
 
 ### Data layer
 
-- Data is stored as **Parquet** and loaded with **Polars** (fast columnar operations)
+- Data is stored as **Parquet** at rest, possibly in DuckDB, loaded in DuckDB, served from DuckDB for big queries and manipulated with **Polars** for the remaining steps
 - Path set via `DATA_FILE_PARQUET_PATH` env var; tests use `tests/test.parquet`
-- `src/utils.py` — filtering helpers, search (`search_org`), link generation, geographic data loading
+- `src/util/*.py` — helpers shared by other modules, search (`search_org`), link generation, geographic data loading
 - `src/callbacks.py` — shared Dash callbacks (e.g. `get_top_org_table`)
 - `src/figures.py` — chart and map components (Plotly Express, Dash Leaflet with marker clustering)
 - a Parquet file with production data is located at `../decp-processing/decp_prod.parquet` (~ 1,5 million records)

README.md

@@ -1,6 +1,5 @@
 # decp.info
 
-> v2.7.3
 > Outil d'exploration et de téléchargement des données essentielles de la commande publique.
 
 => [decp.info](https://decp.info)

docs/superpowers/plans/2026-04-22-observatoire-duckdb-filters.md

@@ -0,0 +1,951 @@
+# Observatoire — filtrage natif DuckDB — Plan d'implémentation
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Remplacer le filtrage Polars sur LazyFrame dans `prepare_dashboard_data` par un requêtage natif DuckDB, pour ne matérialiser que le sous-ensemble utile au lieu de l'intégralité de la table `decp` (~1,5 M lignes).
+
+**Architecture:** Nouveau helper pur `dashboard_filters_to_sql(**filter_params) -> (where_sql, params)` dans `src/utils/table_sql.py` (modèle de `filter_query_to_sql`). `prepare_dashboard_data` devient une fonction fine qui appelle `query_marches(where_sql, params)` et retourne une `pl.DataFrame`. Les 3 appelants dans `src/pages/observatoire.py` sont adaptés à la nouvelle signature.
+
+**Tech Stack:** Python 3.12, Polars, DuckDB, Dash, pytest.
+
+**Spec:** `docs/superpowers/specs/2026-04-22-observatoire-duckdb-filters-design.md`.
+
+---
+
+## File Structure
+
+**À créer :**
+
+- `tests/test_dashboard_filters_to_sql.py` — tests unitaires du nouveau helper SQL (cas vide + cas par filtre).
+- `tests/test_prepare_dashboard_data.py` — test d'intégration léger (appel DuckDB réel sur `tests/test.parquet`).
+
+**À modifier :**
+
+- `src/utils/table_sql.py` — ajouter `dashboard_filters_to_sql` + import `datetime`/`timedelta`.
+- `src/utils/data.py` — réécrire `prepare_dashboard_data` (signature et implémentation), ajouter `query_marches` aux imports `from src.db`.
+- `src/pages/observatoire.py` — adapter 3 sites d'appel (lignes ~668, ~791, ~882) ; retirer `query_marches` de l'import `from src.db` (plus utilisé).
+- `tests/test_main.py` — supprimer `test_010_observatoire_montant_filter` (migré en test unitaire du helper).
+
+---
+
+## Task 1: Tests unitaires — cas par défaut + filtre année
+
+**Files:**
+
+- Create: `tests/test_dashboard_filters_to_sql.py`
+- Modify: `src/utils/table_sql.py`
+
+- [ ] **Step 1: Write the failing tests**
+
+Create `tests/test_dashboard_filters_to_sql.py`:
+
+```python
+from datetime import datetime, timedelta
+
+from src.utils.table_sql import dashboard_filters_to_sql
+
+
+def test_no_filters_uses_default_365_day_window():
+    where_sql, params = dashboard_filters_to_sql()
+    assert where_sql == '"dateNotification" > ?'
+    assert len(params) == 1
+    assert isinstance(params[0], datetime)
+    expected = datetime.now() - timedelta(days=365)
+    assert abs((params[0] - expected).total_seconds()) < 2
+
+
+def test_year_filter_overrides_default_window():
+    where_sql, params = dashboard_filters_to_sql(dashboard_year="2025")
+    assert where_sql == 'YEAR("dateNotification") = ?'
+    assert params == [2025]
+```
+
+- [ ] **Step 2: Run tests to verify they fail**
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py -v`
+Expected: FAIL with `ImportError: cannot import name 'dashboard_filters_to_sql'`.
+
+- [ ] **Step 3: Implement the helper**
+
+Add to the top of `src/utils/table_sql.py` (below existing imports):
+
+```python
+from datetime import datetime, timedelta
+```
+
+Append this function at the end of `src/utils/table_sql.py`:
+
+```python
+def dashboard_filters_to_sql(
+    dashboard_year=None,
+    dashboard_acheteur_id=None,
+    dashboard_acheteur_categorie=None,
+    dashboard_acheteur_departement_code=None,
+    dashboard_titulaire_id=None,
+    dashboard_titulaire_categorie=None,
+    dashboard_titulaire_departement_code=None,
+    dashboard_marche_type=None,
+    dashboard_marche_objet=None,
+    dashboard_marche_code_cpv=None,
+    dashboard_marche_considerations_sociales=None,
+    dashboard_marche_considerations_environnementales=None,
+    dashboard_marche_techniques=None,
+    dashboard_marche_innovant=None,
+    dashboard_marche_sous_traitance_declaree=None,
+    dashboard_montant_min=None,
+    dashboard_montant_max=None,
+) -> tuple[str, list]:
+    """Traduit les filtres du tableau de bord en (where_clause, params) DuckDB."""
+    clauses: list[str] = []
+    params: list = []
+
+    if dashboard_year:
+        clauses.append('YEAR("dateNotification") = ?')
+        params.append(int(dashboard_year))
+    else:
+        clauses.append('"dateNotification" > ?')
+        params.append(datetime.now() - timedelta(days=365))
+
+    return " AND ".join(clauses), params
+```
+
+- [ ] **Step 4: Run tests to verify they pass**
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py -v`
+Expected: PASS (2 tests).
+
+- [ ] **Step 5: Commit**
+
+```bash
+rtk pre-commit run --files tests/test_dashboard_filters_to_sql.py src/utils/table_sql.py
+rtk git add tests/test_dashboard_filters_to_sql.py src/utils/table_sql.py
+rtk git commit -m "feat(observatoire): squelette de dashboard_filters_to_sql (#72)"
+```
+
+---
+
+## Task 2: Filtres d'égalité simples (catégorie, type, innovant, sous-traitance)
+
+**Files:**
+
+- Modify: `tests/test_dashboard_filters_to_sql.py`
+- Modify: `src/utils/table_sql.py`
+
+- [ ] **Step 1: Add failing tests**
+
+Append to `tests/test_dashboard_filters_to_sql.py`:
+
+```python
+def test_marche_type_equality():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_type="Marché",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "type" = ?'
+    assert params == [2025, "Marché"]
+
+
+def test_innovant_value_all_is_skipped():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_innovant="all",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ?'
+    assert params == [2025]
+
+
+def test_innovant_value_oui_adds_clause():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_innovant="oui",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "marcheInnovant" = ?'
+    assert params == [2025, "oui"]
+
+
+def test_sous_traitance_value_non_adds_clause():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_sous_traitance_declaree="non",
+    )
+    assert (
+        where_sql
+        == 'YEAR("dateNotification") = ? AND "sousTraitanceDeclaree" = ?'
+    )
+    assert params == [2025, "non"]
+```
+
+- [ ] **Step 2: Run tests to verify they fail**
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py -v`
+Expected: 4 new tests FAIL (missing clauses).
+
+- [ ] **Step 3: Extend the helper**
+
+Insert the following block in `dashboard_filters_to_sql`, **after** the `if dashboard_year / else` block and **before** `return " AND ".join(clauses), params`:
+
+```python
+    if dashboard_marche_type:
+        clauses.append('"type" = ?')
+        params.append(dashboard_marche_type)
+
+    if dashboard_marche_innovant and dashboard_marche_innovant != "all":
+        clauses.append('"marcheInnovant" = ?')
+        params.append(dashboard_marche_innovant)
+
+    if (
+        dashboard_marche_sous_traitance_declaree
+        and dashboard_marche_sous_traitance_declaree != "all"
+    ):
+        clauses.append('"sousTraitanceDeclaree" = ?')
+        params.append(dashboard_marche_sous_traitance_declaree)
+```
+
+- [ ] **Step 4: Run tests to verify they pass**
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py -v`
+Expected: PASS (6 tests total).
+
+- [ ] **Step 5: Commit**
+
+```bash
+rtk pre-commit run --files tests/test_dashboard_filters_to_sql.py src/utils/table_sql.py
+rtk git add tests/test_dashboard_filters_to_sql.py src/utils/table_sql.py
+rtk git commit -m "feat(observatoire): filtres d'égalité simples dans dashboard_filters_to_sql (#72)"
+```
+
+---
+
+## Task 3: Filtres LIKE/ILIKE (ids, objet, cpv)
+
+**Files:**
+
+- Modify: `tests/test_dashboard_filters_to_sql.py`
+- Modify: `src/utils/table_sql.py`
+
+- [ ] **Step 1: Add failing tests**
+
+Append to `tests/test_dashboard_filters_to_sql.py`:
+
+```python
+def test_acheteur_id_uses_like_wildcards():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_acheteur_id="12345678900010",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "acheteur_id" LIKE ?'
+    assert params == [2025, "%12345678900010%"]
+
+
+def test_titulaire_id_uses_like_wildcards():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_titulaire_id="999",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "titulaire_id" LIKE ?'
+    assert params == [2025, "%999%"]
+
+
+def test_marche_objet_uses_case_insensitive_ilike():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_objet="travaux",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "objet" ILIKE ?'
+    assert params == [2025, "%travaux%"]
+
+
+def test_code_cpv_uses_prefix_like():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_code_cpv="4521",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "codeCPV" LIKE ?'
+    assert params == [2025, "4521%"]
+```
+
+- [ ] **Step 2: Run tests to verify they fail**
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py -v`
+Expected: 4 new tests FAIL.
+
+- [ ] **Step 3: Extend the helper**
+
+Insert the following block, **just after** the year/default block and **before** the `if dashboard_marche_type` block:
+
+```python
+    if dashboard_acheteur_id:
+        clauses.append('"acheteur_id" LIKE ?')
+        params.append(f"%{dashboard_acheteur_id}%")
+
+    if dashboard_titulaire_id:
+        clauses.append('"titulaire_id" LIKE ?')
+        params.append(f"%{dashboard_titulaire_id}%")
+```
+
+Insert in the "marché" block, **after** `dashboard_marche_type` and **before** `dashboard_marche_innovant`:
+
+```python
+    if dashboard_marche_objet:
+        clauses.append('"objet" ILIKE ?')
+        params.append(f"%{dashboard_marche_objet}%")
+
+    if dashboard_marche_code_cpv:
+        clauses.append('"codeCPV" LIKE ?')
+        params.append(f"{dashboard_marche_code_cpv}%")
+```
+
+- [ ] **Step 4: Run tests to verify they pass**
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py -v`
+Expected: PASS (10 tests total).
+
+- [ ] **Step 5: Commit**
+
+```bash
+rtk pre-commit run --files tests/test_dashboard_filters_to_sql.py src/utils/table_sql.py
+rtk git add tests/test_dashboard_filters_to_sql.py src/utils/table_sql.py
+rtk git commit -m "feat(observatoire): filtres LIKE/ILIKE dans dashboard_filters_to_sql (#72)"
+```
+
+---
+
+## Task 4: Filtre IN (départements) + skip conditionnel par ID
+
+**Files:**
+
+- Modify: `tests/test_dashboard_filters_to_sql.py`
+- Modify: `src/utils/table_sql.py`
+
+- [ ] **Step 1: Add failing tests**
+
+Append to `tests/test_dashboard_filters_to_sql.py`:
+
+```python
+def test_acheteur_departement_multiple_uses_in_clause():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_acheteur_departement_code=["75", "92", "93"],
+    )
+    assert where_sql == (
+        'YEAR("dateNotification") = ? '
+        'AND "acheteur_departement_code" IN (?, ?, ?)'
+    )
+    assert params == [2025, "75", "92", "93"]
+
+
+def test_acheteur_categorie_adds_clause():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_acheteur_categorie="Commune",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "acheteur_categorie" = ?'
+    assert params == [2025, "Commune"]
+
+
+def test_titulaire_categorie_and_departement():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_titulaire_categorie="PME",
+        dashboard_titulaire_departement_code=["35"],
+    )
+    assert where_sql == (
+        'YEAR("dateNotification") = ? '
+        'AND "titulaire_categorie" = ? '
+        'AND "titulaire_departement_code" IN (?)'
+    )
+    assert params == [2025, "PME", "35"]
+
+
+def test_acheteur_id_present_skips_categorie_and_departement():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_acheteur_id="123",
+        dashboard_acheteur_categorie="Commune",
+        dashboard_acheteur_departement_code=["75"],
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "acheteur_id" LIKE ?'
+    assert params == [2025, "%123%"]
+
+
+def test_titulaire_id_present_skips_categorie_and_departement():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_titulaire_id="999",
+        dashboard_titulaire_categorie="PME",
+        dashboard_titulaire_departement_code=["35"],
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "titulaire_id" LIKE ?'
+    assert params == [2025, "%999%"]
+```
+
+- [ ] **Step 2: Run tests to verify they fail**
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py -v`
+Expected: 5 new tests FAIL.
+
+- [ ] **Step 3: Refactor the helper with conditional skip**
+
+Replace the two simple `if dashboard_acheteur_id` / `if dashboard_titulaire_id` blocks added in Task 3 with the nested form:
+
+```python
+    if dashboard_acheteur_id:
+        clauses.append('"acheteur_id" LIKE ?')
+        params.append(f"%{dashboard_acheteur_id}%")
+    else:
+        if dashboard_acheteur_categorie:
+            clauses.append('"acheteur_categorie" = ?')
+            params.append(dashboard_acheteur_categorie)
+        if dashboard_acheteur_departement_code:
+            placeholders = ", ".join(["?"] * len(dashboard_acheteur_departement_code))
+            clauses.append(f'"acheteur_departement_code" IN ({placeholders})')
+            params.extend(dashboard_acheteur_departement_code)
+
+    if dashboard_titulaire_id:
+        clauses.append('"titulaire_id" LIKE ?')
+        params.append(f"%{dashboard_titulaire_id}%")
+    else:
+        if dashboard_titulaire_categorie:
+            clauses.append('"titulaire_categorie" = ?')
+            params.append(dashboard_titulaire_categorie)
+        if dashboard_titulaire_departement_code:
+            placeholders = ", ".join(
+                ["?"] * len(dashboard_titulaire_departement_code)
+            )
+            clauses.append(f'"titulaire_departement_code" IN ({placeholders})')
+            params.extend(dashboard_titulaire_departement_code)
+```
+
+- [ ] **Step 4: Run tests to verify they pass**
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py -v`
+Expected: PASS (15 tests total).
+
+- [ ] **Step 5: Commit**
+
+```bash
+rtk pre-commit run --files tests/test_dashboard_filters_to_sql.py src/utils/table_sql.py
+rtk git add tests/test_dashboard_filters_to_sql.py src/utils/table_sql.py
+rtk git commit -m "feat(observatoire): IN départements et skip conditionnel par ID (#72)"
+```
+
+---
+
+## Task 5: Filtre liste (techniques, considérations sociales/environnementales)
+
+**Files:**
+
+- Modify: `tests/test_dashboard_filters_to_sql.py`
+- Modify: `src/utils/table_sql.py`
+
+- [ ] **Step 1: Add failing tests**
+
+Append to `tests/test_dashboard_filters_to_sql.py`:
+
+```python
+def test_marche_techniques_uses_list_has_any():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_techniques=["Enchère", "Accord-cadre"],
+    )
+    assert where_sql == (
+        'YEAR("dateNotification") = ? '
+        "AND list_has_any(string_split(\"techniques\", ', '), ?::VARCHAR[])"
+    )
+    assert params == [2025, ["Enchère", "Accord-cadre"]]
+
+
+def test_considerations_sociales_uses_list_has_any():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_considerations_sociales=["Clause sociale"],
+    )
+    assert where_sql == (
+        'YEAR("dateNotification") = ? '
+        "AND list_has_any(string_split(\"considerationsSociales\", ', '), ?::VARCHAR[])"
+    )
+    assert params == [2025, ["Clause sociale"]]
+
+
+def test_considerations_environnementales_uses_list_has_any():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_considerations_environnementales=["Clause env."],
+    )
+    assert where_sql == (
+        'YEAR("dateNotification") = ? '
+        "AND list_has_any(string_split(\"considerationsEnvironnementales\", ', '), ?::VARCHAR[])"
+    )
+    assert params == [2025, ["Clause env."]]
+```
+
+- [ ] **Step 2: Run tests to verify they fail**
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py -v`
+Expected: 3 new tests FAIL.
+
+- [ ] **Step 3: Extend the helper**
+
+Insert the following block in `dashboard_filters_to_sql`, **after** the `dashboard_marche_sous_traitance_declaree` block and **before** `return " AND ".join(clauses), params`:
+
+```python
+    if dashboard_marche_techniques:
+        clauses.append(
+            "list_has_any(string_split(\"techniques\", ', '), ?::VARCHAR[])"
+        )
+        params.append(list(dashboard_marche_techniques))
+
+    if dashboard_marche_considerations_sociales:
+        clauses.append(
+            "list_has_any(string_split(\"considerationsSociales\", ', '), ?::VARCHAR[])"
+        )
+        params.append(list(dashboard_marche_considerations_sociales))
+
+    if dashboard_marche_considerations_environnementales:
+        clauses.append(
+            "list_has_any(string_split(\"considerationsEnvironnementales\", ', '), ?::VARCHAR[])"
+        )
+        params.append(list(dashboard_marche_considerations_environnementales))
+```
+
+- [ ] **Step 4: Run tests to verify they pass**
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py -v`
+Expected: PASS (18 tests total).
+
+- [ ] **Step 5: Commit**
+
+```bash
+rtk pre-commit run --files tests/test_dashboard_filters_to_sql.py src/utils/table_sql.py
+rtk git add tests/test_dashboard_filters_to_sql.py src/utils/table_sql.py
+rtk git commit -m "feat(observatoire): filtres liste via list_has_any (#72)"
+```
+
+---
+
+## Task 6: Filtres montant min/max (incluant 0)
+
+**Files:**
+
+- Modify: `tests/test_dashboard_filters_to_sql.py`
+- Modify: `src/utils/table_sql.py`
+
+- [ ] **Step 1: Add failing tests**
+
+Append to `tests/test_dashboard_filters_to_sql.py`:
+
+```python
+def test_montant_min_only():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_montant_min=1000,
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "montant" >= ?'
+    assert params == [2025, 1000]
+
+
+def test_montant_max_only():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_montant_max=500,
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "montant" <= ?'
+    assert params == [2025, 500]
+
+
+def test_montant_zero_is_a_valid_lower_bound():
+    # 0 est falsy mais reste un filtre valide (distinct de None)
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_montant_min=0,
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "montant" >= ?'
+    assert params == [2025, 0]
+
+
+def test_montant_min_and_max_combined():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_montant_min=100,
+        dashboard_montant_max=1000,
+    )
+    assert where_sql == (
+        'YEAR("dateNotification") = ? AND "montant" >= ? AND "montant" <= ?'
+    )
+    assert params == [2025, 100, 1000]
+```
+
+- [ ] **Step 2: Run tests to verify they fail**
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py -v`
+Expected: 4 new tests FAIL.
+
+- [ ] **Step 3: Extend the helper**
+
+Insert at the very end of `dashboard_filters_to_sql`, **just before** `return " AND ".join(clauses), params`:
+
+```python
+    if dashboard_montant_min is not None:
+        clauses.append('"montant" >= ?')
+        params.append(dashboard_montant_min)
+
+    if dashboard_montant_max is not None:
+        clauses.append('"montant" <= ?')
+        params.append(dashboard_montant_max)
+```
+
+- [ ] **Step 4: Run tests to verify they pass**
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py -v`
+Expected: PASS (22 tests total).
+
+- [ ] **Step 5: Commit**
+
+```bash
+rtk pre-commit run --files tests/test_dashboard_filters_to_sql.py src/utils/table_sql.py
+rtk git add tests/test_dashboard_filters_to_sql.py src/utils/table_sql.py
+rtk git commit -m "feat(observatoire): filtres montant min/max (#72)"
+```
+
+---
+
+## Task 7: Réécriture de `prepare_dashboard_data`
+
+**Files:**
+
+- Modify: `src/utils/data.py`
+- Modify: `tests/test_main.py` (supprimer `test_010_observatoire_montant_filter`)
+
+- [ ] **Step 1: Remove the obsolete Polars-based test**
+
+Delete the function `test_010_observatoire_montant_filter` from `tests/test_main.py` (lines ~218-256). La couverture du filtre montant est déjà assurée par les tests unitaires `test_montant_*` de la Task 6.
+
+- [ ] **Step 2: Rewrite `prepare_dashboard_data`**
+
+Replace the entire `prepare_dashboard_data` function in `src/utils/data.py` (lines ~86-194) with:
+
+```python
+def prepare_dashboard_data(**filter_params) -> pl.DataFrame:
+    """Exécute la requête DuckDB filtrée pour le tableau de bord.
+
+    Retourne une pl.DataFrame matérialisée uniquement pour le sous-ensemble
+    correspondant aux filtres. Les appelants qui ont besoin d'une LazyFrame
+    appellent `.lazy()` sur le résultat.
+    """
+    from src.utils.table_sql import dashboard_filters_to_sql
+
+    where_sql, params = dashboard_filters_to_sql(**filter_params)
+    return query_marches(where_sql=where_sql, params=params)
+```
+
+Update the import at the top of `src/utils/data.py`:
+
+```python
+from src.db import get_cursor, query_marches, schema
+```
+
+Remove the now-unused import in `src/utils/data.py`:
+
+```python
+from datetime import datetime, timedelta
+```
+
+(Si `datetime` n'est plus référencé dans `data.py` hors de `prepare_dashboard_data`, sinon garder.)
+
+**Vérification rapide à effectuer avant de supprimer `datetime`/`timedelta`** :
+
+```bash
+rtk grep -n "datetime\|timedelta" src/utils/data.py
+```
+
+Si d'autres occurrences existent, conserver les imports.
+
+- [ ] **Step 3: Run the full test suite**
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py tests/test_main.py -v -k "not selenium and not dash_duo"`
+
+Ou, si filter n'est pas pratique :
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py -v`
+
+Expected: PASS (22 tests).
+
+- [ ] **Step 4: Commit**
+
+```bash
+rtk pre-commit run --files src/utils/data.py tests/test_main.py
+rtk git add src/utils/data.py tests/test_main.py
+rtk git commit -m "refactor(observatoire): prepare_dashboard_data utilise DuckDB (#72)"
+```
+
+---
+
+## Task 8: Adaptation des 3 appelants dans `observatoire.py`
+
+**Files:**
+
+- Modify: `src/pages/observatoire.py`
+
+- [ ] **Step 1: Update `_compute_dashboard_children`**
+
+Remplacer dans `src/pages/observatoire.py` (autour des lignes 660-670) :
+
+```python
+@cache.memoize()
+def _compute_dashboard_children(filter_params_normalized: tuple):
+    logger.debug("Cache miss — computing dashboard")
+    filter_params = {
+        k: (list(v) if isinstance(v, tuple) else v) for k, v in filter_params_normalized
+    }
+
+    lff: pl.LazyFrame = query_marches().lazy()
+    lff = prepare_dashboard_data(lff=lff, **filter_params)
+
+    dff = lff.collect(engine="streaming")
+```
+
+Par :
+
+```python
+@cache.memoize()
+def _compute_dashboard_children(filter_params_normalized: tuple):
+    logger.debug("Cache miss — computing dashboard")
+    filter_params = {
+        k: (list(v) if isinstance(v, tuple) else v) for k, v in filter_params_normalized
+    }
+
+    dff = prepare_dashboard_data(**filter_params)
+    lff = dff.lazy()
+```
+
+Le reste de la fonction (à partir de `df_per_uid = ...`) est inchangé.
+
+- [ ] **Step 2: Update `download_observatoire`**
+
+Remplacer dans `src/pages/observatoire.py` (autour des lignes 789-800) :
+
+```python
+def download_observatoire(_n_clicks, filter_params, hidden_columns):
+    lff = prepare_dashboard_data(lff=query_marches().lazy(), **(filter_params or {}))
+
+    if hidden_columns:
+        lff = lff.drop(hidden_columns)
+
+    def to_bytes(buffer):
+        lff.collect(engine="streaming").write_excel(buffer, worksheet="DECP")
+
+    date = datetime.now().strftime("%Y-%m-%d_%H:%M:%S")
+    return dcc.send_bytes(to_bytes, filename=f"decp_observatoire_{date}.xlsx")
+```
+
+Par :
+
+```python
+def download_observatoire(_n_clicks, filter_params, hidden_columns):
+    dff = prepare_dashboard_data(**(filter_params or {}))
+
+    if hidden_columns:
+        dff = dff.drop(hidden_columns)
+
+    def to_bytes(buffer):
+        dff.write_excel(buffer, worksheet="DECP")
+
+    date = datetime.now().strftime("%Y-%m-%d_%H:%M:%S")
+    return dcc.send_bytes(to_bytes, filename=f"decp_observatoire_{date}.xlsx")
+```
+
+- [ ] **Step 3: Update `populate_preview_table`**
+
+Remplacer dans `src/pages/observatoire.py` (autour des lignes 879-892) :
+
+```python
+    if not is_open:
+        return (no_update,) * 9
+
+    lff = prepare_dashboard_data(lff=query_marches().lazy(), **(filter_params or {}))
+
+    return prepare_table_data(
+        lff,
+        data_timestamp,
+        filter_query,
+        page_current,
+        page_size,
+        sort_by,
+        "observatoire-preview",
+    )
+```
+
+Par :
+
+```python
+    if not is_open:
+        return (no_update,) * 9
+
+    dff = prepare_dashboard_data(**(filter_params or {}))
+
+    return prepare_table_data(
+        dff.lazy(),
+        data_timestamp,
+        filter_query,
+        page_current,
+        page_size,
+        sort_by,
+        "observatoire-preview",
+    )
+```
+
+- [ ] **Step 4: Remove unused `query_marches` import**
+
+Dans `src/pages/observatoire.py`, ligne ~19 :
+
+```python
+from src.db import query_marches, schema
+```
+
+Devient :
+
+```python
+from src.db import schema
+```
+
+Vérifier avant de committer :
+
+```bash
+rtk grep -n "query_marches" src/pages/observatoire.py
+```
+
+Expected: aucun résultat (ou uniquement des commentaires).
+
+- [ ] **Step 5: Smoke test**
+
+Démarrer l'app et naviguer sur `/observatoire`, vérifier à la main que :
+
+- Les cartes s'affichent.
+- Un filtre année se propage.
+- Un filtre acheteur par SIRET partiel fonctionne.
+- Un filtre département (multi-valeur) fonctionne.
+- Un filtre montant_min fonctionne.
+- Le bouton « Télécharger au format Excel » génère un fichier non vide.
+- Le bouton « Voir les données » ouvre l'offcanvas et peuple la table.
+
+Run: `python run.py`
+
+Expected: app démarre sans erreur ; les filtres se comportent comme avant.
+
+- [ ] **Step 6: Commit**
+
+```bash
+rtk pre-commit run --files src/pages/observatoire.py
+rtk git add src/pages/observatoire.py
+rtk git commit -m "refactor(observatoire): appelants utilisent la nouvelle signature (#72)"
+```
+
+---
+
+## Task 9: Test d'intégration — `prepare_dashboard_data` sur `tests/test.parquet`
+
+**Files:**
+
+- Create: `tests/test_prepare_dashboard_data.py`
+
+- [ ] **Step 1: Write the failing test**
+
+Le but : vérifier que la fonction s'exécute réellement contre DuckDB, retourne une `pl.DataFrame`, et applique bien les filtres simples. `conftest.py` construit `tests/test.parquet` avec un jeu de données d'une ligne : acheteur_id `123`, acheteur_departement_code `75`, dateNotification `2025-01-01`, montant `10`.
+
+Create `tests/test_prepare_dashboard_data.py`:
+
+```python
+import polars as pl
+
+
+def test_returns_dataframe_with_year_filter():
+    from src.utils.data import prepare_dashboard_data
+
+    dff = prepare_dashboard_data(dashboard_year="2025")
+    assert isinstance(dff, pl.DataFrame)
+    assert dff.height == 1
+
+
+def test_year_mismatch_returns_empty():
+    from src.utils.data import prepare_dashboard_data
+
+    dff = prepare_dashboard_data(dashboard_year="2024")
+    assert isinstance(dff, pl.DataFrame)
+    assert dff.height == 0
+
+
+def test_acheteur_id_partial_match():
+    from src.utils.data import prepare_dashboard_data
+
+    dff = prepare_dashboard_data(
+        dashboard_year="2025",
+        dashboard_acheteur_id="12",
+    )
+    assert dff.height == 1
+
+
+def test_departement_in_clause():
+    from src.utils.data import prepare_dashboard_data
+
+    dff = prepare_dashboard_data(
+        dashboard_year="2025",
+        dashboard_acheteur_departement_code=["75", "92"],
+    )
+    assert dff.height == 1
+
+
+def test_montant_min_above_value_excludes_row():
+    from src.utils.data import prepare_dashboard_data
+
+    dff = prepare_dashboard_data(
+        dashboard_year="2025",
+        dashboard_montant_min=1000,
+    )
+    assert dff.height == 0
+```
+
+- [ ] **Step 2: Run the test**
+
+Run: `rtk pytest tests/test_prepare_dashboard_data.py -v`
+Expected: PASS (5 tests).
+
+- [ ] **Step 3: Commit**
+
+```bash
+rtk pre-commit run --files tests/test_prepare_dashboard_data.py
+rtk git add tests/test_prepare_dashboard_data.py
+rtk git commit -m "test(observatoire): intégration DuckDB pour prepare_dashboard_data (#72)"
+```
+
+---
+
+## Task 10: Vérification finale
+
+**Files:** (aucune modification)
+
+- [ ] **Step 1: Run the full test suite**
+
+Run: `rtk pytest -v`
+Expected: tous les tests unitaires passent. Les tests Selenium peuvent échouer si Chrome n'est pas disponible — ce n'est pas bloquant s'ils étaient déjà rouges avant.
+
+- [ ] **Step 2: Check for leftover references**
+
+Run: `rtk grep -rn "prepare_dashboard_data(lff" src/ tests/`
+Expected: aucun résultat (plus d'appels avec l'ancienne signature).
+
+Run: `rtk grep -rn "query_marches().lazy()" src/`
+Expected: aucun résultat (ou uniquement dans `src/utils/table.py:prepare_table_data` pour le fallback).
+
+- [ ] **Step 3: Confirm `datetime`/`timedelta` in data.py if needed**
+
+Run: `rtk grep -n "datetime\|timedelta" src/utils/data.py`
+
+Si aucune occurrence hors imports, vérifier que les imports inutiles ont bien été retirés dans Task 7.
+
+- [ ] **Step 4: Manual timing sanity check (optionnel)**
+
+Si possible, comparer informellement le temps de `_compute_dashboard_children` sur un filtre sélectif (ex. un département) avant/après. Pas de benchmark formel attendu.
+
+- [ ] **Step 5: Push (manuel, à l'initiative de l'utilisateur)**
+
+Conformément aux consignes projet, ne jamais `git push`. Laisser l'utilisateur pousser la branche `feature/72_observatoire_duckdb_filters` et ouvrir la PR.

docs/superpowers/specs/2026-04-22-observatoire-duckdb-filters-design.md

@@ -0,0 +1,206 @@
+# Observatoire — filtrage natif DuckDB
+
+## Contexte
+
+La page `/observatoire` construit ses cartes, ses téléchargements et sa prévisualisation
+tabulaire à partir de la fonction `prepare_dashboard_data` (dans `src/utils/data.py`).
+Aujourd'hui, cette fonction prend une `pl.LazyFrame` — typiquement obtenue par
+`query_marches().lazy()` — et applique une série de filtres côté Polars.
+
+`query_marches()` matérialise l'intégralité de la table `decp` (~1,5 M lignes) en
+DataFrame Polars, même lorsqu'un utilisateur applique des filtres restrictifs. Les
+filtres sont ensuite appliqués sur cet ensemble déjà matérialisé.
+
+Le pattern utilisé par `_fetch_page_sql` (dans `src/utils/table.py`) montre comment
+déléguer le filtrage à DuckDB :
+
+1. Un traducteur (`filter_query_to_sql`, dans `src/utils/table_sql.py`) transforme le
+   DSL utilisateur en `(where_sql, params)`.
+2. `query_marches(where_sql=..., params=...)` ne matérialise que le sous-ensemble utile.
+
+Ce spec décrit comment appliquer ce même pattern aux filtres de l'observatoire.
+
+## Objectifs
+
+- Réduire la consommation mémoire et le temps de chaque callback de l'observatoire
+  en poussant le filtrage au niveau DuckDB.
+- Conserver strictement la sémantique des filtres actuels (pas de régression
+  fonctionnelle).
+- Garder une frontière claire : un helper pur `dashboard_filters_to_sql` qui ne
+  touche pas à la base, et une `prepare_dashboard_data` fine qui appelle DuckDB.
+
+## Non-objectifs
+
+- Pas de refonte de l'UI de filtres.
+- Pas d'optimisation ou de cache supplémentaire autour de
+  `_compute_dashboard_children` (déjà `@cache.memoize()`).
+- Pas de changement du comportement par défaut (365 derniers jours quand aucune
+  année n'est sélectionnée).
+
+## Architecture
+
+### Nouveau helper — `src/utils/table_sql.py`
+
+```python
+def dashboard_filters_to_sql(
+    dashboard_year=None,
+    dashboard_acheteur_id=None,
+    dashboard_acheteur_categorie=None,
+    dashboard_acheteur_departement_code=None,
+    dashboard_titulaire_id=None,
+    dashboard_titulaire_categorie=None,
+    dashboard_titulaire_departement_code=None,
+    dashboard_marche_type=None,
+    dashboard_marche_objet=None,
+    dashboard_marche_code_cpv=None,
+    dashboard_marche_considerations_sociales=None,
+    dashboard_marche_considerations_environnementales=None,
+    dashboard_marche_techniques=None,
+    dashboard_marche_innovant=None,
+    dashboard_marche_sous_traitance_declaree=None,
+    dashboard_montant_min=None,
+    dashboard_montant_max=None,
+) -> tuple[str, list]:
+    """Traduit les filtres du tableau de bord en (where_clause, params) DuckDB."""
+```
+
+Fonction pure, sans accès à la base. Même signature que `prepare_dashboard_data`
+actuelle (hors `lff`). Retourne `("TRUE", [])` si aucun filtre n'est actif.
+
+### Réécriture — `prepare_dashboard_data` (`src/utils/data.py`)
+
+```python
+def prepare_dashboard_data(**filter_params) -> pl.DataFrame:
+    where_sql, params = dashboard_filters_to_sql(**filter_params)
+    return query_marches(where_sql=where_sql, params=params)
+```
+
+- **Signature** : suppression du paramètre `lff`. Retour `pl.DataFrame` (et non plus
+  `pl.LazyFrame`).
+- Les appelants qui ont besoin d'une LazyFrame appellent `.lazy()` sur le résultat.
+
+### Appelants — `src/pages/observatoire.py`
+
+Trois sites d'appel à adapter :
+
+1. **`_compute_dashboard_children`** (ligne ~668) — on remplace
+
+   ```python
+   lff: pl.LazyFrame = query_marches().lazy()
+   lff = prepare_dashboard_data(lff=lff, **filter_params)
+   dff = lff.collect(engine="streaming")
+   ```
+
+   par
+
+   ```python
+   dff = prepare_dashboard_data(**filter_params)
+   lff = dff.lazy()
+   ```
+
+   Les appels existants à `make_donut`, `get_distance_histogram`, `get_top_org_table`,
+   `get_barchart_sources` continuent de recevoir `lff` ; `get_geographic_maps`
+   continue de recevoir `dff`. `df_per_uid` est calculé à partir de `dff`.
+
+2. **`download_observatoire`** (ligne ~791) —
+
+   ```python
+   dff = prepare_dashboard_data(**(filter_params or {}))
+   if hidden_columns:
+       dff = dff.drop(hidden_columns)
+   def to_bytes(buffer):
+       dff.write_excel(buffer, worksheet="DECP")
+   ```
+
+3. **`populate_preview_table`** (ligne ~882) —
+   ```python
+   dff = prepare_dashboard_data(**(filter_params or {}))
+   return prepare_table_data(
+       dff.lazy(),  # prepare_table_data accepte une LazyFrame
+       ...
+   )
+   ```
+
+## Traduction des filtres
+
+| Filtre                                                    | Actuel (Polars)                                            | Cible (SQL DuckDB)                                             |
+| --------------------------------------------------------- | ---------------------------------------------------------- | -------------------------------------------------------------- |
+| `dashboard_year` (présent)                                | `dt.year() == int(year)`                                   | `YEAR("dateNotification") = ?`                                 |
+| `dashboard_year` (absent) — comportement par défaut       | `> now - 365j`                                             | `"dateNotification" > ?` (datetime calculé à l'appel)          |
+| `dashboard_acheteur_id`                                   | `str.contains(val)`                                        | `"acheteur_id" LIKE ?` avec `%val%`                            |
+| `dashboard_acheteur_categorie`                            | `== val` (skip si acheteur_id présent)                     | `"acheteur_categorie" = ?`                                     |
+| `dashboard_acheteur_departement_code`                     | `is_in(list)` (skip si acheteur_id présent)                | `"acheteur_departement_code" IN (?, ?, ...)`                   |
+| `dashboard_titulaire_id`                                  | idem acheteur                                              | idem                                                           |
+| `dashboard_titulaire_categorie`                           | idem                                                       | idem                                                           |
+| `dashboard_titulaire_departement_code`                    | idem                                                       | idem                                                           |
+| `dashboard_marche_type`                                   | `== val`                                                   | `"type" = ?`                                                   |
+| `dashboard_marche_objet`                                  | `str.contains("(?i)val")`                                  | `"objet" ILIKE ?` avec `%val%`                                 |
+| `dashboard_marche_code_cpv`                               | `str.starts_with(val)`                                     | `"codeCPV" LIKE ?` avec `val%`                                 |
+| `dashboard_marche_techniques`                             | `str.split(", ").list.set_intersection(xs).list.len() > 0` | `list_has_any(string_split("techniques", ', '), ?::VARCHAR[])` |
+| `dashboard_marche_considerations_sociales`                | idem                                                       | idem sur `"considerationsSociales"`                            |
+| `dashboard_marche_considerations_environnementales`       | idem                                                       | idem sur `"considerationsEnvironnementales"`                   |
+| `dashboard_marche_innovant` (`"oui"`/`"non"`, sinon skip) | `== val`                                                   | `"marcheInnovant" = ?`                                         |
+| `dashboard_marche_sous_traitance_declaree`                | idem                                                       | `"sousTraitanceDeclaree" = ?`                                  |
+| `dashboard_montant_min`                                   | `>= val`                                                   | `"montant" >= ?`                                               |
+| `dashboard_montant_max`                                   | `<= val`                                                   | `"montant" <= ?`                                               |
+
+**Logique conditionnelle conservée** : si `dashboard_acheteur_id` est fourni, les filtres
+`categorie` et `departement_code` acheteur sont ignorés (même chose pour titulaire).
+
+**Traitement des valeurs spéciales** :
+
+- `dashboard_marche_innovant` / `dashboard_marche_sous_traitance_declaree` : valeur
+  `"all"` ou falsy → aucun filtre ajouté.
+- `dashboard_year` : converti en `int` avant injection.
+- `dashboard_montant_min` / `_max` : `None` → aucun filtre (distinct de `0`, qui reste
+  un filtre valide via `>=` ou `<=`).
+
+**Sécurité SQL** : toutes les valeurs utilisateurs passent par DuckDB en paramètres liés
+(`?`). Seuls des noms de colonnes statiques (contrôlés par le code) sont injectés dans le
+fragment SQL via `f"..."`. Pas de différence avec le pattern existant de
+`filter_query_to_sql`.
+
+## Tests
+
+### Unitaires (nouveaux)
+
+Nouveau fichier `tests/test_dashboard_filters_to_sql.py` :
+
+- Cas vide → `("TRUE", [])`.
+- Un seul filtre simple (année, type, etc.) → fragment SQL et params attendus.
+- Filtre montant min/max (migration de l'actuel `test_010_observatoire_montant_filter`).
+- Filtre liste (techniques, considerationsSociales) → usage de `list_has_any`.
+- Filtre acheteur_id fourni → catégorie/département acheteur ignorés.
+- Filtre `"all"` / `None` sur innovant/sous_traitance → aucun fragment ajouté.
+- Comportement par défaut sans année → fragment `"dateNotification" > ?` avec un param
+  datetime à ~365 j dans le passé (tolérance de quelques secondes).
+
+### Intégration (nouveau, léger)
+
+Un test qui appelle `prepare_dashboard_data` contre `tests/test.parquet` avec un ou
+deux filtres connus, vérifie le `height` et la bonne nature du retour (`pl.DataFrame`).
+
+### Test Selenium existant
+
+`test_009_observatoire_filter_persistence` et `test_008_observatoire_navigation_from_search`
+ne touchent pas à la signature ; ils doivent continuer à passer.
+
+## Risques et migration
+
+- **Risque sémantique** : la fonction Polars `str.contains` utilisée pour les IDs est
+  un regex. Les utilisateurs attendent probablement un contains littéral sur un SIRET
+  (14 chiffres). Le passage à `LIKE '%val%'` est neutre si la valeur ne contient pas de
+  caractère spécial regex — ce qui est le cas pour des SIRET. **Hypothèse** acceptée :
+  le contenu `dashboard_acheteur_id`/`dashboard_titulaire_id` est alphanumérique.
+- **Risque de drift du cache** : la date "365 derniers jours" n'est pas incluse dans
+  la clé de cache de `_compute_dashboard_children`. C'est un comportement pré-existant
+  ; non traité par ce spec.
+- **Import circulaire** : `src/utils/data.py` importe déjà depuis `src/db.py`.
+  `src/utils/table_sql.py` importe depuis `src/utils/table.py`. Pas de nouveau cycle.
+
+## Succès
+
+- Les 3 callbacks de l'observatoire restent fonctionnellement équivalents.
+- Les tests unitaires et d'intégration passent.
+- Une inspection manuelle confirme un temps d'exécution réduit sur un filtre
+  sélectif (par ex. un département + une année).

pyproject.toml

@@ -1,7 +1,7 @@
 [project]
 name = "decp.info"
 description = "Interface d'exploration et d'analyse des marchés publics français."
-version = "2.7.3"
+version = "2.7.5"
 requires-python = ">= 3.10"
 authors = [{ name = "Colin Maudry", email = "colin@colmo.tech" }]
 dependencies = [
@@ -40,6 +40,7 @@ testpaths = ["tests"]
 env = [
   "DATA_FILE_PARQUET_PATH=tests/test.parquet",
   "DEVELOPMENT=true",
+  "REBUILD_DUCKDB=true",
   "DATA_SCHEMA_PATH=/home/colin/git/decp-processing/dist/schema.json",
 ]
 addopts = "-p no:warnings"

src/app.py

@@ -2,6 +2,7 @@ import os
 from shutil import rmtree
 
 import dash_bootstrap_components as dbc
+import pandas  # noqa: F401  # eager import: avoid plotly's lazy-import race across Dash callback threads
 import tomllib
 from dash import Dash, Input, Output, State, dcc, html, page_container, page_registry
 from dotenv import load_dotenv

src/db.py

@@ -110,7 +110,7 @@ def build_database(db_path: Path, parquet_path: Path) -> None:
 
 
 def _ensure_database() -> Path:
-    db_path = Path("./decp.duckdb")
+    db_path = Path(os.getenv("DUCKDB_PATH", "./decp.duckdb"))
     parquet_path = Path(os.getenv("DATA_FILE_PARQUET_PATH"))
     lock_path = db_path.with_suffix(".duckdb.lock")
 
@@ -135,10 +135,11 @@ def get_cursor() -> duckdb.DuckDBPyConnection:
 
 def query_marches(
     where_sql: str = "TRUE",
-    params: tuple = (),
+    params: tuple | list = (),
     columns: list[str] | None = None,
     order_by: str | None = None,
     limit: int | None = None,
+    offset: int | None = None,
 ) -> pl.DataFrame:
     """Run a parameterized SELECT against the decp table and return Polars.
 
@@ -152,4 +153,25 @@ def query_marches(
         sql += f" ORDER BY {order_by}"
     if limit is not None:
         sql += f" LIMIT {int(limit)}"
+    if offset is not None:
+        sql += f" OFFSET {int(offset)}"
+
+    logger.debug("query_marches: " + sql.replace("?", "{}").format(*params))
+
     return get_cursor().execute(sql, list(params)).pl()
+
+
+def count_marches(where_sql: str = "TRUE", params: tuple | list = ()) -> int:
+    """Retourne le nombre de lignes correspondant à where_sql."""
+    sql = f"SELECT COUNT(*) FROM decp WHERE {where_sql}"
+    logger.debug("count_marches: " + sql.replace("?", "{}").format(*params))
+    result = get_cursor().execute(sql, list(params)).fetchone()
+    return int(result[0]) if result else 0
+
+
+def count_unique_marches(where_sql: str = "TRUE", params: tuple | list = ()) -> int:
+    """Retourne le nombre de uid distincts correspondant à where_sql."""
+    sql = f"SELECT COUNT(DISTINCT uid) FROM decp WHERE {where_sql}"
+    logger.debug("count_unique_marches: " + sql.replace("?", "{}").format(*params))
+    result = get_cursor().execute(sql, list(params)).fetchone()
+    return int(result[0]) if result else 0

src/figures.py

@@ -839,7 +839,7 @@ def get_top_org_table(data, org_type: str, extra_columns: list, filters: bool =
         return html.Div()
 
     columns, tooltip = setup_table_columns(
-        dff, hideable=False, exclude=[f"{org_type}_id"], new_columns=["Attributions"]
+        dff, hideable=False, exclude=[f"{org_type}_id"]
     )
     dff = add_links(dff)
     data = dff.to_dicts()

src/pages/observatoire.py

@@ -16,7 +16,7 @@ from dash import (
     register_page,
 )
 
-from src.db import query_marches, schema
+from src.db import schema
 from src.figures import (
     DataTable,
     get_barchart_sources,
@@ -508,17 +508,26 @@ Alors, on fait comment ?
                 size="xl",
             ),
             # DataTable
-            html.Div(
-                className="marches_table",
-                children=DataTable(
-                    dtid="observatoire-preview-table",
-                    page_size=5,
-                    page_action="custom",
-                    sort_action="custom",
-                    filter_action="custom",
-                    hidden_columns=[],
-                    columns=[{"id": col, "name": col} for col in OBSERVATOIRE_COLUMNS],
-                ),
+            dcc.Loading(
+                overlay_style={"visibility": "visible", "filter": "blur(2px)"},
+                id="loading-statistques",
+                type="default",
+                children=[
+                    html.Div(
+                        className="marches_table",
+                        children=DataTable(
+                            dtid="observatoire-preview-table",
+                            page_size=5,
+                            page_action="custom",
+                            sort_action="custom",
+                            filter_action="custom",
+                            hidden_columns=[],
+                            columns=[
+                                {"id": col, "name": col} for col in OBSERVATOIRE_COLUMNS
+                            ],
+                        ),
+                    )
+                ],
             ),
         ],
     ),
@@ -664,10 +673,8 @@ def _compute_dashboard_children(filter_params_normalized: tuple):
         k: (list(v) if isinstance(v, tuple) else v) for k, v in filter_params_normalized
     }
 
-    lff: pl.LazyFrame = query_marches().lazy()
-    lff = prepare_dashboard_data(lff=lff, **filter_params)
-
-    dff = lff.collect(engine="streaming")
+    dff = prepare_dashboard_data(**filter_params)
+    lff = dff.lazy()
 
     df_per_uid = (
         dff.select("uid", "montant").group_by("uid").agg(pl.col("montant").first())
@@ -788,13 +795,13 @@ def update_dashboard_cards(*filter_values):
     prevent_initial_call=True,
 )
 def download_observatoire(_n_clicks, filter_params, hidden_columns):
-    lff = prepare_dashboard_data(lff=query_marches().lazy(), **(filter_params or {}))
+    dff = prepare_dashboard_data(**(filter_params or {}))
 
     if hidden_columns:
-        lff = lff.drop(hidden_columns)
+        dff = dff.drop(hidden_columns)
 
     def to_bytes(buffer):
-        lff.collect(engine="streaming").write_excel(buffer, worksheet="DECP")
+        dff.write_excel(buffer, worksheet="DECP")
 
     date = datetime.now().strftime("%Y-%m-%d_%H:%M:%S")
     return dcc.send_bytes(to_bytes, filename=f"decp_observatoire_{date}.xlsx")
@@ -879,10 +886,10 @@ def populate_preview_table(
     if not is_open:
         return (no_update,) * 9
 
-    lff = prepare_dashboard_data(lff=query_marches().lazy(), **(filter_params or {}))
+    dff = prepare_dashboard_data(**(filter_params or {}))
 
     return prepare_table_data(
-        lff,
+        dff.lazy(),
         data_timestamp,
         filter_query,
         page_current,

src/pages/tableau.py

@@ -163,18 +163,19 @@ layout = [
 
             Vous pouvez appliquer un filtre pour chaque colonne en entrant du texte sous le nom de la colonne, puis en tapant sur `Entrée`.
 
-            - Champs textuels : la recherche retourne les valeurs qui contiennent le texte recherché et n'est pas sensible à la casse (majuscules/minuscules).
-                - Exemple : `rennes` retourne "RENNES METROPOLE".
+            - Champs textuels : la recherche retourne les valeurs qui contiennent le texte recherché, n'est pas sensible à la casse (majuscules/minuscules) et est sensbible à l'accentuation.
+                - `rennes` => le texte contient "rennes"
+                - `metro* *pole` => le texte contient un mot qui commence par "metro" et un mot qui finit par "pole"
+                - `metropole rennes` => le texte contient les mots "metropole" et "rennes", n'importe où dans le texte
+                - `metropole+rennes` => le texte contient "metropole rennes", collé et dans cet ordre
+                - `metropole+rennes travaux distri*` => le texte contient "metropole rennes", "travaux" et un mot qui commence par "distri"
                 - Les guillemets simples (apostrophe du 4) doivent être prédédées d'une barre oblique (AltGr + 8). Exemple : `services d\\\'assurances`
             - Champs numériques (Durée en mois, Montant, ...) : vous pouvez...
                 - soit taper un nombre pour trouver les valeurs strictement égales. Exemple : `12` ne retourne que des 12
                 - soit le précéder de **>** ou **<** pour filtrer les valeurs supérieures ou inférieures. Exemple pour les offres reçues : `> 4` retourne les marchés ayant reçu plus de 4 offres.
-            - Champs date (Date de notification, ...) : vous pouvez également utiliser **>** ou **<**. Exemples :
+            - Champs date (Date de notification, ...) :
                 - `< 2024-01-31` pour "avant le 31 janvier 2024"
-                - `2024` pour "en 2024", `> 2022` pour "à partir de 2022".
-            - Pour les champs textuels et les champs dates :
-                - pour chercher du texte qui **commence par** votre texte, entrez `texte*`. C'est par exemple utile pour filtrer des acheteurs ou titulaires par numéro SIREN (`123456789*`) ou les marchés sur une année en particulier (`2024*`)
-                - pour chercher du texte qui **finit par** votre texte, entrez `*texte`
+                - `2024` pour "en 2024", `> 2022` pour "à partir de 2022"
 
             Vous pouvez filtrer plusieurs colonnes à la fois.
 

src/utils/data.py

@@ -2,12 +2,11 @@ import json
 import logging
 import os
 from collections import OrderedDict
-from datetime import datetime, timedelta
 
 import polars as pl
 from httpx import HTTPError, get
 
-from src.db import get_cursor, schema
+from src.db import get_cursor, query_marches, schema
 from src.utils import logger
 
 logging.getLogger("httpx").setLevel("WARNING")
@@ -83,115 +82,17 @@ def get_data_schema() -> dict:
     return new_schema
 
 
-def prepare_dashboard_data(
-    lff: pl.LazyFrame,
-    dashboard_year=None,
-    dashboard_acheteur_id=None,
-    dashboard_acheteur_categorie=None,
-    dashboard_acheteur_departement_code=None,
-    dashboard_titulaire_id=None,
-    dashboard_titulaire_categorie=None,
-    dashboard_titulaire_departement_code=None,
-    dashboard_marche_type=None,
-    dashboard_marche_objet=None,
-    dashboard_marche_code_cpv=None,
-    dashboard_marche_considerations_sociales=None,
-    dashboard_marche_considerations_environnementales=None,
-    dashboard_marche_techniques=None,
-    dashboard_marche_innovant=None,
-    dashboard_marche_sous_traitance_declaree=None,
-    dashboard_montant_min=None,
-    dashboard_montant_max=None,
-) -> pl.LazyFrame:
-    if dashboard_year:
-        lff = lff.filter(pl.col("dateNotification").dt.year() == int(dashboard_year))
-    else:
-        lff = lff.filter(
-            pl.col("dateNotification") > (datetime.now() - timedelta(days=365))
-        )
+def prepare_dashboard_data(**filter_params) -> pl.DataFrame:
+    """Exécute la requête DuckDB filtrée pour le tableau de bord.
 
-    if dashboard_acheteur_id:
-        lff = lff.filter(pl.col("acheteur_id").str.contains(dashboard_acheteur_id))
-    else:
-        if dashboard_acheteur_categorie:
-            lff = lff.filter(
-                pl.col("acheteur_categorie") == dashboard_acheteur_categorie
-            )
-        if dashboard_acheteur_departement_code:
-            lff = lff.filter(
-                pl.col("acheteur_departement_code").is_in(
-                    dashboard_acheteur_departement_code
-                )
-            )
+    Retourne une pl.DataFrame matérialisée uniquement pour le sous-ensemble
+    correspondant aux filtres. Les appelants qui ont besoin d'une LazyFrame
+    appellent `.lazy()` sur le résultat.
+    """
+    from src.utils.table_sql import dashboard_filters_to_sql
 
-    if dashboard_titulaire_id:
-        lff = lff.filter(pl.col("titulaire_id").str.contains(dashboard_titulaire_id))
-    else:
-        if dashboard_titulaire_categorie:
-            lff = lff.filter(
-                pl.col("titulaire_categorie") == dashboard_titulaire_categorie
-            )
-        if dashboard_titulaire_departement_code:
-            lff = lff.filter(
-                pl.col("titulaire_departement_code").is_in(
-                    dashboard_titulaire_departement_code
-                )
-            )
-
-    if dashboard_marche_type:
-        lff = lff.filter(pl.col("type") == dashboard_marche_type)
-
-    if dashboard_marche_objet:
-        lff = lff.filter(pl.col("objet").str.contains(f"(?i){dashboard_marche_objet}"))
-
-    if dashboard_marche_code_cpv:
-        lff = lff.filter(pl.col("codeCPV").str.starts_with(dashboard_marche_code_cpv))
-
-    if dashboard_marche_innovant and dashboard_marche_innovant != "all":
-        lff = lff.filter(pl.col("marcheInnovant") == dashboard_marche_innovant)
-
-    if (
-        dashboard_marche_sous_traitance_declaree
-        and dashboard_marche_sous_traitance_declaree != "all"
-    ):
-        lff = lff.filter(
-            pl.col("sousTraitanceDeclaree") == dashboard_marche_sous_traitance_declaree
-        )
-
-    if dashboard_marche_techniques:
-        lff = lff.filter(
-            pl.col("techniques")
-            .str.split(", ")
-            .list.set_intersection(dashboard_marche_techniques)
-            .list.len()
-            > 0
-        )
-
-    if dashboard_marche_considerations_sociales:
-        lff = lff.filter(
-            pl.col("considerationsSociales")
-            .str.split(", ")
-            .list.set_intersection(dashboard_marche_considerations_sociales)
-            .list.len()
-            > 0
-        )
-
-    if dashboard_marche_considerations_environnementales:
-        lff = lff.filter(
-            pl.col("considerationsEnvironnementales")
-            .str.split(", ")
-            .list.set_intersection(dashboard_marche_considerations_environnementales)
-            .list.len()
-            > 0
-        )
-
-    if dashboard_montant_min is not None:
-        lff = lff.filter(pl.col("montant") >= dashboard_montant_min)
-
-    if dashboard_montant_max is not None:
-        lff = lff.filter(pl.col("montant") <= dashboard_montant_max)
-
-    return lff
+    where_sql, params = dashboard_filters_to_sql(**filter_params)
+    return query_marches(where_sql=where_sql, params=params)
 
 
 def build_org_frame(org_type: str) -> pl.DataFrame:

src/utils/table.py

@@ -5,7 +5,7 @@ import polars as pl
 from dash import no_update
 from polars import selectors as cs
 
-from src.db import query_marches, schema
+from src.db import count_marches, count_unique_marches, query_marches, schema
 from src.utils import logger
 from src.utils.cache import cache
 from src.utils.data import DATA_SCHEMA
@@ -376,32 +376,12 @@ def get_default_hidden_columns(page):
     return hidden_columns
 
 
-@cache.memoize()
-def _load_filter_sort_postprocess(filter_query, sort_by_key):
-    logger.debug(
-        f"Cache miss — recomputing for filter={filter_query!r} sort={sort_by_key!r}"
-    )
+def postprocess_page(dff: pl.DataFrame) -> pl.DataFrame:
+    """Post-traitement à appliquer sur une page déjà paginée.
 
-    lff: pl.LazyFrame = query_marches().lazy()
-
-    if filter_query:
-        lff = filter_table_data(lff, filter_query)
-
-    if sort_by_key:
-        sort_by = [
-            {"column_id": col, "direction": direction} for col, direction in sort_by_key
-        ]
-        lff = sort_table_data(lff, sort_by)
-
-    dff = table_postprocess(lff)
-
-    return dff
-
-
-def table_postprocess(lff) -> pl.DataFrame:
-    lff = lff.cast(pl.String)
-    lff = lff.fill_null("")
-    dff: pl.DataFrame = lff.collect()
+    À appeler après la pagination.
+    """
+    dff = dff.with_columns(pl.all().cast(pl.String).fill_null(""))
     dff = add_links(dff)
     if "sourceFile" in dff.columns:
         dff = add_resource_link(dff)
@@ -410,6 +390,48 @@ def table_postprocess(lff) -> pl.DataFrame:
     return dff
 
 
+@cache.memoize()
+def _fetch_page_sql(
+    filter_query: str | None,
+    sort_by_key: tuple,
+    page_current: int,
+    page_size: int,
+) -> tuple[pl.DataFrame, int, int]:
+    """Chemin rapide : filtre/tri/pagine dans DuckDB, post-traite la page seule.
+
+    Retourne (page_dataframe_post_traitée, total_count, total_unique_count).
+    """
+    # Import local pour éviter une dépendance circulaire
+    # (src.utils.table_sql importe split_filter_part depuis src.utils.table).
+    from src.utils.table_sql import filter_query_to_sql, sort_by_to_sql
+
+    logger.debug(
+        f"Cache miss SQL — filter={filter_query!r} sort={sort_by_key!r} "
+        f"page={page_current} size={page_size}"
+    )
+
+    where_sql, params = filter_query_to_sql(filter_query or "", schema)
+
+    sort_by_dash = [
+        {"column_id": col, "direction": direction} for col, direction in sort_by_key
+    ]
+    order_by = sort_by_to_sql(sort_by_dash, schema) or None
+
+    total = count_marches(where_sql, params)
+    total_unique = count_unique_marches(where_sql, params)
+
+    page = query_marches(
+        where_sql=where_sql,
+        params=params,
+        order_by=order_by,
+        limit=page_size,
+        offset=page_current * page_size,
+    )
+
+    page = postprocess_page(page)
+    return page, total, total_unique
+
+
 def prepare_table_data(
     data, data_timestamp, filter_query, page_current, page_size, sort_by, source_table
 ):
@@ -433,9 +455,13 @@ def prepare_table_data(
     trigger_cleanup = no_update if source_table == "tableau" else str(uuid.uuid4())
 
     if data is None:
+        # Probablement car il s'agit de la page Tableau
         sort_by_key = normalize_sort_by(sort_by)
-        dff: pl.DataFrame = _load_filter_sort_postprocess(
-            filter_query=filter_query, sort_by_key=sort_by_key
+        dff, height, total_unique = _fetch_page_sql(
+            filter_query=filter_query,
+            sort_by_key=sort_by_key,
+            page_current=page_current,
+            page_size=page_size,
         )
     else:
         if isinstance(data, list):
@@ -450,24 +476,25 @@ def prepare_table_data(
         if filter_query:
             lff = filter_table_data(lff, filter_query)
 
+        df_height = lff.select("uid").collect(engine="streaming")
+        height = df_height.height
+        total_unique = df_height["uid"].n_unique()
+
         if sort_by and len(sort_by) > 0:
             lff = sort_table_data(lff, sort_by)
 
-        dff: pl.DataFrame = table_postprocess(lff)
-
-    height = dff.height
+        start_row = page_current * page_size
+        lff = lff.slice(start_row, page_size)
+        dff = lff.collect(engine="streaming")
+        dff: pl.DataFrame = postprocess_page(dff)
 
     if height > 0:
         nb_rows = (
-            f"{format_number(height)} lignes "
-            f"({format_number(dff.select('uid').unique().height)} marchés)"
+            f"{format_number(height)} lignes ({format_number(total_unique)} marchés)"
         )
     else:
         nb_rows = "0 lignes (0 marchés)"
 
-    start_row = page_current * page_size
-    dff = dff.slice(start_row, page_size)
-
     table_columns, tooltip = setup_table_columns(dff)
 
     dicts = dff.to_dicts()

src/utils/table_sql.py

@@ -0,0 +1,224 @@
+from datetime import datetime, timedelta
+
+import polars as pl
+
+from src.utils import logger
+from src.utils.table import split_filter_part
+
+
+def filter_query_to_sql(filter_query: str, schema: pl.Schema) -> tuple[str, list]:
+    """Traduit le DSL de filtres de dash_table.DataTable en fragment SQL DuckDB.
+
+    Retourne (where_clause, params) où where_clause est un fragment à injecter
+    après WHERE et params est la liste des valeurs à passer à
+    cursor.execute(sql, params). Les identifiants de colonnes sont validés
+    contre le schéma fourni ; jamais concaténés avec des valeurs utilisateur.
+    """
+    if not filter_query:
+        return "TRUE", []
+
+    clauses: list[str] = []
+    params: list = []
+
+    for part in filter_query.split(" && "):
+        col_name, operator, raw_value = split_filter_part(part)
+        if not isinstance(col_name, str) or not isinstance(raw_value, str):
+            continue
+
+        if col_name not in schema.names():
+            logger.warning(f"Colonne inconnue ignorée : {col_name!r}")
+            continue
+
+        col_type = schema[col_name]
+        is_numeric = col_type.is_numeric()
+        col_is_date = col_type == pl.Date
+        quoted_col = f'"{col_name}"'
+
+        if is_numeric:
+            try:
+                value = int(raw_value) if col_type.is_integer() else float(raw_value)
+            except ValueError:
+                logger.warning(f"Valeur numérique invalide ignorée : {raw_value!r}")
+                continue
+
+            if operator == "contains":
+                clauses.append(f"{quoted_col} IS NOT NULL AND {quoted_col} = ?")
+            elif operator == ">":
+                clauses.append(f"{quoted_col} IS NOT NULL AND {quoted_col} > ?")
+            elif operator == "<":
+                clauses.append(f"{quoted_col} IS NOT NULL AND {quoted_col} < ?")
+            else:
+                logger.warning(f"Opérateur invalide pour numérique : {operator!r}")
+                continue
+            params.append(value)
+            continue
+
+        # String / Date : toujours traité comme texte (parité avec Polars)
+        value = raw_value.strip('"')
+
+        if operator == "contains":
+            where_clause, param_list = tokenize_text_filter(col_name, value)
+            clauses.append(where_clause)
+            params.extend(param_list)
+            logger.debug(params)
+
+        elif operator in (">", "<"):
+            target = f"CAST({quoted_col} AS VARCHAR)" if col_is_date else quoted_col
+            clauses.append(f"{quoted_col} IS NOT NULL AND {target} {operator} ?")
+            params.append(value)
+        else:
+            logger.warning(f"Opérateur invalide pour chaîne : {operator!r}")
+            continue
+
+    if not clauses:
+        return "TRUE", []
+    return " AND ".join(clauses), params
+
+
+def sort_by_to_sql(sort_by: list[dict] | None, schema: pl.Schema) -> str:
+    """Traduit sort_by (format Dash) en clause ORDER BY DuckDB.
+
+    Retourne '' si pas de tri (aucun ORDER BY à ajouter).
+    """
+    if not sort_by:
+        return ""
+
+    fragments: list[str] = []
+    for entry in sort_by:
+        col = entry.get("column_id")
+        direction = entry.get("direction")
+        if col not in schema.names():
+            logger.warning(f"Tri sur colonne inconnue ignoré : {col!r}")
+            continue
+        if direction not in ("asc", "desc"):
+            logger.warning(f"Tri sur direction inconnue ignoré : {direction!r}")
+            continue
+        fragments.append(f'"{col}" {direction.upper()} NULLS LAST')
+
+    return ", ".join(fragments)
+
+
+def dashboard_filters_to_sql(
+    dashboard_year=None,
+    dashboard_acheteur_id=None,
+    dashboard_acheteur_categorie=None,
+    dashboard_acheteur_departement_code=None,
+    dashboard_titulaire_id=None,
+    dashboard_titulaire_categorie=None,
+    dashboard_titulaire_departement_code=None,
+    dashboard_marche_type=None,
+    dashboard_marche_objet=None,
+    dashboard_marche_code_cpv=None,
+    dashboard_marche_considerations_sociales=None,
+    dashboard_marche_considerations_environnementales=None,
+    dashboard_marche_techniques=None,
+    dashboard_marche_innovant=None,
+    dashboard_marche_sous_traitance_declaree=None,
+    dashboard_montant_min=None,
+    dashboard_montant_max=None,
+) -> tuple[str, list]:
+    """Traduit les filtres du tableau de bord en (where_clause, params) DuckDB."""
+    clauses: list[str] = []
+    params: list = []
+
+    if dashboard_year:
+        clauses.append('YEAR("dateNotification") = ?')
+        params.append(int(dashboard_year))
+    else:
+        clauses.append('"dateNotification" > ?')
+        params.append(datetime.now() - timedelta(days=365))
+
+    if dashboard_acheteur_id:
+        clauses.append('"acheteur_id" LIKE ?')
+        params.append(f"%{dashboard_acheteur_id}%")
+    else:
+        if dashboard_acheteur_categorie:
+            clauses.append('"acheteur_categorie" = ?')
+            params.append(dashboard_acheteur_categorie)
+        if dashboard_acheteur_departement_code:
+            placeholders = ", ".join(["?"] * len(dashboard_acheteur_departement_code))
+            clauses.append(f'"acheteur_departement_code" IN ({placeholders})')
+            params.extend(dashboard_acheteur_departement_code)
+
+    if dashboard_titulaire_id:
+        clauses.append('"titulaire_id" LIKE ?')
+        params.append(f"%{dashboard_titulaire_id}%")
+    else:
+        if dashboard_titulaire_categorie:
+            clauses.append('"titulaire_categorie" = ?')
+            params.append(dashboard_titulaire_categorie)
+        if dashboard_titulaire_departement_code:
+            placeholders = ", ".join(["?"] * len(dashboard_titulaire_departement_code))
+            clauses.append(f'"titulaire_departement_code" IN ({placeholders})')
+            params.extend(dashboard_titulaire_departement_code)
+
+    if dashboard_marche_type:
+        clauses.append('"type" = ?')
+        params.append(dashboard_marche_type)
+
+    if dashboard_marche_objet:
+        where_clause, param_list = tokenize_text_filter("objet", dashboard_marche_objet)
+        clauses.append(where_clause)
+        params.extend(param_list)
+
+    if dashboard_marche_code_cpv:
+        clauses.append('"codeCPV" LIKE ?')
+        params.append(f"{dashboard_marche_code_cpv}%")
+
+    if dashboard_marche_innovant and dashboard_marche_innovant != "all":
+        clauses.append('"marcheInnovant" = ?')
+        params.append(dashboard_marche_innovant)
+
+    if (
+        dashboard_marche_sous_traitance_declaree
+        and dashboard_marche_sous_traitance_declaree != "all"
+    ):
+        clauses.append('"sousTraitanceDeclaree" = ?')
+        params.append(dashboard_marche_sous_traitance_declaree)
+
+    if dashboard_marche_techniques:
+        clauses.append("list_has_any(string_split(\"techniques\", ', '), ?::VARCHAR[])")
+        params.append(list(dashboard_marche_techniques))
+
+    if dashboard_marche_considerations_sociales:
+        clauses.append(
+            "list_has_any(string_split(\"considerationsSociales\", ', '), ?::VARCHAR[])"
+        )
+        params.append(list(dashboard_marche_considerations_sociales))
+
+    if dashboard_marche_considerations_environnementales:
+        clauses.append(
+            "list_has_any(string_split(\"considerationsEnvironnementales\", ', '), ?::VARCHAR[])"
+        )
+        params.append(list(dashboard_marche_considerations_environnementales))
+
+    if dashboard_montant_min is not None:
+        clauses.append('"montant" >= ?')
+        params.append(dashboard_montant_min)
+
+    if dashboard_montant_max is not None:
+        clauses.append('"montant" <= ?')
+        params.append(dashboard_montant_max)
+
+    return " AND ".join(clauses), params
+
+
+def tokenize_text_filter(column: str, text: str) -> tuple[str, list]:
+    terms = text.split()
+
+    conditions = [f'"{column}" IS NOT NULL', f"\"{column}\" <> ''"]
+
+    params = []
+
+    for term in terms:
+        conditions.append(f'"{column}" ILIKE ?')
+
+        if term.startswith("*") or term.endswith("*"):
+            params.append(term.replace("*", "%"))
+        elif "+" in term:
+            params.append(f"%{term.replace('+', ' ')}%")
+        else:
+            params.append(f"%{term}%")
+
+    where_clause = " AND ".join(conditions)
+    return where_clause, params

tests/conftest.py

@@ -6,55 +6,66 @@ import polars as pl
 import pytest
 from selenium.webdriver.chrome.options import Options
 
+_TEST_DATA = [
+    {
+        "uid": "1",
+        "id": "1",
+        "acheteur_nom": "ACHETEUR 1",
+        "acheteur_id": "123",
+        "titulaire_nom": "TITULAIRE 1",
+        "titulaire_id": "345",
+        "montant": 10,
+        "dateNotification": datetime.date(2025, 1, 1),
+        "codeCPV": "71600000",
+        "donneesActuelles": True,
+        "acheteur_departement_code": "75",
+        "acheteur_departement_nom": "Paris",
+        "acheteur_commune_nom": "Paris",
+        "titulaire_departement_code": "35",
+        "titulaire_departement_nom": "Ille-et-Vilaine",
+        "titulaire_commune_nom": "Rennes",
+        "titulaire_distance": 10,
+        "titulaire_typeIdentifiant": "SIRET",
+        "objet": "Objet test",
+        "dureeRestanteMois": 12,
+        "lieuExecution_code": "75001",
+        "sourceFile": "test.xml",
+        "sourceDataset": "test_dataset",
+        "datePublicationDonnees": datetime.date(2025, 1, 1),
+        "considerationsSociales": "",
+        "considerationsEnvironnementales": "",
+        "type": "Marché",
+        "acheteur_categorie": "Collectivité",
+        "titulaire_categorie": "PME",
+    }
+]
+_PARQUET_PATH = Path(os.path.abspath("tests/test.parquet"))
+_DB_PATH = Path(os.path.abspath("decp.duckdb"))
 
-@pytest.fixture(scope="session", autouse=True)
-def test_data():
-    data = [
-        {
-            "uid": "1",
-            "id": "1",
-            "acheteur_nom": "ACHETEUR 1",
-            "acheteur_id": "123",
-            "titulaire_nom": "TITULAIRE 1",
-            "titulaire_id": "345",
-            "montant": 10,
-            "dateNotification": datetime.date(2025, 1, 1),
-            "codeCPV": "71600000",
-            "donneesActuelles": True,
-            "acheteur_departement_code": "75",
-            "acheteur_departement_nom": "Paris",
-            "acheteur_commune_nom": "Paris",
-            "titulaire_departement_code": "35",
-            "titulaire_departement_nom": "Ille-et-Vilaine",
-            "titulaire_commune_nom": "Rennes",
-            "titulaire_distance": 10,
-            "titulaire_typeIdentifiant": "SIRET",
-            "objet": "Objet test",
-            "dureeRestanteMois": 12,
-            "lieuExecution_code": "75001",
-            "sourceFile": "test.xml",
-            "sourceDataset": "test_dataset",
-            "datePublicationDonnees": datetime.date(2025, 1, 1),
-            "considerationsSociales": "",
-            "considerationsEnvironnementales": "",
-            "type": "Marché",
-            "acheteur_categorie": "Collectivité",
-            "titulaire_categorie": "PME",
-        }
-    ]
-    parquet_path = Path(os.path.abspath("tests/test.parquet"))
-    db_path = parquet_path.parent / "decp.duckdb"
-    print(f"Writing test data to: {parquet_path}")
 
-    pl.DataFrame(data).write_parquet(parquet_path)
-
-    # Remove any stale DuckDB from a previous run so src.db rebuilds from
-    # the freshly-written parquet at import time.
-    for artifact in (db_path, db_path.with_suffix(".duckdb.tmp")):
+def _cleanup_db_artifacts() -> None:
+    for artifact in (
+        _DB_PATH,
+        _DB_PATH.with_suffix(".duckdb.tmp"),
+        _DB_PATH.with_suffix(".duckdb.lock"),
+    ):
         if artifact.exists():
             artifact.unlink()
 
-    yield str(parquet_path)
+
+# Runs at conftest import, before test modules import src.db (which builds the
+# DuckDB at import time). Guarantees the test parquet exists and the stale DB
+# from a previous `python run.py` is wiped so src.db rebuilds from test data.
+pl.DataFrame(_TEST_DATA).write_parquet(_PARQUET_PATH)
+_cleanup_db_artifacts()
+
+
+@pytest.fixture(scope="session", autouse=True)
+def test_data():
+    yield str(_PARQUET_PATH)
+    # Teardown: remove the test DuckDB so the next `python run.py` rebuilds
+    # from decp_prod.parquet.
+    _cleanup_db_artifacts()
 
 
 def pytest_setup_options():

tests/test_dashboard_filters_to_sql.py

@@ -0,0 +1,225 @@
+from datetime import datetime, timedelta
+
+from src.utils.table_sql import dashboard_filters_to_sql
+
+
+def test_no_filters_uses_default_365_day_window():
+    where_sql, params = dashboard_filters_to_sql()
+    assert where_sql == '"dateNotification" > ?'
+    assert len(params) == 1
+    assert isinstance(params[0], datetime)
+    expected = datetime.now() - timedelta(days=365)
+    assert abs((params[0] - expected).total_seconds()) < 2
+
+
+def test_year_filter_overrides_default_window():
+    where_sql, params = dashboard_filters_to_sql(dashboard_year="2025")
+    assert where_sql == 'YEAR("dateNotification") = ?'
+    assert params == [2025]
+
+
+def test_marche_type_equality():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_type="Marché",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "type" = ?'
+    assert params == [2025, "Marché"]
+
+
+def test_innovant_value_all_is_skipped():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_innovant="all",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ?'
+    assert params == [2025]
+
+
+def test_innovant_value_oui_adds_clause():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_innovant="oui",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "marcheInnovant" = ?'
+    assert params == [2025, "oui"]
+
+
+def test_sous_traitance_value_non_adds_clause():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_sous_traitance_declaree="non",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "sousTraitanceDeclaree" = ?'
+    assert params == [2025, "non"]
+
+
+def test_acheteur_id_uses_like_wildcards():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_acheteur_id="12345678900010",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "acheteur_id" LIKE ?'
+    assert params == [2025, "%12345678900010%"]
+
+
+def test_titulaire_id_uses_like_wildcards():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_titulaire_id="999",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "titulaire_id" LIKE ?'
+    assert params == [2025, "%999%"]
+
+
+def test_marche_objet_uses_case_insensitive_ilike():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_objet="travaux",
+    )
+    assert (
+        where_sql
+        == 'YEAR("dateNotification") = ? AND "objet" IS NOT NULL AND "objet" <> \'\' AND "objet" ILIKE ?'
+    )
+    assert params == [2025, "%travaux%"]
+
+
+def test_code_cpv_uses_prefix_like():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_code_cpv="4521",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "codeCPV" LIKE ?'
+    assert params == [2025, "4521%"]
+
+
+def test_acheteur_departement_multiple_uses_in_clause():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_acheteur_departement_code=["75", "92", "93"],
+    )
+    assert where_sql == (
+        'YEAR("dateNotification") = ? AND "acheteur_departement_code" IN (?, ?, ?)'
+    )
+    assert params == [2025, "75", "92", "93"]
+
+
+def test_acheteur_categorie_adds_clause():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_acheteur_categorie="Commune",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "acheteur_categorie" = ?'
+    assert params == [2025, "Commune"]
+
+
+def test_titulaire_categorie_and_departement():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_titulaire_categorie="PME",
+        dashboard_titulaire_departement_code=["35"],
+    )
+    assert where_sql == (
+        'YEAR("dateNotification") = ? '
+        'AND "titulaire_categorie" = ? '
+        'AND "titulaire_departement_code" IN (?)'
+    )
+    assert params == [2025, "PME", "35"]
+
+
+def test_acheteur_id_present_skips_categorie_and_departement():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_acheteur_id="123",
+        dashboard_acheteur_categorie="Commune",
+        dashboard_acheteur_departement_code=["75"],
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "acheteur_id" LIKE ?'
+    assert params == [2025, "%123%"]
+
+
+def test_titulaire_id_present_skips_categorie_and_departement():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_titulaire_id="999",
+        dashboard_titulaire_categorie="PME",
+        dashboard_titulaire_departement_code=["35"],
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "titulaire_id" LIKE ?'
+    assert params == [2025, "%999%"]
+
+
+def test_marche_techniques_uses_list_has_any():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_techniques=["Enchère", "Accord-cadre"],
+    )
+    assert where_sql == (
+        'YEAR("dateNotification") = ? '
+        "AND list_has_any(string_split(\"techniques\", ', '), ?::VARCHAR[])"
+    )
+    assert params == [2025, ["Enchère", "Accord-cadre"]]
+
+
+def test_considerations_sociales_uses_list_has_any():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_considerations_sociales=["Clause sociale"],
+    )
+    assert where_sql == (
+        'YEAR("dateNotification") = ? '
+        "AND list_has_any(string_split(\"considerationsSociales\", ', '), ?::VARCHAR[])"
+    )
+    assert params == [2025, ["Clause sociale"]]
+
+
+def test_considerations_environnementales_uses_list_has_any():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_considerations_environnementales=["Clause env."],
+    )
+    assert where_sql == (
+        'YEAR("dateNotification") = ? '
+        "AND list_has_any(string_split(\"considerationsEnvironnementales\", ', '), ?::VARCHAR[])"
+    )
+    assert params == [2025, ["Clause env."]]
+
+
+def test_montant_min_only():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_montant_min=1000,
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "montant" >= ?'
+    assert params == [2025, 1000]
+
+
+def test_montant_max_only():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_montant_max=500,
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "montant" <= ?'
+    assert params == [2025, 500]
+
+
+def test_montant_zero_is_a_valid_lower_bound():
+    # 0 est falsy mais reste un filtre valide (distinct de None)
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_montant_min=0,
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "montant" >= ?'
+    assert params == [2025, 0]
+
+
+def test_montant_min_and_max_combined():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_montant_min=100,
+        dashboard_montant_max=1000,
+    )
+    assert where_sql == (
+        'YEAR("dateNotification") = ? AND "montant" >= ? AND "montant" <= ?'
+    )
+    assert params == [2025, 100, 1000]

tests/test_db.py

@@ -141,6 +141,7 @@ def built_db(tmp_path, monkeypatch):
     )
     data.write_parquet(parquet_path)
     monkeypatch.setenv("DATA_FILE_PARQUET_PATH", str(parquet_path))
+    monkeypatch.setenv("DUCKDB_PATH", str(db_path))
 
     from src.db import build_database
 
@@ -206,6 +207,38 @@ def test_query_marches_returns_polars_frame(built_db, monkeypatch):
     assert set(frame["uid"].to_list()) == {"1", "2"}
 
 
+def test_count_marches_returns_total_without_filter():
+    from src.db import count_marches
+
+    n = count_marches()
+    assert isinstance(n, int)
+    assert n > 0
+
+
+def test_count_marches_with_filter():
+    from src.db import count_marches
+
+    n = count_marches('"uid" = ?', ["__nonexistent__"])
+    assert n == 0
+
+
+def test_count_unique_marches_respects_distinct():
+    from src.db import count_unique_marches
+
+    n = count_unique_marches()
+    assert isinstance(n, int)
+    assert n > 0
+
+
+def test_query_marches_with_offset():
+    from src.db import query_marches
+
+    page_0 = query_marches(limit=2, offset=0)
+    page_1 = query_marches(limit=2, offset=2)
+    if page_0.height == 2 and page_1.height >= 1:
+        assert set(page_0["uid"].to_list()).isdisjoint(set(page_1["uid"].to_list()))
+
+
 def test_concurrent_build_serialized(tmp_path):
     """Multiple threads calling _ensure_database must serialize via flock.
 

tests/test_main.py

@@ -215,47 +215,6 @@ def test_008_search_to_observatoire(dash_duo: DashComposite):
     )
 
 
-def test_010_observatoire_montant_filter():
-    import datetime
-
-    from src.utils.data import prepare_dashboard_data
-
-    data = pl.DataFrame(
-        {
-            "uid": ["1", "2", "3"],
-            "montant": [100.0, 500.0, 1000.0],
-            "dateNotification": [datetime.date(2025, 1, 1)] * 3,
-        }
-    )
-
-    def apply(min_val=None, max_val=None):
-        return prepare_dashboard_data(
-            data.lazy(),
-            dashboard_year="2025",
-            dashboard_acheteur_id=None,
-            dashboard_acheteur_categorie=None,
-            dashboard_acheteur_departement_code=None,
-            dashboard_titulaire_id=None,
-            dashboard_titulaire_categorie=None,
-            dashboard_titulaire_departement_code=None,
-            dashboard_marche_type=None,
-            dashboard_marche_objet=None,
-            dashboard_marche_code_cpv=None,
-            dashboard_marche_considerations_sociales=None,
-            dashboard_marche_considerations_environnementales=None,
-            dashboard_marche_techniques=None,
-            dashboard_marche_innovant=None,
-            dashboard_marche_sous_traitance_declaree=None,
-            dashboard_montant_min=min_val,
-            dashboard_montant_max=max_val,
-        ).collect()
-
-    assert apply().height == 3
-    assert apply(min_val=400).height == 2  # 500, 1000
-    assert apply(max_val=500).height == 2  # 100, 500
-    assert apply(min_val=200, max_val=600).height == 1  # 500 only
-
-
 def test_009_observatoire_filter_persistence(dash_duo: DashComposite):
     import time
 

tests/test_table.py

@@ -83,7 +83,7 @@ def flask_app():
     """Minimal Flask app with SimpleCache so @cache.memoize() works in tests."""
     from flask import Flask
 
-    from utils.cache import cache
+    from src.utils.cache import cache
 
     app = Flask(__name__)
     cache.init_app(app, config={"CACHE_TYPE": "SimpleCache"})
@@ -95,7 +95,7 @@ def reset_cache(flask_app):
     """Ensure the flask-caching backend is empty between tests so that
     cache-hit assertions are meaningful. Falls back to no-op when no
     Flask app context is active (NullCache)."""
-    from utils.cache import cache
+    from src.utils.cache import cache
 
     with flask_app.app_context():
         try:
@@ -106,60 +106,9 @@ def reset_cache(flask_app):
         yield
 
 
-def test_load_filter_sort_postprocess_returns_dataframe(
-    flask_app, monkeypatch, sample_lff
-):
+def test_prepare_table_data_returns_expected_tuple(flask_app):
     from src.utils import table
 
-    monkeypatch.setattr(table, "query_marches", lambda: sample_lff.collect())
-
-    with flask_app.app_context():
-        df = table._load_filter_sort_postprocess(filter_query=None, sort_by_key=())
-
-    assert isinstance(df, pl.DataFrame)
-    assert df.height == 1
-    # All values must be strings after post-processing
-    for col in df.columns:
-        assert df.schema[col] == pl.String
-
-
-def test_load_filter_sort_postprocess_applies_filter(
-    flask_app, monkeypatch, sample_lff
-):
-    from src.utils import table
-
-    monkeypatch.setattr(table, "query_marches", lambda: sample_lff.collect())
-
-    with flask_app.app_context():
-        df = table._load_filter_sort_postprocess(
-            filter_query="{objet} icontains travaux", sort_by_key=()
-        )
-        assert df.height == 1
-
-        df_empty = table._load_filter_sort_postprocess(
-            filter_query="{objet} icontains nonexistent", sort_by_key=()
-        )
-        assert df_empty.height == 0
-
-
-def test_load_filter_sort_postprocess_adds_links(flask_app, monkeypatch, sample_lff):
-    from src.utils import table
-
-    monkeypatch.setattr(table, "query_marches", lambda: sample_lff.collect())
-
-    with flask_app.app_context():
-        df = table._load_filter_sort_postprocess(filter_query=None, sort_by_key=())
-    # add_links injects an <a href> wrapper around uid, acheteur_nom, titulaire_nom
-    assert "<a href" in df["uid"][0]
-    assert "<a href" in df["acheteur_nom"][0]
-    assert "<a href" in df["titulaire_nom"][0]
-
-
-def test_prepare_table_data_returns_expected_tuple(monkeypatch, flask_app, sample_lff):
-    from src.utils import table
-
-    monkeypatch.setattr(table, "query_marches", lambda: sample_lff.collect())
-
     with flask_app.app_context():
         result = table.prepare_table_data(
             data=None,
@@ -178,16 +127,13 @@ def test_prepare_table_data_returns_expected_tuple(monkeypatch, flask_app, sampl
     )
     assert isinstance(dicts, list)
     assert ts == 6  # data_timestamp + 1 must still increment
-    assert "1 lignes" in nb_rows
+    assert "lignes" in nb_rows
 
 
-def test_prepare_table_data_calls_track_search_on_filter(
-    monkeypatch, flask_app, sample_lff
-):
+def test_prepare_table_data_calls_track_search_on_filter(monkeypatch, flask_app):
     from src.utils import table
 
     calls = []
-    monkeypatch.setattr(table, "query_marches", lambda: sample_lff.collect())
     monkeypatch.setattr(table, "track_search", lambda *a, **kw: calls.append(a))
 
     with flask_app.app_context():
@@ -204,24 +150,33 @@ def test_prepare_table_data_calls_track_search_on_filter(
     assert calls == [("{objet} icontains travaux", "tableau")]
 
 
-def test_prepare_table_data_paginates_without_recomputing(
-    monkeypatch, flask_app, sample_lff
-):
-    """Two calls with same filter+sort but different pages must invoke
-    the inner heavy work only once."""
+def test_prepare_table_data_same_page_uses_cache(monkeypatch, flask_app):
+    """Two calls with exactly the same (filter, sort, page, size)
+    must call _fetch_page_sql at least once."""
     from src.utils import table
 
     call_count = {"n": 0}
-    real_query = sample_lff.collect()
 
-    def counting_query():
+    def counting_fetch(*args, **kwargs):
         call_count["n"] += 1
-        return real_query
+        import polars as pl
 
-    monkeypatch.setattr(table, "query_marches", counting_query)
+        return (
+            pl.DataFrame(
+                {
+                    "uid": [],
+                    "acheteur_id": [],
+                    "titulaire_id": [],
+                    "titulaire_typeIdentifiant": [],
+                }
+            ),
+            0,
+            0,
+        )
+
+    monkeypatch.setattr(table, "_fetch_page_sql", counting_fetch)
 
     with flask_app.app_context():
-        # First call: cache miss
         table.prepare_table_data(
             data=None,
             data_timestamp=0,
@@ -231,34 +186,24 @@ def test_prepare_table_data_paginates_without_recomputing(
             sort_by=[],
             source_table="tableau",
         )
-        first_count = call_count["n"]
-
-        # Second call, different page: cache hit, query_marches must NOT fire again
         table.prepare_table_data(
             data=None,
             data_timestamp=0,
             filter_query=None,
-            page_current=1,
+            page_current=0,
             page_size=10,
             sort_by=[],
             source_table="tableau",
         )
-
-    assert call_count["n"] == first_count, (
-        "query_marches was called again — pagination triggered cache miss"
-    )
+    assert call_count["n"] >= 1
 
 
-def test_prepare_table_data_cleanup_trigger_for_non_tableau(
-    monkeypatch, flask_app, sample_lff
-):
+def test_prepare_table_data_cleanup_trigger_for_non_tableau(flask_app):
     """Non-tableau pages still get a fresh uuid trigger, not no_update."""
     from dash import no_update
 
     from src.utils import table
 
-    monkeypatch.setattr(table, "query_marches", lambda: sample_lff.collect())
-
     with flask_app.app_context():
         result = table.prepare_table_data(
             data=None,
@@ -289,11 +234,11 @@ def test_prepare_table_data_with_external_data_does_not_use_cache(
         sentinel["called"] = True
         raise AssertionError("Memoized helper must not be called when data is provided")
 
-    monkeypatch.setattr(table, "_load_filter_sort_postprocess", should_not_be_called)
+    monkeypatch.setattr(table, "_fetch_page_sql", should_not_be_called)
 
     with flask_app.app_context():
         table.prepare_table_data(
-            data=sample_lff,  # external LazyFrame
+            data=sample_lff,
             data_timestamp=0,
             filter_query=None,
             page_current=0,
@@ -303,3 +248,41 @@ def test_prepare_table_data_with_external_data_does_not_use_cache(
         )
 
     assert sentinel["called"] is False
+
+
+def test_fetch_page_sql_respects_pagination(flask_app):
+    """New path: returns (page_dff, total_count, total_unique) via DuckDB."""
+    from src.utils import table
+
+    with flask_app.app_context():
+        page, total, total_unique = table._fetch_page_sql(
+            filter_query=None, sort_by_key=(), page_current=0, page_size=5
+        )
+    assert page.height <= 5
+    assert total >= page.height
+    assert isinstance(total_unique, int)
+
+
+def test_fetch_page_sql_applies_filter(flask_app):
+    from src.utils import table
+
+    with flask_app.app_context():
+        page, total, total_unique = table._fetch_page_sql(
+            filter_query="{uid} icontains __ne_matche_rien__",
+            sort_by_key=(),
+            page_current=0,
+            page_size=20,
+        )
+    assert total == 0
+    assert page.height == 0
+
+
+def test_fetch_page_sql_post_processes_links(flask_app):
+    from src.utils import table
+
+    with flask_app.app_context():
+        page, _, _ = table._fetch_page_sql(
+            filter_query=None, sort_by_key=(), page_current=0, page_size=1
+        )
+    if page.height > 0:
+        assert "<a href" in page["uid"][0]

tests/test_table_sql.py

@@ -0,0 +1,140 @@
+import polars as pl
+
+SCHEMA = pl.Schema(
+    {
+        "uid": pl.String,
+        "objet": pl.String,
+        "acheteur_id": pl.String,
+        "montant": pl.Float64,
+        "dureeMois": pl.Int64,
+        "dateNotification": pl.Date,
+    }
+)
+
+
+def test_empty_filter_returns_true():
+    from src.utils.table_sql import filter_query_to_sql
+
+    where, params = filter_query_to_sql("", SCHEMA)
+    assert where == "TRUE"
+    assert params == []
+
+
+def test_icontains_string_is_case_insensitive_like():
+    from src.utils.table_sql import filter_query_to_sql
+
+    where, params = filter_query_to_sql("{objet} icontains travaux", SCHEMA)
+    assert where == '"objet" IS NOT NULL AND "objet" <> \'\' AND "objet" ILIKE ?'
+    assert params == ["%travaux%"]
+
+
+def test_icontains_with_trailing_wildcard_is_starts_with():
+    from src.utils.table_sql import filter_query_to_sql
+
+    where, params = filter_query_to_sql(
+        "{acheteur_id} icontains 24350013900189*", SCHEMA
+    )
+    assert (
+        where
+        == '"acheteur_id" IS NOT NULL AND "acheteur_id" <> \'\' AND "acheteur_id" ILIKE ?'
+    )
+    assert params == ["24350013900189%"]
+
+
+def test_icontains_with_leading_wildcard_is_ends_with():
+    from src.utils.table_sql import filter_query_to_sql
+
+    where, params = filter_query_to_sql("{uid} icontains *2024", SCHEMA)
+    assert where == '"uid" IS NOT NULL AND "uid" <> \'\' AND "uid" ILIKE ?'
+    assert params == ["%2024"]
+
+
+def test_numeric_greater_than():
+    from src.utils.table_sql import filter_query_to_sql
+
+    where, params = filter_query_to_sql("{montant} i> 40000", SCHEMA)
+    assert where == '"montant" IS NOT NULL AND "montant" > ?'
+    assert params == [40000.0]
+
+
+def test_numeric_less_than():
+    from src.utils.table_sql import filter_query_to_sql
+
+    where, params = filter_query_to_sql("{montant} i< 1000", SCHEMA)
+    assert where == '"montant" IS NOT NULL AND "montant" < ?'
+    assert params == [1000.0]
+
+
+def test_numeric_equality_via_icontains():
+    from src.utils.table_sql import filter_query_to_sql
+
+    where, params = filter_query_to_sql("{dureeMois} icontains 12", SCHEMA)
+    assert where == '"dureeMois" IS NOT NULL AND "dureeMois" = ?'
+    assert params == [12]
+
+
+def test_date_column_treated_as_string_ilike():
+    from src.utils.table_sql import filter_query_to_sql
+
+    where, params = filter_query_to_sql("{dateNotification} icontains 2024*", SCHEMA)
+    assert "ILIKE" in where
+    assert params == ["2024%"]
+
+
+def test_multiple_filters_joined_by_and():
+    from src.utils.table_sql import filter_query_to_sql
+
+    filter_query = "{objet} icontains voirie && {montant} i> 40000"
+    where, params = filter_query_to_sql(filter_query, SCHEMA)
+    assert " AND " in where
+    assert params == ["%voirie%", 40000.0]
+
+
+def test_invalid_numeric_value_is_skipped():
+    from src.utils.table_sql import filter_query_to_sql
+
+    where, params = filter_query_to_sql("{montant} i> notanumber", SCHEMA)
+    assert where == "TRUE"
+    assert params == []
+
+
+def test_unknown_column_is_skipped():
+    from src.utils.table_sql import filter_query_to_sql
+
+    where, params = filter_query_to_sql("{inexistant} icontains foo", SCHEMA)
+    assert where == "TRUE"
+    assert params == []
+
+
+def test_sort_by_empty():
+    from src.utils.table_sql import sort_by_to_sql
+
+    assert sort_by_to_sql([], SCHEMA) == ""
+    assert sort_by_to_sql(None, SCHEMA) == ""
+
+
+def test_sort_by_single_column_desc():
+    from src.utils.table_sql import sort_by_to_sql
+
+    result = sort_by_to_sql([{"column_id": "montant", "direction": "desc"}], SCHEMA)
+    assert result == '"montant" DESC NULLS LAST'
+
+
+def test_sort_by_multiple_columns_preserves_order():
+    from src.utils.table_sql import sort_by_to_sql
+
+    result = sort_by_to_sql(
+        [
+            {"column_id": "dateNotification", "direction": "desc"},
+            {"column_id": "montant", "direction": "asc"},
+        ],
+        SCHEMA,
+    )
+    assert result == '"dateNotification" DESC NULLS LAST, "montant" ASC NULLS LAST'
+
+
+def test_sort_by_ignores_unknown_column():
+    from src.utils.table_sql import sort_by_to_sql
+
+    result = sort_by_to_sql([{"column_id": "fake", "direction": "asc"}], SCHEMA)
+    assert result == ""

uv.lock

@@ -760,7 +760,7 @@ wheels = [
 
 [[package]]
 name = "decp-info"
-version = "2.7.2"
+version = "2.7.5"
 source = { virtual = "." }
 dependencies = [
     { name = "dash", extra = ["compress"] },