Merge branch 'release/2.7.7'

Omnikles publie via l'API DUME de l'AIFE, ne reste que Klekoon. Mais bon, ça fait des mauvaises vibes sur la page de le préciser :)

.github/workflows/deploy.yaml

@@ -20,7 +20,7 @@ jobs:
     environment: ${{ github.ref_name }}
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v3
+        uses: actions/checkout@v5
 
       - name: Set up SSH key
         run: |

.gitignore

@@ -3,5 +3,11 @@
 __pycache__
 .idea
 .venv
+.worktrees
 build
 .env
+
+# DuckDB runtime artifacts (regenerated from decp_prod.parquet at startup)
+**/decp.duckdb
+**/decp.duckdb.tmp
+**/decp.duckdb.lock

.template.env

@@ -1,14 +1,26 @@
 DATA_FILE_PARQUET_PATH=https://www.data.gouv.fr/fr/datasets/r/11cea8e8-df3e-4ed1-932b-781e2635e432
+DUCKDB_PATH=./decp.duckdb
 PORT=8050
 DEVELOPMENT=True
 SOURCE_STATS_CSV_PATH="https://www.data.gouv.fr/api/1/datasets/r/8ded94de-3b80-4840-a5bb-7faad1c9c234"
 
+# Annonce dans l'en-tête du site
+ANNOUNCEMENTS=
+
 # Chemin vers le schéma de données
 DATA_SCHEMA_PATH=https://www.data.gouv.fr/api/1/datasets/r/9a4144c0-ee44-4dec-bee5-bbef38191d9a
 
+# Colonnes masquées par défaut
+DISPLAYED_COLUMNS="uid, acheteur_id, acheteur_nom, montant, objet, titulaire_nom, titulaire_id, dateNotification, dureeMois, acheteur_departement_code, sourceDataset"
+
 # Formulaire de contact
 SENDER_SERVER_DOMAIN="mail.example.com" # serveur SMTP
 LOGIN_PASSWORD="" # mot de passe du serveur
 LOGIN_EMAIL="connect@example.fr" # adresse utilisée pour se connecter au serveur SMTP
 FROM_EMAIL="from@example.com" # adresse d'envoi des emails (From)
 TO_EMAIL="to@example.com" # adresse de destination des emails (To)
+
+# Matomo
+MATOMO_ID_SITE=
+MATOMO_BASE_URL=
+MATOMO_TOKEN=

CHANGELOG.md

@@ -0,0 +1,221 @@
+##### 2.7.7 (11 mai 2026)
+
+- Suppression des mentions sur les profils d'acheteur. Omnikles/Safetender publie via l'API DUME et Klekoon ne publie pas, mais c'est peut-être pas le seul, donc je préfère supprimer et refaire un tour.
+
+##### 2.7.6 (5 mai 2026)
+
+- Correction du problème de filtre par date dans les tableaux
+- Retour des cartes dans les pages acheteur et titulaire
+- Possibilité de chercher un SIRET/SIREN avec des espaces dans les champs `SIRET acheteur` et `Identifiant titulaire`
+
+##### 2.7.5 (24 avril 2026)
+
+- Amélioration des permormances de l'observatoire
+- Possibilité dans observatoire (champ objet) et tableau (tous champs texte) de soit chercher des mots présents, soit une suite de mot précise (voir mode d'emploi dans Tableau)
+- Ajout d'une animation pendant le chargement de la prévisualisation des données de l'observatoire
+
+##### 2.7.4 (22 avril 2026)
+
+- Utilisation élargie de DuckDB au détriment de Polars => bien meilleure perf ([#72](https://github.com/ColinMaudry/decp.info/issues/72)
+
+##### 2.7.3 (20 avril 2026)
+
+- Mise en cache des vues tableau par ensemble de filtres et de tris
+- Résolution du bug d'écriture du fichier de vérouillage de la base de données
+
+##### 2.7.2 (19 avril 2026)
+
+- Chargement des données depuis une base DuckDB plutôt qu'en mémoire (plus de stabilité) ([#71](https://github.com/ColinMaudry/decp.info/issues/71))
+- Mise en cache des vue sur l'observatoire pour un chargement plus rapide (remise à zéro quotidienne)
+- Correction de bug : la liste de colonnes par défaut est bien appliquée plutôt qu'afficher toutes les colonnes
+- Quelques corrections de bugs d'affichage
+- Refactorisation des fonctions utilitaires (`utils.py` approchait des 1 000 lignes)
+
+##### 2.7.1 (23 mars 2026)
+
+- Correction du partage de données filtrées entre dashboard et vue des données
+
+#### 2.7.0 (23 mars 2026)
+
+- Remplacement de la page Statistiques par l'observatoire
+- Généralisation de la grille dash (`dbc.Row`, `dbc.Col`)
+- Ajout de l'histogramme de distances aux pages acheteur et titulaire
+- Ajout de la colonne `acheteur_categorie` (commune, État, etc.)
+
+##### 2.6.2 (22 février 2026)
+
+- Correction du téléchargemnent buggé dans /tableau
+
+##### 2.6.1 (17 février 2026)
+
+- Corrections la création des liens canoniques (SEO)
+
+#### 2.6.0 (5 février 2026)
+
+- Suite de la refonte graphique
+- Persistence des filtres, des tris et des choix de colonnes sur toutes les pages
+- Joli tableau pour choisir les colonnes à afficher
+- Meilleure gestion des acheteurs et titulaires absents de la base SIRENE
+- Amélioration du SEO (liens canoniques)
+
+##### 2.5.1 (29 janvier 2026)
+
+- Mise en production un peu hâtive ([#67](https://github.com/ColinMaudry/decp.info/issues/67), [#68](https://github.com/ColinMaudry/decp.info/issues/68))
+
+#### 2.5.0 (29 janvier 2026)
+
+- Refonte graphique et amélioration des textes d'aide
+- Amélioration du filtrage du tableau à partir d'une URL
+- Renforcement du SEO avec une arborescence permettant l'accès aux marchés et des snippets JSON-LD
+- Suppression de la dépendance à Google Fonts grâce à [Bunny Fonts](https://fonts.bunny.net) 🇪🇺 🇸🇮
+
+##### 2.4.1 (22 janvier 2026)
+
+- Meilleure gestion des colonnes absentes du schéma
+
+#### 2.4.0 (22 janvier 2026)
+
+- Site à peu près utilisable sur petit écran (smartphone) ([#63](https://github.com/ColinMaudry/decp.info/issues/63))
+- Ajout de nouvelles statistiques dans [/statistiques](https://decp.info/statistiques) (stats par année, doublons par source)
+- Amélioration du référencement Web (sitemap, titres, descriptions) ([#50](https://github.com/ColinMaudry/decp.info/issues/50))
+- Possibilité dans les champs non-numériques de filtrer le texte selon son début ou sa fin (`text*` et `*text`)
+- Ajout d'une table des matières dans la page [À propos](https://decp.infi/a-propos) ([#36](https://github.com/ColinMaudry/decp.info/issues/36))
+- Désactivation du bloquage des robot d'agents de LLM (robots.txt)
+
+##### 2.3.1 (16 janvier 2026)
+
+- Les champs absents du [schéma](https://www.data.gouv.fr/datasets/donnees-essentielles-de-la-commande-publique-consolidees-format-tabulaire?resource_id=9a4144c0-ee44-4dec-bee5-bbef38191d9a) sont ignorés pour éviter les erreurs
+
+#### 2.3.0 (24 décembre 2025)
+
+- Possibilité de filtrer, trier etc. dans les vues acheteur et titulaire
+- Possibilité de partager les filtres, tris et choix de colonnes via une adresse Web ([exemple](https://decp.info/tableau?filtres=%7Bobjet%7D+icontains+%22d%C3%A9corations+de+no%C3%ABl%22+%26%26+%7BdateNotification%7D+icontains+2025&colonnes=uid%2Cacheteur_id%2Cacheteur_nom%2Ctitulaire_id%2Ctitulaire_nom%2Cobjet%2Cmontant%2CdateNotification%2Cdistance%2Cacheteur_departement_code))
+- Possibilité de filtrer une colonne avec plusieurs mots
+
+##### 2.2.3 (4 décembre 2025)
+
+- mise à jour de l'adresse email de contact (colmo.tech)
+- message sur l'indisponibilité des données MINEF
+
+##### 2.2.2 (22 novembre 2025)
+
+- Correction d'un bug dans le téléchargement Excel
+
+##### 2.2.1 (15 novembre 2025)
+
+- Le moteur de recherche ignore les tirets ("franche comté" trouve "Bourgogne-Franche-Comté)
+- Phrase "tagline" au-dessus du champ de recherche
+- Les infos de Contact rebasculent dans À propos
+- Police de caractère "Open Sans" généralisée
+
+#### 2.2.0 (13 novembre 2025)
+
+- Moteur de recherche (acheteurs et titulaires) en page d'accueil ([#58](https://github.com/ColinMaudry/decp.info/issues/58))
+- Top acheteurs / titulaires par montant attribué/remporté (([#55](https://github.com/ColinMaudry/decp.info/issues/55)))
+- Moins de colonnes affichées par défaut dans Tableau ([#54](https://github.com/ColinMaudry/decp.info/issues/54))
+
+##### 2.1.7 (11 novembre 2025)
+
+- Remplacement du formulaire de contact par une adresse email
+
+##### 2.1.6 (15 octobre 2025)
+
+- Stabilisation de la vue marché
+
+##### 2.1.5 (10 octobre 2025)
+
+- réparation des filtres (notamment < > sur les montants)
+- remplacement des valeurs "null" dans les tableaux par des cellules vides
+
+##### 2.1.4 (8 octobre 2025)
+
+- possibilité de filtrer sur le champ "Source"
+- création automatique d'une release Github quand je push un tag
+
+##### 2.1.3 (4 octobre 2025)
+
+- tentative d'auto-release à chaque création de tag git
+- adaptation au format TableSchema
+
+##### 2.1.2 (3 octobre 2025)
+
+- dataframe global plutôt que lazyframe, pour plus de résilience et charger toutes les données en mémoire
+
+##### 2.1.1 (1er octobre 2025)
+
+- ajout d'une section dans À propos sur la qualité et l'exhaustivité des données ([#43](https://github.com/ColinMaudry/decp.info/issues/43))
+- ajout du nombre de marchés en plus du nombre de lignes dans la vue Tableau
+
+#### 2.1.0 (30 septembre 2025)
+
+- Ajout des vues [acheteur](https://decp.info/acheteurs/24350013900189) ([#28](https://github.com/ColinMaudry/decp.info/issues/28)), [titulaire](https://decp.info/titulaires/51903758414786) ([#35](https://github.com/ColinMaudry/decp.info/issues/35)) et [marché](https://decp.info/marches/532239472000482025S00004) ([#40](https://github.com/ColinMaudry/decp.info/issues/40)) 🔎
+- Ajout des balises HTML meta Open Graph et Twitter ([#39](https://github.com/ColinMaudry/decp.info/issues/39)) pour de beaux aperçus de liens 🖼️
+- Formulaire de contact ([#48](https://github.com/ColinMaudry/decp.info/issues/48)) 📨
+- Nom de colonnes plus_agréables ([#33](https://github.com/ColinMaudry/decp.info/issues/33)) 💅
+- Définition des colonnes quand vous passez votre souris sur les en-têtes ([#33](https://github.com/ColinMaudry/decp.info/issues/33)) 📖
+- Affichage du numéro de version près du logo et lien vers ici 🤓
+- Variables globales uniquement en lecture (😁)
+
+##### 2.0.1 (23 septembre 2025)
+
+- Bloquage du bouton de téléchargement si trop de lignes (+ 65000) [#38](https://github.com/ColinMaudry/decp.info/issues/38)
+- Amélioration du script de déploiement (deploy.sh)
+- Meilleures instructions d'installation et lancement
+- Coquilles 🐚
+
+### 2.0.0 (23 septembre 2025)
+
+- détails des sources de données
+- section "À propos" plus développée
+- correction de bugs dans les filtres de la data table
+
+#### 2.0.0-alpha
+
+- Data table fonctionnelle
+
+### 1.5.0 (28/01/2023
+
+- fixation des dépendances Python pour plus de stabilité en cas de réinstallation (Pipfile)
+
+#### 1.4.1 (14/06/2021)
+
+- ajout des traductions des opérations de filtrage à toutes les vues, pas seulement /db/decp
+
+### 1.4.0 (14/06/2021)
+
+- traduction des opérations de filtrage (ex : contains => contient)
+- élargissement des menus de filtrage
+- correction du titre de la page des notes de versions
+
+### 1.3.0 (03/06/2021)
+
+- utilisation de noms de colonnes plus lisibles dans l'application
+- suppression des références à la licence et aux données source sur la page d'accueil
+- correction des liens vers le code source
+- correction de l'indentation des puces dans les notes de version
+
+### 1.2.0 (28/05/2021)
+
+- ajout d'une page "Notes de version"
+- meilleur lien pour la documentation des champs
+- déplacement du code de decp.info depuis [ColinMaudry/decp-table-schema-utils](https://github.com/ColinMaudry/decp-table-schema-utils) vers [ColinMaudry/decp.info](https://github.com/ColinMaudry/decp.info)
+
+### 1.1.0 (25/05/2021)
+
+- ajout de nouvelles vues :
+  - Marchés publics sans leurs titulaires : vue dédiée aux titulaires de marchés avec des données provenant du répertoire SIRENE
+  - Données sur les titulaires et géolocalisation : vue sans les titulaires pour analyser les nombres de marchés et les montants
+- amélioration de la page d'accueil
+- développement de la page "db" avec description des vues et liste des colonnes
+- les codes APE sont cliquables
+- ajout des mentions légales
+- ajout d'un formulatire d'inscription à une lettre d'information
+- correction de bugs :
+  - correction du format de certaines dates dans les données
+
+### 1.0.0
+
+- publication sur <https://decp.info>
+- ajout d'une vue équivalente au format DECP réglementaire
+- personnalisation de datasette
+- script de conversion quotidien basé sur [dataflows](https://github.com/datahq/dataflows)

CLAUDE.md

@@ -0,0 +1,98 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+**decp.info** is a French public procurement data explorer — a Dash (Python) web app for browsing, filtering, and visualizing _Données Essentielles de la Commande Publique_ (DECP). The UI is in French.
+
+## Commands
+
+### Setup
+
+Setting up the virtual environment:
+
+```bash
+python -m venv .venv # s'il n'existe pas déjà
+source .venv/bin/activate
+rtk pip install -U pip > /dev/null 2>&1
+rtk pip install -e . --group=dev
+```
+
+Environment variables:
+
+```bash
+cp .template.env .env   # then customize .env
+```
+
+### Development
+
+```bash
+python run.py         # starts Dash app
+```
+
+### Production
+
+```bash
+gunicorn app:server
+```
+
+### Tests
+
+```bash
+rtk pytest                  # run all tests (some are Selenium-based integration tests)
+rtk pytest tests/test_main.py::test_001_logo_and_search   # run a single test
+```
+
+Tests require a running Chrome/Chromium browser. They use `DashComposite` from `dash[testing]` with Selenium WebDriver.
+
+## Architecture
+
+### Multi-page Dash app
+
+- `src/app.py` — creates the Dash app instance, navbar, SEO endpoints (robots.txt, sitemap.xml), Matomo analytics
+- `src/pages/*.py` — each page registers itself with `@register_page()` and o.wns its own layout and callbacks
+- `run.py` — dev entry point; exports `server` (Flask) for gunicorn
+
+### Module imports
+
+- always import modules from the app starting with `src.` (e.g. `src.utils.`, `src.pages.recherche`, etc.), NOT `utils.cache` or `pages.observatoire`.
+
+### Key pages
+
+| Page              | URL             | Purpose                                |
+| ----------------- | --------------- | -------------------------------------- |
+| `recherche.py`    | `/`             | Search homepage for buyers/contractors |
+| `acheteur.py`     | `/acheteur`     | Buyer detail with stats, charts, maps  |
+| `titulaire.py`    | `/titulaire`    | Contractor detail                      |
+| `tableau.py`      | `/tableau`      | Filterable data table with exports     |
+| `marche.py`       | `/marche`       | Individual contract detail             |
+| `observatoire.py` | `/observatoire` | An interactive analytics dashboard     |
+
+### Data layer
+
+- Data is stored as **Parquet** at rest, possibly in DuckDB, loaded in DuckDB, served from DuckDB for big queries and manipulated with **Polars** for the remaining steps
+- Path set via `DATA_FILE_PARQUET_PATH` env var; tests use `tests/test.parquet`
+- `src/util/*.py` — helpers shared by other modules, search (`search_org`), link generation, geographic data loading
+- `src/callbacks.py` — shared Dash callbacks (e.g. `get_top_org_table`)
+- `src/figures.py` — chart and map components (Plotly Express, Dash Leaflet with marker clustering)
+- a Parquet file with production data is located at `../decp-processing/decp_prod.parquet` (~ 1,5 million records)
+- the TableSchema of the dataset with the list of field and their definition is located at `../decp-processing/reference/base_schema.json`
+- `tests/test.parquet` is very small and may not contain all possible columns, only those necessary for testing
+
+### UI stack
+
+- **Dash 3.4** + **Dash Bootstrap Components** for layout
+- **Plotly Express** for charts
+- **Dash Leaflet** + **Dash Extensions** for interactive maps with clustering
+- Custom CSS in `src/assets/css/`
+
+### Environment
+
+- `DEVELOPMENT=true` enables debug logging and is set automatically during tests
+- `.env` file is required at runtime (copy from `template.env`)
+
+### Deployment
+
+- `main` branch → manual deploy to decp.info via GitHub Actions
+- `dev` branch → auto-deploy to test.decp.info via GitHub Actions

README.md

@@ -1,27 +1,21 @@
 # decp.info
 
-> v2.1.2
+> Outil d'exploration et de téléchargement des données essentielles de la commande publique.
-
-Outil d'exploration et de téléchargement des données essentielles de la commande publique.
 
 => [decp.info](https://decp.info)
 
 ## Installation et lancement
 
 ```shell
-python -m venv .venv
-source .venv/bin/activate
-pip install .
-
 # Copie et personnalisation du .env
 cp template.env .env
 nano .env
 
 # Pour la production
-gunicorn app:server
+uv run gunicorn app:server
 
 # Pour avoir le debuggage et le hot reload
-python run.py
+uv run run.py
 ```
 
 ## Déploiement
@@ -38,85 +32,4 @@ Ne pas oublier de mettre à jour les fichier .env.
 
 ## Notes de version
 
-##### 2.1.2 (3 octobre 2025)
+Voir [CHANGELOG](https://github.com/ColinMaudry/decp.info/blob/main/CHANGELOG.md).
-
-- dataframe global plutôt que lazyframe, pour plus de résilience et charger toutes les données en mémoire
-
-##### 2.1.1 (1er octobre 2025)
-
-- ajout d'une section dans À propos sur la qualité et l'exhaustivité des données ([#43](https://github.com/ColinMaudry/decp.info/issues/43))
-- ajout du nombre de marchés en plus du nombre de lignes dans la vue Tableau
-
-#### 2.1.0 (30 septembre 2025)
-
-- Ajout des vues [acheteur](https://decp.info/acheteurs/24350013900189) ([#28](https://github.com/ColinMaudry/decp.info/issues/28)), [titulaire](https://decp.info/titulaires/51903758414786) ([#35](https://github.com/ColinMaudry/decp.info/issues/35)) et [marché](https://decp.info/marches/532239472000482025S00004) ([#40](https://github.com/ColinMaudry/decp.info/issues/40)) 🔎
-- Ajout des balises HTML meta Open Graph et Twitter ([#39](https://github.com/ColinMaudry/decp.info/issues/39)) pour de beaux aperçus de liens 🖼️
-- Formulaire de contact ([#48](https://github.com/ColinMaudry/decp.info/issues/48)) 📨
-- Nom de colonnes plus_agréables ([#33](https://github.com/ColinMaudry/decp.info/issues/33)) 💅
-- Définition des colonnes quand vous passez votre souris sur les en-têtes ([#33](https://github.com/ColinMaudry/decp.info/issues/33)) 📖
-- Affichage du numéro de version près du logo et lien vers ici 🤓
-- Variables globales uniquement en lecture (😁)
-
-##### 2.0.1 (23 septembre 2025)
-
-- Bloquage du bouton de téléchargement si trop de lignes (+ 65000) [#38](https://github.com/ColinMaudry/decp.info/issues/38)
-- Amélioration du script de déploiement (deploy.sh)
-- Meilleures instructions d'installation et lancement
-- Coquilles 🐚
-
-### 2.0.0 (23 septembre 2025)
-
-- détails des sources de données
-- section "À propos" plus développée
-- correction de bugs dans les filtres de la data table
-
-#### 2.0.0-alpha
-
-- Data table fonctionnelle
-
-### 1.5.0 (28/01/2023
-
-- fixation des dépendances Python pour plus de stabilité en cas de réinstallation (Pipfile)
-
-#### 1.4.1 (14/06/2021)
-
-- ajout des traductions des opérations de filtrage à toutes les vues, pas seulement /db/decp
-
-### 1.4.0 (14/06/2021)
-
-- traduction des opérations de filtrage (ex : contains => contient)
-- élargissement des menus de filtrage
-- correction du titre de la page des notes de versions
-
-### 1.3.0 (03/06/2021)
-
-- utilisation de noms de colonnes plus lisibles dans l'application
-- suppression des références à la licence et aux données source sur la page d'accueil
-- correction des liens vers le code source
-- correction de l'indentation des puces dans les notes de version
-
-### 1.2.0 (28/05/2021)
-
-- ajout d'une page "Notes de version"
-- meilleur lien pour la documentation des champs
-- déplacement du code de decp.info depuis [ColinMaudry/decp-table-schema-utils](https://github.com/ColinMaudry/decp-table-schema-utils) vers [ColinMaudry/decp.info](https://github.com/ColinMaudry/decp.info)
-
-### 1.1.0 (25/05/2021)
-
-- ajout de nouvelles vues :
-  - Marchés publics sans leurs titulaires : vue dédiée aux titulaires de marchés avec des données provenant du répertoire SIRENE
-  - Données sur les titulaires et géolocalisation : vue sans les titulaires pour analyser les nombres de marchés et les montants
-- amélioration de la page d'accueil
-- développement de la page "db" avec description des vues et liste des colonnes
-- les codes APE sont cliquables
-- ajout des mentions légales
-- ajout d'un formulatire d'inscription à une lettre d'information
-- correction de bugs :
-  - correction du format de certaines dates dans les données
-
-### 1.0.0
-
-- publication sur https://decp.info
-- ajout d'une vue équivalente au format DECP réglementaire
-- personnalisation de datasette
-- script de conversion quotidien basé sur [dataflows](https://github.com/datahq/dataflows)

data/departements.json

@@ -391,8 +391,16 @@
     "departement": "La Réunion",
     "region": "La Réunion"
   },
+  "975": {
+    "departement": "Saint-Pierre-et-Miquelon",
+    "region": "Saint-Pierre-et-Miquelon"
+  },
   "976": {
     "departement": "Mayotte",
     "region": "Mayotte"
+  },
+  "977": {
+    "departement": "Saint-Barthelemy",
+    "region": "Saint-Barthelemy"
   }
 }

docs/superpowers/plans/2026-03-18-observatoire-link.md

@@ -0,0 +1,473 @@
+# Observatoire Link from Search & Tableau Results — Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Let users jump from search/tableau results to the observatoire page, pre-filtered for a given organization, via a 📊 link in the `_nom` columns.
+
+**Architecture:** Modify `add_links()` in `src/utils.py` to append an observatoire link to `_nom` columns. Add two callbacks to `src/pages/observatoire.py` for bidirectional URL ↔ filter sync using the existing `dcc.Location(id="dashboard_url")`. Add a share URL input and clipboard button to the observatoire layout.
+
+**Tech Stack:** Dash 3.4, Polars, `urllib.parse`, `dcc.Location`, `dcc.Clipboard`
+
+**Spec:** `docs/superpowers/specs/2026-03-18-observatoire-link-from-search-design.md`
+
+---
+
+### Task 1: Add observatoire link to `acheteur_nom` in `add_links()`
+
+**Files:**
+
+- Modify: `src/utils.py:82-91` (the `acheteur_` block inside `add_links()`)
+- Test: `tests/test_main.py`
+
+**Context:** The `add_links()` function loops over column names. The `if col.startswith("acheteur_")` block (lines 82-91) currently wraps both `acheteur_nom` and `acheteur_id` in a detail page link. We must only append the observatoire link when `col == "acheteur_nom"`.
+
+- [ ] **Step 1: Write a unit test for the observatoire link in acheteur_nom**
+
+In `tests/test_main.py`, add a test that calls `add_links()` on a minimal DataFrame and checks the `acheteur_nom` column contains both the detail link and the observatoire link, while `acheteur_id` does NOT contain the observatoire link.
+
+```python
+def test_004_add_links_observatoire_acheteur():
+    import polars as pl
+
+    from src.utils import add_links
+
+    dff = pl.DataFrame(
+        {
+            "acheteur_id": ["a1"],
+            "acheteur_nom": ["ACHETEUR 1"],
+        }
+    )
+    result = add_links(dff)
+    nom_value = result["acheteur_nom"][0]
+    id_value = result["acheteur_id"][0]
+
+    # acheteur_nom should contain detail link + observatoire link
+    assert "/acheteurs/a1" in nom_value
+    assert "ACHETEUR 1" in nom_value
+    assert '/observatoire?acheteur_id=a1' in nom_value
+    assert "📊" in nom_value
+
+    # acheteur_id should NOT contain observatoire link
+    assert "/observatoire" not in id_value
+```
+
+- [ ] **Step 2: Run test to verify it fails**
+
+Run: `source .venv/bin/activate && pytest tests/test_main.py::test_004_add_links_observatoire_acheteur -v`
+Expected: FAIL — `'/observatoire?acheteur_id=a1'` not found in the output string.
+
+- [ ] **Step 3: Implement the observatoire link for acheteur_nom**
+
+In `src/utils.py`, modify the `if col.startswith("acheteur_")` block (lines 82-91). Gate the observatoire link append on `col == "acheteur_nom"`:
+
+```python
+            if col.startswith("acheteur_"):
+                detail_link = (
+                    '<a href = "/acheteurs/'
+                    + pl.col("acheteur_id")
+                    + '">'
+                    + pl.col(col)
+                    + "</a>"
+                )
+                if col == "acheteur_nom":
+                    detail_link = (
+                        detail_link
+                        + ' <a href="/observatoire?acheteur_id='
+                        + pl.col("acheteur_id")
+                        + '" title="Voir dans l\'observatoire">📊</a>'
+                    )
+                dff = dff.with_columns(detail_link.alias(col))
+```
+
+- [ ] **Step 4: Run test to verify it passes**
+
+Run: `source .venv/bin/activate && pytest tests/test_main.py::test_004_add_links_observatoire_acheteur -v`
+Expected: PASS
+
+- [ ] **Step 5: Update `test_001` to account for the new emoji in cell text**
+
+The existing `test_001` asserts `result_table.find_element(...).text == name` for `acheteur_nom`. The cell text now includes "📊" from the observatoire link. Update the assertion in `tests/test_main.py` to use `startswith` instead of exact match:
+
+```python
+        assert result_table.find_element(
+            by=By.CSS_SELECTOR, value=f'td[data-dash-column="{org_type}_nom"]'
+        ).text.startswith(
+            name
+        ), f"The search result should have the right {org_type} name"
+```
+
+- [ ] **Step 6: Run `test_001` to verify it still passes**
+
+Run: `source .venv/bin/activate && pytest tests/test_main.py::test_001_logo_and_search -v`
+Expected: PASS
+
+- [ ] **Step 7: Commit**
+
+```bash
+git add src/utils.py tests/test_main.py
+git commit -m "Ajout du lien observatoire dans acheteur_nom via add_links() #65"
+```
+
+---
+
+### Task 2: Add observatoire link to `titulaire_nom` in `add_links()`
+
+**Files:**
+
+- Modify: `src/utils.py:64-81` (the `titulaire_` block inside `add_links()`)
+- Test: `tests/test_main.py`
+
+**Context:** The `titulaire_` block (lines 64-81) uses a `pl.when().then().otherwise()` pattern because it guards on `titulaire_typeIdentifiant` being SIRET or null. The observatoire link must be appended inside the `.then()` branch, and only when `col == "titulaire_nom"`. Note: this block requires `titulaire_typeIdentifiant` to be present in the DataFrame.
+
+- [ ] **Step 1: Write a unit test for the observatoire link in titulaire_nom**
+
+```python
+def test_005_add_links_observatoire_titulaire():
+    import polars as pl
+
+    from src.utils import add_links
+
+    dff = pl.DataFrame(
+        {
+            "titulaire_id": ["t1"],
+            "titulaire_nom": ["TITULAIRE 1"],
+            "titulaire_typeIdentifiant": ["SIRET"],
+        }
+    )
+    result = add_links(dff)
+    nom_value = result["titulaire_nom"][0]
+    id_value = result["titulaire_id"][0]
+
+    # titulaire_nom should contain detail link + observatoire link
+    assert "/titulaires/t1" in nom_value
+    assert "TITULAIRE 1" in nom_value
+    assert '/observatoire?titulaire_id=t1' in nom_value
+    assert "📊" in nom_value
+
+    # titulaire_id should NOT contain observatoire link
+    assert "/observatoire" not in id_value
+```
+
+- [ ] **Step 2: Run test to verify it fails**
+
+Run: `source .venv/bin/activate && pytest tests/test_main.py::test_005_add_links_observatoire_titulaire -v`
+Expected: FAIL — `'/observatoire?titulaire_id=t1'` not found.
+
+- [ ] **Step 3: Implement the observatoire link for titulaire_nom**
+
+In `src/utils.py`, modify the `if col.startswith("titulaire_")` block (lines 64-81). The `.then()` branch must build the link differently when `col == "titulaire_nom"`:
+
+```python
+            if col.startswith("titulaire_"):
+                detail_link = (
+                    '<a href = "/titulaires/'
+                    + pl.col("titulaire_id")
+                    + '">'
+                    + pl.col(col)
+                    + "</a>"
+                )
+                if col == "titulaire_nom":
+                    detail_link = (
+                        detail_link
+                        + ' <a href="/observatoire?titulaire_id='
+                        + pl.col("titulaire_id")
+                        + '" title="Voir dans l\'observatoire">📊</a>'
+                    )
+                dff = dff.with_columns(
+                    pl.when(
+                        pl.Expr.or_(
+                            pl.col("titulaire_typeIdentifiant").is_null(),
+                            pl.col("titulaire_typeIdentifiant") == "SIRET",
+                        )
+                    )
+                    .then(detail_link)
+                    .otherwise(pl.col(col))
+                    .alias(col)
+                )
+```
+
+- [ ] **Step 4: Run test to verify it passes**
+
+Run: `source .venv/bin/activate && pytest tests/test_main.py::test_005_add_links_observatoire_titulaire -v`
+Expected: PASS
+
+- [ ] **Step 5: Run all tests so far to check for regressions**
+
+Run: `source .venv/bin/activate && pytest tests/test_main.py::test_004_add_links_observatoire_acheteur tests/test_main.py::test_005_add_links_observatoire_titulaire -v`
+Expected: both PASS
+
+- [ ] **Step 6: Commit**
+
+```bash
+git add src/utils.py tests/test_main.py
+git commit -m "Ajout du lien observatoire dans titulaire_nom via add_links() #65"
+```
+
+---
+
+### Task 3: Observatoire Callback A — URL → Inputs (page load)
+
+**Files:**
+
+- Modify: `src/pages/observatoire.py` (add import + new callback after line 281)
+- Test: `tests/test_main.py`
+
+**Context:** The existing `dcc.Location(id="dashboard_url")` is in the observatoire layout. A new callback reads `dashboard_url.search` on page load, parses query params, and sets `dashboard_acheteur_id.value` and/or `dashboard_titulaire_id.value`. It also clears `dashboard_url.search` to `""` to prevent re-triggering. Two imports must be added: `import urllib.parse` at the top of the file, and `no_update` to the existing `from dash import ...` line (currently: `from dash import ALL, Input, Output, State, callback, ctx, dcc, html, register_page` — add `no_update` to this).
+
+- [ ] **Step 1: Write a Selenium test for URL → Input sync**
+
+This test navigates to `/observatoire?acheteur_id=a1` and verifies the SIRET input gets populated.
+
+```python
+def test_006_observatoire_url_to_input(dash_duo: DashComposite):
+    from src.app import app
+
+    dash_duo.start_server(app)
+    dash_duo.wait_for_text_to_equal(".logo > h1", "decp.info", timeout=4)
+
+    # Navigate to observatoire with acheteur_id query param
+    dash_duo.wait_for_page(f"{dash_duo.server_url}/observatoire?acheteur_id=a1")
+    dash_duo.wait_for_element("#dashboard_acheteur_id", timeout=4)
+
+    acheteur_input = dash_duo.find_element("#dashboard_acheteur_id")
+    dash_duo.wait_for_text_to_equal(
+        "#dashboard_acheteur_id", "", timeout=4
+    )  # Wait for callback
+    import time
+    time.sleep(1)  # Allow callback chain to complete
+
+    assert acheteur_input.get_attribute("value") == "a1", (
+        "acheteur_id input should be populated from URL param"
+    )
+```
+
+- [ ] **Step 2: Run test to verify it fails**
+
+Run: `source .venv/bin/activate && pytest tests/test_main.py::test_006_observatoire_url_to_input -v`
+Expected: FAIL — the input value is empty because no callback reads URL params yet.
+
+- [ ] **Step 3: Implement Callback A**
+
+Add `import urllib.parse` to the imports at the top of `src/pages/observatoire.py` (after line 1). Also add `no_update` to the existing dash import line:
+
+```python
+from dash import ALL, Input, Output, State, callback, ctx, dcc, html, no_update, register_page
+```
+
+Add the callback after the `layout` list ends, before existing callbacks:
+
+```python
+@callback(
+    Output("dashboard_acheteur_id", "value"),
+    Output("dashboard_titulaire_id", "value"),
+    Output("dashboard_url", "search"),
+    Input("dashboard_url", "search"),
+)
+def restore_filters_from_url(search):
+    if not search:
+        return no_update, no_update, no_update
+
+    params = urllib.parse.parse_qs(search.lstrip("?"))
+
+    acheteur_id = params.get("acheteur_id", [None])[0] or no_update
+    titulaire_id = params.get("titulaire_id", [None])[0] or no_update
+
+    return acheteur_id, titulaire_id, ""
+```
+
+- [ ] **Step 4: Run test to verify it passes**
+
+Run: `source .venv/bin/activate && pytest tests/test_main.py::test_006_observatoire_url_to_input -v`
+Expected: PASS
+
+- [ ] **Step 5: Commit**
+
+```bash
+git add src/pages/observatoire.py tests/test_main.py
+git commit -m "Callback URL → filtres sur la page observatoire #65"
+```
+
+---
+
+### Task 4: Observatoire Callback B — Inputs → shareable URL + layout
+
+**Files:**
+
+- Modify: `src/pages/observatoire.py` (add layout components + new callback)
+- Test: `tests/test_main.py`
+
+**Context:** Following the tableau.py pattern (lines 237-238 for layout, lines 399-450 for callback), add a hidden `share-url` input and a `copy-container` div to the observatoire layout. The callback listens to the ID inputs and builds a shareable URL. Component IDs must be unique across the app, so use `observatoire-share-url` and `observatoire-copy-container` to avoid collisions with tableau's `share-url` and `copy-container`.
+
+- [ ] **Step 1: Write a test for the shareable URL generation**
+
+```python
+def test_007_observatoire_share_url(dash_duo: DashComposite):
+    from src.app import app
+
+    dash_duo.start_server(app)
+    dash_duo.wait_for_text_to_equal(".logo > h1", "decp.info", timeout=4)
+
+    # Navigate to observatoire with acheteur_id query param
+    dash_duo.wait_for_page(f"{dash_duo.server_url}/observatoire?acheteur_id=a1")
+    dash_duo.wait_for_element("#observatoire-share-url", timeout=4)
+
+    import time
+    time.sleep(1)  # Allow callback chain to complete
+
+    share_url_input = dash_duo.find_element("#observatoire-share-url")
+    share_url_value = share_url_input.get_attribute("value")
+
+    assert "acheteur_id=a1" in share_url_value, (
+        f"Share URL should contain acheteur_id param, got: {share_url_value}"
+    )
+```
+
+- [ ] **Step 2: Run test to verify it fails**
+
+Run: `source .venv/bin/activate && pytest tests/test_main.py::test_007_observatoire_share_url -v`
+Expected: FAIL — `#observatoire-share-url` element does not exist yet.
+
+- [ ] **Step 3: Add layout components to observatoire**
+
+In `src/pages/observatoire.py`, add the share URL input and copy container inside the filters column (after the download button, before the closing `]` of the `id="filters"` children list, around line 264):
+
+```python
+                                    dcc.Input(
+                                        id="observatoire-share-url",
+                                        readOnly=True,
+                                        style={"display": "none"},
+                                    ),
+                                    html.Div(id="observatoire-copy-container"),
+```
+
+- [ ] **Step 4: Implement Callback B**
+
+Add after Callback A in `src/pages/observatoire.py`:
+
+```python
+@callback(
+    Output("observatoire-share-url", "value"),
+    Output("observatoire-copy-container", "children"),
+    Input("dashboard_acheteur_id", "value"),
+    Input("dashboard_titulaire_id", "value"),
+    State("dashboard_url", "href"),
+    prevent_initial_call=True,
+)
+def sync_observatoire_share_url(acheteur_id, titulaire_id, href):
+    if not href:
+        return no_update, no_update
+
+    base_url = href.split("?")[0]
+
+    params = {}
+    if acheteur_id:
+        params["acheteur_id"] = acheteur_id
+    if titulaire_id:
+        params["titulaire_id"] = titulaire_id
+
+    query_string = urllib.parse.urlencode(params)
+    full_url = f"{base_url}?{query_string}" if query_string else base_url
+
+    copy_button = dcc.Clipboard(
+        id="btn-copy-observatoire-url",
+        target_id="observatoire-share-url",
+        title="Copier l'URL de cette vue",
+        style={
+            "display": "inline-block",
+            "fontSize": 20,
+            "verticalAlign": "top",
+            "cursor": "pointer",
+        },
+        className="fa fa-link",
+        children=[
+            dbc.Button(
+                "Partager",
+                className="btn btn-primary mt-2",
+                title="Copier l'adresse de cette vue filtrée pour la partager.",
+            )
+        ],
+    )
+
+    return full_url, copy_button
+```
+
+- [ ] **Step 5: Run test to verify it passes**
+
+Run: `source .venv/bin/activate && pytest tests/test_main.py::test_007_observatoire_share_url -v`
+Expected: PASS
+
+- [ ] **Step 6: Run all tests to check for regressions**
+
+Run: `source .venv/bin/activate && pytest tests/test_main.py -v`
+Expected: all tests PASS
+
+- [ ] **Step 7: Commit**
+
+```bash
+git add src/pages/observatoire.py tests/test_main.py
+git commit -m "URL partageable pour la page observatoire #65"
+```
+
+---
+
+### Task 5: End-to-end integration test
+
+**Files:**
+
+- Test: `tests/test_main.py`
+
+**Context:** Verify the full flow: search for an organization on the homepage, see the 📊 link in results, click it, arrive on the observatoire with the correct input populated.
+
+- [ ] **Step 1: Write end-to-end test**
+
+```python
+def test_008_search_to_observatoire(dash_duo: DashComposite):
+    from src.app import app
+
+    dash_duo.start_server(app)
+    dash_duo.wait_for_text_to_equal(".logo > h1", "decp.info", timeout=4)
+
+    # Search for an acheteur
+    search_bar = dash_duo.find_element("#search")
+    search_bar.send_keys("ACHETEUR 1")
+    search_bar.send_keys(Keys.ENTER)
+
+    dash_duo.wait_for_element("#results_acheteur_datatable", timeout=2)
+
+    # Find the observatoire link in acheteur_nom column
+    observatoire_link = dash_duo.find_element(
+        '#results_acheteur_datatable td[data-dash-column="acheteur_nom"] a[href*="observatoire"]'
+    )
+    assert "📊" in observatoire_link.text
+
+    # Click the observatoire link
+    observatoire_link.click()
+
+    # Wait for observatoire page to load
+    dash_duo.wait_for_element("#dashboard_acheteur_id", timeout=4)
+
+    import time
+    time.sleep(1)  # Allow callback chain to complete
+
+    acheteur_input = dash_duo.find_element("#dashboard_acheteur_id")
+    assert acheteur_input.get_attribute("value") == "a1", (
+        "acheteur_id input should be populated after navigating from search"
+    )
+```
+
+- [ ] **Step 2: Run end-to-end test**
+
+Run: `source .venv/bin/activate && pytest tests/test_main.py::test_008_search_to_observatoire -v`
+Expected: PASS
+
+- [ ] **Step 3: Run the full test suite**
+
+Run: `source .venv/bin/activate && pytest tests/test_main.py -v`
+Expected: all tests PASS
+
+- [ ] **Step 4: Commit**
+
+```bash
+git add tests/test_main.py
+git commit -m "Test e2e : recherche → observatoire #65"
+```

docs/superpowers/plans/2026-04-19-tableau-prepare-table-data-cache.md

@@ -0,0 +1,838 @@
+# Tableau prepare_table_data Cache Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Make page navigation, sort changes, and repeated filter visits in the `/tableau` page near-instant by memoizing the expensive filter+sort+post-process pipeline inside `prepare_table_data`.
+
+**Architecture:** Extract a memoized inner function `_load_filter_sort_postprocess(filter_query, sort_by_key)` that performs the heavy work (load full data, filter, sort, collect, cast-to-string, fill-null, add HTML links, format values) and returns a fully post-processed Polars DataFrame. The outer `prepare_table_data` becomes a thin wrapper that handles non-deterministic side effects (`track_search`, `uuid.uuid4()` for cleanup trigger, `data_timestamp + 1`) and pagination. The memoized helper only runs when no `data` argument is passed (i.e., the Tableau path). Other callers (`acheteur`, `titulaire`, `observatoire`) keep the current uncached path because they pass an externally-provided LazyFrame that is not safely hashable for cache keys.
+
+**Tech Stack:** Polars (LazyFrame, DataFrame), Flask-Caching (`@cache.memoize()` on `FileSystemCache` already configured in `src/app.py:38`), pytest for unit tests.
+
+**Git**: the issue id is #72, add the reference in commit messages.
+
+---
+
+## Background and constraints
+
+Read these before starting; they explain why the design takes the shape it does.
+
+1. **Cache infrastructure is already wired.** `src/cache.py` defines `cache = Cache()`. `src/app.py:38-48` initializes it with `FileSystemCache`, default 24h timeout, `CACHE_THRESHOLD=300`. The cache directory is wiped on every restart (`rmtree` at `src/app.py:36`), so cache always starts empty.
+
+2. **Existing pattern to mirror.** `src/pages/observatoire.py:650-660` already uses `@cache.memoize()` plus a `_normalize_filter_params` helper that converts a dict of filters into a hashable tuple. This plan applies the same idiom to `sort_by` (which is a `list[dict]` from Dash DataTable).
+
+3. **Non-deterministic outputs that MUST stay outside the memoized function:**
+
+   - `data_timestamp + 1` (increments each call; would freeze if cached)
+   - `trigger_cleanup = str(uuid.uuid4())` (intentionally unique per call to fire the clientside filter-cleanup callback)
+   - `track_search(filter_query, source_table)` — Matomo HTTP POST, currently called inside `filter_table_data` at `src/utils/table.py:214`. Must fire on every user action including cache hits.
+
+4. **Tracking call site move.** `track_search` must move OUT of `filter_table_data` and into each caller, otherwise cache hits would silently skip Matomo tracking. Current callers of `filter_table_data` to update:
+
+   - `src/utils/table.py:402` (inside `prepare_table_data`)
+   - `src/pages/tableau.py:325` (`download_data` callback)
+   - `src/pages/acheteur.py:427` (`download_data_acheteur` callback)
+   - `src/pages/titulaire.py:443` (`download_data_titulaire` callback)
+
+5. **Why Tableau-only caching.** `prepare_table_data` is also called from `acheteur.py`, `titulaire.py`, `observatoire.py`. Those callers pass a pre-filtered LazyFrame or list-of-dicts as `data`. Hashing arbitrary LazyFrames or large lists for memoization is impractical. The fix gates on `data is None` (the Tableau path) and leaves the other paths byte-for-byte identical.
+
+6. **Cache key composition.** The memoized function takes only `(filter_query, sort_by_key)`. `page_current` and `page_size` are intentionally NOT in the key — pagination happens in the outer wrapper after retrieving the cached, fully post-processed frame. This means every page click and page-size change is a cache hit (the whole point of the change).
+
+7. **Pickling.** Flask-Caching pickles arguments to form keys and pickles return values to disk. Polars `DataFrame` pickles cleanly. `LazyFrame` does not — so the memoized function must `.collect()` before returning.
+
+8. **File path expectations.** All paths below are relative to repo root `/home/colin/git/decp.info`. Run all commands from there.
+
+---
+
+## File Structure
+
+- **Modify** `src/utils/table.py` — extract memoized helper, refactor `prepare_table_data`, remove `track_search` call from `filter_table_data`.
+- **Modify** `src/pages/tableau.py` — add explicit `track_search` call in `download_data`.
+- **Modify** `src/pages/acheteur.py` — add explicit `track_search` call in `download_data_acheteur`.
+- **Modify** `src/pages/titulaire.py` — add explicit `track_search` call in `download_data_titulaire`.
+- **Create** `tests/test_table.py` — unit tests for new helpers and refactored `prepare_table_data`.
+
+---
+
+## Task 1: Set up unit tests for table.py
+
+**Files:**
+
+- Create: `tests/test_table.py`
+
+This task scaffolds a non-Selenium pytest module so subsequent tasks can do TDD without booting a Dash server. The conftest already writes a small `tests/test.parquet` fixture (see `tests/conftest.py:10`); reuse it.
+
+- [ ] **Step 1: Write the failing test**
+
+Create `tests/test_table.py` with:
+
+```python
+import os
+
+import polars as pl
+import pytest
+
+
+@pytest.fixture
+def sample_lff():
+    """Small LazyFrame with the columns needed by add_links / format_values."""
+    return pl.LazyFrame(
+        [
+            {
+                "uid": "u1",
+                "id": "u1",
+                "acheteur_id": "12345678900011",
+                "acheteur_nom": "Mairie de Test",
+                "titulaire_id": "98765432100022",
+                "titulaire_nom": "Entreprise Test",
+                "titulaire_typeIdentifiant": "SIRET",
+                "objet": "Travaux divers",
+                "montant": 12500.0,
+                "dateNotification": "2025-03-15",
+                "codeCPV": "45000000",
+                "dureeRestanteMois": 6,
+                "titulaire_distance": 42.0,
+            }
+        ]
+    )
+
+
+def test_table_module_imports():
+    from src.utils import table
+
+    assert hasattr(table, "prepare_table_data")
+```
+
+- [ ] **Step 2: Run test to verify it passes (sanity check)**
+
+Run: `uv run pytest tests/test_table.py -v`
+Expected: PASS for `test_table_module_imports`. (Selenium is not invoked because no `dash_duo` fixture is used.)
+
+- [ ] **Step 3: Commit**
+
+```bash
+git add tests/test_table.py
+git commit -m "test: scaffold unit tests for table utilities"
+```
+
+---
+
+## Task 2: Move track_search out of filter_table_data
+
+**Files:**
+
+- Modify: `src/utils/table.py:210-274` (remove `track_search` import usage at line 214)
+- Modify: `src/pages/tableau.py:317-334` (`download_data` callback)
+- Modify: `src/pages/acheteur.py:425-430` area (`download_data_acheteur` callback)
+- Modify: `src/pages/titulaire.py:441-446` area (`download_data_titulaire` callback)
+- Modify: `tests/test_table.py` (add a test that confirms `filter_table_data` no longer calls Matomo)
+
+`track_search` must move out so that the soon-to-be-memoized helper does not swallow tracking on cache hits. We do this BEFORE introducing caching so that the diff is small and verifiable on its own.
+
+- [ ] **Step 1: Write the failing test**
+
+Append to `tests/test_table.py`:
+
+```python
+def test_filter_table_data_does_not_call_track_search(monkeypatch, sample_lff):
+    from src.utils import table
+
+    calls = []
+    monkeypatch.setattr(table, "track_search", lambda *a, **kw: calls.append(a))
+
+    result = table.filter_table_data(
+        sample_lff, "{objet} icontains travaux", "tableau"
+    ).collect()
+
+    assert calls == []
+    assert result.height == 1
+```
+
+- [ ] **Step 2: Run test to verify it fails**
+
+Run: `uv run pytest tests/test_table.py::test_filter_table_data_does_not_call_track_search -v`
+Expected: FAIL (`assert calls == []` fails because `filter_table_data` currently calls `track_search` at line 214).
+
+- [ ] **Step 3: Remove the track_search call from filter_table_data**
+
+Edit `src/utils/table.py` — find this block:
+
+```python
+def filter_table_data(
+    lff: pl.LazyFrame, filter_query: str, filter_source: str
+) -> pl.LazyFrame:
+    _schema = lff.collect_schema()
+    track_search(filter_query, filter_source)
+    filtering_expressions = filter_query.split(" && ")
+```
+
+Remove the `track_search(filter_query, filter_source)` line. Result:
+
+```python
+def filter_table_data(
+    lff: pl.LazyFrame, filter_query: str, filter_source: str
+) -> pl.LazyFrame:
+    _schema = lff.collect_schema()
+    filtering_expressions = filter_query.split(" && ")
+```
+
+The `filter_source` parameter remains in the signature (avoids changing all callers in this task). It becomes unused; that is acceptable since callers will pass it again later if needed. Do NOT remove the `from src.utils.tracking import track_search` import yet — `prepare_table_data` will use it in Task 5.
+
+- [ ] **Step 4: Add explicit track_search calls in download callbacks**
+
+In `src/pages/tableau.py`, find:
+
+```python
+def download_data(n_clicks, filter_query, sort_by, hidden_columns: list = None):
+    lff: pl.LazyFrame = query_marches().lazy()
+
+    # Les colonnes masquées sont supprimées
+    if hidden_columns:
+        lff = lff.drop(hidden_columns)
+
+    if filter_query:
+        lff = filter_table_data(lff, filter_query, "tab download")
+```
+
+Insert a `track_search` call so behavior is preserved. First add the import at the top of `src/pages/tableau.py` next to other `src.utils` imports:
+
+```python
+from src.utils.tracking import track_search
+```
+
+Then change the body:
+
+```python
+def download_data(n_clicks, filter_query, sort_by, hidden_columns: list = None):
+    lff: pl.LazyFrame = query_marches().lazy()
+
+    # Les colonnes masquées sont supprimées
+    if hidden_columns:
+        lff = lff.drop(hidden_columns)
+
+    if filter_query:
+        track_search(filter_query, "tab download")
+        lff = filter_table_data(lff, filter_query, "tab download")
+```
+
+Repeat the same pattern in `src/pages/acheteur.py` (search for `filter_table_data(lff, filter_query, "ach download")`):
+
+Add import:
+
+```python
+from src.utils.tracking import track_search
+```
+
+Wrap the call:
+
+```python
+    if filter_query:
+        track_search(filter_query, "ach download")
+        lff = filter_table_data(lff, filter_query, "ach download")
+```
+
+Repeat in `src/pages/titulaire.py` (search for `filter_table_data(lff, filter_query, "titu download")`):
+
+Add import:
+
+```python
+from src.utils.tracking import track_search
+```
+
+Wrap the call:
+
+```python
+    if filter_query:
+        track_search(filter_query, "titu download")
+        lff = filter_table_data(lff, filter_query, "titu download")
+```
+
+- [ ] **Step 5: Run test to verify it passes**
+
+Run: `uv run pytest tests/test_table.py::test_filter_table_data_does_not_call_track_search -v`
+Expected: PASS.
+
+- [ ] **Step 6: Run full unit test file to verify no regressions**
+
+Run: `uv run pytest tests/test_table.py -v`
+Expected: All tests in `test_table.py` PASS.
+
+- [ ] **Step 7: Commit**
+
+```bash
+git add src/utils/table.py src/pages/tableau.py src/pages/acheteur.py src/pages/titulaire.py tests/test_table.py
+git commit -m "refactor: move track_search out of filter_table_data into callers"
+```
+
+---
+
+## Task 3: Add normalize_sort_by helper
+
+**Files:**
+
+- Modify: `src/utils/table.py` (add helper near other utility functions, e.g. after `dates_to_strings`)
+- Modify: `tests/test_table.py` (add tests)
+
+A cache key must be hashable. Dash DataTable's `sort_by` is a `list[dict]` like `[{"column_id": "montant", "direction": "asc"}, ...]`, which is not hashable. We mirror the `_normalize_filter_params` idiom from `src/pages/observatoire.py:650-657`.
+
+- [ ] **Step 1: Write the failing tests**
+
+Append to `tests/test_table.py`:
+
+```python
+def test_normalize_sort_by_handles_empty():
+    from src.utils.table import normalize_sort_by
+
+    assert normalize_sort_by(None) == ()
+    assert normalize_sort_by([]) == ()
+
+
+def test_normalize_sort_by_returns_hashable_tuple():
+    from src.utils.table import normalize_sort_by
+
+    sort_by = [
+        {"column_id": "montant", "direction": "desc"},
+        {"column_id": "dateNotification", "direction": "asc"},
+    ]
+    key = normalize_sort_by(sort_by)
+
+    assert key == (("montant", "desc"), ("dateNotification", "asc"))
+    # Must be hashable so that flask-caching can build a cache key from it
+    hash(key)
+
+
+def test_normalize_sort_by_preserves_order():
+    """Order matters for sort: [A, B] != [B, A]."""
+    from src.utils.table import normalize_sort_by
+
+    a_then_b = normalize_sort_by(
+        [{"column_id": "a", "direction": "asc"}, {"column_id": "b", "direction": "asc"}]
+    )
+    b_then_a = normalize_sort_by(
+        [{"column_id": "b", "direction": "asc"}, {"column_id": "a", "direction": "asc"}]
+    )
+    assert a_then_b != b_then_a
+```
+
+- [ ] **Step 2: Run tests to verify they fail**
+
+Run: `uv run pytest tests/test_table.py -v -k normalize_sort_by`
+Expected: FAIL with `ImportError` for `normalize_sort_by`.
+
+- [ ] **Step 3: Implement normalize_sort_by**
+
+Edit `src/utils/table.py`. Add this function immediately after the `dates_to_strings` function (around line 148):
+
+```python
+def normalize_sort_by(sort_by) -> tuple:
+    """Convert Dash DataTable sort_by (list[dict]) into a hashable tuple
+    suitable for use as a cache key. Order is preserved because it determines
+    sort precedence."""
+    if not sort_by:
+        return ()
+    return tuple((entry["column_id"], entry["direction"]) for entry in sort_by)
+```
+
+- [ ] **Step 4: Run tests to verify they pass**
+
+Run: `uv run pytest tests/test_table.py -v -k normalize_sort_by`
+Expected: 3 PASS.
+
+- [ ] **Step 5: Commit**
+
+```bash
+git add src/utils/table.py tests/test_table.py
+git commit -m "feat: add normalize_sort_by hashable cache-key helper"
+```
+
+---
+
+## Task 4: Extract memoized post-process helper
+
+**Files:**
+
+- Modify: `src/utils/table.py` (add `_load_filter_sort_postprocess`, decorate with `@cache.memoize()`, import `cache`)
+- Modify: `tests/test_table.py` (add tests)
+
+Introduce the function whose result will live in the FileSystemCache. Inputs: `(filter_query, sort_by_key)`. Output: a fully post-processed, unpaginated Polars DataFrame ready to slice and convert to dicts.
+
+This task does NOT yet wire the helper into `prepare_table_data` — that happens in Task 5. Splitting these tasks keeps each diff small and testable.
+
+- [ ] **Step 1: Write the failing tests**
+
+Append to `tests/test_table.py`:
+
+```python
+@pytest.fixture(autouse=True)
+def reset_cache():
+    """Ensure the flask-caching backend is empty between tests so that
+    cache-hit assertions are meaningful. Falls back to no-op when no
+    Flask app context is active (NullCache)."""
+    from utils.cache import cache
+
+    try:
+        cache.clear()
+    except RuntimeError:
+        # No app context — cache is NullCache, nothing to clear
+        pass
+    yield
+
+
+def test_load_filter_sort_postprocess_returns_dataframe(monkeypatch, sample_lff):
+    from src.utils import table
+
+    monkeypatch.setattr(
+        table, "query_marches", lambda: sample_lff.collect()
+    )
+
+    df = table._load_filter_sort_postprocess(filter_query=None, sort_by_key=())
+
+    assert isinstance(df, pl.DataFrame)
+    assert df.height == 1
+    # All values must be strings after post-processing
+    for col in df.columns:
+        assert df.schema[col] == pl.String
+
+
+def test_load_filter_sort_postprocess_applies_filter(monkeypatch, sample_lff):
+    from src.utils import table
+
+    monkeypatch.setattr(
+        table, "query_marches", lambda: sample_lff.collect()
+    )
+
+    df = table._load_filter_sort_postprocess(
+        filter_query="{objet} icontains travaux", sort_by_key=()
+    )
+    assert df.height == 1
+
+    df_empty = table._load_filter_sort_postprocess(
+        filter_query="{objet} icontains nonexistent", sort_by_key=()
+    )
+    assert df_empty.height == 0
+
+
+def test_load_filter_sort_postprocess_adds_links(monkeypatch, sample_lff):
+    from src.utils import table
+
+    monkeypatch.setattr(
+        table, "query_marches", lambda: sample_lff.collect()
+    )
+
+    df = table._load_filter_sort_postprocess(filter_query=None, sort_by_key=())
+    # add_links injects an <a href> wrapper around uid, acheteur_nom, titulaire_nom
+    assert "<a href" in df["uid"][0]
+    assert "<a href" in df["acheteur_nom"][0]
+    assert "<a href" in df["titulaire_nom"][0]
+```
+
+- [ ] **Step 2: Run tests to verify they fail**
+
+Run: `uv run pytest tests/test_table.py -v -k load_filter_sort_postprocess`
+Expected: FAIL with `AttributeError: module 'src.utils.table' has no attribute '_load_filter_sort_postprocess'`.
+
+- [ ] **Step 3: Implement the helper**
+
+Edit `src/utils/table.py`. Add this import near the top, with the other `src.` imports:
+
+```python
+from utils.cache import cache
+```
+
+Then add the helper function. Place it ABOVE `prepare_table_data` (around line 370, just before `def prepare_table_data`):
+
+```python
+@cache.memoize()
+def _load_filter_sort_postprocess(filter_query, sort_by_key):
+    """Memoized core of the Tableau page pipeline.
+
+    Loads the full marchés dataset, applies filter and sort, materializes,
+    then runs the per-row post-processing (cast to string, fill nulls, add
+    HTML links, format values). Returns an unpaginated Polars DataFrame.
+
+    Inputs MUST be hashable: filter_query is str|None, sort_by_key is the
+    tuple produced by normalize_sort_by(). Pagination intentionally lives
+    in the outer wrapper so that page changes are cache hits.
+    """
+    logger.debug(f"Cache miss — recomputing for filter={filter_query!r} sort={sort_by_key!r}")
+
+    lff: pl.LazyFrame = query_marches().lazy()
+
+    if filter_query:
+        lff = filter_table_data(lff, filter_query, "tableau")
+
+    if sort_by_key:
+        sort_by = [
+            {"column_id": col, "direction": direction}
+            for col, direction in sort_by_key
+        ]
+        lff = sort_table_data(lff, sort_by)
+
+
+    # The remaining steps are cheap per-row operations that we run ONCE here
+    # so that pagination in the outer function is a pure slice + to_dicts.
+    lff = lff.cast(pl.String)
+    lff = lff.fill_null("")
+
+    dff: pl.DataFrame = lff.collect()
+
+    dff = add_links(dff)
+    if "sourceFile" in dff.columns:
+        dff = add_resource_link(dff)
+    if dff.height > 0:
+        dff = format_values(dff)
+
+    return dff
+```
+
+- [ ] **Step 4: Run tests to verify they pass**
+
+Run: `uv run pytest tests/test_table.py -v -k load_filter_sort_postprocess`
+Expected: 3 PASS.
+
+- [ ] **Step 5: Run the full test_table.py to catch regressions**
+
+Run: `uv run pytest tests/test_table.py -v`
+Expected: All PASS.
+
+- [ ] **Step 6: Commit**
+
+```bash
+git add src/utils/table.py tests/test_table.py
+git commit -m "feat: add memoized _load_filter_sort_postprocess helper"
+```
+
+---
+
+## Task 5: Wire the memoized helper into prepare_table_data
+
+**Files:**
+
+- Modify: `src/utils/table.py` — replace the body of `prepare_table_data` so the Tableau path uses the cache
+- Modify: `tests/test_table.py` — add tests covering the new flow
+
+The outer function keeps its signature unchanged so callers in `acheteur.py`, `titulaire.py`, `observatoire.py`, `tableau.py` need no updates. When `data is None` (the Tableau case), use the memoized helper; otherwise fall through to the original logic.
+
+- [ ] **Step 1: Write the failing tests**
+
+Append to `tests/test_table.py`:
+
+```python
+def test_prepare_table_data_returns_expected_tuple(monkeypatch, sample_lff):
+    from src.utils import table
+
+    monkeypatch.setattr(
+        table, "query_marches", lambda: sample_lff.collect()
+    )
+
+    result = table.prepare_table_data(
+        data=None,
+        data_timestamp=5,
+        filter_query=None,
+        page_current=0,
+        page_size=20,
+        sort_by=[],
+        source_table="tableau",
+    )
+
+    # Same arity as before: 9 outputs
+    assert len(result) == 9
+    dicts, columns, tooltip, ts, nb_rows, dl_disabled, dl_text, dl_title, cleanup = result
+    assert isinstance(dicts, list)
+    assert ts == 6  # data_timestamp + 1 must still increment
+    assert "1 lignes" in nb_rows
+
+
+def test_prepare_table_data_calls_track_search_on_filter(monkeypatch, sample_lff):
+    from src.utils import table
+
+    calls = []
+    monkeypatch.setattr(
+        table, "query_marches", lambda: sample_lff.collect()
+    )
+    monkeypatch.setattr(table, "track_search", lambda *a, **kw: calls.append(a))
+
+    table.prepare_table_data(
+        data=None,
+        data_timestamp=0,
+        filter_query="{objet} icontains travaux",
+        page_current=0,
+        page_size=20,
+        sort_by=[],
+        source_table="tableau",
+    )
+
+    assert calls == [("{objet} icontains travaux", "tableau")]
+
+
+def test_prepare_table_data_paginates_without_recomputing(monkeypatch, sample_lff):
+    """Two calls with same filter+sort but different pages must invoke
+    the inner heavy work only once."""
+    from src.utils import table
+
+    call_count = {"n": 0}
+    real_query = sample_lff.collect()
+
+    def counting_query():
+        call_count["n"] += 1
+        return real_query
+
+    monkeypatch.setattr(table, "query_marches", counting_query)
+
+    # First call: cache miss
+    table.prepare_table_data(
+        data=None,
+        data_timestamp=0,
+        filter_query=None,
+        page_current=0,
+        page_size=10,
+        sort_by=[],
+        source_table="tableau",
+    )
+    first_count = call_count["n"]
+
+    # Second call, different page: cache hit, query_marches must NOT fire again
+    table.prepare_table_data(
+        data=None,
+        data_timestamp=0,
+        filter_query=None,
+        page_current=1,
+        page_size=10,
+        sort_by=[],
+        source_table="tableau",
+    )
+
+    assert call_count["n"] == first_count, (
+        "query_marches was called again — pagination triggered cache miss"
+    )
+
+
+def test_prepare_table_data_cleanup_trigger_for_non_tableau(monkeypatch, sample_lff):
+    """Non-tableau pages still get a fresh uuid trigger, not no_update."""
+    from dash import no_update
+
+    from src.utils import table
+
+    monkeypatch.setattr(
+        table, "query_marches", lambda: sample_lff.collect()
+    )
+
+    result = table.prepare_table_data(
+        data=None,
+        data_timestamp=0,
+        filter_query="{objet} icontains travaux",
+        page_current=0,
+        page_size=20,
+        sort_by=[],
+        source_table="acheteur",
+    )
+
+    cleanup = result[8]
+    assert cleanup is not no_update
+    assert isinstance(cleanup, str)
+    assert len(cleanup) >= 32  # uuid4 hex string
+
+
+def test_prepare_table_data_with_external_data_does_not_use_cache(
+    monkeypatch, sample_lff
+):
+    """When a caller passes data (acheteur/titulaire/observatoire path),
+    bypass the memoized helper entirely."""
+    from src.utils import table
+
+    sentinel = {"called": False}
+
+    def should_not_be_called(*a, **kw):
+        sentinel["called"] = True
+        raise AssertionError("Memoized helper must not be called when data is provided")
+
+    monkeypatch.setattr(
+        table, "_load_filter_sort_postprocess", should_not_be_called
+    )
+
+    table.prepare_table_data(
+        data=sample_lff,  # external LazyFrame
+        data_timestamp=0,
+        filter_query=None,
+        page_current=0,
+        page_size=20,
+        sort_by=[],
+        source_table="acheteur",
+    )
+
+    assert sentinel["called"] is False
+```
+
+- [ ] **Step 2: Run tests to verify they fail**
+
+Run: `uv run pytest tests/test_table.py -v -k prepare_table_data`
+Expected: At least the cache-hit (`paginates_without_recomputing`) and `track_search`-routing tests FAIL because the current `prepare_table_data` re-runs the full pipeline on every call and routes tracking through `filter_table_data` (which Task 2 already neutralized — so tracking would be lost without the new explicit call).
+
+- [ ] **Step 3: Refactor prepare_table_data**
+
+Edit `src/utils/table.py`. Replace the entire `prepare_table_data` function body with:
+
+```python
+def prepare_table_data(
+    data, data_timestamp, filter_query, page_current, page_size, sort_by, source_table
+):
+    """
+    Préparation des données pour les datatables.
+
+    Pour la page Tableau (data is None), le calcul lourd (chargement complet,
+    filtre, tri, post-traitement) est mémorisé via _load_filter_sort_postprocess.
+    Les changements de page deviennent ainsi des cache hits.
+
+    Pour les autres pages (data fourni), le chemin original est conservé : la
+    LazyFrame externe n'est pas hashable et le coût de filtre/tri y est déjà
+    minime puisque les données sont pré-restreintes.
+    """
+    logger.debug(" + + + + + + + + + + + + + + + + + + ")
+
+    # Side effect non-cacheable : le tracking doit firer sur chaque action
+    # utilisateur, y compris sur cache hit.
+    if filter_query:
+        track_search(filter_query, source_table)
+
+    # Trigger uuid pour les pages autres que tableau (clientside cleanup)
+    trigger_cleanup = (
+        no_update if source_table == "tableau" else str(uuid.uuid4())
+    )
+
+    if data is None:
+        # Tableau path : utilise le cache
+        sort_by_key = normalize_sort_by(sort_by)
+        dff: pl.DataFrame = _load_filter_sort_postprocess(
+            filter_query=filter_query, sort_by_key=sort_by_key
+        )
+    else:
+        # acheteur / titulaire / observatoire path : code original, non caché
+        if isinstance(data, list):
+            lff: pl.LazyFrame = pl.LazyFrame(
+                data, strict=False, infer_schema_length=5000
+            )
+        elif isinstance(data, pl.LazyFrame):
+            lff = data
+        else:
+            lff = query_marches().lazy()
+
+        if filter_query:
+            lff = filter_table_data(lff, filter_query, source_table)
+
+        if sort_by and len(sort_by) > 0:
+            lff = sort_table_data(lff, sort_by)
+
+        dff = lff.collect()
+        dff = dff.cast(pl.String)
+        dff = dff.fill_null("")
+        dff = add_links(dff)
+        if "sourceFile" in dff.columns:
+            dff = add_resource_link(dff)
+        if dff.height > 0:
+            dff = format_values(dff)
+
+    height = dff.height
+
+    if height > 0:
+        nb_rows = (
+            f"{format_number(height)} lignes "
+            f"({format_number(dff.select('uid').unique().height)} marchés)"
+        )
+    else:
+        nb_rows = "0 lignes (0 marchés)"
+
+    # Pagination — toujours hors cache pour rester sur des cache hits
+    start_row = page_current * page_size
+    dff = dff.slice(start_row, page_size)
+
+    table_columns, tooltip = setup_table_columns(dff)
+
+    dicts = dff.to_dicts()
+
+    download_disabled, download_text, download_title = get_button_properties(height)
+
+    return (
+        dicts,
+        table_columns,
+        tooltip,
+        data_timestamp + 1,
+        nb_rows,
+        download_disabled,
+        download_text,
+        download_title,
+        trigger_cleanup,
+    )
+```
+
+Notes on what changed vs the original at `src/utils/table.py:372-458`:
+
+- `track_search` now called explicitly at the top, on every invocation (not via `filter_table_data`).
+- `data is None` branch delegates the heavy work to the memoized helper.
+- `data is not None` branch is functionally identical to the original (pagination still happens after collect+post-process).
+- The post-processing (`cast`, `fill_null`, `add_links`, `add_resource_link`, `format_values`) is now done in BOTH branches before `nb_rows` calculation. In the cached branch this was already done inside `_load_filter_sort_postprocess`; in the uncached branch we keep doing it inline. This means `nb_rows` and `dff.select('uid').unique().height` operate on the post-processed frame in both branches, matching the original semantics.
+
+- [ ] **Step 4: Run all unit tests**
+
+Run: `uv run pytest tests/test_table.py -v`
+Expected: All PASS, including `test_prepare_table_data_paginates_without_recomputing`.
+
+- [ ] **Step 5: Run the full repo test suite to catch regressions**
+
+Run: `uv run pytest -v`
+Expected: All PASS. Selenium tests (`tests/test_main.py`) require Chrome/Chromium; if the executor lacks a browser, those tests will error/skip — note the failures and rerun in an environment with Chrome before declaring done.
+
+- [ ] **Step 6: Commit**
+
+```bash
+git add src/utils/table.py tests/test_table.py
+git commit -m "perf(tableau): memoize filter+sort+postprocess pipeline"
+```
+
+---
+
+## Task 6: Manual smoke test in the browser
+
+**Files:** none modified.
+
+Type checks and unit tests cannot validate that page navigation actually feels faster. This task is explicitly a hands-on verification.
+
+- [ ] **Step 1: Start the dev server**
+
+Run: `uv run run.py`
+Wait for `Dash is running on http://...`.
+
+- [ ] **Step 2: Open the Tableau page and warm the cache**
+
+1. Open `http://localhost:8050/tableau` (or whatever port the dev server prints).
+2. With no filter applied, wait for the first page to load fully. This is the cold-cache load (slow expected).
+3. Open the browser devtools Network panel.
+
+- [ ] **Step 3: Verify pagination is fast**
+
+1. Click "page 2" / "page 3" / "page 4" in the table footer in quick succession.
+2. Each navigation should return data in well under 1 second (in the original code each took several seconds).
+3. In the dev server logs, look for the line `Cache miss — recomputing for filter=...` from `_load_filter_sort_postprocess`. It should appear ONCE for the initial load and NOT appear again as you change pages.
+
+- [ ] **Step 4: Verify a new filter triggers exactly one cache miss**
+
+1. In the table, type a filter into one of the columns (e.g. `paris` in `acheteur_commune_nom`) and press Enter.
+2. The dev log should show ONE new `Cache miss — recomputing` line.
+3. Change page within the filtered view — no new cache miss line should appear.
+
+- [ ] **Step 5: Verify filter cleanup trigger still fires**
+
+1. Open `http://localhost:8050/acheteur?id=<some_acheteur_id>` (use any valid id from the dataset).
+2. Apply a filter on the embedded table.
+3. The clientside callback for filter cleanup (`src/assets/dash_clientside.js` `clean_filters`) should still rewrite the filter operators (e.g. `contains` → `icontains`). If it doesn't fire, the `trigger_cleanup` uuid is broken — investigate.
+
+- [ ] **Step 6: Verify download still works**
+
+1. On the Tableau page, click "Télécharger au format Excel" (the button must be enabled — apply a filter that brings the row count under 65,000).
+2. The downloaded XLSX must open and contain the filtered rows.
+
+- [ ] **Step 7: Stop the dev server**
+
+Ctrl-C.
+
+- [ ] **Step 8: If all checks pass, this completes the implementation**
+
+No commit — this task is verification only. Report results to the user.

docs/superpowers/plans/2026-04-22-observatoire-duckdb-filters.md

@@ -0,0 +1,951 @@
+# Observatoire — filtrage natif DuckDB — Plan d'implémentation
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Remplacer le filtrage Polars sur LazyFrame dans `prepare_dashboard_data` par un requêtage natif DuckDB, pour ne matérialiser que le sous-ensemble utile au lieu de l'intégralité de la table `decp` (~1,5 M lignes).
+
+**Architecture:** Nouveau helper pur `dashboard_filters_to_sql(**filter_params) -> (where_sql, params)` dans `src/utils/table_sql.py` (modèle de `filter_query_to_sql`). `prepare_dashboard_data` devient une fonction fine qui appelle `query_marches(where_sql, params)` et retourne une `pl.DataFrame`. Les 3 appelants dans `src/pages/observatoire.py` sont adaptés à la nouvelle signature.
+
+**Tech Stack:** Python 3.12, Polars, DuckDB, Dash, pytest.
+
+**Spec:** `docs/superpowers/specs/2026-04-22-observatoire-duckdb-filters-design.md`.
+
+---
+
+## File Structure
+
+**À créer :**
+
+- `tests/test_dashboard_filters_to_sql.py` — tests unitaires du nouveau helper SQL (cas vide + cas par filtre).
+- `tests/test_prepare_dashboard_data.py` — test d'intégration léger (appel DuckDB réel sur `tests/test.parquet`).
+
+**À modifier :**
+
+- `src/utils/table_sql.py` — ajouter `dashboard_filters_to_sql` + import `datetime`/`timedelta`.
+- `src/utils/data.py` — réécrire `prepare_dashboard_data` (signature et implémentation), ajouter `query_marches` aux imports `from src.db`.
+- `src/pages/observatoire.py` — adapter 3 sites d'appel (lignes ~668, ~791, ~882) ; retirer `query_marches` de l'import `from src.db` (plus utilisé).
+- `tests/test_main.py` — supprimer `test_010_observatoire_montant_filter` (migré en test unitaire du helper).
+
+---
+
+## Task 1: Tests unitaires — cas par défaut + filtre année
+
+**Files:**
+
+- Create: `tests/test_dashboard_filters_to_sql.py`
+- Modify: `src/utils/table_sql.py`
+
+- [ ] **Step 1: Write the failing tests**
+
+Create `tests/test_dashboard_filters_to_sql.py`:
+
+```python
+from datetime import datetime, timedelta
+
+from src.utils.table_sql import dashboard_filters_to_sql
+
+
+def test_no_filters_uses_default_365_day_window():
+    where_sql, params = dashboard_filters_to_sql()
+    assert where_sql == '"dateNotification" > ?'
+    assert len(params) == 1
+    assert isinstance(params[0], datetime)
+    expected = datetime.now() - timedelta(days=365)
+    assert abs((params[0] - expected).total_seconds()) < 2
+
+
+def test_year_filter_overrides_default_window():
+    where_sql, params = dashboard_filters_to_sql(dashboard_year="2025")
+    assert where_sql == 'YEAR("dateNotification") = ?'
+    assert params == [2025]
+```
+
+- [ ] **Step 2: Run tests to verify they fail**
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py -v`
+Expected: FAIL with `ImportError: cannot import name 'dashboard_filters_to_sql'`.
+
+- [ ] **Step 3: Implement the helper**
+
+Add to the top of `src/utils/table_sql.py` (below existing imports):
+
+```python
+from datetime import datetime, timedelta
+```
+
+Append this function at the end of `src/utils/table_sql.py`:
+
+```python
+def dashboard_filters_to_sql(
+    dashboard_year=None,
+    dashboard_acheteur_id=None,
+    dashboard_acheteur_categorie=None,
+    dashboard_acheteur_departement_code=None,
+    dashboard_titulaire_id=None,
+    dashboard_titulaire_categorie=None,
+    dashboard_titulaire_departement_code=None,
+    dashboard_marche_type=None,
+    dashboard_marche_objet=None,
+    dashboard_marche_code_cpv=None,
+    dashboard_marche_considerations_sociales=None,
+    dashboard_marche_considerations_environnementales=None,
+    dashboard_marche_techniques=None,
+    dashboard_marche_innovant=None,
+    dashboard_marche_sous_traitance_declaree=None,
+    dashboard_montant_min=None,
+    dashboard_montant_max=None,
+) -> tuple[str, list]:
+    """Traduit les filtres du tableau de bord en (where_clause, params) DuckDB."""
+    clauses: list[str] = []
+    params: list = []
+
+    if dashboard_year:
+        clauses.append('YEAR("dateNotification") = ?')
+        params.append(int(dashboard_year))
+    else:
+        clauses.append('"dateNotification" > ?')
+        params.append(datetime.now() - timedelta(days=365))
+
+    return " AND ".join(clauses), params
+```
+
+- [ ] **Step 4: Run tests to verify they pass**
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py -v`
+Expected: PASS (2 tests).
+
+- [ ] **Step 5: Commit**
+
+```bash
+rtk pre-commit run --files tests/test_dashboard_filters_to_sql.py src/utils/table_sql.py
+rtk git add tests/test_dashboard_filters_to_sql.py src/utils/table_sql.py
+rtk git commit -m "feat(observatoire): squelette de dashboard_filters_to_sql (#72)"
+```
+
+---
+
+## Task 2: Filtres d'égalité simples (catégorie, type, innovant, sous-traitance)
+
+**Files:**
+
+- Modify: `tests/test_dashboard_filters_to_sql.py`
+- Modify: `src/utils/table_sql.py`
+
+- [ ] **Step 1: Add failing tests**
+
+Append to `tests/test_dashboard_filters_to_sql.py`:
+
+```python
+def test_marche_type_equality():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_type="Marché",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "type" = ?'
+    assert params == [2025, "Marché"]
+
+
+def test_innovant_value_all_is_skipped():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_innovant="all",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ?'
+    assert params == [2025]
+
+
+def test_innovant_value_oui_adds_clause():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_innovant="oui",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "marcheInnovant" = ?'
+    assert params == [2025, "oui"]
+
+
+def test_sous_traitance_value_non_adds_clause():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_sous_traitance_declaree="non",
+    )
+    assert (
+        where_sql
+        == 'YEAR("dateNotification") = ? AND "sousTraitanceDeclaree" = ?'
+    )
+    assert params == [2025, "non"]
+```
+
+- [ ] **Step 2: Run tests to verify they fail**
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py -v`
+Expected: 4 new tests FAIL (missing clauses).
+
+- [ ] **Step 3: Extend the helper**
+
+Insert the following block in `dashboard_filters_to_sql`, **after** the `if dashboard_year / else` block and **before** `return " AND ".join(clauses), params`:
+
+```python
+    if dashboard_marche_type:
+        clauses.append('"type" = ?')
+        params.append(dashboard_marche_type)
+
+    if dashboard_marche_innovant and dashboard_marche_innovant != "all":
+        clauses.append('"marcheInnovant" = ?')
+        params.append(dashboard_marche_innovant)
+
+    if (
+        dashboard_marche_sous_traitance_declaree
+        and dashboard_marche_sous_traitance_declaree != "all"
+    ):
+        clauses.append('"sousTraitanceDeclaree" = ?')
+        params.append(dashboard_marche_sous_traitance_declaree)
+```
+
+- [ ] **Step 4: Run tests to verify they pass**
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py -v`
+Expected: PASS (6 tests total).
+
+- [ ] **Step 5: Commit**
+
+```bash
+rtk pre-commit run --files tests/test_dashboard_filters_to_sql.py src/utils/table_sql.py
+rtk git add tests/test_dashboard_filters_to_sql.py src/utils/table_sql.py
+rtk git commit -m "feat(observatoire): filtres d'égalité simples dans dashboard_filters_to_sql (#72)"
+```
+
+---
+
+## Task 3: Filtres LIKE/ILIKE (ids, objet, cpv)
+
+**Files:**
+
+- Modify: `tests/test_dashboard_filters_to_sql.py`
+- Modify: `src/utils/table_sql.py`
+
+- [ ] **Step 1: Add failing tests**
+
+Append to `tests/test_dashboard_filters_to_sql.py`:
+
+```python
+def test_acheteur_id_uses_like_wildcards():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_acheteur_id="12345678900010",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "acheteur_id" LIKE ?'
+    assert params == [2025, "%12345678900010%"]
+
+
+def test_titulaire_id_uses_like_wildcards():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_titulaire_id="999",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "titulaire_id" LIKE ?'
+    assert params == [2025, "%999%"]
+
+
+def test_marche_objet_uses_case_insensitive_ilike():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_objet="travaux",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "objet" ILIKE ?'
+    assert params == [2025, "%travaux%"]
+
+
+def test_code_cpv_uses_prefix_like():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_code_cpv="4521",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "codeCPV" LIKE ?'
+    assert params == [2025, "4521%"]
+```
+
+- [ ] **Step 2: Run tests to verify they fail**
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py -v`
+Expected: 4 new tests FAIL.
+
+- [ ] **Step 3: Extend the helper**
+
+Insert the following block, **just after** the year/default block and **before** the `if dashboard_marche_type` block:
+
+```python
+    if dashboard_acheteur_id:
+        clauses.append('"acheteur_id" LIKE ?')
+        params.append(f"%{dashboard_acheteur_id}%")
+
+    if dashboard_titulaire_id:
+        clauses.append('"titulaire_id" LIKE ?')
+        params.append(f"%{dashboard_titulaire_id}%")
+```
+
+Insert in the "marché" block, **after** `dashboard_marche_type` and **before** `dashboard_marche_innovant`:
+
+```python
+    if dashboard_marche_objet:
+        clauses.append('"objet" ILIKE ?')
+        params.append(f"%{dashboard_marche_objet}%")
+
+    if dashboard_marche_code_cpv:
+        clauses.append('"codeCPV" LIKE ?')
+        params.append(f"{dashboard_marche_code_cpv}%")
+```
+
+- [ ] **Step 4: Run tests to verify they pass**
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py -v`
+Expected: PASS (10 tests total).
+
+- [ ] **Step 5: Commit**
+
+```bash
+rtk pre-commit run --files tests/test_dashboard_filters_to_sql.py src/utils/table_sql.py
+rtk git add tests/test_dashboard_filters_to_sql.py src/utils/table_sql.py
+rtk git commit -m "feat(observatoire): filtres LIKE/ILIKE dans dashboard_filters_to_sql (#72)"
+```
+
+---
+
+## Task 4: Filtre IN (départements) + skip conditionnel par ID
+
+**Files:**
+
+- Modify: `tests/test_dashboard_filters_to_sql.py`
+- Modify: `src/utils/table_sql.py`
+
+- [ ] **Step 1: Add failing tests**
+
+Append to `tests/test_dashboard_filters_to_sql.py`:
+
+```python
+def test_acheteur_departement_multiple_uses_in_clause():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_acheteur_departement_code=["75", "92", "93"],
+    )
+    assert where_sql == (
+        'YEAR("dateNotification") = ? '
+        'AND "acheteur_departement_code" IN (?, ?, ?)'
+    )
+    assert params == [2025, "75", "92", "93"]
+
+
+def test_acheteur_categorie_adds_clause():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_acheteur_categorie="Commune",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "acheteur_categorie" = ?'
+    assert params == [2025, "Commune"]
+
+
+def test_titulaire_categorie_and_departement():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_titulaire_categorie="PME",
+        dashboard_titulaire_departement_code=["35"],
+    )
+    assert where_sql == (
+        'YEAR("dateNotification") = ? '
+        'AND "titulaire_categorie" = ? '
+        'AND "titulaire_departement_code" IN (?)'
+    )
+    assert params == [2025, "PME", "35"]
+
+
+def test_acheteur_id_present_skips_categorie_and_departement():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_acheteur_id="123",
+        dashboard_acheteur_categorie="Commune",
+        dashboard_acheteur_departement_code=["75"],
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "acheteur_id" LIKE ?'
+    assert params == [2025, "%123%"]
+
+
+def test_titulaire_id_present_skips_categorie_and_departement():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_titulaire_id="999",
+        dashboard_titulaire_categorie="PME",
+        dashboard_titulaire_departement_code=["35"],
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "titulaire_id" LIKE ?'
+    assert params == [2025, "%999%"]
+```
+
+- [ ] **Step 2: Run tests to verify they fail**
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py -v`
+Expected: 5 new tests FAIL.
+
+- [ ] **Step 3: Refactor the helper with conditional skip**
+
+Replace the two simple `if dashboard_acheteur_id` / `if dashboard_titulaire_id` blocks added in Task 3 with the nested form:
+
+```python
+    if dashboard_acheteur_id:
+        clauses.append('"acheteur_id" LIKE ?')
+        params.append(f"%{dashboard_acheteur_id}%")
+    else:
+        if dashboard_acheteur_categorie:
+            clauses.append('"acheteur_categorie" = ?')
+            params.append(dashboard_acheteur_categorie)
+        if dashboard_acheteur_departement_code:
+            placeholders = ", ".join(["?"] * len(dashboard_acheteur_departement_code))
+            clauses.append(f'"acheteur_departement_code" IN ({placeholders})')
+            params.extend(dashboard_acheteur_departement_code)
+
+    if dashboard_titulaire_id:
+        clauses.append('"titulaire_id" LIKE ?')
+        params.append(f"%{dashboard_titulaire_id}%")
+    else:
+        if dashboard_titulaire_categorie:
+            clauses.append('"titulaire_categorie" = ?')
+            params.append(dashboard_titulaire_categorie)
+        if dashboard_titulaire_departement_code:
+            placeholders = ", ".join(
+                ["?"] * len(dashboard_titulaire_departement_code)
+            )
+            clauses.append(f'"titulaire_departement_code" IN ({placeholders})')
+            params.extend(dashboard_titulaire_departement_code)
+```
+
+- [ ] **Step 4: Run tests to verify they pass**
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py -v`
+Expected: PASS (15 tests total).
+
+- [ ] **Step 5: Commit**
+
+```bash
+rtk pre-commit run --files tests/test_dashboard_filters_to_sql.py src/utils/table_sql.py
+rtk git add tests/test_dashboard_filters_to_sql.py src/utils/table_sql.py
+rtk git commit -m "feat(observatoire): IN départements et skip conditionnel par ID (#72)"
+```
+
+---
+
+## Task 5: Filtre liste (techniques, considérations sociales/environnementales)
+
+**Files:**
+
+- Modify: `tests/test_dashboard_filters_to_sql.py`
+- Modify: `src/utils/table_sql.py`
+
+- [ ] **Step 1: Add failing tests**
+
+Append to `tests/test_dashboard_filters_to_sql.py`:
+
+```python
+def test_marche_techniques_uses_list_has_any():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_techniques=["Enchère", "Accord-cadre"],
+    )
+    assert where_sql == (
+        'YEAR("dateNotification") = ? '
+        "AND list_has_any(string_split(\"techniques\", ', '), ?::VARCHAR[])"
+    )
+    assert params == [2025, ["Enchère", "Accord-cadre"]]
+
+
+def test_considerations_sociales_uses_list_has_any():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_considerations_sociales=["Clause sociale"],
+    )
+    assert where_sql == (
+        'YEAR("dateNotification") = ? '
+        "AND list_has_any(string_split(\"considerationsSociales\", ', '), ?::VARCHAR[])"
+    )
+    assert params == [2025, ["Clause sociale"]]
+
+
+def test_considerations_environnementales_uses_list_has_any():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_considerations_environnementales=["Clause env."],
+    )
+    assert where_sql == (
+        'YEAR("dateNotification") = ? '
+        "AND list_has_any(string_split(\"considerationsEnvironnementales\", ', '), ?::VARCHAR[])"
+    )
+    assert params == [2025, ["Clause env."]]
+```
+
+- [ ] **Step 2: Run tests to verify they fail**
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py -v`
+Expected: 3 new tests FAIL.
+
+- [ ] **Step 3: Extend the helper**
+
+Insert the following block in `dashboard_filters_to_sql`, **after** the `dashboard_marche_sous_traitance_declaree` block and **before** `return " AND ".join(clauses), params`:
+
+```python
+    if dashboard_marche_techniques:
+        clauses.append(
+            "list_has_any(string_split(\"techniques\", ', '), ?::VARCHAR[])"
+        )
+        params.append(list(dashboard_marche_techniques))
+
+    if dashboard_marche_considerations_sociales:
+        clauses.append(
+            "list_has_any(string_split(\"considerationsSociales\", ', '), ?::VARCHAR[])"
+        )
+        params.append(list(dashboard_marche_considerations_sociales))
+
+    if dashboard_marche_considerations_environnementales:
+        clauses.append(
+            "list_has_any(string_split(\"considerationsEnvironnementales\", ', '), ?::VARCHAR[])"
+        )
+        params.append(list(dashboard_marche_considerations_environnementales))
+```
+
+- [ ] **Step 4: Run tests to verify they pass**
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py -v`
+Expected: PASS (18 tests total).
+
+- [ ] **Step 5: Commit**
+
+```bash
+rtk pre-commit run --files tests/test_dashboard_filters_to_sql.py src/utils/table_sql.py
+rtk git add tests/test_dashboard_filters_to_sql.py src/utils/table_sql.py
+rtk git commit -m "feat(observatoire): filtres liste via list_has_any (#72)"
+```
+
+---
+
+## Task 6: Filtres montant min/max (incluant 0)
+
+**Files:**
+
+- Modify: `tests/test_dashboard_filters_to_sql.py`
+- Modify: `src/utils/table_sql.py`
+
+- [ ] **Step 1: Add failing tests**
+
+Append to `tests/test_dashboard_filters_to_sql.py`:
+
+```python
+def test_montant_min_only():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_montant_min=1000,
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "montant" >= ?'
+    assert params == [2025, 1000]
+
+
+def test_montant_max_only():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_montant_max=500,
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "montant" <= ?'
+    assert params == [2025, 500]
+
+
+def test_montant_zero_is_a_valid_lower_bound():
+    # 0 est falsy mais reste un filtre valide (distinct de None)
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_montant_min=0,
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "montant" >= ?'
+    assert params == [2025, 0]
+
+
+def test_montant_min_and_max_combined():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_montant_min=100,
+        dashboard_montant_max=1000,
+    )
+    assert where_sql == (
+        'YEAR("dateNotification") = ? AND "montant" >= ? AND "montant" <= ?'
+    )
+    assert params == [2025, 100, 1000]
+```
+
+- [ ] **Step 2: Run tests to verify they fail**
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py -v`
+Expected: 4 new tests FAIL.
+
+- [ ] **Step 3: Extend the helper**
+
+Insert at the very end of `dashboard_filters_to_sql`, **just before** `return " AND ".join(clauses), params`:
+
+```python
+    if dashboard_montant_min is not None:
+        clauses.append('"montant" >= ?')
+        params.append(dashboard_montant_min)
+
+    if dashboard_montant_max is not None:
+        clauses.append('"montant" <= ?')
+        params.append(dashboard_montant_max)
+```
+
+- [ ] **Step 4: Run tests to verify they pass**
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py -v`
+Expected: PASS (22 tests total).
+
+- [ ] **Step 5: Commit**
+
+```bash
+rtk pre-commit run --files tests/test_dashboard_filters_to_sql.py src/utils/table_sql.py
+rtk git add tests/test_dashboard_filters_to_sql.py src/utils/table_sql.py
+rtk git commit -m "feat(observatoire): filtres montant min/max (#72)"
+```
+
+---
+
+## Task 7: Réécriture de `prepare_dashboard_data`
+
+**Files:**
+
+- Modify: `src/utils/data.py`
+- Modify: `tests/test_main.py` (supprimer `test_010_observatoire_montant_filter`)
+
+- [ ] **Step 1: Remove the obsolete Polars-based test**
+
+Delete the function `test_010_observatoire_montant_filter` from `tests/test_main.py` (lines ~218-256). La couverture du filtre montant est déjà assurée par les tests unitaires `test_montant_*` de la Task 6.
+
+- [ ] **Step 2: Rewrite `prepare_dashboard_data`**
+
+Replace the entire `prepare_dashboard_data` function in `src/utils/data.py` (lines ~86-194) with:
+
+```python
+def prepare_dashboard_data(**filter_params) -> pl.DataFrame:
+    """Exécute la requête DuckDB filtrée pour le tableau de bord.
+
+    Retourne une pl.DataFrame matérialisée uniquement pour le sous-ensemble
+    correspondant aux filtres. Les appelants qui ont besoin d'une LazyFrame
+    appellent `.lazy()` sur le résultat.
+    """
+    from src.utils.table_sql import dashboard_filters_to_sql
+
+    where_sql, params = dashboard_filters_to_sql(**filter_params)
+    return query_marches(where_sql=where_sql, params=params)
+```
+
+Update the import at the top of `src/utils/data.py`:
+
+```python
+from src.db import get_cursor, query_marches, schema
+```
+
+Remove the now-unused import in `src/utils/data.py`:
+
+```python
+from datetime import datetime, timedelta
+```
+
+(Si `datetime` n'est plus référencé dans `data.py` hors de `prepare_dashboard_data`, sinon garder.)
+
+**Vérification rapide à effectuer avant de supprimer `datetime`/`timedelta`** :
+
+```bash
+rtk grep -n "datetime\|timedelta" src/utils/data.py
+```
+
+Si d'autres occurrences existent, conserver les imports.
+
+- [ ] **Step 3: Run the full test suite**
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py tests/test_main.py -v -k "not selenium and not dash_duo"`
+
+Ou, si filter n'est pas pratique :
+
+Run: `rtk pytest tests/test_dashboard_filters_to_sql.py -v`
+
+Expected: PASS (22 tests).
+
+- [ ] **Step 4: Commit**
+
+```bash
+rtk pre-commit run --files src/utils/data.py tests/test_main.py
+rtk git add src/utils/data.py tests/test_main.py
+rtk git commit -m "refactor(observatoire): prepare_dashboard_data utilise DuckDB (#72)"
+```
+
+---
+
+## Task 8: Adaptation des 3 appelants dans `observatoire.py`
+
+**Files:**
+
+- Modify: `src/pages/observatoire.py`
+
+- [ ] **Step 1: Update `_compute_dashboard_children`**
+
+Remplacer dans `src/pages/observatoire.py` (autour des lignes 660-670) :
+
+```python
+@cache.memoize()
+def _compute_dashboard_children(filter_params_normalized: tuple):
+    logger.debug("Cache miss — computing dashboard")
+    filter_params = {
+        k: (list(v) if isinstance(v, tuple) else v) for k, v in filter_params_normalized
+    }
+
+    lff: pl.LazyFrame = query_marches().lazy()
+    lff = prepare_dashboard_data(lff=lff, **filter_params)
+
+    dff = lff.collect(engine="streaming")
+```
+
+Par :
+
+```python
+@cache.memoize()
+def _compute_dashboard_children(filter_params_normalized: tuple):
+    logger.debug("Cache miss — computing dashboard")
+    filter_params = {
+        k: (list(v) if isinstance(v, tuple) else v) for k, v in filter_params_normalized
+    }
+
+    dff = prepare_dashboard_data(**filter_params)
+    lff = dff.lazy()
+```
+
+Le reste de la fonction (à partir de `df_per_uid = ...`) est inchangé.
+
+- [ ] **Step 2: Update `download_observatoire`**
+
+Remplacer dans `src/pages/observatoire.py` (autour des lignes 789-800) :
+
+```python
+def download_observatoire(_n_clicks, filter_params, hidden_columns):
+    lff = prepare_dashboard_data(lff=query_marches().lazy(), **(filter_params or {}))
+
+    if hidden_columns:
+        lff = lff.drop(hidden_columns)
+
+    def to_bytes(buffer):
+        lff.collect(engine="streaming").write_excel(buffer, worksheet="DECP")
+
+    date = datetime.now().strftime("%Y-%m-%d_%H:%M:%S")
+    return dcc.send_bytes(to_bytes, filename=f"decp_observatoire_{date}.xlsx")
+```
+
+Par :
+
+```python
+def download_observatoire(_n_clicks, filter_params, hidden_columns):
+    dff = prepare_dashboard_data(**(filter_params or {}))
+
+    if hidden_columns:
+        dff = dff.drop(hidden_columns)
+
+    def to_bytes(buffer):
+        dff.write_excel(buffer, worksheet="DECP")
+
+    date = datetime.now().strftime("%Y-%m-%d_%H:%M:%S")
+    return dcc.send_bytes(to_bytes, filename=f"decp_observatoire_{date}.xlsx")
+```
+
+- [ ] **Step 3: Update `populate_preview_table`**
+
+Remplacer dans `src/pages/observatoire.py` (autour des lignes 879-892) :
+
+```python
+    if not is_open:
+        return (no_update,) * 9
+
+    lff = prepare_dashboard_data(lff=query_marches().lazy(), **(filter_params or {}))
+
+    return prepare_table_data(
+        lff,
+        data_timestamp,
+        filter_query,
+        page_current,
+        page_size,
+        sort_by,
+        "observatoire-preview",
+    )
+```
+
+Par :
+
+```python
+    if not is_open:
+        return (no_update,) * 9
+
+    dff = prepare_dashboard_data(**(filter_params or {}))
+
+    return prepare_table_data(
+        dff.lazy(),
+        data_timestamp,
+        filter_query,
+        page_current,
+        page_size,
+        sort_by,
+        "observatoire-preview",
+    )
+```
+
+- [ ] **Step 4: Remove unused `query_marches` import**
+
+Dans `src/pages/observatoire.py`, ligne ~19 :
+
+```python
+from src.db import query_marches, schema
+```
+
+Devient :
+
+```python
+from src.db import schema
+```
+
+Vérifier avant de committer :
+
+```bash
+rtk grep -n "query_marches" src/pages/observatoire.py
+```
+
+Expected: aucun résultat (ou uniquement des commentaires).
+
+- [ ] **Step 5: Smoke test**
+
+Démarrer l'app et naviguer sur `/observatoire`, vérifier à la main que :
+
+- Les cartes s'affichent.
+- Un filtre année se propage.
+- Un filtre acheteur par SIRET partiel fonctionne.
+- Un filtre département (multi-valeur) fonctionne.
+- Un filtre montant_min fonctionne.
+- Le bouton « Télécharger au format Excel » génère un fichier non vide.
+- Le bouton « Voir les données » ouvre l'offcanvas et peuple la table.
+
+Run: `python run.py`
+
+Expected: app démarre sans erreur ; les filtres se comportent comme avant.
+
+- [ ] **Step 6: Commit**
+
+```bash
+rtk pre-commit run --files src/pages/observatoire.py
+rtk git add src/pages/observatoire.py
+rtk git commit -m "refactor(observatoire): appelants utilisent la nouvelle signature (#72)"
+```
+
+---
+
+## Task 9: Test d'intégration — `prepare_dashboard_data` sur `tests/test.parquet`
+
+**Files:**
+
+- Create: `tests/test_prepare_dashboard_data.py`
+
+- [ ] **Step 1: Write the failing test**
+
+Le but : vérifier que la fonction s'exécute réellement contre DuckDB, retourne une `pl.DataFrame`, et applique bien les filtres simples. `conftest.py` construit `tests/test.parquet` avec un jeu de données d'une ligne : acheteur_id `123`, acheteur_departement_code `75`, dateNotification `2025-01-01`, montant `10`.
+
+Create `tests/test_prepare_dashboard_data.py`:
+
+```python
+import polars as pl
+
+
+def test_returns_dataframe_with_year_filter():
+    from src.utils.data import prepare_dashboard_data
+
+    dff = prepare_dashboard_data(dashboard_year="2025")
+    assert isinstance(dff, pl.DataFrame)
+    assert dff.height == 1
+
+
+def test_year_mismatch_returns_empty():
+    from src.utils.data import prepare_dashboard_data
+
+    dff = prepare_dashboard_data(dashboard_year="2024")
+    assert isinstance(dff, pl.DataFrame)
+    assert dff.height == 0
+
+
+def test_acheteur_id_partial_match():
+    from src.utils.data import prepare_dashboard_data
+
+    dff = prepare_dashboard_data(
+        dashboard_year="2025",
+        dashboard_acheteur_id="12",
+    )
+    assert dff.height == 1
+
+
+def test_departement_in_clause():
+    from src.utils.data import prepare_dashboard_data
+
+    dff = prepare_dashboard_data(
+        dashboard_year="2025",
+        dashboard_acheteur_departement_code=["75", "92"],
+    )
+    assert dff.height == 1
+
+
+def test_montant_min_above_value_excludes_row():
+    from src.utils.data import prepare_dashboard_data
+
+    dff = prepare_dashboard_data(
+        dashboard_year="2025",
+        dashboard_montant_min=1000,
+    )
+    assert dff.height == 0
+```
+
+- [ ] **Step 2: Run the test**
+
+Run: `rtk pytest tests/test_prepare_dashboard_data.py -v`
+Expected: PASS (5 tests).
+
+- [ ] **Step 3: Commit**
+
+```bash
+rtk pre-commit run --files tests/test_prepare_dashboard_data.py
+rtk git add tests/test_prepare_dashboard_data.py
+rtk git commit -m "test(observatoire): intégration DuckDB pour prepare_dashboard_data (#72)"
+```
+
+---
+
+## Task 10: Vérification finale
+
+**Files:** (aucune modification)
+
+- [ ] **Step 1: Run the full test suite**
+
+Run: `rtk pytest -v`
+Expected: tous les tests unitaires passent. Les tests Selenium peuvent échouer si Chrome n'est pas disponible — ce n'est pas bloquant s'ils étaient déjà rouges avant.
+
+- [ ] **Step 2: Check for leftover references**
+
+Run: `rtk grep -rn "prepare_dashboard_data(lff" src/ tests/`
+Expected: aucun résultat (plus d'appels avec l'ancienne signature).
+
+Run: `rtk grep -rn "query_marches().lazy()" src/`
+Expected: aucun résultat (ou uniquement dans `src/utils/table.py:prepare_table_data` pour le fallback).
+
+- [ ] **Step 3: Confirm `datetime`/`timedelta` in data.py if needed**
+
+Run: `rtk grep -n "datetime\|timedelta" src/utils/data.py`
+
+Si aucune occurrence hors imports, vérifier que les imports inutiles ont bien été retirés dans Task 7.
+
+- [ ] **Step 4: Manual timing sanity check (optionnel)**
+
+Si possible, comparer informellement le temps de `_compute_dashboard_children` sur un filtre sélectif (ex. un département) avant/après. Pas de benchmark formel attendu.
+
+- [ ] **Step 5: Push (manuel, à l'initiative de l'utilisateur)**
+
+Conformément aux consignes projet, ne jamais `git push`. Laisser l'utilisateur pousser la branche `feature/72_observatoire_duckdb_filters` et ouvrir la PR.

docs/superpowers/plans/2026-04-28-organization-maps.md

@@ -0,0 +1,108 @@
+# Plan: Ajouter des cartes de localisation aux pages acheteur et titulaire
+
+## Date: 2026-04-28
+
+## Statut: Approuvé
+
+## Objectif: Ajouter des cartes interactives montrant la localisation des organisations sur les pages acheteur et titulaire
+
+## Contexte
+
+- Les pages acheteur et titulaire ont déjà des placeholders pour les cartes (`acheteur_map` et `titulaire_map`)
+- La fonction `point_on_map()` existe déjà dans `src/figures.py` mais utilise un centrage fixe sur la France
+- Les données de localisation proviennent de l'API Annuaire des Entreprises
+- Les codes départementaux sont disponibles et plus fiables que les coordonnées pour la détection de région
+
+## Exigences
+
+### 1. Carte interactive
+
+- **Localisation**: Colonne de droite dans la section d'informations sur l'organisation
+- **Taille**: 400px de largeur × 300px de hauteur (fixe)
+- **Contenu**: Carte centrée sur la France ou le département d'outre-mer approprié avec un point rouge à l'emplacement de l'organisation
+- **Niveau de zoom**: Approprié pour montrer l'Hexagone ou le département d'outre-mer spécifique
+- **Style**: Fond de carte clair avec point rouge visible
+- **Interactivité**: Carte zoomable et déplaçable (pas de configuration statique)
+
+### 2. Sources de données
+
+- Utiliser les colonnes `acheteur_latitude` et `acheteur_longitude` pour les pages acheteur
+- Utiliser les colonnes `titulaire_latitude` et `titulaire_longitude` pour les pages titulaire
+- Utiliser les codes départementaux (`acheteur_departement_code`, `titulaire_departement_code`) pour la détection de région
+- Solution de repli: Si les coordonnées ou codes départementaux sont manquants ou invalides, afficher une div vide
+
+### 3. Détection de région
+
+- **Départements métropolitains**: Codes à 2 caractères (ex: "75" pour Paris) → Carte Hexagone
+- **Départements d'outre-mer**:
+  - "971" → Guadeloupe
+  - "972" → Martinique
+  - "973" → Guyane
+  - "974" → La Réunion
+  - "976" → Mayotte
+- **Code département manquant**: Retourner une div vide (pas de détection basée sur les coordonnées)
+
+### 4. Gestion des erreurs
+
+- Coordonnées invalides → div vide
+- Code département manquant → div vide
+- Échec de l'API Annuaire → div vide (comportement existant)
+- Format de code département invalide → div vide
+
+## Implémentation
+
+### Fichiers à modifier
+
+#### 1. `src/figures.py` - Améliorer la fonction `point_on_map()`
+
+**Ligne 178-209**: Remplacer la fonction existante par une version améliorée avec:
+
+- Détection de région basée sur les codes départementaux
+- Configuration de carte interactive (zoomable)
+- Point plus grand (size=15)
+- Commentaires en français
+
+#### 2. `src/pages/acheteur.py` - Mettre à jour le callback
+
+**Ligne 249-297**: Modifier `update_acheteur_infos()` pour:
+
+- Extraire le code département du code postal
+- Passer le code département à `point_on_map()`
+- Ajouter des commentaires en français
+
+#### 3. `src/pages/titulaire.py` - Mettre à jour le callback
+
+**Ligne 259-297**: Modifier `update_titulaire_infos()` pour:
+
+- Extraire le code département du code postal
+- Passer le code département à `point_on_map()`
+- Ajouter des commentaires en français
+
+## Plan de Test
+
+### Cas de test prioritaires
+
+1. **Organisation métropolitaine**: Code département "75" (Paris) → Carte Hexagone
+2. **Organisation à La Réunion**: Code département "974" → Carte centrée sur La Réunion
+3. **Code département manquant**: Retourne une div vide
+4. **Coordonnées invalides**: Retourne une div vide
+5. **Interactivité**: Vérifier zoom et déplacement
+
+### Critères d'acceptation
+
+- [ ] Cartes fonctionnelles avec codes départementaux valides
+- [ ] Div vide pour codes manquants/invalides
+- [ ] Cartes correctement centrées et zoomées
+- [ ] Interactivité (zoom et déplacement)
+- [ ] Point de localisation visible (size=15)
+
+## Approbation
+
+Plan approuvé avec spécifications:
+
+- Réutiliser et améliorer `point_on_map`
+- Retourner div vide sans code département
+- Point légèrement plus grand
+- Cartes zoomables
+- Utiliser codes départementaux pour détection de région
+- Commentaires en français

docs/superpowers/specs/2026-03-18-distance-histogram-design.md

@@ -0,0 +1,64 @@
+# Distance Histogram — Design Spec
+
+**Date:** 2026-03-18
+**Branch:** feature/65_observatoire
+
+## Goal
+
+Display the distribution of distances (in km) between buyers and winning contractors, to help users assess whether a buyer or contractor tends to deal locally or at a national scale.
+
+## Data
+
+- Column: `titulaire_distance` (`Int64`, km)
+- Measured at address level — values are always > 0, no zero-handling needed
+- Already selected in the observatoire LazyFrame via `cs.starts_with("titulaire")`
+- Already available on acheteur and titulaire detail pages
+
+## Figure Function
+
+**Location:** `src/figures.py`
+
+**Signature:**
+
+```python
+def get_distance_histogram(lff: pl.LazyFrame) -> dcc.Graph:
+```
+
+**Behaviour:**
+
+- Collects `titulaire_distance` from the LazyFrame, drops nulls
+- If the resulting DataFrame is empty after dropping nulls, `px.histogram` produces a blank figure without errors — no guard logic needed. The order of operations must be: drop nulls → log-transform → histogram
+- Drop nulls first, then pre-log-transform the column (`pl.col("titulaire_distance").log(10)`) so bins are truly equal-width on a log scale. Use `px.histogram` with `nbins=50` on the transformed values
+- Set custom X-axis tick values at powers of 10 (1, 10, 100, 1000, 10000) with km labels, using `fig.update_xaxes(tickvals=[0,1,2,3,4], ticktext=["1","10","100","1 000","10 000"])`
+- Y axis: count of contracts
+- French axis labels: x = `"Distance (km)"`, y = `"Nombre de marchés"`
+- Returns a `dcc.Graph`
+
+## Integration
+
+### Observatoire (`src/pages/observatoire.py`)
+
+- `get_distance_histogram` imported and called inside `udpate_dashboard_cards`
+- Result wrapped in `make_card(title="Distance acheteur–titulaire", subtitle="en nombre de marchés, échelle logarithmique", fig=...)`
+- Card appended to the `cards` list alongside existing donuts and charts
+- No changes to the data pipeline — `titulaire_distance` is already in the LazyFrame
+
+### Acheteur page (`src/pages/acheteur.py`)
+
+The acheteur page uses a `dcc.Store` (`acheteur_data`) that holds serialised contract rows as a list of dicts. The integration follows the existing pattern used by other chart callbacks on this page:
+
+- Add a new `html.Div(id="acheteur-distance-histogram")` placeholder in the layout
+- Add a new callback with `Input("acheteur_data", "data")` that:
+  - Reconstructs `pl.LazyFrame(data)` from the store
+  - Calls `get_distance_histogram(lff)`
+  - Wraps the result in `make_card(...)` and returns it to the placeholder div
+
+### Titulaire page (`src/pages/titulaire.py`)
+
+Same pattern as acheteur: `dcc.Store` (`titulaire_data`) → new callback → `html.Div` placeholder.
+
+## Out of Scope
+
+- Filtering by distance range (could be a future filter on the observatoire page)
+- Showing distance on a map or as a trend over time
+- Bucket-based (named zone) grouping

docs/superpowers/specs/2026-03-18-observatoire-link-from-search-design.md

@@ -0,0 +1,78 @@
+# Observatoire Link from Search & Tableau Results
+
+## Problem
+
+Users searching for an organization (acheteur or titulaire) on the search page or browsing the tableau cannot jump directly to the observatoire page filtered for that organization. They must manually navigate and re-enter the identifier.
+
+## Solution
+
+Extend `add_links()` in `src/utils.py` to append an observatoire link (📊 emoji) to `_nom` columns, and add bidirectional URL parameter sync to the observatoire page.
+
+## Changes
+
+### 1. `src/utils.py` — `add_links()` modification
+
+The existing `add_links()` loop iterates over `["uid", "acheteur_nom", "titulaire_nom", "acheteur_id", "titulaire_id"]`. The `if col.startswith("acheteur_")` and `if col.startswith("titulaire_")` blocks match both `_nom` and `_id` columns. The observatoire link must only be appended to `_nom` columns, so it must be gated on `col == "acheteur_nom"` or `col == "titulaire_nom"` explicitly.
+
+For `acheteur_nom`, append an observatoire link after the existing detail page link:
+
+```
+Before: <a href="/acheteurs/12345678901234">Ville de Paris</a>
+After:  <a href="/acheteurs/12345678901234">Ville de Paris</a> <a href="/observatoire?acheteur_id=12345678901234" title="Voir dans l'observatoire">📊</a>
+```
+
+For `titulaire_nom`, same pattern but only when the existing `typeIdentifiant` guard passes (SIRET or null):
+
+```
+Before: <a href="/titulaires/12345678901234">Entreprise X</a>
+After:  <a href="/titulaires/12345678901234">Entreprise X</a> <a href="/observatoire?titulaire_id=12345678901234" title="Voir dans l'observatoire">📊</a>
+```
+
+The identifier used in the observatoire link (`acheteur_id` / `titulaire_id`) is the same `pl.col("acheteur_id")` / `pl.col("titulaire_id")` column value already used for the detail page link.
+
+The `_id` and `uid` columns are unchanged.
+
+### 2. `src/pages/observatoire.py` — URL parameter handling
+
+#### Callback A: URL → Inputs (page load)
+
+- Trigger: `Input("dashboard_url", "search")`
+- Outputs: `Output("dashboard_acheteur_id", "value")`, `Output("dashboard_titulaire_id", "value")`, `Output("dashboard_url", "search")` (to clear it)
+- `prevent_initial_call=False` (must fire on page load to read URL params)
+- If `search` is empty or None: return `no_update` for all outputs
+- Otherwise: parse query params with `urllib.parse.parse_qs`
+- Set `dashboard_acheteur_id` from `?acheteur_id=` param, or `no_update` if absent
+- Set `dashboard_titulaire_id` from `?titulaire_id=` param, or `no_update` if absent
+- Return `""` for `dashboard_url.search` to clear the URL and prevent re-triggering
+- No validation of param values — consistent with existing input handling in the observatoire callbacks
+
+#### Callback B: Inputs → shareable URL
+
+- Trigger: `Input("dashboard_acheteur_id", "value")`, `Input("dashboard_titulaire_id", "value")`
+- State: `State("dashboard_url", "href")` for base URL
+- `prevent_initial_call=True` (avoid generating URL on initial empty state)
+- Build query string with `urllib.parse.urlencode`, omitting empty values
+- Write full URL to a new `share-url` input component
+- Render a `dcc.Clipboard` + share button (same pattern as tableau.py)
+
+#### Callback chain
+
+When navigating from search with `?acheteur_id=123`: Callback A fires on page load, sets input values, clears URL search. The input value changes then trigger both the existing `udpate_dashboard_cards` callback and Callback B. Dash handles this chaining deterministically — no race condition.
+
+#### Layout additions
+
+- A `dcc.Input(id="share-url", ...)` (hidden or read-only) to hold the shareable URL
+- A `dcc.Clipboard` share/copy button near the filters
+
+### 3. Reuse of existing `dcc.Location`
+
+The existing `dcc.Location(id="dashboard_url")` component is reused — no new Location component needed.
+
+## Future extension
+
+The bidirectional URL sync pattern is designed to extend to all observatoire filters (year, categories, departments, market type, etc.) by adding more params to both callbacks.
+
+## Files touched
+
+- `src/utils.py` — modify `add_links()`
+- `src/pages/observatoire.py` — add 2 callbacks, add share-url + clipboard to layout

docs/superpowers/specs/2026-03-21-observatoire-full-url-sharing-design.md

@@ -0,0 +1,121 @@
+# Observatoire: Full URL Sharing for All Filters
+
+## Problem
+
+The "Partager" button on `/observatoire` currently only encodes `acheteur_id` and `titulaire_id` in the shareable URL. The other 15 filter parameters are lost, so a shared link does not reproduce the sender's filtered view.
+
+## Goal
+
+Extend URL sharing so that **all 17 filter parameters** are encoded in the URL and restored when a recipient opens it. The recipient sees exactly what the sender intended — URL params replace all local filter state.
+
+## Approach
+
+Flat query parameters with short, readable keys. Multi-value filters use repeated keys (native to `urllib.parse`). Only non-default values appear in the URL.
+
+## URL Parameter Mapping
+
+| Component ID                                       | URL key          | Type            | Default (omitted) |
+| -------------------------------------------------- | ---------------- | --------------- | ----------------- |
+| `dashboard_year`                                   | `annee`          | single          | `None`            |
+| `dashboard_acheteur_id`                            | `acheteur_id`    | single          | `None`            |
+| `dashboard_acheteur_categorie`                     | `acheteur_cat`   | single          | `None`            |
+| `dashboard_acheteur_departement_code`              | `acheteur_dept`  | multi           | `[]`/`None`       |
+| `dashboard_titulaire_id`                           | `titulaire_id`   | single          | `None`            |
+| `dashboard_titulaire_categorie`                    | `titulaire_cat`  | single          | `None`            |
+| `dashboard_titulaire_departement_code`             | `titulaire_dept` | multi           | `[]`/`None`       |
+| `dashboard_marche_type`                            | `type`           | single          | `None`            |
+| `dashboard_marche_objet`                           | `objet`          | single          | `None`            |
+| `dashboard_marche_code_cpv`                        | `cpv`            | single          | `None`            |
+| `dashboard_montant_min`                            | `montant_min`    | single (number) | `None`            |
+| `dashboard_montant_max`                            | `montant_max`    | single (number) | `None`            |
+| `dashboard_marche_techniques`                      | `techniques`     | multi           | `[]`/`None`       |
+| `dashboard_marche_innovant`                        | `innovant`       | single          | `"all"`           |
+| `dashboard_marche_sousTraitanceDeclaree`           | `sous_traitance` | single          | `"all"`           |
+| `dashboard_marche_considerationsSociales`          | `social`         | multi           | `[]`/`None`       |
+| `dashboard_marche_considerationsEnvironnementales` | `env`            | multi           | `[]`/`None`       |
+
+Example URL:
+
+```
+/observatoire?annee=2024&acheteur_id=12345678901234&acheteur_dept=75&acheteur_dept=13&montant_min=10000&innovant=oui
+```
+
+## Data Structure
+
+A list of tuples defines the mapping, used by both callbacks to avoid scattered string literals:
+
+```python
+FILTER_PARAMS = [
+    # (component_id, url_key, is_multi, default_value)
+    ("dashboard_year", "annee", False, None),
+    ("dashboard_acheteur_id", "acheteur_id", False, None),
+    ("dashboard_acheteur_categorie", "acheteur_cat", False, None),
+    ("dashboard_acheteur_departement_code", "acheteur_dept", True, None),
+    ("dashboard_titulaire_id", "titulaire_id", False, None),
+    ("dashboard_titulaire_categorie", "titulaire_cat", False, None),
+    ("dashboard_titulaire_departement_code", "titulaire_dept", True, None),
+    ("dashboard_marche_type", "type", False, None),
+    ("dashboard_marche_objet", "objet", False, None),
+    ("dashboard_marche_code_cpv", "cpv", False, None),
+    ("dashboard_montant_min", "montant_min", False, None),
+    ("dashboard_montant_max", "montant_max", False, None),
+    ("dashboard_marche_techniques", "techniques", True, None),
+    ("dashboard_marche_innovant", "innovant", False, "all"),
+    ("dashboard_marche_sousTraitanceDeclaree", "sous_traitance", False, "all"),
+    ("dashboard_marche_considerationsSociales", "social", True, None),
+    ("dashboard_marche_considerationsEnvironnementales", "env", True, None),
+]
+```
+
+## Callback Changes
+
+### 1. `sync_observatoire_share_url` (line 575)
+
+**Current:** Takes `acheteur_id` and `titulaire_id` as Inputs.
+
+**New:** Takes all 17 filter values as Inputs (same as `udpate_dashboard_cards`). Builds the URL using `FILTER_PARAMS`, skipping default values. Uses `urllib.parse.urlencode(params, doseq=True)` for multi-value params.
+
+### 2. `restore_filters` (line 539)
+
+**Current:** Extracts only `acheteur_id` and `titulaire_id` from URL.
+
+**New:**
+
+- Iterates over `FILTER_PARAMS` to extract all values from `parse_qs`
+- For multi-value params: reads the full list from `parse_qs` (returns lists natively)
+- For number params (`montant_min`, `montant_max`): casts to `float`
+- The guard condition changes from `if acheteur_id or titulaire_id` to "if any URL param is present" — this is necessary so URLs like `?annee=2024&montant_min=10000` (without an ID) work correctly
+- When **any** URL param is present: returns explicit values for all 17 outputs — the URL value for params present, `None`/default for params absent. This ensures "URL replaces all" semantics.
+- When **no** URL params are present: returns `(no_update,) * 17` (preserving local persistence)
+- Radio buttons (`innovant`, `sous_traitance`): value from URL if present, otherwise `"all"` (their default)
+
+### 3. Layout bug fix
+
+Remove the duplicate `dcc.Input(id="observatoire-share-url")` (lines 413-422 — two identical elements).
+
+## Backward Compatibility
+
+Old URLs with only `?acheteur_id=...` or `?titulaire_id=...` continue to work — the new `restore_filters` will read those keys and reset all others to defaults, which is the same effective behavior as before.
+
+Links generated by `add_links()` in `src/utils.py` (used on search results to link to `/observatoire?acheteur_id=...`) are unaffected.
+
+## Test Changes
+
+### Fix broken test `test_010_observatoire_montant_filter`
+
+This test imports `_apply_filters` from `pages.observatoire`, which no longer exists (replaced by `prepare_dashboard_data` in `src/utils.py`). Fix:
+
+- Replace import with `from src.utils import prepare_dashboard_data`
+- Update the call to match `prepare_dashboard_data`'s signature: rename `marche_type` keyword to `type`, and add missing params `objet`, `code_cpv`, `techniques`, `marche_innovant`, `sous_traitance_declaree` (all as `None`)
+
+### New test: multi-param URL round-trip
+
+Add a test that navigates to `/observatoire?annee=2024&acheteur_id=<test_id>&montant_min=10000` and verifies that:
+
+- `dashboard_year` dropdown shows "2024"
+- `dashboard_acheteur_id` input contains the test ID
+- `dashboard_montant_min` input contains "10000"
+
+### Update existing tests
+
+Tests `test_006` and `test_007` validate `acheteur_id` round-trip. These should continue to pass without changes since `acheteur_id` keeps the same URL key.

docs/superpowers/specs/2026-04-15-duckdb-migration-design.md

@@ -0,0 +1,196 @@
+# DuckDB migration — design spec
+
+**Date:** 2026-04-15
+**Branch:** dev
+**Status:** Approved, ready for planning
+
+## Goal
+
+Replace the global Polars dataframes that `src/utils.py` materializes at import time (`df` and the five derived frames, lines 891–913) with a DuckDB database on disk. The main table holds ~1.5M rows from `decp_prod.parquet`. Per-request queries pull only what each page needs, dramatically reducing steady-state RSS memory.
+
+Polars stays the primary API for small result sets and post-processing. DuckDB carries the heavy filtering, joining, and aggregation.
+
+## Approach summary
+
+- **Approach A — compatibility layer.** A new `src/db.py` module exposes a `query_marches(where_sql, params, columns, ...)` helper that runs SQL and returns a `pl.DataFrame`. Most existing `df.filter(pl.col(...) == x)` call sites translate mechanically to `query_marches("col = ?", (x,))`. The shape of downstream Polars code is unchanged.
+- **Two small helpers stay in memory.** `df_acheteurs` and `df_titulaires` (tens of thousands of rows, consumed by the autocomplete search on every keystroke) are kept as module-level Polars frames. They are populated from DuckDB at import time, not from Parquet.
+- **Four derived tables live in DuckDB**, built at startup alongside the main table: `acheteurs_marches`, `titulaires_marches`, `acheteurs_departement`, `titulaires_departement`.
+- **Connection model.** One read-only `duckdb.connect(..., read_only=True)` at module load, shared across the process. `conn.cursor()` per Dash callback for thread-safety. The read-write connection is short-lived and only used during the startup build phase.
+
+## Cache invalidation rule
+
+At startup, rebuild the DuckDB file if:
+
+1. **The DB file does not exist**, OR
+2. **`decp_prod.parquet.mtime > duckdb.mtime`**, **unless** `DEVELOPMENT=true` and `REBUILD_DUCKDB != true` — in which case the DB stays as-is (fast dev reloads).
+
+Production auto-rebuilds when the source Parquet is newer. Development keeps a stable DB across reloads unless the developer explicitly sets `REBUILD_DUCKDB=true` to force a rebuild.
+
+## Concurrency
+
+Multi-worker Gunicorn startup and crashed-mid-build scenarios are handled by a file lock, not by polling for the tmp file's existence:
+
+```python
+with open(DB_PATH.with_suffix(".duckdb.lock"), "w") as lock_fd:
+    fcntl.flock(lock_fd, fcntl.LOCK_EX)          # blocks if another worker is building
+    if should_rebuild(DB_PATH, PARQUET_PATH):
+        build_database(DB_PATH, PARQUET_PATH)
+conn = duckdb.connect(str(DB_PATH), read_only=True)
+```
+
+- Worker A acquires the lock, builds, atomically renames tmp → final, releases the lock.
+- Worker B blocks on `flock`, then re-checks `should_rebuild`, sees the fresh DB, skips building.
+- `fcntl.flock` is auto-released on process death, so a crash never deadlocks the next worker.
+- `build_database` unlinks any pre-existing tmp file before starting (safe because it holds the lock) — handles an abandoned tmp from a crashed previous build.
+
+## Build logic
+
+The build keeps **one source of truth** for transforms by reusing the existing Polars pipeline:
+
+```python
+def build_database(db_path, parquet_path):
+    tmp_path = db_path.with_suffix(".duckdb.tmp")
+    if tmp_path.exists():
+        tmp_path.unlink()
+    frame = get_decp_data()   # existing function in utils.py
+    with duckdb.connect(str(tmp_path)) as w:
+        w.register("frame", frame)
+        w.execute("CREATE TABLE decp AS SELECT * FROM frame")
+        w.execute("CREATE TABLE acheteurs_marches AS "
+                  "SELECT DISTINCT uid, objet, acheteur_id FROM decp "
+                  "ORDER BY acheteur_id")
+        w.execute("CREATE TABLE titulaires_marches AS "
+                  "SELECT DISTINCT uid, objet, titulaire_id FROM decp "
+                  "ORDER BY titulaire_id")
+        w.execute("CREATE TABLE acheteurs_departement AS "
+                  "SELECT DISTINCT acheteur_id, acheteur_nom, acheteur_departement_code "
+                  "FROM decp ORDER BY acheteur_nom")
+        w.execute("CREATE TABLE titulaires_departement AS "
+                  "SELECT DISTINCT titulaire_id, titulaire_nom, titulaire_departement_code "
+                  "FROM decp ORDER BY titulaire_nom")
+    os.replace(tmp_path, db_path)
+```
+
+Why Polars, not SQL, for the row-level transforms:
+
+- `booleans_to_strings` is not a simple cast — it replaces `true`/`false` with `"oui"`/`"non"` on every boolean column. Reimplementing in SQL risks drifting from the Polars version.
+- The null-name replacement (`acheteur_nom`, `titulaire_nom` → `"[Identifiant non reconnu dans la base INSEE]"`) is also easier to keep identical in Polars.
+- `w.register("frame", frame)` is zero-copy. The memory spike is one-time during build and released when the write connection closes.
+
+`os.replace` is atomic on POSIX — the read-only connection that opens next always sees a complete DB.
+
+## Module layout
+
+### New: `src/db.py`
+
+```python
+conn: duckdb.DuckDBPyConnection       # read-only, module-level
+schema: pl.Schema                     # from conn.execute("SELECT * FROM decp LIMIT 0").pl().schema
+
+def get_cursor() -> duckdb.DuckDBPyConnection: ...
+def query_marches(where_sql: str = "TRUE",
+                  params: tuple = (),
+                  columns: list[str] | None = None,
+                  order_by: str | None = None,
+                  limit: int | None = None) -> pl.DataFrame: ...
+def should_rebuild(db_path: Path, parquet_path: Path) -> bool: ...
+def build_database(db_path: Path, parquet_path: Path) -> None: ...
+```
+
+Only imports: `polars`, `duckdb`, `os`, `fcntl`, `pathlib`, `logging`. No app modules — prevents circular imports.
+
+### Changes to `src/utils.py`
+
+- `df: pl.DataFrame = get_decp_data()` — **removed** (after migration).
+- `df_acheteurs`, `df_titulaires` — **kept as Polars globals**, populated via DuckDB at import time. The query mirrors today's `get_org_data(df, org_type)`: select all columns whose name starts with `acheteur_` (or `titulaire_`) except the `_latitude` / `_longitude` pair, plus `COUNT(*) AS "Marchés"`, grouped by the same set. Implementation can either:
+
+  - enumerate the columns by filtering `schema.names()` at import time and build the `SELECT` / `GROUP BY` strings, or
+  - call `get_org_data()` once against a small Polars frame returned by `SELECT <org_ cols> FROM decp`.
+
+  Feeds `search_org` unchanged.
+
+- `df_acheteurs_marches`, `df_titulaires_marches`, `df_acheteurs_departement`, `df_titulaires_departement` — **removed** as Python globals. Call sites query the corresponding DuckDB tables.
+- `schema` — imported from `src/db.py` (stays a `pl.Schema` — so `schema.names()` and dtype lookups both work, no call-site changes beyond `acheteur.py:303`).
+- `columns` — replaced with `schema.names()`.
+- `get_decp_data()` — **kept** (used by `build_database`).
+- `get_org_data()` — can be removed once `df_acheteurs` / `df_titulaires` are populated from DuckDB directly.
+
+### Call-site translations
+
+| Before (Polars global)                                       | After                                                                             |
+| ------------------------------------------------------------ | --------------------------------------------------------------------------------- |
+| `df.filter(pl.col("acheteur_id") == aid)`                    | `query_marches("acheteur_id = ?", (aid,))`                                        |
+| `df.filter(pl.col("uid") == uid).row(0, named=True)`         | `query_marches("uid = ?", (uid,)).row(0, named=True)`                             |
+| `df.select("uid","objet","acheteur_id").filter(...)`         | `query_marches("...", (...), columns=["uid","objet","acheteur_id"])`              |
+| `df.columns`                                                 | `schema.names()`                                                                  |
+| `df_acheteurs_marches.filter(...)`                           | `get_cursor().execute("SELECT ... FROM acheteurs_marches WHERE ...", [...]).pl()` |
+| `pl.DataFrame(schema=df.collect_schema())` (acheteur.py:303) | `pl.DataFrame(schema=schema)`                                                     |
+
+Heavy dashboard aggregations (observatoire, tableau full-scan) use raw SQL via `get_cursor().execute(...).pl()` rather than the helper.
+
+## Configuration
+
+- **`DATA_FILE_PARQUET_PATH`** — unchanged.
+- **DuckDB file location** — computed: `Path(DATA_FILE_PARQUET_PATH).parent / "decp.duckdb"`. No new env var.
+- **`REBUILD_DUCKDB`** — new, optional, default `false`. In development, setting this to `true` forces a rebuild when the parquet is newer.
+- **`DEVELOPMENT`** — unchanged; now also gates the auto-rebuild behavior per the rule above.
+
+## Testing
+
+- `tests/conftest.py` (or a startup hook in `src/db.py`) ensures the test run builds the DuckDB in a temp directory derived from the parquet path — `tests/test.parquet` → `tests/decp.duckdb`. This file is added to `.gitignore`.
+- Tests already set `DEVELOPMENT=true`; they must also set `REBUILD_DUCKDB=true` on cold test runs to force a fresh build from the test parquet.
+- The existing Selenium suite exercises every page and is the primary acceptance signal.
+
+## Migration order
+
+Incremental — `df` global coexists with `src/db.py` until every page is migrated.
+
+1. **Add `src/db.py`** (build, lock, `query_marches`, `schema`). `df` global unchanged.
+2. **Migrate `marche.py`** — single-row lookup by `uid`, one call site.
+3. **Migrate `acheteur.py`, `titulaire.py`** — filter by id.
+4. **Migrate `arbre/departement.py`, `arbre/liste_marches_org.py`** — use the new derived DuckDB tables.
+5. **Migrate `tableau.py`** — may need raw SQL.
+6. **Migrate `observatoire.py`** — heaviest aggregations, most likely raw SQL.
+7. **Migrate `figures.py`** — uses `df` in chart generation.
+8. **Remove** `df`, `df_*_marches`, `df_*_departement` globals, `get_org_data()`, and the `df = get_decp_data()` call from `utils.py`. Move `schema` / `columns` exports to `src/db.py`.
+
+### Verification gates
+
+- `uv run pytest` green after every page migration.
+- Manual smoke test via `uv run run.py` of the migrated page before proceeding.
+- RSS memory measurement (`ps -o rss`) of a cold `gunicorn app:server` with the prod parquet, before and after, to confirm the memory reduction.
+
+## Out of scope
+
+- Changes to `src/cache.py` (flask-caching stays).
+- The in-progress observatoire-localstorage-filters work on `dev`.
+- Schema changes to the parquet.
+- SQL views beyond the four derived tables.
+- Multi-database or replication setups.
+
+## Risks and mitigations
+
+| Risk                                                                          | Mitigation                                                                                                                       |
+| ----------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
+| `booleans_to_strings` reimplemented in SQL and drifts from Polars version     | Transforms stay in Polars via `w.register("frame", frame)`. One source of truth.                                                 |
+| Two Gunicorn workers rebuild concurrently                                     | `fcntl.flock` serializes the build; second worker re-checks and skips.                                                           |
+| Crashed build leaves stale `.tmp` file                                        | Build unlinks any pre-existing tmp before starting (safe under lock).                                                            |
+| `schema` shape change breaks `acheteur.py:303`                                | `schema` stays a `pl.Schema` object, not a list. One call site (`collect_schema()` → module `schema`) updated.                   |
+| Test runs inherit a stale DuckDB from a previous run with a different parquet | Tests force `REBUILD_DUCKDB=true` on cold runs; test DB added to `.gitignore`.                                                   |
+| Read-only connection opened before build finishes in another worker           | Lock held across build + rename; read-only `connect` happens after lock release. Atomic `os.replace` guarantees a complete file. |
+
+## Outcome
+
+### Memory impact
+
+Memory measurement against the production parquet (`decp_prod.parquet`, ~1.5M rows) requires a running gunicorn process with access to the production data file. The measurement was deferred to the post-merge smoke test on the staging server (test.decp.info).
+
+**Expected reduction:** The removed globals (`df`, `df_acheteurs_departement`, `df_titulaires_departement`, `df_acheteurs_marches`, `df_titulaires_marches`) previously materialised the full 1.5M-row Parquet in memory as multiple Polars frames. At ~300 bytes/row × 5 frames, steady-state RSS reduction is estimated at **1–2 GB per worker**. The retained `df_acheteurs` and `df_titulaires` (autocomplete search) represent only the distinct-organisation subset (~tens of thousands of rows) and are negligible.
+
+**What remains in memory:**
+
+- `df_acheteurs` — distinct acheteurs with Marchés count (populated from DuckDB at startup)
+- `df_titulaires` — same for titulaires
+- DuckDB's own page cache (disk-backed, grows under load, evicted by OS)
+
+All per-request data is fetched from DuckDB and discarded after the callback returns.

docs/superpowers/specs/2026-04-22-observatoire-duckdb-filters-design.md

@@ -0,0 +1,206 @@
+# Observatoire — filtrage natif DuckDB
+
+## Contexte
+
+La page `/observatoire` construit ses cartes, ses téléchargements et sa prévisualisation
+tabulaire à partir de la fonction `prepare_dashboard_data` (dans `src/utils/data.py`).
+Aujourd'hui, cette fonction prend une `pl.LazyFrame` — typiquement obtenue par
+`query_marches().lazy()` — et applique une série de filtres côté Polars.
+
+`query_marches()` matérialise l'intégralité de la table `decp` (~1,5 M lignes) en
+DataFrame Polars, même lorsqu'un utilisateur applique des filtres restrictifs. Les
+filtres sont ensuite appliqués sur cet ensemble déjà matérialisé.
+
+Le pattern utilisé par `_fetch_page_sql` (dans `src/utils/table.py`) montre comment
+déléguer le filtrage à DuckDB :
+
+1. Un traducteur (`filter_query_to_sql`, dans `src/utils/table_sql.py`) transforme le
+   DSL utilisateur en `(where_sql, params)`.
+2. `query_marches(where_sql=..., params=...)` ne matérialise que le sous-ensemble utile.
+
+Ce spec décrit comment appliquer ce même pattern aux filtres de l'observatoire.
+
+## Objectifs
+
+- Réduire la consommation mémoire et le temps de chaque callback de l'observatoire
+  en poussant le filtrage au niveau DuckDB.
+- Conserver strictement la sémantique des filtres actuels (pas de régression
+  fonctionnelle).
+- Garder une frontière claire : un helper pur `dashboard_filters_to_sql` qui ne
+  touche pas à la base, et une `prepare_dashboard_data` fine qui appelle DuckDB.
+
+## Non-objectifs
+
+- Pas de refonte de l'UI de filtres.
+- Pas d'optimisation ou de cache supplémentaire autour de
+  `_compute_dashboard_children` (déjà `@cache.memoize()`).
+- Pas de changement du comportement par défaut (365 derniers jours quand aucune
+  année n'est sélectionnée).
+
+## Architecture
+
+### Nouveau helper — `src/utils/table_sql.py`
+
+```python
+def dashboard_filters_to_sql(
+    dashboard_year=None,
+    dashboard_acheteur_id=None,
+    dashboard_acheteur_categorie=None,
+    dashboard_acheteur_departement_code=None,
+    dashboard_titulaire_id=None,
+    dashboard_titulaire_categorie=None,
+    dashboard_titulaire_departement_code=None,
+    dashboard_marche_type=None,
+    dashboard_marche_objet=None,
+    dashboard_marche_code_cpv=None,
+    dashboard_marche_considerations_sociales=None,
+    dashboard_marche_considerations_environnementales=None,
+    dashboard_marche_techniques=None,
+    dashboard_marche_innovant=None,
+    dashboard_marche_sous_traitance_declaree=None,
+    dashboard_montant_min=None,
+    dashboard_montant_max=None,
+) -> tuple[str, list]:
+    """Traduit les filtres du tableau de bord en (where_clause, params) DuckDB."""
+```
+
+Fonction pure, sans accès à la base. Même signature que `prepare_dashboard_data`
+actuelle (hors `lff`). Retourne `("TRUE", [])` si aucun filtre n'est actif.
+
+### Réécriture — `prepare_dashboard_data` (`src/utils/data.py`)
+
+```python
+def prepare_dashboard_data(**filter_params) -> pl.DataFrame:
+    where_sql, params = dashboard_filters_to_sql(**filter_params)
+    return query_marches(where_sql=where_sql, params=params)
+```
+
+- **Signature** : suppression du paramètre `lff`. Retour `pl.DataFrame` (et non plus
+  `pl.LazyFrame`).
+- Les appelants qui ont besoin d'une LazyFrame appellent `.lazy()` sur le résultat.
+
+### Appelants — `src/pages/observatoire.py`
+
+Trois sites d'appel à adapter :
+
+1. **`_compute_dashboard_children`** (ligne ~668) — on remplace
+
+   ```python
+   lff: pl.LazyFrame = query_marches().lazy()
+   lff = prepare_dashboard_data(lff=lff, **filter_params)
+   dff = lff.collect(engine="streaming")
+   ```
+
+   par
+
+   ```python
+   dff = prepare_dashboard_data(**filter_params)
+   lff = dff.lazy()
+   ```
+
+   Les appels existants à `make_donut`, `get_distance_histogram`, `get_top_org_table`,
+   `get_barchart_sources` continuent de recevoir `lff` ; `get_geographic_maps`
+   continue de recevoir `dff`. `df_per_uid` est calculé à partir de `dff`.
+
+2. **`download_observatoire`** (ligne ~791) —
+
+   ```python
+   dff = prepare_dashboard_data(**(filter_params or {}))
+   if hidden_columns:
+       dff = dff.drop(hidden_columns)
+   def to_bytes(buffer):
+       dff.write_excel(buffer, worksheet="DECP")
+   ```
+
+3. **`populate_preview_table`** (ligne ~882) —
+   ```python
+   dff = prepare_dashboard_data(**(filter_params or {}))
+   return prepare_table_data(
+       dff.lazy(),  # prepare_table_data accepte une LazyFrame
+       ...
+   )
+   ```
+
+## Traduction des filtres
+
+| Filtre                                                    | Actuel (Polars)                                            | Cible (SQL DuckDB)                                             |
+| --------------------------------------------------------- | ---------------------------------------------------------- | -------------------------------------------------------------- |
+| `dashboard_year` (présent)                                | `dt.year() == int(year)`                                   | `YEAR("dateNotification") = ?`                                 |
+| `dashboard_year` (absent) — comportement par défaut       | `> now - 365j`                                             | `"dateNotification" > ?` (datetime calculé à l'appel)          |
+| `dashboard_acheteur_id`                                   | `str.contains(val)`                                        | `"acheteur_id" LIKE ?` avec `%val%`                            |
+| `dashboard_acheteur_categorie`                            | `== val` (skip si acheteur_id présent)                     | `"acheteur_categorie" = ?`                                     |
+| `dashboard_acheteur_departement_code`                     | `is_in(list)` (skip si acheteur_id présent)                | `"acheteur_departement_code" IN (?, ?, ...)`                   |
+| `dashboard_titulaire_id`                                  | idem acheteur                                              | idem                                                           |
+| `dashboard_titulaire_categorie`                           | idem                                                       | idem                                                           |
+| `dashboard_titulaire_departement_code`                    | idem                                                       | idem                                                           |
+| `dashboard_marche_type`                                   | `== val`                                                   | `"type" = ?`                                                   |
+| `dashboard_marche_objet`                                  | `str.contains("(?i)val")`                                  | `"objet" ILIKE ?` avec `%val%`                                 |
+| `dashboard_marche_code_cpv`                               | `str.starts_with(val)`                                     | `"codeCPV" LIKE ?` avec `val%`                                 |
+| `dashboard_marche_techniques`                             | `str.split(", ").list.set_intersection(xs).list.len() > 0` | `list_has_any(string_split("techniques", ', '), ?::VARCHAR[])` |
+| `dashboard_marche_considerations_sociales`                | idem                                                       | idem sur `"considerationsSociales"`                            |
+| `dashboard_marche_considerations_environnementales`       | idem                                                       | idem sur `"considerationsEnvironnementales"`                   |
+| `dashboard_marche_innovant` (`"oui"`/`"non"`, sinon skip) | `== val`                                                   | `"marcheInnovant" = ?`                                         |
+| `dashboard_marche_sous_traitance_declaree`                | idem                                                       | `"sousTraitanceDeclaree" = ?`                                  |
+| `dashboard_montant_min`                                   | `>= val`                                                   | `"montant" >= ?`                                               |
+| `dashboard_montant_max`                                   | `<= val`                                                   | `"montant" <= ?`                                               |
+
+**Logique conditionnelle conservée** : si `dashboard_acheteur_id` est fourni, les filtres
+`categorie` et `departement_code` acheteur sont ignorés (même chose pour titulaire).
+
+**Traitement des valeurs spéciales** :
+
+- `dashboard_marche_innovant` / `dashboard_marche_sous_traitance_declaree` : valeur
+  `"all"` ou falsy → aucun filtre ajouté.
+- `dashboard_year` : converti en `int` avant injection.
+- `dashboard_montant_min` / `_max` : `None` → aucun filtre (distinct de `0`, qui reste
+  un filtre valide via `>=` ou `<=`).
+
+**Sécurité SQL** : toutes les valeurs utilisateurs passent par DuckDB en paramètres liés
+(`?`). Seuls des noms de colonnes statiques (contrôlés par le code) sont injectés dans le
+fragment SQL via `f"..."`. Pas de différence avec le pattern existant de
+`filter_query_to_sql`.
+
+## Tests
+
+### Unitaires (nouveaux)
+
+Nouveau fichier `tests/test_dashboard_filters_to_sql.py` :
+
+- Cas vide → `("TRUE", [])`.
+- Un seul filtre simple (année, type, etc.) → fragment SQL et params attendus.
+- Filtre montant min/max (migration de l'actuel `test_010_observatoire_montant_filter`).
+- Filtre liste (techniques, considerationsSociales) → usage de `list_has_any`.
+- Filtre acheteur_id fourni → catégorie/département acheteur ignorés.
+- Filtre `"all"` / `None` sur innovant/sous_traitance → aucun fragment ajouté.
+- Comportement par défaut sans année → fragment `"dateNotification" > ?` avec un param
+  datetime à ~365 j dans le passé (tolérance de quelques secondes).
+
+### Intégration (nouveau, léger)
+
+Un test qui appelle `prepare_dashboard_data` contre `tests/test.parquet` avec un ou
+deux filtres connus, vérifie le `height` et la bonne nature du retour (`pl.DataFrame`).
+
+### Test Selenium existant
+
+`test_009_observatoire_filter_persistence` et `test_008_observatoire_navigation_from_search`
+ne touchent pas à la signature ; ils doivent continuer à passer.
+
+## Risques et migration
+
+- **Risque sémantique** : la fonction Polars `str.contains` utilisée pour les IDs est
+  un regex. Les utilisateurs attendent probablement un contains littéral sur un SIRET
+  (14 chiffres). Le passage à `LIKE '%val%'` est neutre si la valeur ne contient pas de
+  caractère spécial regex — ce qui est le cas pour des SIRET. **Hypothèse** acceptée :
+  le contenu `dashboard_acheteur_id`/`dashboard_titulaire_id` est alphanumérique.
+- **Risque de drift du cache** : la date "365 derniers jours" n'est pas incluse dans
+  la clé de cache de `_compute_dashboard_children`. C'est un comportement pré-existant
+  ; non traité par ce spec.
+- **Import circulaire** : `src/utils/data.py` importe déjà depuis `src/db.py`.
+  `src/utils/table_sql.py` importe depuis `src/utils/table.py`. Pas de nouveau cycle.
+
+## Succès
+
+- Les 3 callbacks de l'observatoire restent fonctionnellement équivalents.
+- Les tests unitaires et d'intégration passent.
+- Une inspection manuelle confirme un temps d'exécution réduit sur un filtre
+  sélectif (par ex. un département + une année).

pyproject.toml

@@ -1,25 +1,47 @@
 [project]
 name = "decp.info"
 description = "Interface d'exploration et d'analyse des marchés publics français."
-version = "2.1.2"
+version = "2.7.7"
 requires-python = ">= 3.10"
-authors = [
+authors = [{ name = "Colin Maudry", email = "colin@colmo.tech" }]
-    { name = "Colin Maudry", email = "colin+decp@maudry.com" }
-]
 dependencies = [
-    "dash==3.2.0",
+  "dash==3.4.0",
-    "dash[compress]",
+  "dash[compress]",
-    "polars",
+  "polars",
-    "gunicorn",
+  "gunicorn",
-    "dash-bootstrap-components",
+  "dash-bootstrap-components",
-    "python-dotenv",
+  "python-dotenv",
-    "xlsxwriter",
+  "xlsxwriter",
-    "plotly[express]",
+  "plotly[express]",
-    "httpx",
+  "httpx",
-    "pandas" # utilisé pour la création de certains graphiques
+  "pandas",                    # utilisé pour la création de certains graphiques
+  "unidecode",
+  "dash-leaflet",
+  "dash-extensions",
+  "duckdb",
+  "flask-caching",
+  "pyarrow>=23.0.1",
+  "flask-cors>=6.0.2",
 ]
 
-[project.optional-dependencies]
+[dependency-groups]
 dev = [
-    "pre-commit",
+  "pytest",
+  "pytest-env",
+  "pre-commit",
+  "selenium",
+  "webdriver-manager",
+  "dash[testing]",
+  "fastexcel",
 ]
+
+[tool.pytest.ini_options]
+pythonpath = ["src"]
+testpaths = ["tests"]
+env = [
+  "DATA_FILE_PARQUET_PATH=tests/test.parquet",
+  "DEVELOPMENT=true",
+  "REBUILD_DUCKDB=true",
+  "DATA_SCHEMA_PATH=/home/colin/git/decp-processing/dist/schema.json",
+]
+addopts = "-p no:warnings"

run.py

@@ -1,7 +1,10 @@
+from flask_cors import CORS
+
 from src.app import app
 
 # To use `gunicorn run:server` (prod)
 server = app.server
+CORS(server)
 
 # To use `python run.py` (dev)
 if __name__ == "__main__":

src/app.py

@@ -1,37 +1,84 @@
-import logging
+import os
+from shutil import rmtree
 
 import dash_bootstrap_components as dbc
+import pandas  # noqa: F401  # eager import: avoid plotly's lazy-import race across Dash callback threads
 import tomllib
-from dash import Dash, dcc, html, page_container, page_registry
+from dash import Dash, Input, Output, State, dcc, html, page_container, page_registry
 from dotenv import load_dotenv
-from flask import send_from_directory
+from flask import Response
+
+from src.utils import DEVELOPMENT
+from src.utils.cache import cache
 
 load_dotenv()
 
-app = Dash(
+# if os.getenv("PYTEST_CURRENT_TEST"):
-    external_stylesheets=[dbc.themes.SIMPLEX],
+#     os.environ["DATA_FILE_PARQUET_PATH"]
+
+META_TAGS = [
+    {"name": "viewport", "content": "width=device-width, initial-scale=1"},
+    {
+        "name": "keywords",
+        "content": "commande publique, decp, marchés publics, données essentielles",
+    },
+]
+
+if DEVELOPMENT:
+    META_TAGS.append({"name": "robots", "content": "noindex"})
+
+app: Dash = Dash(
     title="decp.info",
     use_pages=True,
     compress=True,
+    meta_tags=META_TAGS,
+)
+
+cache_dir = os.getenv("CACHE_DIR", "/tmp/decp-cache")
+
+if os.path.exists(cache_dir):
+    rmtree(cache_dir)
+
+cache.init_app(
+    app.server,
+    config={
+        "CACHE_TYPE": "FileSystemCache",
+        "CACHE_DIR": cache_dir,
+        "CACHE_DEFAULT_TIMEOUT": int(
+            os.getenv("CACHE_DEFAULT_TIMEOUT", 3600 * 24)
+        ),  # 24h par défaut
+        "CACHE_THRESHOLD": 300,
+    },
 )
-# COSMO (belle font, blue),
-# UNITED (rouge, ubuntu font),
-# LUMEN (gros séparateur, blue clair),
-# SIMPLEX (rouge, séparateur)
 
 
 # robots.txt
 @app.server.route("/robots.txt")
 def robots():
-    return send_from_directory("./assets", "robots.txt", mimetype="text/plain")
+    text = """User-agent: *
+Allow: /
+    """
+    return Response(text, mimetype="text/plain")
 
 
-logger = logging.getLogger("decp.info")
+@app.server.route("/sitemap.xml")
-logging.basicConfig(
+def sitemap():
-    format="%(asctime)s %(levelname)-8s %(message)s",
+    base_url = "https://decp.info"
-    level=logging.INFO,
+    pages = [
-    datefmt="%Y-%m-%d %H:%M:%S",
+        "/",
-)
+        "/observatoire",
+        "/tableau",
+        "/a-propos",
+    ]
+    xml = '<?xml version="1.0" encoding="UTF-8"?>\n'
+    xml += '<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">\n'
+    for page in pages:
+        xml += "  <url>\n"
+        xml += f"    <loc>{base_url}{page}</loc>\n"
+        xml += "  </url>\n"
+    xml += "</urlset>"
+    return Response(xml, mimetype="text/xml")
+
 
 with open("./pyproject.toml", "rb") as f:
     pyproject = tomllib.load(f)
@@ -46,6 +93,7 @@ app.index_string = """
         <title>{%title%}</title>
         {%favicon%}
         {%css%}
+        <!-- canonical link -->
     </head>
     <body>
         {%app_entry%}
@@ -72,39 +120,93 @@ app.index_string = """
 </html>
 """
 
-app.layout = html.Div(
+navbar = dbc.Navbar(
-    [
+    dbc.Container(
-        html.Div(
+        fluid=True,
-            [
+        children=[
-                html.Div(
+            dbc.NavItem(
-                    [
+                children=[
-                        html.A(children=html.H1("decp.info"), href="/"),
+                    html.Div(
-                        html.P(
+                        [
-                            children=html.A(
+                            dcc.Link(html.H1("decp.info"), href="/", className="logo"),
-                                version,
+                            html.P(
-                                href="https://github.com/ColinMaudry/decp.info?tab=readme-ov-file#notes-de-version",
+                                [
-                                target="_blank",
+                                    html.A(
+                                        version,
+                                        href="https://github.com/ColinMaudry/decp.info/blob/main/CHANGELOG.md",
+                                    )
+                                ],
+                                className="version",
                             ),
-                            className="version",
+                        ],
-                        ),
+                        className="logo-wrapper",
-                    ],
+                    )
-                    className="logo",
+                ],
-                ),
+                style={"minWidth": "230px"},
-                html.Div(
+            ),
+            dbc.Nav(
+                children=[
+                    dcc.Markdown(
+                        os.getenv("ANNOUNCEMENTS"),
+                        id="announcements",
+                        dangerously_allow_html=True,
+                    ),
+                ],
+                style={
+                    "maxWidth": "1200px",
+                    "display": "inline-block",
+                },
+                navbar=True,
+                id="announcements-nav",
+            ),
+            dbc.NavbarToggler(id="navbar-toggler"),
+            dbc.Collapse(
+                dbc.Nav(
                     [
-                        dcc.Link(
+                        dbc.NavItem(
-                            page["name"], href=page["relative_path"], className="nav"
+                            dbc.NavLink(
+                                page["name"].replace(" ", " "),
+                                href=page["relative_path"],
+                                active="exact",
+                            )
                         )
                         for page in page_registry.values()
-                        if page["name"] not in ["Acheteur", "Titulaire", "Marché"]
+                        if page["name"]
-                    ]
+                        in ["Recherche", "À propos", "Tableau", "Observatoire"]
+                    ],
+                    className="ms-auto",
+                    navbar=True,
                 ),
-            ],
+                id="navbar-collapse",
-            className="navbar",
+                navbar=True,
+            ),
+        ],
+    ),
+    color="light",
+    dark=False,
+    className="mb-4",
+    expand="lg",
+)
+
+app.layout = html.Div(
+    [
+        navbar,
+        dbc.Container(
+            page_container,
+            fluid=True,
+            id="page-content-container",
+            className="mb-4",
         ),
-        page_container,
     ]
 )
 
-if __name__ == "__main__":
+
-    app.run(debug=True)
+@app.callback(
+    Output("navbar-collapse", "is_open"),
+    [Input("navbar-toggler", "n_clicks")],
+    [State("navbar-collapse", "is_open")],
+)
+def toggle_navbar_collapse(n, is_open):
+    if n:
+        return not is_open
+    return is_open

src/assets/copy.svg

@@ -0,0 +1 @@
+<svg aria-hidden="true" focusable="false" data-prefix="far" data-icon="copy" class="svg-inline--fa fa-copy fa-w-14 " role="img" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><path fill="currentColor" d="M433.941 65.941l-51.882-51.882A48 48 0 0 0 348.118 0H176c-26.51 0-48 21.49-48 48v48H48c-26.51 0-48 21.49-48 48v320c0 26.51 21.49 48 48 48h224c26.51 0 48-21.49 48-48v-48h80c26.51 0 48-21.49 48-48V99.882a48 48 0 0 0-14.059-33.941zM266 464H54a6 6 0 0 1-6-6V150a6 6 0 0 1 6-6h74v224c0 26.51 21.49 48 48 48h96v42a6 6 0 0 1-6 6zm128-96H182a6 6 0 0 1-6-6V54a6 6 0 0 1 6-6h106v88c0 13.255 10.745 24 24 24h88v202a6 6 0 0 1-6 6zm6-256h-64V48h9.632c1.591 0 3.117.632 4.243 1.757l48.368 48.368a6 6 0 0 1 1.757 4.243V112z"></path></svg>

src/assets/css/style.css

@@ -0,0 +1,548 @@
+@import url(https://fonts.bunny.net/css?family=fira-code:400|inter:400,600);
+
+/* ==========================================================================
+   Variables
+   ========================================================================== */
+:root {
+  --bs-font-monospace: "Fira Code";
+  --primary-color: rgb(179, 56, 33);
+  --primary-color-text: #b33821;
+}
+
+/* ==========================================================================
+   Base & Reset
+   ========================================================================== */
+body {
+  font-family: "Inter", sans-serif;
+  font-weight: 400;
+  background-color: rgb(255 240 240 / 40%);
+  -moz-osx-font-smoothing: grayscale;
+  -webkit-font-smoothing: antialiased;
+  font-smooth: always;
+  font-display: swap;
+}
+
+strong,
+b {
+  font-weight: 600 !important;
+}
+
+h1,
+h2,
+h3,
+h4,
+h5 {
+  font-weight: 600;
+}
+
+h3 {
+  margin: 36px 0 20px 0;
+}
+
+/* Base Button Styles
+button {
+  font-weight: 400;
+  background-color: #fff;
+  border-radius: 3px;
+  appearance: auto;
+  border: solid var(--primary-color) 1px;
+}  */
+
+button.btn.btn-primary,
+button.show-hide {
+  display: block;
+  border-radius: 3px;
+  outline: 0;
+  color: #fff;
+  border: 0;
+  height: 30px;
+  padding-top: 2px;
+  background-image: linear-gradient(
+    rgb(209, 96, 73),
+    rgb(179, 56, 33) 26%,
+    rgb(159, 36, 22)
+  );
+}
+
+button.btn.btn-primary:hover,
+button.show-hide:hover {
+  background-image: linear-gradient(
+    rgb(239, 126, 103),
+    rgb(209, 86, 63) 26%,
+    rgb(189, 66, 52)
+  );
+}
+
+button[disabled] {
+  border-color: #ccc;
+  color: #666;
+}
+
+button:hover:not([disabled]) {
+  background-color: #fee;
+}
+
+/* Global Link Styles */
+#_pages_content a {
+  color: #993333;
+}
+
+/* ==========================================================================
+   Layout
+   ========================================================================== */
+#_pages_content {
+  padding: 28px 24px 0 24px;
+}
+
+#header > * {
+  margin: 0 0 20px 0px;
+}
+
+/* ==========================================================================
+   Components
+   ========================================================================== */
+
+/* --- Navigation & Header --- */
+a.logo {
+  color: black;
+  text-decoration: none;
+}
+
+a.logo > h1 {
+  font-weight: 400;
+  margin: 0;
+  line-height: 1;
+}
+
+.logo-wrapper {
+  display: flex;
+  align-items: baseline;
+}
+
+p.version {
+  margin: 0 0 0 12px;
+  font-family: "Fira Code";
+  font-size: 0.9rem;
+}
+
+p.version > a {
+  text-decoration: none;
+  margin-top: 10px;
+}
+
+.navbar-brand {
+  margin-right: 2px;
+}
+
+.navbar-nav .nav-link.active {
+  font-weight: 600;
+}
+
+#announcements {
+  margin: 25px 40px 0 60px;
+  font-size: 90%;
+  max-width: 900px;
+}
+
+#announcements p {
+  margin-bottom: 0.2rem;
+}
+
+.seeBorder {
+  border: dotted 1px green;
+}
+
+/* --- Search Page --- */
+.tagline {
+  text-align: center;
+  font-size: 120%;
+  display: block;
+  margin-top: 50px;
+}
+
+#search {
+  margin: 30px auto 0px auto;
+  width: 500px;
+  font-size: 16px;
+  height: 30px;
+  display: block;
+}
+
+.search_options {
+  margin: 16px auto;
+  width: 450px;
+}
+
+.search_options input {
+  margin-right: 12px;
+}
+
+/* --- Dashboard inputs --- */
+
+.Select--multi .Select-value {
+  color: var(--primary-color) !important;
+  background-color: rgba(255, 240, 240, 0.4) !important;
+}
+
+#filters .row > * {
+  margin-bottom: 6px;
+}
+
+#filters input[type="text"],
+#filters input[type="number"] {
+  border: 1px #ccc solid;
+  border-radius: 3px;
+  padding-left: 8px;
+}
+
+/* --- Tables (Dash & Custom) --- */
+
+/* Table Menu (Exports etc) */
+.table-menu {
+  font-size: 16px;
+  margin: 12px 0 12px 0;
+  display: flex;
+  align-items: center;
+  flex-wrap: wrap;
+}
+
+.table-menu > * {
+  margin: 8px 16px 8px 0;
+}
+
+#source_table {
+  margin-bottom: 25px;
+}
+
+#source_table p {
+  line-height: 1.5;
+}
+
+/* Dash Table Overrides */
+.column-header--sort {
+  margin-left: 3px;
+}
+
+dash-table-container dash-spreadsheet-menu table.cell-table {
+  margin-right: 8px;
+  margin-lef: 8px;
+}
+
+table.cell-table,
+table.cell-table tr {
+  border-color: #fff;
+  padding: 0;
+  border-collapse: separate !important;
+  /* Required for border-radius */
+  border-spacing: 0;
+}
+
+table.cell-table th {
+  border-collapse: separate !important;
+  border-spacing: 0;
+}
+
+.dash-table-container p {
+  margin-bottom: 0;
+}
+
+.dash-table-container
+  .dash-spreadsheet-container
+  .dash-spreadsheet-inner
+  th.dash-header,
+.dash-table-container
+  .dash-spreadsheet-container
+  .dash-spreadsheet-inner
+  th.dash-select-header {
+  margin: 0;
+  color: white;
+  font-family: "Inter", sans-serif;
+  text-align: left;
+  font-weight: 600;
+  padding: 2px 12px 4px 2px;
+  border: 1px solid rgb(179, 56, 33) !important;
+  background-color: rgb(179, 56, 33);
+  border-bottom: none !important;
+  height: 32px;
+}
+
+.dash-table-container
+  .dash-spreadsheet-container
+  .dash-spreadsheet-inner
+  table.cell-table
+  tr:first-of-type
+  th.dash-header:first-of-type,
+.dash-table-container
+  .dash-spreadsheet-container
+  .dash-spreadsheet-inner
+  table.cell-table
+  tr:first-of-type
+  th.dash-select-header:first-of-type {
+  border-top-left-radius: 3px !important;
+}
+
+.dash-table-container
+  .dash-spreadsheet-container
+  .dash-spreadsheet-inner
+  table.cell-table
+  tr:first-of-type
+  th.dash-header:last-of-type {
+  border-top-right-radius: 3px !important;
+}
+
+/* Dash Filters */
+.dash-table-container
+  .dash-spreadsheet-container
+  .dash-spreadsheet-inner
+  .cell-table
+  .dash-filter
+  input[type="text"] {
+  border-color: #ccc;
+  border-style: solid;
+  border-width: 1px;
+  border-radius: 3px;
+  height: 28px;
+  font-family: "Fira Code";
+  caret-color: #000;
+  background-color: rgb(250 250 250);
+  text-align: left !important;
+  padding: 1px 2px 0 2px;
+  vertical-align: center;
+}
+
+.dash-table-container
+  .dash-spreadsheet-container
+  .dash-spreadsheet-inner
+  .cell-table
+  .dash-filter
+  input[type="text"]::placeholder {
+  color: #999;
+}
+
+.dash-filter--case {
+  display: none;
+}
+
+.dash-table-container
+  .dash-spreadsheet-container
+  .dash-spreadsheet-inner
+  .cell-table
+  th.dash-filter {
+  background-color: #ccc;
+}
+
+/* Custom Marches Table */
+.dash-table-container
+  .dash-spreadsheet-container
+  .dash-spreadsheet-inner
+  .cell-table
+  td {
+  padding-left: 5px;
+  padding-right: 5px;
+}
+
+.dash-table-container
+  .dash-spreadsheet-container
+  .dash-spreadsheet-inner
+  td
+  div.dash-cell-value.cell-markdown {
+  font-family: "Inter", sans-serif !important;
+  font-weight: 400;
+}
+
+.marches_table.stuck {
+  position: relative;
+  right: 200px;
+}
+
+.marches_table .cell-table tr:nth-child(even) td {
+  background-color: rgb(255 240 240 / 40%);
+}
+
+/* Column Visibility Menu */
+.column-actions {
+  margin-right: 8px;
+}
+
+.column-header--hide {
+  display: none;
+}
+
+button.show-hide {
+  position: relative;
+  width: 180px;
+  margin: 0 0 10px 0;
+  display: none;
+}
+
+/*
+.show-hide::before {
+  background: inherit;
+  content: "Colonnes affichées";
+  position: absolute;
+  left: 5px;
+  right: 5px;
+
+
+#column_list .show-hide,
+#table .show-hide {
+  display: none;
+}
+
+.show-hide-menu-item > input {
+  margin-right: 10px;
+} */
+
+#btn-copy-url:before {
+}
+
+/* Dropdowns */
+.Select-placeholder {
+  color: #333 !important;
+}
+
+/* Checkboxes */
+
+input[type="checkbox"] {
+  height: 17px;
+  width: 17px;
+}
+
+/* Tooltips */
+.dash-tooltip,
+.dash-table-tooltip {
+  color: #333;
+  width: 400px !important;
+  max-width: 400px !important;
+  height: 150px !important;
+  max-height: 150px !important;
+  overflow: hidden;
+}
+
+.dash-tooltip pre,
+.dash-tooltip code {
+  overflow: hidden;
+  height: 150px;
+  text-wrap: wrap;
+  font-family: "Inter", sans-serif;
+}
+
+/* --- Organization Cards (Grid Items) --- */
+
+#cards .card {
+  margin-bottom: 16px;
+}
+
+.org_infos > p {
+  margin: 8px 0;
+}
+
+/* --- About Page (A Propos) --- */
+.a-propos-container {
+  display: flex;
+  flex-wrap: wrap;
+  align-items: flex-start;
+  position: relative;
+  max-width: 1200px;
+  margin: 0 auto;
+}
+
+.a-propos-content {
+  flex: 1 1 70%;
+  max-width: 75%;
+  padding-right: 40px;
+}
+
+.a-propos-toc {
+  flex: 0 0 25%;
+  max-width: 25%;
+  /* Keeps it from growing too large */
+  position: sticky;
+  top: 40px;
+  /* Sticks 40px from the top of the viewport */
+  border-left: 2px solid #333;
+  /* Dark vertical line like hedgedoc */
+  padding-left: 15px;
+  margin-top: 40px;
+  background-color: #fff;
+  /* Aligns visually with the first header */
+}
+
+/* TOC Links */
+.toc-link {
+  display: block;
+  color: #666;
+  text-decoration: none;
+  font-size: 0.9em;
+  padding: 2px 0;
+  transition: color 0.2s, font-weight 0.2s;
+  line-height: 1.4;
+}
+
+.toc-link:hover {
+  color: #000;
+  text-decoration: none;
+}
+
+.toc-active {
+  color: #000;
+  font-weight: bold;
+}
+
+.toc-level-2 {
+  margin-left: 15px;
+  font-size: 0.85em;
+}
+
+.toc-header {
+  font-weight: bold;
+  margin-bottom: 10px;
+  display: block;
+  color: #333;
+}
+
+/* --- Misc & Utility --- */
+#instructions {
+  max-width: 1000px;
+}
+
+details > div {
+  padding-top: 24px;
+}
+
+summary > h4 {
+  margin: 0;
+  display: inline;
+}
+
+/* ==========================================================================
+   Media Queries
+   ========================================================================== */
+
+@media (max-width: 992px) {
+  /* Navigation */
+  #announcements-nav {
+    display: none !important;
+  }
+
+  /* About Page */
+  .a-propos-content {
+    max-width: 100%;
+    padding-right: 0;
+    flex: 1 1 100%;
+  }
+
+  .a-propos-toc {
+    display: none;
+  }
+}
+
+input[type="number"]::-webkit-outer-spin-button,
+input[type="number"]::-webkit-inner-spin-button {
+  -webkit-appearance: none;
+  margin: 0;
+}
+
+input[type="number"] {
+  -moz-appearance: textfield;
+}

src/assets/dash_clientside.js

@@ -0,0 +1,132 @@
+window.dash_clientside = Object.assign({}, window.dash_clientside, {
+  leaflet: {
+    pointToLayer: function (feature, latlng, context) {
+      return L.circleMarker(latlng, {
+        radius: 5,
+        fillColor: feature.properties.marker_color,
+        color: "white",
+        weight: 1,
+        opacity: 1,
+        fillOpacity: 0.8,
+      }).bindTooltip(feature.properties.tooltip);
+    },
+    clusterToLayer: function (feature, latlng, index, context) {
+      console.log(feature);
+      console.log(index);
+      console.log(context);
+
+      const count = feature.properties.point_count;
+      const size = count < 100 ? 30 : count < 1000 ? 40 : 50;
+      const color = "#555"; // Default cluster color
+      const icon = L.divIcon({
+        html: `<div style="background-color: ${context.fillColor}; width: ${size}px; height: ${size}px; border-radius: 50%; display: flex; align-items:center; justify-content:center; color: white; border: 2px solid white; font-weight: bold;">${count}</div>`,
+        className: "marker-cluster",
+        iconSize: L.point(size, size),
+      });
+      return L.marker(latlng, { icon: icon });
+    },
+  },
+  clientside: {
+    clean_filters: function (trigger) {
+      if (!trigger) {
+        return window.dash_clientside.no_update;
+      }
+
+      // Helper to set value on a React text input
+      const setNativeValue = (element, value) => {
+        const valueSetter = Object.getOwnPropertyDescriptor(
+          element,
+          "value"
+        ).set;
+        const prototype = Object.getPrototypeOf(element);
+        const prototypeValueSetter = Object.getOwnPropertyDescriptor(
+          prototype,
+          "value"
+        ).set;
+
+        if (valueSetter && valueSetter !== prototypeValueSetter) {
+          prototypeValueSetter.call(element, value);
+        } else {
+          valueSetter.call(element, value);
+        }
+
+        element.dispatchEvent(new Event("input", { bubbles: true }));
+      };
+
+      const cleanInputs = () => {
+        const inputs = document.querySelectorAll(
+          '.dash-filter input[type="text"]'
+        );
+        inputs.forEach((input) => {
+          let val = input.value;
+          let original = val;
+
+          // Remove "icontains " prefix
+          if (/^icontains\s+/i.test(val)) {
+            val = val.replace(/^icontains\s+/i, "");
+            // Check for surrounding quotes (single or double) and remove them
+            if (
+              (val.startsWith('"') && val.endsWith('"')) ||
+              (val.startsWith("'") && val.endsWith("'"))
+            ) {
+              val = val.substring(1, val.length - 1);
+            }
+          }
+          // Handle relational operators (i<, s>, i<=, etc.)
+          else if (/^[is][<>]=?/i.test(val)) {
+            val = val.substring(1);
+          }
+
+          if (val !== original) {
+            try {
+              // Try setting it the React-friendly way
+              setNativeValue(input, val);
+            } catch (e) {
+              // Fallback to direct assignment if fancy way fails
+              input.value = val;
+            }
+          }
+        });
+      };
+
+      // Use MutationObserver to wait for table to appear/update
+      const observer = new MutationObserver((mutations) => {
+        cleanInputs();
+      });
+
+      const target = document.querySelector(".dash-table-container");
+      if (target) {
+        observer.observe(target, {
+          childList: true,
+          subtree: true,
+          attributes: true,
+          attributeFilter: ["value"],
+        });
+
+        // Disconnect after 5 seconds
+        setTimeout(() => {
+          observer.disconnect();
+        }, 5000);
+
+        // Also try immediately just in case
+        cleanInputs();
+      } else {
+        // Poll briefly if container not found yet
+        const checkInterval = setInterval(() => {
+          const t = document.querySelector(".dash-table-container");
+          if (t) {
+            clearInterval(checkInterval);
+            observer.observe(t, { childList: true, subtree: true });
+            setTimeout(() => observer.disconnect(), 5000);
+            cleanInputs();
+          }
+        }, 200);
+
+        // Stop polling after 2s if still nothing
+        setTimeout(() => clearInterval(checkInterval), 2000);
+      }
+
+      return window.dash_clientside.no_update;
+    },
+  },
+});

src/assets/robots.txt

@@ -1,41 +0,0 @@
-# START YOAST BLOCK
-# Copié depuis https://next.ink/robots.txt
-# ---------------------------
-User-agent: *
-Allow: /
-
-# ---------------------------
-# END YOAST BLOCK
-
-User-agent: GPTBot
-Disallow: /
-User-agent: ChatGPT-User
-Disallow: /
-User-agent: Google-Extended
-Disallow: /
-User-agent: PerplexityBot
-Disallow: /
-User-agent: Amazonbot
-Disallow: /
-User-agent: ClaudeBot
-Disallow: /
-User-Agent: FacebookBot
-Disallow: /
-User-Agent: Applebot
-Disallow: /
-User-agent: anthropic-ai
-Disallow: /
-User-agent: Bytespider
-Disallow: /
-User-agent: Claude-Web
-Disallow: /
-User-agent: Diffbot
-Disallow: /
-User-agent: ImagesiftBot
-Disallow: /
-User-agent: Omgilibot
-Disallow: /
-User-agent: Omgili
-Disallow: /
-User-agent: YouBot
-Disallow: /

src/assets/style.css

@@ -1,240 +0,0 @@
-/* Change la marge bout d'export  */
-.table-menu {
-  font-size: 16px;
-  margin: 12px;
-  height: 36px;
-}
-
-.table-menu > * {
-  margin: 8px;
-  float: left;
-}
-
-#source_table p {
-  line-height: 1.5;
-}
-
-#source_table {
-  margin-bottom: 25px;
-}
-
-#instructions {
-  max-width: 1000px;
-}
-
-details > div {
-  padding-top: 24px;
-}
-
-/* Logo et version */
-
-div.logo {
-  width: 230px;
-}
-
-p.version {
-  float: left;
-  margin-top: 21px;
-  margin-left: 12px;
-}
-
-p.version > a {
-  text-decoration: none;
-}
-
-div.logo h1 {
-  float: left;
-}
-
-div.logo > a {
-  text-decoration: none;
-  color: black;
-}
-
-/* Réduire la taille du texte de la colonne Objet */
-
-td[data-dash-column="objet"] {
-  font-size: 85%;
-}
-
-/* Couleur des en-têtes */
-.dash-table-container
-  .dash-spreadsheet-container
-  .dash-spreadsheet-inner
-  th.dash-header {
-  background-color: #b33821;
-  color: white;
-  font-family: sans-serif;
-  text-align: left;
-}
-
-.dash-table-container
-  .dash-spreadsheet-container
-  .dash-spreadsheet-inner
-  th.dash-filter {
-  background-color: #f0afa3;
-}
-
-.dash-table-container p {
-  margin-bottom: 0;
-}
-
-.dash-filter--case {
-  display: none;
-}
-
-.dash-tooltip,
-.dash-table-tooltip {
-  color: #333;
-  width: 400px !important;
-  max-width: 400px !important;
-  height: 150px !important;
-  max-height: 150px !important;
-  overflow: hidden;
-}
-
-.dash-tooltip pre,
-.dash-tooltip code {
-  overflow: hidden;
-  height: 150px;
-  text-wrap: wrap;
-  font-family: sans-serif;
-}
-
-/* Menu de masquage des colonnes */
-.column-actions {
-  margin-right: 8px;
-}
-
-.column-header--hide {
-  display: none;
-}
-
-.show-hide {
-  position: relative;
-  width: 180px;
-  margin: 0 0 10px 10px;
-}
-
-.show-hide::before {
-  background: inherit;
-  content: "Colonnes affichées";
-  position: absolute;
-  left: 5px;
-  right: 5px;
-}
-
-.show-hide-menu-item > input {
-  margin-right: 10px;
-}
-
-/* Alternance des couleurs pour les lignes */
-.marches_table {
-  font-family: sans-serif;
-}
-.marches_table .cell-table tr:nth-child(even) td {
-  background-color: #feeeee;
-  font-family: sans-serif;
-}
-
-#header > *,
-.dash-spreadsheet-menu button.export {
-  margin: 0 0 20px 20px;
-}
-
-/* Menu de navigation */
-
-.navbar {
-  width: 100%;
-  height: 100px;
-  padding: 0 10px;
-  border: 0;
-  border-bottom: solid 1px black;
-}
-
-a.nav {
-  float: left;
-  margin-right: 40px;
-  font-size: 120%;
-}
-
-h3 {
-  margin: 36px 0 24px 0;
-}
-
-summary > h3 {
-  margin: 0;
-  display: inline;
-}
-
-#_pages_content {
-  padding-top: 28px;
-}
-
-/* Vue acheteur/titulaire    */
-.wrapper {
-  display: grid;
-  grid-gap: 10px;
-  margin-bottom: 50px;
-  justify-content: space-between;
-}
-
-.wrapper > div {
-}
-
-.org_title {
-  grid-column: 1 / 3;
-  grid-row: 1;
-}
-
-.org_year {
-  grid-column: 3;
-  grid-row: 1;
-}
-
-.org_infos {
-  grid-column: 1;
-  grid-row: 2;
-}
-
-.org_infos > p {
-  margin: 8px 0;
-}
-
-.org_stats {
-  grid-column: 2;
-  grid-row: 2;
-}
-
-.org_map {
-  grid-column: 3;
-  grid-row: 2;
-}
-
-/* Vue marché */
-
-.marche_infos p {
-  margin-top: 6px;
-  margin-bottom: 6px;
-}
-
-.marche_title {
-  grid-column: 1/3;
-  grid-row: 1;
-}
-
-.marche_map {
-  grid-column: 3;
-  grid-row: 2;
-  width: 400px;
-}
-
-.marche_infos_1 {
-  grid-column: 1;
-  grid-row: 2;
-}
-
-.marche_infos_2 {
-  grid-column: 2;
-  grid-row: 2;
-}

src/db.py

@@ -0,0 +1,177 @@
+import fcntl
+import os
+from pathlib import Path
+from time import sleep
+
+import duckdb
+import polars as pl
+import polars.selectors as cs
+from polars.exceptions import ComputeError
+
+from src.utils import logger
+
+
+def should_rebuild(db_path: Path, parquet_path: Path) -> bool:
+    db_path = Path(db_path)
+    parquet_path = Path(parquet_path)
+    if not db_path.exists():
+        return True
+    dev = os.getenv("DEVELOPMENT", "False").lower() == "true"
+    force = os.getenv("REBUILD_DUCKDB", "False").lower() == "true"
+    if dev and not force:
+        return False
+    return parquet_path.stat().st_mtime > db_path.stat().st_mtime
+
+
+def _load_source_frame(parquet_path: Path) -> pl.DataFrame:
+    """Read the source parquet and apply the row-level transforms.
+
+    Kept here (not in utils.py) so src.db has no dependency on utils.
+    Mirrors the behavior previously in utils.get_decp_data().
+    """
+    try:
+        lff: pl.LazyFrame = pl.scan_parquet(str(parquet_path))
+    except ComputeError:
+        logger.info("Lecture du parquet échouée, nouvelle tentative dans 10s...")
+        sleep(10)
+        lff = pl.scan_parquet(str(parquet_path))
+
+    lff = lff.sort(by=["dateNotification", "uid"], descending=True, nulls_last=True)
+    lff = lff.filter(pl.col("donneesActuelles")).drop("donneesActuelles")
+
+    # booleans_to_strings: true → "oui", false → "non"
+    lff = lff.with_columns(
+        pl.col(cs.Boolean)
+        .cast(pl.String)
+        .str.replace("true", "oui")
+        .str.replace("false", "non")
+    )
+
+    for col in ["acheteur_nom", "titulaire_nom"]:
+        lff = lff.with_columns(
+            pl.when(pl.col(col).is_null())
+            .then(pl.lit("[Identifiant non reconnu dans la base INSEE]"))
+            .otherwise(pl.col(col))
+            .name.keep()
+        )
+
+    return lff.collect()
+
+
+def build_database(db_path: Path, parquet_path: Path) -> None:
+    """Build the DuckDB database atomically under an exclusive lock.
+
+    Caller MUST hold the fcntl.flock on the .lock file.
+    """
+    db_path = Path(db_path)
+    parquet_path = Path(parquet_path)
+    tmp_path = db_path.with_suffix(".duckdb.tmp")
+    staging_parquet = db_path.with_suffix(".staging.parquet")
+    if tmp_path.exists():
+        tmp_path.unlink()
+
+    logger.info(f"Construction de la base DuckDB à partir de {parquet_path}...")
+    frame = _load_source_frame(parquet_path)
+
+    # Write transformed frame as parquet so DuckDB can read it natively
+    # (avoids pyarrow dependency for the Polars→DuckDB handoff)
+    frame.write_parquet(str(staging_parquet))
+    try:
+        with duckdb.connect(str(tmp_path)) as w:
+            w.execute(
+                f"CREATE TABLE decp AS SELECT * FROM read_parquet('{staging_parquet}')"
+            )
+            w.execute(
+                "CREATE TABLE acheteurs_marches AS "
+                "SELECT DISTINCT uid, objet, acheteur_id FROM decp "
+                "ORDER BY acheteur_id"
+            )
+            w.execute(
+                "CREATE TABLE titulaires_marches AS "
+                "SELECT DISTINCT uid, objet, titulaire_id FROM decp "
+                "ORDER BY titulaire_id"
+            )
+            w.execute(
+                "CREATE TABLE acheteurs_departement AS "
+                "SELECT DISTINCT acheteur_id, acheteur_nom, acheteur_departement_code "
+                "FROM decp ORDER BY acheteur_nom"
+            )
+            w.execute(
+                "CREATE TABLE titulaires_departement AS "
+                "SELECT DISTINCT titulaire_id, titulaire_nom, titulaire_departement_code "
+                "FROM decp ORDER BY titulaire_nom"
+            )
+    finally:
+        if staging_parquet.exists():
+            staging_parquet.unlink()
+
+    os.replace(tmp_path, db_path)
+    logger.info(f"Base DuckDB construite : {db_path}")
+
+
+def _ensure_database() -> Path:
+    db_path = Path(os.getenv("DUCKDB_PATH", "./decp.duckdb"))
+    parquet_path = Path(os.getenv("DATA_FILE_PARQUET_PATH"))
+    lock_path = db_path.with_suffix(".duckdb.lock")
+
+    with open(lock_path, "w") as lock_fd:
+        fcntl.flock(lock_fd, fcntl.LOCK_EX)
+        if should_rebuild(db_path, parquet_path):
+            build_database(db_path, parquet_path)
+        else:
+            logger.debug("Base de données déjà disponible et à jour.")
+    return db_path
+
+
+DB_PATH = _ensure_database()
+conn: duckdb.DuckDBPyConnection = duckdb.connect(str(DB_PATH), read_only=True)
+schema: pl.Schema = conn.execute("SELECT * FROM decp LIMIT 0").pl().schema
+
+
+def get_cursor() -> duckdb.DuckDBPyConnection:
+    """Return a per-request cursor that shares the process-wide connection."""
+    return conn.cursor()
+
+
+def query_marches(
+    where_sql: str = "TRUE",
+    params: tuple | list = (),
+    columns: list[str] | None = None,
+    order_by: str | None = None,
+    limit: int | None = None,
+    offset: int | None = None,
+) -> pl.DataFrame:
+    """Run a parameterized SELECT against the decp table and return Polars.
+
+    `where_sql` and `order_by` are trusted SQL fragments (callers are internal
+    code, never user input). `params` values are passed through DuckDB's
+    parameter binding.
+    """
+    cols = ", ".join(columns) if columns else "*"
+    sql = f"SELECT {cols} FROM decp WHERE {where_sql}"
+    if order_by:
+        sql += f" ORDER BY {order_by}"
+    if limit is not None:
+        sql += f" LIMIT {int(limit)}"
+    if offset is not None:
+        sql += f" OFFSET {int(offset)}"
+
+    logger.debug("query_marches: " + sql.replace("?", "{}").format(*params))
+
+    return get_cursor().execute(sql, list(params)).pl()
+
+
+def count_marches(where_sql: str = "TRUE", params: tuple | list = ()) -> int:
+    """Retourne le nombre de lignes correspondant à where_sql."""
+    sql = f"SELECT COUNT(*) FROM decp WHERE {where_sql}"
+    logger.debug("count_marches: " + sql.replace("?", "{}").format(*params))
+    result = get_cursor().execute(sql, list(params)).fetchone()
+    return int(result[0]) if result else 0
+
+
+def count_unique_marches(where_sql: str = "TRUE", params: tuple | list = ()) -> int:
+    """Retourne le nombre de uid distincts correspondant à where_sql."""
+    sql = f"SELECT COUNT(DISTINCT uid) FROM decp WHERE {where_sql}"
+    logger.debug("count_unique_marches: " + sql.replace("?", "{}").format(*params))
+    result = get_cursor().execute(sql, list(params)).fetchone()
+    return int(result[0]) if result else 0

src/figures.py

@@ -1,73 +1,77 @@
-import json
+from datetime import datetime
+from typing import Literal
+from urllib.error import HTTPError, URLError
 
+import dash_bootstrap_components as dbc
+import dash_leaflet as dl
+import dash_leaflet.express as dlx
+import numpy as np
 import plotly.express as px
+import plotly.graph_objects as go
 import polars as pl
 from dash import dash_table, dcc, html
+from dash_extensions.javascript import Namespace
+from polars.exceptions import ColumnNotFoundError
+
+from src.db import schema
+from src.utils import logger
+from src.utils.data import DATA_SCHEMA, DEPARTEMENTS_GEOJSON
+from src.utils.table import add_links, format_number, setup_table_columns
 
 
-def get_map_count_marches(df: pl.DataFrame):
+def get_yearly_statistics(statistics, today_str) -> html.Div:
-    lf = df.lazy()
+    # Build DataFrame from statistics
-    lf = lf.with_columns(
+    years = list(reversed(range(2018, int(today_str.split("/")[-1]) + 1)))
-        pl.col("lieuExecution_code").str.head(2).str.zfill(2).alias("Département")
+    data = []
-    )
+    for year in years:
-    lf = (
+        year_str = str(year)
-        lf.select(["uid", "Département"])
+        stat = statistics[year_str]
-        .drop_nulls()
+        data.append(
-        .unique(subset="uid")
+            {
-        .group_by("Département")
+                "Année": year_str,
-        .len("uid")
+                "Marchés et accord-cadres": format_number(
-    )
+                    stat["nb_notifications_marches"]
-    # Suppression des infos pour les DOM/TOM pour l'instant
+                ),
-    lf = lf.remove(pl.col("Département").is_in(["97", "98"]))
+                "Acheteurs": format_number(stat["nb_acheteurs_uniques"]),
+                "Titulaires": format_number(stat["nb_titulaires_uniques"]),
+            }
+        )
 
-    with open("./data/departements-1000m.geojson") as f:
+    dff = pl.DataFrame(data)
-        departements = json.load(f)
 
-    # Ajout de feature.id
+    # Create Dash DataTable
-    for f in departements["features"]:
+    table = dash_table.DataTable(
-        f["id"] = f["properties"]["code"]
+        data=dff.to_dicts(),
-
+        columns=[
-    df = lf.collect(engine="streaming")
+            {"name": "Année", "id": "Année"},
-
+            {"name": "Marchés et accord-cadres", "id": "Marchés et accord-cadres"},
-    fig = px.choropleth(
+            {"name": "Acheteurs", "id": "Acheteurs"},
-        df,
+            {"name": "Titulaires", "id": "Titulaires"},
-        geojson=departements,
+        ],
-        locations="Département",
+        page_size=10,
-        color="uid",
+        sort_action="none",
-        color_continuous_scale="Reds",
+        filter_action="none",
-        title="Nombres de marchés attribués par département (lieu d'exécution)",
+        style_header={"fontFamily": "Inter", "fontSize": "16px"},
-        range_color=(df["uid"].min(), df["uid"].max()),
+        style_cell={"fontFamily": "Inter", "fontSize": "16px"},
-        labels={"uid": "Marchés attribués"},
-        scope="europe",
-        width=900,
-        height=700,
     )
 
-    fig.update_geos(fitbounds="locations", visible=False)
+    return html.Div(children=table, className="marches_table")
-    fig.update_layout(
-        mapbox={
-            "style": "carto-positron",
-            "center": {"lon": 10, "lat": 10},
-            "zoom": 1,
-            "domain": {"x": [0, 1], "y": [0, 1]},
-        }
-    )
-    return fig
 
 
-def get_barchart_sources(df: pl.DataFrame, type_date: str):
+def get_barchart_sources(lff: pl.LazyFrame, type_date: str):
-    lf = df.lazy()
     labels = {
         "dateNotification": "notification",
         "datePublicationDonnees": "publication des données",
     }
 
-    lf = lf.select("uid", type_date, "sourceDataset")
+    now_year = datetime.now().year
 
-    lf = lf.unique("uid")
+    lff = lff.select("uid", type_date, "sourceDataset")
+
+    lff = lff.unique("uid")
 
     # Rassemblement des datasets Atexo pour ne pas surcharger le graphique
-    lf = lf.with_columns(
+    lff = lff.with_columns(
         pl.when(pl.col("sourceDataset").str.starts_with("atexo"))
         .then(pl.lit("plateformes atexo"))
         .otherwise(pl.col("sourceDataset"))
@@ -75,38 +79,33 @@ def get_barchart_sources(df: pl.DataFrame, type_date: str):
     )
 
     # Rassemblement des datasets AWS pour ne pas surcharger le graphique
-    lf = lf.with_columns(
+    lff = lff.with_columns(
         pl.when(pl.col("sourceDataset").str.contains(r"aws|marches\-publics.info"))
         .then(pl.lit("aws"))
         .otherwise(pl.col("sourceDataset"))
         .alias("sourceDataset")
     )
 
-    lf = lf.with_columns(pl.col(type_date).dt.year().alias("annee"))
+    lff = lff.with_columns(pl.col(type_date).dt.year().alias("annee"))
-    lf = lf.filter(
+    lff = lff.filter(
-        pl.col(type_date).is_not_null() & pl.col("annee").is_between(2019, 2025)
+        pl.col(type_date).is_not_null() & pl.col("annee").is_between(2019, now_year)
     )
-    lf = lf.with_columns(pl.col(type_date).cast(pl.String).str.head(7))
+    lff = lff.with_columns(pl.col(type_date).cast(pl.String).str.head(7))
-    lf = (
+    lff = (
-        lf.group_by([type_date, "sourceDataset"])
+        lff.group_by([type_date, "sourceDataset"])
         .len()
         .sort(by=[type_date, "len"], descending=True)
     )
 
-    # lf = lf.with_columns(
+    lff = lff.sort(by=["sourceDataset"], descending=False)
-    #     pl.when(pl.col("sourceDataset").is_null()).then(
-    #         pl.lit("Source inconnue")).alias("sourceDataset")
-    #     )
 
-    lf = lf.sort(by=["sourceDataset"], descending=False)
+    dff: pl.DataFrame = lff.collect(engine="streaming")
-    df: pl.DataFrame = lf.collect(engine="streaming")
 
     fig = px.bar(
-        df,
+        dff,
         x=type_date,
         y="len",
         color="sourceDataset",
-        title=f"Nombre de marchés attribués par date de {labels[type_date]} et source de données",
         labels={
             "len": "Nombre de marchés",
             type_date: f"Mois de {labels[type_date]}",
@@ -114,12 +113,17 @@ def get_barchart_sources(df: pl.DataFrame, type_date: str):
         },
     )
 
-    return fig
+    graph = dcc.Graph(figure=fig)
+
+    return graph
 
 
 def get_sources_tables(source_path) -> html.Div:
-    df = pl.read_csv(source_path)
+    try:
-    df = df.with_columns(
+        dff = pl.read_csv(source_path)
+    except (URLError, HTTPError):
+        return html.Div("Erreur de connexion")
+    dff = dff.with_columns(
         (
             pl.lit('<a href = "')
             + pl.col("url")
@@ -128,20 +132,29 @@ def get_sources_tables(source_path) -> html.Div:
             + pl.lit("</a>")
         ).alias("nom")
     )
-    df = df.drop("url")
+    dff = dff.drop("url", "unique")
-    df = df.sort(by=["nb_marchés"], descending=True)
+    dff = dff.sort(by=["nb_marchés"], descending=True)
+
+    columns = {
+        "nom": "Nom de la source",
+        "organisation": "Responsable de publication",
+        "nb_marchés": "Nb de marchés",
+        "nb_acheteurs": "Nb d'acheteurs",
+        "code": "Code",
+    }
 
     datatable = dash_table.DataTable(
         id="source_table",
+        data=dff.to_dicts(),
         columns=[
             {
-                "name": i,
+                "name": columns[i],
                 "id": i,
                 "presentation": "markdown",
                 "type": "text",
                 "format": {"nully": "N/A"},
             }
-            for i in df.schema.names()
+            for i in dff.schema.names()
         ],
         style_cell_conditional=[
             {
@@ -155,40 +168,734 @@ def get_sources_tables(source_path) -> html.Div:
         ],
         sort_action="native",
         markdown_options={"html": True},
+        style_header={"fontFamily": "Inter", "fontSize": "16px"},
+        style_cell={"fontFamily": "Inter", "fontSize": "16px"},
     )
-    datatable.data = df.to_dicts()
 
     return html.Div(children=datatable)
 
 
-def point_on_map(lat, lon):
+def point_on_map(lat, lon, departement_code=None):
-    lat = float(lat)
+    """Fonction améliorée utilisant les codes départementaux pour la détection de région.
-    lon = float(lon)
 
-    # Create a scatter mapbox or choropleth map
+    Args:
+        lat: Coordonnée de latitude
+        lon: Coordonnée de longitude
+        departement_code: Code du département (ex: '75', '971', etc.)
+
+    Returns:
+        html.Div contenant la carte, ou div vide si invalide
+    """
+    # Validation des coordonnées
+    try:
+        lat = float(lat)
+        lon = float(lon)
+    except (TypeError, ValueError):
+        return html.Div()  # Div vide pour les coordonnées invalides
+
+    # Vérification que les coordonnées sont valides
+    if not (-90 <= lat <= 90) or not (-180 <= lon <= 180):
+        return html.Div()
+
+    # Si aucun code département n'est fourni, retourner une div vide
+    if not departement_code:
+        return html.Div()
+
+    # Détermination de la région en utilisant le code département
+    # Logique identique à get_geographic_maps
+    if departement_code in ["971", "972", "973", "974", "976"]:
+        region_key = departement_code  # Département d'outre-mer
+    elif len(departement_code) == 2:  # Département métropolitain
+        region_key = "Hexagone"
+    else:
+        return html.Div()  # Format de code département invalide
+
+    # Paramètres de carte par région (réutilisés de get_geographic_maps)
+    regions = {
+        "Hexagone": {"center": [46.6, 2.2], "zoom": 5},
+        "971": {"center": [16.23, -61.55], "zoom": 9},  # Guadeloupe
+        "972": {"center": [14.64, -61.02], "zoom": 10},  # Martinique
+        "973": {"center": [3.93, -53.12], "zoom": 7},  # Guyane
+        "974": {"center": [-21.11, 55.53], "zoom": 9},  # La Réunion
+        "976": {"center": [-12.82, 45.16], "zoom": 10},  # Mayotte
+    }
+
+    settings = regions.get(region_key, regions["Hexagone"])
+
+    # Création de la carte
     fig = px.scatter_map(
-        lat=[lat], lon=[lon], height=300, width=400, color=[1], size=[1]
+        lat=[lat],
+        lon=[lon],
+        height=300,
+        # width=400,
+        color=[1],
+        zoom=settings["zoom"],
     )
 
-    fig.update_coloraxes(showscale=False)
+    fig.update_traces(marker=dict(size=10))
 
-    # Set map style (you can use 'open-street-map', 'carto-positron', etc.)
+    # Configuration de la carte (interactive - zoomable)
     fig.update_layout(
-        mapbox_style="light",  # Light, clean background
+        map_style="light",  # Fond de carte clair
         margin={"r": 0, "t": 0, "l": 0, "b": 0},
+        mapbox_center={"lat": settings["center"][0], "lon": settings["center"][1]},
+        mapbox_zoom=settings["zoom"],
+        coloraxis_showscale=False,
     )
 
-    # Optionally, center the map on France
+    return html.Div(
-    fig.update_geos(
+        dcc.Graph(figure=fig, config={"displayModeBar": False}),
-        center=dict(lat=46.603354, lon=1.888334),  # Center of France
-        lataxis_range=[41, 51.5],  # Latitude range for France
-        lonaxis_range=[-5, 10],  # Longitude range for France
     )
 
-    # But scatter_mapbox doesn't use geos, so better to control via zoom/center manually
-    # Let's reset and use proper centering in scatter_mapbox instead:
 
-    fig.update_layout(map_center={"lat": 46.6, "lon": 1.89}, map_zoom=4)
+class DataTable(dash_table.DataTable):
+    def __init__(
+        self,
+        dtid: str,
+        hidden_columns: list[str] | None = None,
+        data: list[dict[str, str | int | float | bool]] | None = None,
+        columns: list[dict[str, str]] | None = None,
+        page_size: int = 20,
+        page_action: Literal["native", "custom", "none"] = "native",
+        sort_action: Literal["native", "custom", "none"] = "native",
+        filter_action: Literal["native", "custom", "none"] = "native",
+        style_cell_conditional: list | None = None,
+        style_cell: dict | None = None,
+        **kwargs,
+    ):
+        # Styles de base
+        style_cell_conditional_common = [
+            {
+                "if": {"column_id": "objet"},
+                "minWidth": "350px",
+                "overflow": "hidden",
+                "lineHeight": "18px",
+                "whiteSpace": "normal",
+            },
+            {
+                "if": {"column_id": "acheteur_id"},
+                "minWidth": "160px",
+                "overflow": "hidden",
+                "whiteSpace": "normal",
+            },
+            {
+                "if": {"column_id": "acheteur_nom"},
+                "minWidth": "250px",
+                "overflow": "hidden",
+                "lineHeight": "18px",
+                "whiteSpace": "normal",
+            },
+            {
+                "if": {"column_id": "titulaire_nom"},
+                "minWidth": "250px",
+                "overflow": "hidden",
+                "lineHeight": "18px",
+                "whiteSpace": "normal",
+            },
+        ]
 
-    graph = dcc.Graph(id="map", figure=fig)
+        style_cell_common = {"fontFamily": "Inter", "fontSize": "16px"}
+
+        for key in DATA_SCHEMA.keys():
+            field = DATA_SCHEMA[key]
+            if field["type"] in ["number", "integer"]:
+                rule = {
+                    "if": {"column_id": field["name"]},
+                    "textAlign": "right",
+                    # "fontFamily": "Fira Code",
+                }
+                style_cell_conditional_common.append(rule)
+
+        style_cell_conditional = (
+            style_cell_conditional or []
+        ) + style_cell_conditional_common
+        if style_cell:
+            style_cell.update(style_cell_common)
+        else:
+            style_cell = style_cell_common
+        style_header = style_cell
+
+        # Initialisation de la classe parente avec les arguments
+        super().__init__(
+            id=dtid,
+            data=data,
+            columns=columns,
+            cell_selectable=False,
+            page_size=page_size,
+            filter_action=filter_action,
+            page_action=page_action,
+            filter_options={
+                "case": "insensitive",
+                "placeholder_text": "Filtre de colonne...",
+            },
+            sort_action=sort_action,
+            sort_mode="multi",
+            row_deletable=False,
+            page_current=0,
+            style_cell_conditional=style_cell_conditional,
+            data_timestamp=0,
+            markdown_options={"html": True},
+            style_header=style_header,
+            style_cell=style_cell,
+            tooltip_duration=8000,
+            tooltip_delay=350,
+            hidden_columns=hidden_columns,
+            **kwargs,  # Possibilité de remplacer des arguments
+        )
+
+
+def get_duplicate_matrix() -> dcc.Graph:
+    """
+    Fonction développée avec l'aide de la LLM Euria d'Infomaniak.
+    :return:
+    """
+    lff = pl.scan_parquet(
+        "https://www.data.gouv.fr/api/1/datasets/r/a545bf6c-8b24-46ed-b49f-a32bf02eaffa"
+    ).sort("sourceDataset")
+    lff = lff.select(
+        ["sourceDataset", "unique"] + sorted(lff.collect_schema().names()[2:])
+    )
+
+    dff = lff.collect()
+
+    # Extract data
+    z_data = dff.select(pl.all().exclude("sourceDataset")).fill_null(0).to_numpy()
+    x_labels = dff.columns[1:]  # columns after "sourceDataset"
+    y_labels = dff["sourceDataset"].to_list()
+
+    # Create heatmap
+    fig = go.Figure(
+        data=go.Heatmap(
+            z=z_data,
+            x=x_labels,
+            y=y_labels,
+            colorscale=[
+                [0.0, "white"],  # 0% → white
+                [0.10, "lightsalmon"],  # 10% → light warm tone
+                [1.0, "darkred"],  # 100% → deep red
+            ],
+            zmin=0,
+            zmax=1,
+            hoverongaps=False,
+            showscale=True,
+            hovertemplate=(
+                "<b>%{z:.0%}</b> des marchés présents dans <b>%{y}</b> sont également présents dans <b>%{x}</b>"
+            ),
+        )
+    )
+
+    # Update layout: make it wider and taller
+    fig.update_layout(
+        title="",
+        xaxis_title="Sources de données",
+        yaxis_title="Sources de données",
+        yaxis=dict(autorange="reversed"),
+        xaxis=dict(tickangle=45, tickfont=dict(size=10)),  # Smaller x-tick labels
+        coloraxis_colorbar=dict(title="Percentage", tickfont=dict(size=10)),
+        width=1000,  # Wider
+        height=1000,  # Taller
+        font=dict(size=11),  # Overall font size
+        margin=dict(l=100, r=50, t=80, b=100),  # Add margin for labels
+    )
+
+    return dcc.Graph(figure=fig)
+
+
+def get_geographic_maps(dff: pl.DataFrame) -> list[dbc.Col] | list:
+    """
+    Génère les cartes géographiques pour l'hexagone et les DOM-TOM.
+    """
+
+    regions: dict = {
+        "Hexagone": {
+            "coordinates": [46.6, 2.2],
+            "zoom_leaflet": 5,
+            "zoom_chloropleth": 1,
+            "name": "Hexagone",
+        },
+        "971": {
+            "coordinates": [16.23, -61.55],
+            "zoom_leaflet": 9,
+            "zoom_chloropleth": 1,
+            "name": "Guadeloupe",
+        },
+        "972": {
+            "coordinates": [14.64, -61.02],
+            "zoom_leaflet": 10,
+            "zoom_chloropleth": 1,
+            "name": "Martinique",
+        },
+        "973": {
+            "coordinates": [3.93, -53.12],
+            "zoom_leaflet": 7,
+            "zoom_chloropleth": 1,
+            "name": "Guyane",
+        },
+        "974": {
+            "coordinates": [-21.11, 55.53],
+            "zoom_leaflet": 9,
+            "zoom_chloropleth": 1,
+            "name": "La Réunion",
+        },
+        "976": {
+            "coordinates": [-12.82, 45.16],
+            "zoom_leaflet": 10,
+            "zoom_chloropleth": 1,
+            "name": "Mayotte",
+        },
+    }
+
+    def make_map_data(region_code: str) -> tuple[list, str | None]:
+        lff: pl.LazyFrame = dff.lazy()
+        if region_code == "Hexagone":
+            lff = lff.filter(
+                (pl.col("acheteur_departement_code").str.len_chars() == 2)
+                & (pl.col("titulaire_departement_code").str.len_chars() == 2)
+            )
+        else:
+            lff = lff.filter(
+                (pl.col("acheteur_departement_code") == code)
+                | (pl.col("titulaire_departement_code") == code)
+            )
+
+        nb_marches = lff.select("uid").collect()["uid"].n_unique()
+
+        if nb_marches == 0:
+            return [], None
+
+        dfs = []
+
+        if (code == "Hexagone" and nb_marches > 30000) or (
+            code != "Hexagone" and nb_marches > 10000
+        ):
+            _map_type: str = "chloropleth"
+
+            lff = lff.rename({"acheteur_departement_code": "Département"})
+            lff = (
+                lff.select(["uid", "Département"])
+                .drop_nulls()
+                .group_by("uid")
+                .agg(pl.col("Département").first())
+                .group_by("Département")
+                .len("uid")
+            )
+            dfs.append(lff.collect())
+        else:
+            _map_type: str = "clusters"
+            for org_type in ["acheteur", "titulaire"]:
+                lff_org = (
+                    lff.select(
+                        "uid",
+                        f"{org_type}_longitude",
+                        f"{org_type}_latitude",
+                        f"{org_type}_nom",
+                    )
+                    .group_by(
+                        f"{org_type}_longitude",
+                        f"{org_type}_latitude",
+                        f"{org_type}_nom",
+                    )
+                    .len("nb_marches")
+                    .filter(
+                        pl.col(f"{org_type}_latitude").is_not_null()
+                        & pl.col(f"{org_type}_longitude").is_not_null()
+                    )
+                )
+
+                markers = []
+
+                # Couleurs accessibles (Okabe-Ito)
+                colors = {
+                    "acheteur": "#E69F00",  # orange
+                    "titulaire": "#56B4E9",  # bleu ciel
+                }
+
+                for row in lff_org.collect().to_dicts():
+                    markers.append(
+                        {
+                            "lat": row[f"{org_type}_latitude"],
+                            "lon": row[f"{org_type}_longitude"],
+                            "tooltip": f"{row[f'{org_type}_nom']} ({row['nb_marches']} marchés)",
+                            "marker_color": colors[org_type],
+                        }
+                    )
+                dfs.append(markers)
+
+        return dfs, _map_type
+
+    cols = []
+
+    for code in regions.keys():
+        regions[code]["data"], map_type = make_map_data(code)
+
+        if map_type == "chloropleth":
+            map_graph = make_chloropleth_map(regions[code])
+        elif map_type == "clusters":
+            map_graph = make_clusters_map(regions[code])
+        elif map_type is None:
+            continue
+        else:
+            raise ValueError(f"Map type '{map_type}' not recognised")
+
+        lg, xl = (12, 8) if code == "Hexagone" else (6, 4)
+
+        col = make_card(regions[code]["name"], fig=map_graph, lg=lg, xl=xl)
+        cols.append(col)
+
+    return cols
+
+
+def make_chloropleth_map(region: dict) -> dcc.Graph:
+    df_map = region["data"][0]
+
+    fig = px.choropleth(
+        df_map,
+        geojson=DEPARTEMENTS_GEOJSON,
+        locations="Département",
+        color="uid",
+        color_continuous_scale="Reds",
+        range_color=(df_map["uid"].min(), df_map["uid"].max()),
+        labels={"uid": "Marchés attribués"},
+        scope="europe",
+    )
+
+    fig.update_geos(fitbounds="locations", visible=False)
+    fig.update_layout(
+        mapbox={
+            "style": "carto-positron",
+            "center": {"lon": 10, "lat": 10},
+            "zoom": 8,
+            "domain": {"x": [0, 1], "y": [0, 1]},
+        }
+    )
+
+    graph = dcc.Graph(figure=fig, config={"displayModeBar": False})
     return graph
+
+
+def make_clusters_map(region: dict) -> dl.Map:
+    # JavaScript functions for styling
+    ns = Namespace("dash_clientside", "leaflet")
+    point_to_layer = ns("pointToLayer")
+    cluster_to_layer = ns("clusterToLayer")
+
+    name = region["name"]
+
+    # Données de la région
+    region_acheteurs = region["data"][0]
+    region_titulaires = region["data"][1]
+
+    # Couleurs
+    color_acheteur = region_acheteurs[0]["marker_color"]
+    color_titulaire = region_titulaires[0]["marker_color"]
+
+    acheteurs_geojson_data = dlx.dicts_to_geojson(region_acheteurs)
+    titulaires_geojson_data = dlx.dicts_to_geojson(region_titulaires)
+
+    center, zoom = region["coordinates"], region["zoom_leaflet"]
+    region_id = name.lower().replace(" ", "-")
+    leaflet_map = dl.Map(
+        [
+            dl.TileLayer(),
+            dl.GeoJSON(
+                data=titulaires_geojson_data,
+                cluster=True,
+                zoomToBoundsOnClick=True,
+                pointToLayer=point_to_layer,
+                clusterToLayer=cluster_to_layer,
+                id=f"geojson-{region_id}-titulaires",
+                options={"fillColor": color_titulaire},
+            ),
+            dl.GeoJSON(
+                data=acheteurs_geojson_data,
+                cluster=True,
+                zoomToBoundsOnClick=True,
+                pointToLayer=point_to_layer,
+                clusterToLayer=cluster_to_layer,
+                id=f"geojson-{region_id}-acheteurs",
+                options={"fillColor": color_acheteur},
+            ),
+        ],
+        center=center,
+        zoom=zoom,
+        style={
+            "width": "100%",
+            "height": "400px" if name == "Hexagone" else "300px",
+        },
+        id=f"map-{region_id}",
+    )
+    return leaflet_map
+
+
+def get_distance_histogram(lff: pl.LazyFrame) -> dcc.Graph:
+    if "titulaire_distance" not in lff.collect_schema().names():
+        dff = pl.DataFrame({"titulaire_distance": pl.Series([], dtype=pl.Float64)})
+    else:
+        dff = (
+            lff.select("titulaire_distance")
+            .drop_nulls()
+            .filter(pl.col("titulaire_distance") > 0)
+            .collect(engine="streaming")
+        )
+    log_distances = dff["titulaire_distance"].log(10).to_numpy()
+
+    fig = go.Figure()
+    if len(log_distances) > 0:
+        counts, bin_edges = np.histogram(log_distances, bins=25)
+        bin_centers = (bin_edges[:-1] + bin_edges[1:]) / 2
+        bin_widths = bin_edges[1:] - bin_edges[:-1]
+        bin_edges_km = 10.0**bin_edges
+
+        def fmt_km(km):
+            if km < 10:
+                return f"{km:.1f}"
+            elif km < 1000:
+                return f"{round(km)}"
+            else:
+                return f"{round(km):,}".replace(",", " ")
+
+        hover_texts = []
+        for i in range(len(counts)):
+            nb = f"{counts[i]:,}".replace(",", " ")
+            hover_texts.append(
+                f"Distance : {fmt_km(bin_edges_km[i])} – {fmt_km(bin_edges_km[i + 1])} km"
+                f"<br>Nombre de marchés : {nb}"
+            )
+
+        fig.add_trace(
+            go.Bar(
+                x=bin_centers,
+                y=counts,
+                width=bin_widths,
+                hovertext=hover_texts,
+                hoverinfo="text",
+            )
+        )
+        fig.update_layout(bargap=0)
+
+    fig.update_layout(margin=dict(r=10, t=10))
+    fig.update_xaxes(
+        tickvals=[0, 1, 2, 3, 4],
+        ticktext=["1", "10", "100", "1 000", "10 000"],
+        title_text="Distance (km)",
+    )
+    fig.update_yaxes(title_text="Nombre de marchés")
+    return dcc.Graph(figure=fig)
+
+
+def get_dashboard_summary_table(dff, dff_per_uid, nb_marches):
+    nb_acheteurs = dff.select("acheteur_id").n_unique()
+    nb_titulaires = dff.select("titulaire_id", "titulaire_typeIdentifiant").n_unique()
+    total_montant = int(dff_per_uid.select(pl.col("montant").sum()).item())
+    median_distance = dff.select(pl.median("titulaire_distance")).item()
+
+    summary_table = [
+        html.P(["Nombre de marchés : ", html.Strong(str(format_number(nb_marches)))]),
+        html.P(
+            [
+                "Nombre d'acheteurs uniques : ",
+                html.Strong(str(format_number(nb_acheteurs))),
+            ]
+        ),
+        html.P(
+            [
+                "Nombre de titulaires uniques : ",
+                html.Strong(str(format_number(nb_titulaires))),
+            ]
+        ),
+        html.P(
+            [
+                "Montant total (",
+                html.Span(
+                    "?",
+                    id={"type": "modal-trigger", "index": "montant"},
+                    style={"cursor": "pointer", "textDecoration": "underline dotted"},
+                ),
+                ") : ",
+                html.Strong(format_number(total_montant) + " €"),
+            ]
+        ),
+        html.P(
+            [
+                "Distance acheteur-titulaire médiane : ",
+                html.Strong(format_number(median_distance) + " km"),
+            ]
+        ),
+    ]
+
+    return summary_table
+
+
+def make_card(
+    title: str, subtitle=None, fig=None, paragraphs=None, lg=6, xl=4
+) -> dbc.Col:
+    children = []
+    if title:
+        children.append(html.H5(title, className="card-title"))
+    if subtitle:
+        children.append(html.H6(subtitle, className="card-subtitle mb-2 text-muted"))
+    if fig is not None:
+        children.append(fig)
+    if paragraphs:
+        for p in paragraphs:
+            p.className = "card-text"
+            children.append(p)
+
+    card = dbc.Col(
+        html.Div(html.Div(className="card-body", children=children), className="card"),
+        lg=lg,
+        xl=xl,
+        # width=width,
+        # className="mb-4",
+    )
+    return card
+
+
+def make_donut(
+    lff: pl.LazyFrame,
+    names_col,
+    per_uid: bool,
+    nulls="?",
+    potentially_many_names: bool = False,
+):
+    title = DATA_SCHEMA[names_col]["title"]
+    lff = lff.rename({names_col: title})
+    lff = lff.select("uid", title)
+
+    if per_uid:
+        lff = lff.group_by("uid").first()
+
+    lff = lff.group_by(title).len("Nombre")
+    lff = lff.with_columns(pl.col(title).replace(None, pl.lit(nulls)))
+    dff = lff.collect(engine="streaming")
+    nb_names = dff[title].n_unique()
+
+    sum_values = dff["Nombre"].sum()
+    dff = dff.with_columns(
+        pl.when((pl.col("Nombre") / sum_values) < 0.01)
+        .then(pl.lit("Autres"))
+        .otherwise(pl.col(title))
+        .alias(title)
+    )
+
+    dff = dff.with_columns(
+        pl.col("Nombre")
+        .map_elements(format_number, return_dtype=pl.String)
+        .alias("Nombre_fmt")
+    )
+    fig = px.pie(
+        dff,
+        values="Nombre",
+        names=title,
+        hole=0.4,
+        color_discrete_sequence=px.colors.qualitative.Safe,
+        custom_data=["Nombre_fmt"],
+    )
+    fig = fig.update_traces(
+        texttemplate="<b>%{label}</b><br><b>%{percent}</b>",
+        hovertemplate="<b>%{label}</b><br>%{customdata[0]}<extra></extra>",
+    )
+    fig = fig.update_layout(showlegend=False, font=dict(size=14))
+    graph = dcc.Graph(figure=fig)
+    if potentially_many_names:
+        return graph, nb_names
+    return graph
+
+
+def make_column_picker(page: str):
+    table_data = []
+    table_columns = [
+        {
+            "id": col,
+            "name": DATA_SCHEMA[col]["title"],
+            "description": DATA_SCHEMA[col]["description"],
+        }
+        for col in schema.names()
+    ]
+    for column in table_columns:
+        new_column = {
+            "id": column["id"],
+            "name": column["name"],
+            "description": DATA_SCHEMA[column["id"]]["description"],
+        }
+        table_data.append(new_column)
+
+    table = (
+        DataTable(
+            row_selectable="multi",
+            data=table_data,
+            filter_action="native",
+            sort_action="none",
+            style_cell={
+                "textAlign": "left",
+            },
+            columns=[
+                {
+                    "name": "Nom",
+                    "id": "name",
+                },
+                {
+                    "name": "Description",
+                    "id": "description",
+                },
+            ],
+            style_cell_conditional=[
+                {
+                    "if": {"column_id": "description"},
+                    "minWidth": "450px",
+                    "overflow": "hidden",
+                    "lineHeight": "18px",
+                    "whiteSpace": "normal",
+                }
+            ],
+            page_action="none",
+            dtid=f"{page}_column_list",
+        ),
+    )
+
+    return table
+
+
+def get_top_org_table(data, org_type: str, extra_columns: list, filters: bool = True):
+    if isinstance(data, pl.LazyFrame):
+        lff = data
+    else:
+        lff = pl.LazyFrame(data, strict=False, infer_schema_length=5000)
+
+    if org_type == "titulaire":
+        extra_columns.append("titulaire_typeIdentifiant")
+    columns = ["uid", f"{org_type}_id", f"{org_type}_nom"] + extra_columns
+
+    lff = lff.select(columns)
+    lff = lff.group_by([f"{org_type}_id", f"{org_type}_nom"] + extra_columns).agg(
+        pl.len().alias("Attributions")
+    )
+    lff = lff.sort(by="Attributions", descending=True, nulls_last=True)
+    lff = lff.cast(pl.String)
+    lff = lff.fill_null("")
+
+    try:
+        dff: pl.DataFrame = lff.collect(engine="streaming")
+    except ColumnNotFoundError:
+        logger.warning(f"get_top_org_table: column not found. {lff.collect_schema()}")
+        return html.Div()
+
+    if dff.height == 0:
+        return html.Div()
+
+    columns, tooltip = setup_table_columns(
+        dff, hideable=False, exclude=[f"{org_type}_id"]
+    )
+    dff = add_links(dff)
+    data = dff.to_dicts()
+    # data = add_links_in_dict(data, f"{org_type}")
+
+    return DataTable(
+        dtid=f"top10_{org_type}",
+        data=data,
+        page_action="native",
+        page_size=10,
+        columns=columns,
+        tooltip_header=tooltip,
+        filter_action="native" if filters else "none",
+    )

src/pages/a-propos.py

@@ -3,27 +3,33 @@ import os
 from dash import dcc, html, register_page
 
 from src.figures import get_sources_tables
-from src.utils import meta_content
+from src.utils.seo import META_CONTENT
 
-name = "À propos"
+NAME = "À propos"
 
 register_page(
     __name__,
     path="/a-propos",
-    title=meta_content["title"],
+    title="À propos | decp.info",
-    name=name,
+    name="À propos",
-    description=meta_content["description"],
+    description="En savoir plus sur decp.info, l'outil d'exploration des données essentielles de la commande publique.",
-    image_url=meta_content["image_url"],
+    image_url=META_CONTENT["image_url"],
     order=5,
 )
 
-layout = [
+layout = html.Div(
-    html.Div(
+    className="container",
-        className="container",
+    children=[
-        children=[
+        html.H2(NAME),
-            html.H2(name),
+        html.Div(
-            dcc.Markdown(
+            className="a-propos-container",
-                """Outil d'exploration libre et gratuit des données de marchés publics, développé par Colin Maudry.
+            children=[
+                # Main Content Column
+                html.Div(
+                    className="a-propos-content",
+                    children=[
+                        dcc.Markdown(
+                            """Outil d'exploration libre et gratuit des données de marchés publics, développé par Colin Maudry.
 
 Ce projet vise à démocratiser l'accès aux données des marchés publics et à un outil performant et gratuit. Si vous le trouvez utile
 j'aimerais beaucoup échanger avec vous pour comprendre vos cas d'usages et vos besoins. Cet outil ne peut rester performant que si je comprends les problèmes qu'il peut aider à résoudre. Ce projet ne peut rester gratuit que grâce au financement du développement de nouvelles fonctionnalités.
@@ -32,20 +38,39 @@ En effet, le potentiel des données d'attribution de marchés et des données qu
 les fonctionnalités actuelles de decp.info. Il est ainsi possible de rajouter
 
 - de nombreuses visualisations de données (cartes, graphiques, tableaux) sur des thématiques variées (vivacité de la concurrence, secteurs d'activité, insertion par l'activité économique (IAE), distance acheteur-titulaire...)
-- la sauvegarde de filtres pour les retrouver plus tard et les partager
 - des alertes par email si des marchés correspondant à certains critères
-- le développement d'une API pour alimenter d'autres logiciels
 - ...et toutes les fonctionnalités auxquelles vous pourrez penser
-   """
+"""
-            ),
+                        ),
-            html.H4("Pour contribuer", id="contribuer"),
+                        html.H4("Consommer les données brutes", id="donnees-brutes"),
-            dcc.Markdown("""
+                        dcc.Markdown(
+                            """
+Vous pouvez consommer les données qui alimentent decp.info
+
+- en les téléchargeant [sur data.gouv.fr](https://www.data.gouv.fr/datasets/donnees-essentielles-de-la-commande-publique-consolidees-format-tabulaire) (Parquet, CSV), pensez à lire la description du jeu de données
+- en interrogeant l'[API REST ouverte](https://www.data.gouv.fr/datasets/donnees-essentielles-de-la-commande-publique-consolidees-format-tabulaire#user-content-api-rest)
+"""
+                        ),
+                        html.H4("Contact", id="contact"),
+                        dcc.Markdown(
+                            """
+- Email : [colin@colmo.tech](mailto:colin@colmo.tech)
+- Bluesky : [@col1m.bsky.social](https://bsky.app/profile/col1m.bsky.social)
+- Mastodon : [col1m@mamot.fr](https://mamot.fr/@col1m)
+- LinkedIn : [colinmaudry](https://www.linkedin.com/in/colinmaudry/)
+"""
+                        ),
+                        html.H4("Pour contribuer", id="contribuer"),
+                        dcc.Markdown(
+                            """
 - via l'achat d'une prestation de service (devis, prestation, facture), vous pouvez financer le développement de [fonctionnalités prévues](https://github.com/ColinMaudry/decp.info/issues), ou d'autres !
 - ma société accepte aussi les dons (pas de réduction d'impôt possible)
-- [écrivez-moi](/contact) et on discute !
+- écrivez-moi et on discute !
-
+"""
-#### Pour explorer le projet
+                        ),
-
+                        html.H4("Pour explorer le projet", id="explorer"),
+                        dcc.Markdown(
+                            """
 - ✉️  [inscription à la liste de diffusion](https://6254d9a3.sibforms.com/serve/MUIFAEonUVkoSVrdgey18CTgLyI16xw4yeu-M-YOUzhWE_AgfQfbgkyT7GvA_RYLro9MfuRqkzQxSvu7-uzbMSv2a2ZQPsliM7wtiiqIL8kR2zOvl6m11fb5qjcOxMAYsLiY_YBi3P7NY95CTJ8vRY4CpsDclF2iLooOElKkTgIgi5nePe7zAIrgiYM5v2EuALlGJZMEG9vBP-Cu) (annonces des mises à jour et évènements, maximum une fois par mois)
 - 💾  [données consolidées en Open Data](https://www.data.gouv.fr/datasets/donnees-essentielles-de-la-commande-publique-consolidees-format-tabulaire/)
 - 🗞️  [mon blog](https://colin.maudry.com)
@@ -53,34 +78,33 @@ les fonctionnalités actuelles de decp.info. Il est ainsi possible de rajouter
 - 🚰  code source
     - [de decp.info](https://github.com/ColinMaudry/decp.info)
     - [du traitement des données](https://github.com/ColinMaudry/decp-processing)
-    """),
+"""
-            html.H4("Qualité et exhaustivité des données", id="qualite-exhausitivite"),
+                        ),
-            dcc.Markdown("""Les données visibles sur ce site proviennent exclusivement de la publication de données ouvertes par les acheteurs publics ou en leur nom, régie par [l'arrêté du 22 décembre 2022](https://www.legifrance.gouv.fr/jorf/id/JORFTEXT000046850496). Leur qualité est donc principalement liée à la qualité de leur saisie par les agents publics, parfois peu aidé·es par la qualité des outils à leur disposition. Je pense que l'analyse de marchés individuels et le comptage de marchés sur des critères autres que financiers sont plutôt fiables. En revanche, certains montants de marché estimés à des valeurs farfelues ([1 euro](https://decp.info/marches/432766947000192025S01301), [1 milliard](https://decp.info/marches/2459004280001320210000000271)) faussent les calculs par aggrégation (sommes, moyennes, médianes) et donc la production de statistiques financières fiables. Acheteurs, acheteuses : s'il vous plaît, essayez d'estimer les montants des marchés publics attribués de manière plus précise.
+                        html.H4(
-
+                            "Qualité et exhaustivité des données",
-Quant à l'exhaustivité, je consolide toutes les sources de données exploitables que j'ai pu identifier (voir [statistiques](/statistiques)). Certains profils d'acheteurs ne publient pas leurs données malgré l'obligation réglementaire :
+                            id="qualite-exhausitivite",
-
+                        ),
-- marches-securises.fr
+                        dcc.Markdown(
-- klekoon.fr (ils y travaillent)
+                            """Les données visibles sur ce site proviennent exclusivement de la publication de données ouvertes par les acheteurs publics ou en leur nom, régie par [l'arrêté du 22 décembre 2022](https://www.legifrance.gouv.fr/jorf/id/JORFTEXT000046850496). Leur qualité est donc principalement liée à la qualité de leur saisie par les agents publics, parfois peu aidé·es par la qualité des outils à leur disposition. Je pense que l'analyse de marchés individuels et le comptage de marchés sur des critères autres que financiers sont plutôt fiables. En revanche, certains montants de marché estimés à des valeurs farfelues ([1 euro](https://decp.info/marches/432766947000192025S01301), [1 milliard](https://decp.info/marches/2459004280001320210000000271)) faussent les calculs par aggrégation (sommes, moyennes, médianes) et donc la production de statistiques financières fiables. Acheteurs, acheteuses : s'il vous plaît, essayez d'estimer les montants des marchés publics attribués de manière plus précise.
-
-**marches-publics.gouv.fr** (PLACE, [AIFE](https://aife.economie.gouv.fr)), la plateforme des marchés de l'État, ne publie plus de données depuis le 1er janvier 2024 et l'entrée en vigueur de l'arrêté. Nous n'avons donc plus de données sur les marchés publics passés par les ministères, ainsi que sur ceux passés par les acheteurs clients d'[achatpublic.com](https://www.achatpublic.com), qui s'appuie sur les moyens de publication de l'AIFE.
-
-**marches-publics.info** (AWS) publie ses données de manière assez sporadique depuis début 2023. Compte tenu de son poids dans le secteur, c'est assez dommageable pour la transparence des marchés publics.
-
-Au milieu de ces mauvaises nouvelles, je tiens à souligner la belle continuité de la publication par la DGFiP des données des marchés publics remontées via le [protocole PES](https://www.collectivites-locales.gouv.fr/finances-locales/le-protocole-dechange-standard-pes). Merci à leurs équipes."""),
-            html.H4("Sources de données ", id="qualite-exhausitivite"),
-            get_sources_tables(os.getenv("SOURCE_STATS_CSV_PATH")),
-            html.H4("Mentions légales", id="mentions-legales"),
-            dcc.Markdown("""
-##### Publication
 
+Quant à l'exhaustivité, je consolide toutes les sources de données exploitables que j'ai pu identifier (voir [ci-dessous](/bin.usr-is-merged/)). Je tiens à souligner la belle continuité de la publication par la DGFiP des données des marchés publics remontées via le [protocole PES](https://www.collectivites-locales.gouv.fr/finances-locales/le-protocole-dechange-standard-pes). Merci à leurs équipes."""
+                        ),
+                        html.H4("Sources de données ", id="sources"),
+                        get_sources_tables(os.getenv("SOURCE_STATS_CSV_PATH")),
+                        html.H4("Mentions légales", id="mentions-legales"),
+                        html.H5("Publication", id="publication"),
+                        dcc.Markdown(
+                            """
 Site Web développé et édité par [SAS Colmo](https://annuaire-entreprises.data.gouv.fr/entreprise/colmo-989393350), 989 393 350 RCS Rennes au capital de 3 000 euros.
 
 Siège social : 1 carrefour Jouaust, 35000 Rennes
 
 Hébergement : serveur situé en France et administré par Scaleway, 8 rue de la Ville l’Evêque, 75008 Paris
-
+"""
-##### Suivi d'audience
+                        ),
-
+                        html.H5("Suivi d'audience", id="audience"),
+                        dcc.Markdown(
+                            """
 Ce site dépose un petit fichier texte (un « cookie ») sur votre ordinateur lorsque vous le consultez ([Wikipédia](https://fr.wikipedia.org/wiki/Cookie_(informatique))). Cela me permet de mesurer le nombre de visites, de distinguer les nouveaux visiteurs des utilisateurs réguliers et ainsi de communiquer sur l'impact de decp.info.
 
 **Ce site n’affiche pas de bannière de consentement aux cookies, pourquoi ?**
@@ -89,21 +113,90 @@ C’est vrai, vous n’avez pas eu à cliquer sur un bloc qui recouvre la moiti
 
 Rien d’exceptionnel, je respecte simplement la loi, qui dit que certains outils de suivi d’audience, correctement configurés pour respecter la vie privée, sont exemptés d’autorisation préalable.
 
-J’utilise pour cela [Matomo](https://matomo.org/), un outil [libre](https://matomo.org/free-software/), paramétré pour être en conformité avec [la recommandation « Cookies »](https://www.cnil.fr/fr/solutions-pour-les-cookies-de-mesure-daudience) de la CNIL. Cela signifie que votre adresse IP, par exemple, est anonymisée avant d’être enregistrée. Il m’est donc impossible d’associer vos visites sur ce site à votre personne."""),
+J’utilise pour cela [Matomo](https://matomo.org/), un outil [libre](https://matomo.org/free-software/), paramétré pour être en conformité avec [la recommandation « Cookies »](https://www.cnil.fr/fr/solutions-pour-les-cookies-de-mesure-daudience) de la CNIL. Cela signifie que votre adresse IP, par exemple, est anonymisée avant d’être enregistrée. Il m’est donc impossible d’associer vos visites sur ce site à votre personne.
-            # Matomo propose cependant ce formulaire si vous souhaitez totalement désactiver le suivi de vos sessions sur ce site :"""),
+
-            #             html.Div(
+J'enregistre également les données suivantes, de manière anonyme, afin de mieux comprendre comment vous utilisez le site et l'améliorer :
-            #                 id="matomo-opt-out",
+
-            #                 style={
+- recherches sur la page d'accueil
-            #                     "border": "1pt solid lightgrey",
+- filtres appliqués aux données
-            #                     "padding": "12px",
+"""
-            #                     "margin": "auto 0 auto 12px",
+                        ),
-            #                     "width": "80%",
+                        html.H5("Attributions", id="attributions"),
-            #                 },
+                        dcc.Markdown("""
-            #                 children=["Vous utilisez un bloqueur de suivi de trafic."],
+                        Les polices de caractères sont distribuées par [Bunny fonts](https://fonts.bunny.net), une alternative européenne et qualitative à Google Fonts.
-            #             ),
+
-            #             html.Script(
+                        - la police de caractère [Inter](https://fonts.bunny.net/family/inter), principale police de ce site, a été créée par The Inter Project Authors ([source](https://github.com/rsms/inter))
-            #                 src="https://analytics.maudry.com/index.php?module=CoreAdminHome&action=optOutJS&divId=matomo-opt-out&language=auto&showIntro=1"
+                        - la police de caractère [Fira Code](https://fonts.bunny.net/family/fira-code), la police à largeure fixe, a été créée par The Fira Code Project Authors (https://github.com/tonsky/FiraCode)
-            #             ),
+                        """),
-        ],
+                        html.H4(
-    )
+                            "Liste des marchés par département", id="liste_marches"
-]
+                        ),
+                        dcc.Markdown(
+                            """
+                            - [Marchés par département](/departements)
+                            """
+                        ),
+                    ],
+                ),
+                # Table of Contents Column
+                html.Div(
+                    className="a-propos-toc",
+                    children=[
+                        html.Div(
+                            [
+                                html.A(
+                                    "Consommer les données brutes",
+                                    href="#donnees-brutes",
+                                    className="toc-link",
+                                ),
+                                html.A(
+                                    "Contact", href="#contact", className="toc-link"
+                                ),
+                                html.A(
+                                    "Pour contribuer",
+                                    href="#contribuer",
+                                    className="toc-link",
+                                ),
+                                html.A(
+                                    "Pour explorer le projet",
+                                    href="#explorer",
+                                    className="toc-link",
+                                ),
+                                html.A(
+                                    "Qualité et exhaustivité des données",
+                                    href="#qualite-exhausitivite",
+                                    className="toc-link",
+                                ),
+                                html.A(
+                                    "Sources de données",
+                                    href="#sources",
+                                    className="toc-link",
+                                ),
+                                html.A(
+                                    "Mentions légales",
+                                    href="#mentions-legales",
+                                    className="toc-link",
+                                ),
+                                html.A(
+                                    "Publication",
+                                    href="#publication",
+                                    className="toc-link toc-level-2",
+                                ),
+                                html.A(
+                                    "Suivi d'audience",
+                                    href="#audience",
+                                    className="toc-link toc-level-2",
+                                ),
+                                html.A(
+                                    "Attributions",
+                                    href="#attributions",
+                                    className="toc-link toc-level-2",
+                                ),
+                            ]
+                        ),
+                    ],
+                ),
+            ],
+        ),
+    ],
+)

src/pages/acheteur.py

@@ -1,99 +1,236 @@
 import datetime
+from typing import Any
 
+import dash_bootstrap_components as dbc
 import polars as pl
-from dash import Input, Output, State, callback, dash_table, dcc, html, register_page
+from dash import (
-
+    ClientsideFunction,
-from src.figures import point_on_map
+    Input,
-from src.utils import (
+    Output,
-    add_links_in_dict,
+    State,
-    df,
+    callback,
-    format_montant,
+    clientside_callback,
-    format_number,
+    dcc,
-    get_annuaire_data,
+    html,
-    get_departement_region,
+    register_page,
-    meta_content,
-    setup_table_columns,
 )
 
+from src.db import query_marches, schema
+from src.figures import (
+    DataTable,
+    get_distance_histogram,
+    get_top_org_table,
+    make_card,
+    make_column_picker,
+    point_on_map,
+)
+from src.utils.data import DF_ACHETEURS, get_annuaire_data, get_departement_region
+from src.utils.frontend import get_button_properties
+from src.utils.seo import META_CONTENT
+from src.utils.table import (
+    COLUMNS,
+    filter_table_data,
+    format_number,
+    get_default_hidden_columns,
+    prepare_table_data,
+    sort_table_data,
+)
+from src.utils.tracking import track_search
+
+
+def get_title(acheteur_id: str | None = None) -> str:
+    acheteur_nom = DF_ACHETEURS.filter(pl.col("acheteur_id") == acheteur_id).select(
+        "acheteur_nom"
+    )
+    if acheteur_nom.height > 0:
+        return f"Marchés publics attribués par {acheteur_nom.item(0, 0)} | decp.info"
+    return "Marchés publics attribués | decp.info"
+
+
 register_page(
     __name__,
     path_template="/acheteurs/<acheteur_id>",
-    title=meta_content["title"],
+    title=get_title,
     name="Acheteur",
-    description=meta_content["description"],
+    description="Consultez les marchés publics attribués par cet acheteur.",
-    image_url=meta_content["image_url"],
+    image_url=META_CONTENT["image_url"],
     order=5,
 )
 
+DATATABLE = html.Div(
+    className="marches_table",
+    children=DataTable(
+        dtid="acheteur_datatable",
+        persistence=True,
+        persistence_type="local",
+        persisted_props=["filter_query", "sort_by"],
+        page_action="custom",
+        filter_action="custom",
+        sort_action="custom",
+        page_size=10,
+        hidden_columns=[],
+        columns=[{"id": col, "name": col} for col in schema.names()],
+    ),
+)
+
 layout = [
     dcc.Store(id="acheteur_data", storage_type="memory"),
-    dcc.Location(id="url", refresh="callback-nav"),
+    dcc.Store(id="acheteur-hidden-columns", storage_type="local"),
+    dcc.Store(id="filter-cleanup-trigger-acheteur"),
+    dcc.Location(id="acheteur_url", refresh="callback-nav"),
     html.Div(
-        className="container",
         children=[
             html.Div(
-                className="wrapper",
+                style={"marginBottom": "50px"},
                 children=[
-                    html.H2(
+                    dbc.Row(
-                        className="org_title",
+                        className="mb-2",
                         children=[
-                            html.Span(id="acheteur_siret"),
+                            dbc.Col(
-                            " - ",
+                                html.H2(
-                            html.Span(id="acheteur_nom"),
+                                    children=[
-                        ],
+                                        html.Span(id="acheteur_siret"),
-                    ),
+                                        " - ",
-                    html.Div(
+                                        html.Span(id="acheteur_nom"),
-                        className="org_year",
+                                    ],
-                        children=dcc.Dropdown(
+                                ),
-                            id="acheteur_year",
+                                width=8,
-                            options=["Toutes"]
-                            + [
-                                str(year)
-                                for year in range(
-                                    2018, int(datetime.date.today().year) + 1
-                                )
-                            ],
-                            placeholder="Année",
-                        ),
-                    ),
-                    html.Div(
-                        className="org_infos",
-                        children=[
-                            # TODO: ajouter le type d'acheteur : commune, CD, CR, etc.
-                            html.P(["Commune : ", html.Strong(id="acheteur_commune")]),
-                            html.P(
-                                [
-                                    "Département : ",
-                                    html.Strong(id="acheteur_departement"),
-                                ]
                             ),
-                            html.P(["Région : ", html.Strong(id="acheteur_region")]),
+                            dbc.Col(
-                            html.A(
+                                dcc.Dropdown(
-                                id="acheteur_lien_annuaire",
+                                    id="acheteur_year",
-                                children="Plus de détails sur l'Annuaire des entreprises",
+                                    options=["Toutes les années"]
-                                target="_blank",
+                                    + [
+                                        str(year)
+                                        for year in range(
+                                            2018, int(datetime.date.today().year) + 1
+                                        )
+                                    ],
+                                    placeholder="Année",
+                                ),
+                                width=4,
                             ),
                         ],
                     ),
-                    html.Div(
+                    dbc.Row(
-                        className="org_stats",
+                        className="mb-2",
                         children=[
-                            html.P(id="acheteur_titre_stats"),
+                            dbc.Col(
-                            html.P(id="acheteur_marches_attribues"),
+                                className="org_infos",
-                            html.P(id="acheteur_titulaires_differents"),
+                                children=[
-                            html.Button(
+                                    # TODO: ajouter le type d'acheteur : commune, CD, CR, etc.
-                                "Téléchargement au format Excel",
+                                    html.P(
-                                id="btn-download-acheteur-data",
+                                        [
+                                            "Commune : ",
+                                            html.Strong(id="acheteur_commune"),
+                                        ]
+                                    ),
+                                    html.P(
+                                        [
+                                            "Département : ",
+                                            html.Strong(id="acheteur_departement"),
+                                        ]
+                                    ),
+                                    html.P(
+                                        ["Région : ", html.Strong(id="acheteur_region")]
+                                    ),
+                                    html.A(
+                                        id="acheteur_lien_annuaire",
+                                        children="Plus de détails sur l'Annuaire des entreprises",
+                                    ),
+                                ],
+                                width=4,
+                            ),
+                            dbc.Col(
+                                children=[
+                                    html.P(id="acheteur_titre_stats"),
+                                    html.P(id="acheteur_marches_attribues"),
+                                    html.P(id="acheteur_titulaires_differents"),
+                                    html.Button(
+                                        "Téléchargement au format Excel",
+                                        id="btn-download-data-acheteur",
+                                        className="btn btn-primary",
+                                    ),
+                                    dcc.Download(id="download-data-acheteur"),
+                                ],
+                                width=4,
+                            ),
+                            dbc.Col(
+                                id="acheteur_map",
+                                width=4,
                             ),
-                            dcc.Download(id="download-acheteur-data"),
                         ],
                     ),
-                    html.Div(className="org_map", id="acheteur_map"),
+                    dbc.Row(
+                        children=[
+                            dbc.Col(
+                                className="marches_table",
+                                id="top10_titulaires",
+                                width=8,
+                            ),
+                            dbc.Col(id="acheteur-distance-histogram", width=4),
+                        ],
+                    ),
                 ],
             ),
             # récupérer les données de l'acheteur sur l'api annuaire
             html.H3("Derniers marchés publics attribués"),
-            html.Div(id="acheteur_last_marches", children=""),
+            dcc.Loading(
+                overlay_style={"visibility": "visible", "filter": "blur(2px)"},
+                id="loading-home",
+                type="default",
+                children=[
+                    html.Div(
+                        [
+                            # Bouton modal des colonnes affichées
+                            dbc.Button(
+                                "Colonnes affichées",
+                                id="acheteur_columns_open",
+                                className="column_list",
+                            ),
+                            html.P("lignes", id="acheteur_nb_rows"),
+                            html.Button(
+                                "Téléchargement désactivé au-delà de 65 000 lignes",
+                                id="btn-download-filtered-data-acheteur",
+                                className="btn btn-primary",
+                                disabled=True,
+                            ),
+                            dcc.Download(id="acheteur-download-filtered-data"),
+                            dbc.Button(
+                                "Remise à zéro",
+                                title="Supprime tous les filtres et les tris. Autrement ils sont conservés même si vous fermez la page.",
+                                id="btn-acheteur-reset",
+                            ),
+                        ],
+                        className="table-menu",
+                    ),
+                    dbc.Modal(
+                        [
+                            dbc.ModalHeader(
+                                dbc.ModalTitle("Choix des colonnes à afficher")
+                            ),
+                            dbc.ModalBody(
+                                id="acheteur_columns_body",
+                                children=make_column_picker("acheteur"),
+                            ),
+                            dbc.ModalFooter(
+                                dbc.Button(
+                                    "Fermer",
+                                    id="acheteur_columns_close",
+                                    className="ms-auto",
+                                    n_clicks=0,
+                                )
+                            ),
+                        ],
+                        id="acheteur_columns",
+                        is_open=False,
+                        fullscreen="md-down",
+                        scrollable=True,
+                        size="xl",
+                    ),
+                    DATATABLE,
+                ],
+            ),
         ],
     ),
 ]
@@ -107,30 +244,51 @@ layout = [
     Output(component_id="acheteur_departement", component_property="children"),
     Output(component_id="acheteur_region", component_property="children"),
     Output(component_id="acheteur_lien_annuaire", component_property="href"),
-    Input(component_id="url", component_property="pathname"),
+    Input(component_id="acheteur_url", component_property="pathname"),
 )
 def update_acheteur_infos(url):
     acheteur_siret = url.split("/")[-1]
-    if len(acheteur_siret) != 14:
+    # if len(acheteur_siret) != 14:
-        acheteur_siret = (
+    #     acheteur_siret = (
-            f"Le SIRET renseigné doit faire 14 caractères ({acheteur_siret})"
+    #         f"Le SIRET renseigné doit faire 14 caractères ({acheteur_siret})"
-        )
+    #     )
     data = get_annuaire_data(acheteur_siret)
-    data_etablissement = data["matching_etablissements"][0]
+    data_etablissement = data.get("matching_etablissements") if data else None
-    acheteur_map = point_on_map(
+    if data_etablissement:
-        data_etablissement["latitude"], data_etablissement["longitude"]
+        data_etablissement = data_etablissement[0]
-    )
+
-    code_departement, nom_departement, nom_region = get_departement_region(
+        # Extraction du code département à partir du code postal
-        data_etablissement["code_postal"]
+        code_postal = data_etablissement.get("code_postal", "")
-    )
+        departement_code = code_postal[:2] if code_postal else None
-    departement = f"{nom_departement} ({code_departement})"
+
-    lien_annuaire = (
+        # Création de la carte avec le code département pour un centrage approprié
-        f"https://annuaire-entreprises.data.gouv.fr/etablissement/{acheteur_siret}"
+        acheteur_map = point_on_map(
-    )
+            data_etablissement["latitude"],
+            data_etablissement["longitude"],
+            departement_code,
+        )
+        code_departement, nom_departement, nom_region = get_departement_region(
+            data_etablissement["code_postal"]
+        )
+        departement = f"{nom_departement} ({code_departement})"
+        lien_annuaire = (
+            f"https://annuaire-entreprises.data.gouv.fr/etablissement/{acheteur_siret}"
+        )
+        raison_sociale = data["nom_raison_sociale"]
+        libelle_commune = data_etablissement["libelle_commune"]
+
+    else:
+        acheteur_map = html.Div()
+        code_departement, nom_departement, nom_region = "", "", ""
+        departement = ""
+        lien_annuaire = ""
+        raison_sociale = ""
+        libelle_commune = ""
+
     return (
         acheteur_siret,
-        data["nom_raison_sociale"],
+        raison_sociale,
-        data_etablissement["libelle_commune"],
+        libelle_commune,
         acheteur_map,
         departement,
         nom_region,
@@ -146,114 +304,85 @@ def update_acheteur_infos(url):
     Input(component_id="acheteur_data", component_property="data"),
 )
 def update_acheteur_stats(data):
-    df = pl.DataFrame(data)
+    dff = pl.DataFrame(data, strict=False, infer_schema_length=5000)
-    if df.height == 0:
+    if dff.height == 0:
-        df = pl.DataFrame(schema=df.collect_schema())
+        dff = pl.DataFrame(schema=schema)
-    df_marches = df.unique("id")
+    df_marches = dff.unique("id")
     nb_marches = format_number(df_marches.height)
     # somme_marches = format_number(int(df_marches.select(pl.sum("montant")).item()))
     marches_attribues = [html.Strong(nb_marches), " marchés et accord-cadres attribués"]
     # + ", pour un total de ", html.Strong(somme_marches + " €")]
     del df_marches
 
-    nb_titulaires = df.unique("titulaire_id").height
+    nb_titulaires = dff.unique("titulaire_id").height
     nb_titulaires = [
         html.Strong(format_number(nb_titulaires)),
         " titulaires (SIRET) différents",
     ]
-    del df
+    del dff
 
     return marches_attribues, nb_titulaires
 
 
 @callback(
     Output(component_id="acheteur_data", component_property="data"),
-    Input(component_id="url", component_property="pathname"),
+    Output("btn-download-data-acheteur", "disabled"),
+    Output("btn-download-data-acheteur", "children"),
+    Output("btn-download-data-acheteur", "title"),
+    Input(component_id="acheteur_url", component_property="pathname"),
     Input(component_id="acheteur_year", component_property="value"),
 )
-def get_acheteur_marches_data(url, acheteur_year: str) -> list[dict]:
+def get_acheteur_marches_data(url, ach_year: str) -> tuple:
     acheteur_siret = url.split("/")[-1]
-    lff = df.lazy()
+    lff = query_marches("acheteur_id = ?", (acheteur_siret,)).lazy()
-    lff = lff.filter(pl.col("acheteur_id") == acheteur_siret)
+    if ach_year and ach_year != "Toutes les années":
-    lff = lff.fill_null("")
+        ach_year = int(ach_year)
-    lff = lff.select(
+        lff = lff.filter(pl.col("dateNotification").dt.year() == ach_year)
-        "id",
+    lff = lff.sort(["dateNotification", "uid"], descending=True, nulls_last=True)
-        "uid",
+    dff: pl.DataFrame = lff.collect(engine="streaming")
-        "objet",
+    download_disabled, download_text, download_title = get_button_properties(dff.height)
-        "dateNotification",
+    data = dff.to_dicts()
-        "titulaire_id",
+    return data, download_disabled, download_text, download_title
-        "titulaire_typeIdentifiant",
-        "titulaire_nom",
-        "montant",
-        "codeCPV",
-        "dureeMois",
-    )
-    if acheteur_year and acheteur_year != "Toutes":
-        lff = lff.filter(
-            pl.col("dateNotification").cast(pl.String).str.starts_with(acheteur_year)
-        )
-    lff = lff.sort(["dateNotification", "id"], descending=True, nulls_last=True)
-
-    data = lff.collect(engine="streaming").to_dicts()
-    return data
 
 
 @callback(
-    Output(component_id="acheteur_last_marches", component_property="children"),
+    Output("acheteur_datatable", "data"),
+    Output("acheteur_datatable", "columns"),
+    Output("acheteur_datatable", "tooltip_header"),
+    Output("acheteur_datatable", "data_timestamp"),
+    Output("acheteur_nb_rows", "children"),
+    Output("btn-download-filtered-data-acheteur", "disabled"),
+    Output("btn-download-filtered-data-acheteur", "children"),
+    Output("btn-download-filtered-data-acheteur", "title"),
+    Output("filter-cleanup-trigger-acheteur", "data"),
+    Input("acheteur_url", "href"),
+    Input("acheteur_data", "data"),
+    Input("acheteur_datatable", "page_current"),
+    Input("acheteur_datatable", "page_size"),
+    Input("acheteur_datatable", "filter_query"),
+    Input("acheteur_datatable", "sort_by"),
+    State("acheteur_datatable", "data_timestamp"),
+)
+def get_last_marches_data(
+    href, data, page_current, page_size, filter_query, sort_by, data_timestamp
+) -> tuple:
+    return prepare_table_data(
+        data, data_timestamp, filter_query, page_current, page_size, sort_by, "acheteur"
+    )
+
+
+@callback(
+    Output(component_id="top10_titulaires", component_property="children"),
     Input(component_id="acheteur_data", component_property="data"),
 )
-def get_last_marches_table(data) -> html.Div:
+def get_top_titulaires(data):
-    dff = pl.DataFrame(data)
+    table = get_top_org_table(data, "titulaire", ["titulaire_distance"])
-    dff = format_montant(dff)
+    return make_card(fig=table, title="Top titulaires", lg=12, xl=12)
-    columns, tooltip = setup_table_columns(
-        dff,
-        hideable=False,
-        exclude=["titulaire_id", "titulaire_typeIdentifiant", "uid"],
-    )
-    data = dff.to_dicts()
-    data = add_links_in_dict(data, "titulaire")
-
-    table = html.Div(
-        className="marches_table",
-        id="acheteur_datatable",
-        children=dash_table.DataTable(
-            data=data,
-            markdown_options={"html": True},
-            page_action="native",
-            filter_action="native",
-            filter_options={"case": "insensitive", "placeholder_text": "Filtrer..."},
-            columns=columns,
-            tooltip_header=tooltip,
-            tooltip_duration=8000,
-            tooltip_delay=350,
-            cell_selectable=False,
-            page_size=10,
-            style_cell_conditional=[
-                {
-                    "if": {"column_id": "objet"},
-                    "minWidth": "300px",
-                    "textAlign": "left",
-                    "overflow": "hidden",
-                    "lineHeight": "14px",
-                    "whiteSpace": "normal",
-                },
-                {
-                    "if": {"column_id": "titulaire_nom"},
-                    "minWidth": "200px",
-                    "textAlign": "left",
-                    "overflow": "hidden",
-                    "lineHeight": "18px",
-                    "whiteSpace": "normal",
-                },
-            ],
-        ),
-    )
-    return table
 
 
 @callback(
-    Output("download-acheteur-data", "data"),
+    Output("download-data-acheteur", "data"),
-    Input("btn-download-acheteur-data", "n_clicks"),
+    Input("btn-download-data-acheteur", "n_clicks"),
     State(component_id="acheteur_data", component_property="data"),
     State(component_id="acheteur_nom", component_property="children"),
     State(component_id="acheteur_year", component_property="value"),
@@ -261,7 +390,7 @@ def get_last_marches_table(data) -> html.Div:
 )
 def download_acheteur_data(
     n_clicks,
-    data: [dict],
+    data: list[dict[str, Any]],
     acheteur_nom: str,
     annee: str,
 ):
@@ -269,8 +398,141 @@ def download_acheteur_data(
 
     def to_bytes(buffer):
         df_to_download.write_excel(
-            buffer, worksheet="DECP" if annee in ["Toutes", None] else annee
+            buffer, worksheet="DECP" if annee in ["Toutes les années", None] else annee
         )
 
     date = datetime.datetime.now().strftime("%Y-%m-%d_%H:%M:%S")
     return dcc.send_bytes(to_bytes, filename=f"decp_{acheteur_nom}_{date}.xlsx")
+
+
+@callback(
+    Output("acheteur-download-filtered-data", "data"),
+    State("acheteur_data", "data"),
+    Input("btn-download-filtered-data-acheteur", "n_clicks"),
+    State("acheteur_nom", "children"),
+    State("acheteur_datatable", "filter_query"),
+    State("acheteur_datatable", "sort_by"),
+    State("acheteur_datatable", "hidden_columns"),
+    prevent_initial_call=True,
+)
+def download_filtered_acheteur_data(
+    data,
+    n_clicks,
+    acheteur_nom,
+    filter_query,
+    sort_by,
+    hidden_columns: list | None = None,
+):
+    lff: pl.LazyFrame = pl.LazyFrame(
+        data
+    )  # start from the full acheteur data, not from paginated table data
+
+    # Les colonnes masquées sont supprimées
+    if hidden_columns:
+        lff = lff.drop(hidden_columns)
+
+    if filter_query:
+        track_search(filter_query, "ach download")
+        lff = filter_table_data(lff, filter_query)
+
+    if len(sort_by) > 0:
+        lff = sort_table_data(lff, sort_by)
+
+    def to_bytes(buffer):
+        lff.collect(engine="streaming").write_excel(buffer, worksheet="DECP")
+
+    date = datetime.datetime.now().strftime("%Y-%m-%d_%H:%M:%S")
+    return dcc.send_bytes(
+        to_bytes, filename=f"decp_filtrées_{acheteur_nom}_{date}.xlsx"
+    )
+
+
+# Pour nettoyer les icontains et i< des filtres
+# voir aussi src/assets/dash_clientside.js
+clientside_callback(
+    ClientsideFunction(
+        namespace="clientside",
+        function_name="clean_filters",
+    ),
+    Output("filter-cleanup-trigger-acheteur", "data", allow_duplicate=True),
+    Input("filter-cleanup-trigger-acheteur", "data"),
+    prevent_initial_call=True,
+)
+
+
+@callback(
+    Output("acheteur-hidden-columns", "data", allow_duplicate=True),
+    Input("acheteur_column_list", "selected_rows"),
+    prevent_initial_call=True,
+)
+def update_hidden_columns_from_checkboxes(selected_columns):
+    if selected_columns:
+        selected_columns = [COLUMNS[i] for i in selected_columns]
+        hidden_columns = [col for col in COLUMNS if col not in selected_columns]
+        return hidden_columns
+    else:
+        return []
+
+
+@callback(
+    Output("acheteur_datatable", "hidden_columns"),
+    Input(
+        "acheteur-hidden-columns",
+        "data",
+    ),
+)
+def store_hidden_columns(hidden_columns):
+    if hidden_columns is None:
+        hidden_columns = get_default_hidden_columns("acheteur")
+    return hidden_columns
+
+
+@callback(
+    Output("acheteur_column_list", "selected_rows"),
+    Input("acheteur_datatable", "hidden_columns"),
+    State("acheteur_column_list", "selected_rows"),  # pour éviter la boucle infinie
+)
+def update_checkboxes_from_hidden_columns(hidden_cols, current_checkboxes):
+    hidden_cols = hidden_cols or get_default_hidden_columns("acheteur")
+
+    # Show all columns that are NOT hidden
+    visible_cols = [COLUMNS.index(col) for col in COLUMNS if col not in hidden_cols]
+    return visible_cols
+
+
+@callback(
+    Output("acheteur_columns", "is_open"),
+    Input("acheteur_columns_open", "n_clicks"),
+    Input("acheteur_columns_close", "n_clicks"),
+    State("acheteur_columns", "is_open"),
+)
+def toggle_acheteur_columns(click_open, click_close, is_open):
+    if click_open or click_close:
+        return not is_open
+    return is_open
+
+
+@callback(
+    Output("acheteur_datatable", "filter_query", allow_duplicate=True),
+    Output("acheteur_datatable", "sort_by"),
+    Input("btn-acheteur-reset", "n_clicks"),
+    prevent_initial_call=True,
+)
+def reset_view(n_clicks):
+    return "", []
+
+
+@callback(
+    Output("acheteur-distance-histogram", "children"),
+    Input("acheteur_data", "data"),
+)
+def update_acheteur_distance_histogram(data):
+    lff = pl.LazyFrame(data, strict=False, infer_schema_length=5000)
+    fig = get_distance_histogram(lff)
+    return make_card(
+        title="Distance acheteur–titulaire",
+        subtitle="en nombre de marchés, échelle logarithmique",
+        fig=fig,
+        lg=12,
+        xl=12,
+    )

src/pages/arbre/departement.py

@@ -0,0 +1,91 @@
+from dash import Input, Output, callback, dcc, html, register_page
+
+from src.db import get_cursor
+from src.utils.data import DEPARTEMENTS
+
+NAME = "Département"
+
+
+def get_title(code):
+    return f"Marchés publics de {DEPARTEMENTS[code]['departement']} | decp.info"
+
+
+def get_description(code):
+    return f"Marchés publics passés dans le département {DEPARTEMENTS[code]['departement']} | decp.info"
+
+
+register_page(
+    __name__,
+    path_template="/departements/<code>",
+    title=get_title,
+    description=get_description,
+    order=50,
+    name=NAME,
+)
+
+layout = html.Div(
+    [
+        dcc.Location(id="departement_url", refresh="callback-nav"),
+        html.Div(id="departement_marches"),
+    ]
+)
+
+
+@callback(
+    Output(component_id="departement_marches", component_property="children"),
+    Input(component_id="departement_url", component_property="pathname"),
+)
+def departement_marches(url):
+    departement = url.split("/")[-1]
+
+    def make_link_list(org_type) -> list:
+        table = (
+            "acheteurs_departement"
+            if org_type == "acheteur"
+            else "titulaires_departement"
+            if org_type == "titulaire"
+            else None
+        )
+        if table is None:
+            raise ValueError
+        col_prefix = org_type
+        rows = (
+            get_cursor()
+            .execute(
+                f"SELECT {col_prefix}_id, {col_prefix}_nom "
+                f"FROM {table} "
+                f"WHERE {col_prefix}_departement_code = ? "
+                f"ORDER BY {col_prefix}_nom",
+                [departement],
+            )
+            .fetchall()
+        )
+
+        link_list = []
+        for org_id, org_nom in rows:
+            li = html.Li(
+                [
+                    dcc.Link(
+                        org_nom,
+                        href=url + f"/{org_type}/{org_id}",
+                        title=f"Marchés publics de {org_nom}",
+                    ),
+                    " ",
+                    dcc.Link(
+                        "(page dédiée)",
+                        href=f"/{org_type}s/{org_id}",
+                        title=f"Page dédiée aux marchés publics de {org_nom}",
+                    ),
+                ]
+            )
+            link_list.append(li)
+        return link_list
+
+    content = [
+        html.H3("Acheteurs publics du département"),
+        html.Ul(make_link_list("acheteur")),
+        html.H3("Titulaires du département"),
+        html.Ul(make_link_list("titulaire")),
+    ]
+
+    return content

src/pages/arbre/departements.py

@@ -0,0 +1,25 @@
+from dash import dcc, html, register_page
+
+from src.utils.data import DEPARTEMENTS
+
+NAME = "Départements"
+
+register_page(
+    __name__,
+    path="/departements",
+    title="Marchés par département | decp.info",
+    name="Départements",
+    description="Tous les marchés publics, classés par départements",
+)
+
+layout = html.Div(
+    [
+        html.H3("Départements"),
+        html.Ul(
+            [
+                html.Li(dcc.Link(d["departement"], href=f"/departements/{k}"))
+                for k, d in DEPARTEMENTS.items()
+            ]
+        ),
+    ]
+)

src/pages/arbre/liste_marches_org.py

@@ -0,0 +1,103 @@
+import polars as pl
+from dash import Input, Output, callback, dcc, html, register_page
+
+from src.db import get_cursor
+from src.utils.data import DF_ACHETEURS, DF_TITULAIRES
+
+NAME = "Liste des marchés publics"
+
+
+def make_org_nom_verbe(org_type, org_id) -> tuple:
+    if org_type == "titulaire":
+        df = DF_TITULAIRES
+        verbe = "remportés"
+    elif org_type == "acheteur":
+        df = DF_ACHETEURS
+        verbe = "attribués"
+    else:
+        raise ValueError
+
+    org_nom = (
+        df.filter(pl.col(f"{org_type}_id") == org_id)
+        .select(f"{org_type}_nom")
+        .item(0, 0)
+    )
+
+    return org_nom, verbe
+
+
+def get_title(code, org_type, org_id):
+    org_nom, verbe = make_org_nom_verbe(org_type, org_id)
+
+    return f"Marchés publics {verbe} par {org_nom} | decp.info"
+
+
+def get_description(code, org_type, org_id):
+    org_nom, verbe = make_org_nom_verbe(org_type, org_id)
+
+    return f"Liste complète des marchés publics {verbe} par {org_nom} et publiés par decp.info. Cliquez sur les liens pour consulter les détails de chaque marché."
+
+
+register_page(
+    __name__,
+    path_template="/departements/<code>/<org_type>/<org_id>",
+    title=get_title,
+    description=get_description,
+    order=40,
+    name=NAME,
+)
+
+layout = html.Div(
+    [
+        dcc.Location(id="liste_marches_url", refresh="callback-nav"),
+        html.Div(id="liste_marches"),
+    ]
+)
+
+
+@callback(
+    Output(component_id="liste_marches", component_property="children"),
+    Input(component_id="liste_marches_url", component_property="pathname"),
+)
+def liste_marches(url):
+    org_type = url.split("/")[-2]
+    org_id = url.split("/")[-1]
+
+    def make_link_list() -> list:
+        table = (
+            "acheteurs_marches"
+            if org_type == "acheteur"
+            else "titulaires_marches"
+            if org_type == "titulaire"
+            else None
+        )
+        if table is None:
+            raise ValueError
+        rows = (
+            get_cursor()
+            .execute(
+                f"SELECT uid, objet FROM {table} WHERE {org_type}_id = ?",
+                [org_id],
+            )
+            .fetchall()
+        )
+
+        return [
+            html.Li(
+                dcc.Link(
+                    objet,
+                    href=f"/marches/{uid}",
+                    title=f"Marchés public attribué : {objet}",
+                )
+            )
+            for uid, objet in rows
+        ]
+
+    nom, verbe = make_org_nom_verbe(org_type, org_id)
+
+    content = [
+        html.H3(f"Marchés publics {verbe} par {nom}"),
+        html.Ul(make_link_list()),
+    ]
+
+    return content

src/pages/contact.py

@@ -1,170 +0,0 @@
-import os
-import re
-import smtplib
-import time
-from email.mime.multipart import MIMEMultipart
-from email.mime.text import MIMEText
-
-from dash import Input, Output, State, callback, dcc, html, register_page
-from flask import request
-
-from src.utils import meta_content
-
-name = "Contact"
-register_page(
-    __name__,
-    path="/contact",
-    title=meta_content["title"],
-    name=name,
-    description=meta_content["description"],
-    image_url=meta_content["image_url"],
-    order=6,
-)
-
-layout = html.Div(
-    className="container",
-    children=[
-        html.H2("Contact", id="contact"),
-        html.P(
-            "Votre message arrivera directement dans ma boîte mail, et je reviendrai vers vous rapidement."
-        ),
-        html.Div(
-            [
-                html.Label("Votre nom"),
-                dcc.Input(
-                    id="input-name",
-                    type="text",
-                    style={"width": "100%", "padding": "8px", "margin": "8px 0"},
-                ),
-                html.Label("Votre adresse email"),
-                dcc.Input(
-                    id="input-email",
-                    type="email",
-                    style={"width": "100%", "padding": "8px", "margin": "8px 0"},
-                ),
-                html.Label("Message"),
-                dcc.Textarea(
-                    id="input-message",
-                    style={
-                        "width": "100%",
-                        "height": 200,
-                        "padding": "8px",
-                        "margin": "8px 0",
-                    },
-                ),
-                html.Button(
-                    "Envoyer    ",
-                    id="submit-button",
-                    n_clicks=0,
-                    style={"marginTop": "10px"},
-                ),
-                html.Div(
-                    id="output-message", style={"marginTop": "10px", "color": "green"}
-                ),
-            ],
-            style={
-                "maxWidth": "600px",
-                "margin": "auto",
-                "padding": "20px",
-                "lineHeight": "20px",
-            },
-        ),
-        dcc.Markdown("""
-- Bluesky : [@col1m.bsky.social](https://bsky.app/profile/col1m.bsky.social)
-- Mastodon : [col1m@mamot.fr](https://mamot.fr/@col1m)
-- LinkedIn : [colinmaudry](https://www.linkedin.com/in/colinmaudry/)
-- venez discuter de la transparence de la commande publique [sur le forum teamopendata.org](https://teamopendata.org/c/commande-publique/101)
-"""),
-    ],
-)
-
-rate_limit_store = {}  # {ip: last_timestamp}
-RATE_LIMIT_WINDOW = 300  # 5 minutes
-
-
-def is_rate_limited():
-    ip = request.remote_addr
-    now = time.time()
-    last_time = rate_limit_store.get(ip)
-
-    if last_time and (now - last_time) < RATE_LIMIT_WINDOW:
-        return True  # rate limited !
-    rate_limit_store[ip] = now  # màj du timestamp
-    return False
-
-
-def sanitize_email(email):
-    return re.match(r"[^@]+@[^@]+\.[^@]+", email)
-
-
-@callback(
-    Output("output-message", "children"),
-    Input("submit-button", "n_clicks"),
-    State("input-name", "value"),
-    State("input-email", "value"),
-    State("input-message", "value"),
-    prevent_initial_call=True,
-)
-def send_email(n_clicks, form_name, form_email, form_message):
-    if not all([form_name, form_email, form_message]):
-        return html.Div(
-            "Veuillez s'il vous plaît remplir tous les champs.", style={"color": "red"}
-        )
-
-    client_ip = request.remote_addr
-    if is_rate_limited():
-        wait_time = int(RATE_LIMIT_WINDOW - (time.time() - rate_limit_store[client_ip]))
-        return html.Div(
-            f"⏳ J'ai mis en place une protection contre le spam, et vous m'avez écrit il y a moins de 5 minutes. "
-            f"Veuillez s'il vous plaît attendre {wait_time} secondes avant de renvoyer un message.",
-            style={"color": "black"},
-        )
-
-    try:
-        # Configuration du serveur SMTP
-        smtp_server = os.getenv("SENDER_SERVER_DOMAIN")
-        smtp_port = 587
-        login_email = os.getenv("LOGIN_EMAIL")
-        from_email = os.getenv("FROM_EMAIL")
-        to_email = os.getenv("TO_EMAIL", from_email)
-        login_password = os.getenv("LOGIN_PASSWORD")
-
-        print(
-            smtp_server,
-            smtp_port,
-            login_email,
-            from_email,
-            to_email,
-            login_password,
-            sep="\n",
-        )
-
-        # Création de l'email
-        email = MIMEMultipart()
-        email["From"] = from_email
-        email["To"] = to_email
-        email["Subject"] = f"[decp.info] Message de {form_name}"
-
-        body = f"""
-        Nom : {form_name}
-        Email : {form_email}
-
-        Message :
-        {form_message}
-        """
-        email.attach(MIMEText(body, "plain"))
-
-        # Send email
-        server = smtplib.SMTP(smtp_server, smtp_port)
-        server.starttls()
-        server.login(login_email, login_password)
-        server.sendmail(from_email, to_email, email.as_string())
-        server.quit()
-
-        return html.Div("✅ Envoi réussi", style={"color": "green"})
-
-    except Exception as e:
-        print(e)
-        return html.Div(
-            f"❌ Échec de l'envoi du message : {str(e)}", style={"color": "red"}
-        )

src/pages/marche.py

@@ -1,56 +1,74 @@
+import json
 from datetime import datetime
 
-import polars as pl
+import dash_bootstrap_components as dbc
 from dash import Input, Output, callback, dcc, html, register_page
 from polars import selectors as cs
 
-from src.utils import data_schema, df, format_montant, meta_content
+from src.db import query_marches
+from src.utils.data import DATA_SCHEMA
+from src.utils.seo import META_CONTENT, make_org_jsonld
+from src.utils.table import format_values, unformat_montant
+
+
+def get_title(uid: str = None) -> str:
+    return f"Marché {uid} | decp.info"
+
 
 register_page(
     __name__,
     path_template="/marches/<uid>",
-    title=meta_content["title"],
+    title=get_title,
     name="Marché",
-    description=meta_content["description"],
+    description="Consultez les détails de ce marché public : montant, acheteur, titulaires, modifications, etc.",
-    image_url=meta_content["image_url"],
+    image_url=META_CONTENT["image_url"],
     order=7,
 )
 
 layout = [
     dcc.Store(id="marche_data"),
     dcc.Store(id="titulaires_data"),
-    dcc.Location(id="url", refresh="callback-nav"),
+    dcc.Location(id="marche_url", refresh="callback-nav"),
-    html.Div(
+    html.Script(type="application/ld+json", id="marche_jsonld"),
-        className="container marche_infos",
+    dbc.Container(
+        className="marche_infos",
         children=[
-            html.P("Vous consultez un résumé des données de ce marché public"),
+            dbc.Row(
-            html.Ul(
+                dbc.Col(
+                    [
+                        html.H1(id="marche_objet", style={"fontSize": "1.5em"}),
+                        html.P(
+                            "Vous consultez un résumé des données de ce marché public"
+                        ),
+                        html.Ul(
+                            [
+                                html.Li(
+                                    "après son attribution aux titulaires qui l'ont remporté à la suite d'un appel d'offres (ou sans appel d'offres via une attribution directe)"
+                                ),
+                                html.Li(
+                                    "après avoir appliqué les éventuelles modifications de montant, durée ou titulaires renseignées par l'acheteur"
+                                ),
+                            ]
+                        ),
+                        html.P(
+                            "Le montant total payé aux titulaires, la durée du marché et la liste des titulaires peuvent cependant encore évoluer jusqu'à la fin de l'exécution du marché."
+                        ),
+                    ]
+                )
+            ),
+            dbc.Row(
                 [
-                    html.Li(
+                    dbc.Col(id="marche_infos_1", width=12, md=4),
-                        "après son attribution aux titulaires qui l'ont remporté à la suite d'un appel d'offres (ou sans appel d'offres via une attribution directe)"
+                    dbc.Col(id="marche_infos_2", width=12, md=4),
-                    ),
+                    dbc.Col(
-                    html.Li(
+                        width=12,
-                        "après avoir appliqué les éventuelles modifications de montant, durée ou titulaires renseignées par l'acheteur"
+                        md=4,
-                    ),
-                ]
-            ),
-            html.P(
-                "Le montant total payé aux titulaires, la durée du marché et la liste des titulaires peuvent cependant encore évoluer jusqu'à la fin de l'exécution du marché."
-            ),
-            html.Div(
-                className="wrapper",
-                children=[
-                    html.Div(
-                        className="marche_map",
-                        id="marche_map",
                         children=[
                             html.H4("Titulaires"),
                             html.Ul(id="marche_infos_titulaires"),
                         ],
                     ),
-                    html.Div(className="marche_infos_1", id="marche_infos_1"),
+                ]
-                    html.Div(className="marche_infos_2", id="marche_infos_2"),
-                ],
             ),
         ],
     ),
@@ -60,30 +78,26 @@ layout = [
 @callback(
     Output("marche_data", "data"),
     Output("titulaires_data", "data"),
-    Input(component_id="url", component_property="pathname"),
+    Input(component_id="marche_url", component_property="pathname"),
 )
 def get_marche_data(url) -> tuple[dict, list]:
     marche_uid = url.split("/")[-1]
 
-    # Récupération des données du marché à partir du df global
+    # Filtre SQL côté DuckDB, puis Polars pour le post-traitement
-    lff = df.lazy()
+    dff_marche = query_marches("uid = ?", (marche_uid,))
-    lff = lff.filter(pl.col("uid") == pl.lit(marche_uid))
+    if dff_marche.height == 0:
+        return {}, []
 
-    # Données des titulaires du marché
+    lff = dff_marche.lazy()
     dff_titulaires = lff.select(cs.starts_with("titulaire")).collect(engine="streaming")
+    dff_marche_unique = lff.unique("uid").collect(engine="streaming")
+    dff_marche_unique = format_values(dff_marche_unique)
 
-    # Données du marché
+    return dff_marche_unique.to_dicts()[0], dff_titulaires.to_dicts()
-    dff_marche = (
-        lff.select(~cs.starts_with("titulaires")).unique().collect(engine="streaming")
-    )
-    dff_marche = format_montant(dff_marche)
-
-    assert dff_marche.height == 1
-
-    return dff_marche.to_dicts()[0], dff_titulaires.to_dicts()
 
 
 @callback(
+    Output("marche_objet", "children"),
     Output("marche_infos_1", "children"),
     Output("marche_infos_2", "children"),
     Output("marche_infos_titulaires", "children"),
@@ -91,11 +105,11 @@ def get_marche_data(url) -> tuple[dict, list]:
     Input("titulaires_data", "data"),
 )
 def update_marche_info(marche, titulaires):
-    def make_parameter(col):
+    def make_parameter(col, bold=True):
-        column_object = data_schema.get(col)
+        column_object = DATA_SCHEMA.get(col)
-        column_name = column_object.get("friendly_name") if column_object else col
+        column_name = column_object.get("title") if column_object else col
 
-        if marche[col]:
+        if col in marche:
             if col == "acheteur_nom":
                 value = html.A(
                     href=f"/acheteurs/{marche['acheteur_id']}",
@@ -109,8 +123,6 @@ def update_marche_info(marche, titulaires):
 
             # Dates
             elif col in ["dateNotification", "datePublicationDonnees"]:
-                print(marche[col])
-
                 value = datetime.fromisoformat(marche[col]).strftime("%d/%m/%Y")
 
             # Listes
@@ -137,12 +149,14 @@ def update_marche_info(marche, titulaires):
         else:
             value = ""
 
-        param_content = html.P([column_name, "  : ", html.Strong(value)])
+        value = html.Strong(value) if bold else value
+        param_content = html.P([column_name, "  : ", value])
         return param_content
 
+    marche_objet = make_parameter("objet", bold=False)
+
     marche_infos = [
         make_parameter("id"),
-        make_parameter("objet"),
         make_parameter("dateNotification"),  # date
         make_parameter("nature"),
         make_parameter("acheteur_nom"),  # lien
@@ -151,6 +165,7 @@ def update_marche_info(marche, titulaires):
         make_parameter("procedure"),
         make_parameter("techniques"),  # list
         make_parameter("dureeMois"),
+        make_parameter("dureeRestanteMois"),
         make_parameter("offresRecues"),
         make_parameter("datePublicationDonnees"),  # date
         make_parameter("formePrix"),
@@ -176,14 +191,71 @@ def update_marche_info(marche, titulaires):
     titulaires_lines = []
     for titulaire in titulaires:
         if titulaire["titulaire_typeIdentifiant"] == "SIRET":
+            categorie = titulaire.get("titulaire_categorie", "")
+            if titulaire.get("titulaire_distance"):
+                distance = str(titulaire.get("titulaire_distance")) + " km"
+            else:
+                distance = ""
+
             content = html.Li(
-                html.A(
+                [
-                    href=f"/titulaires/{titulaire['titulaire_id']}",
+                    html.A(
-                    children=titulaire["titulaire_nom"],
+                        href=f"/titulaires/{titulaire['titulaire_id']}",
-                )
+                        children=titulaire["titulaire_nom"],
+                    ),
+                    f" ({categorie}, {distance})",
+                ]
             )
         else:
             content = html.Li(titulaire["titulaire_nom"])
         titulaires_lines.append(content)
 
-    return marche_infos[:half], marche_infos[half:], titulaires_lines
+    return marche_objet, marche_infos[:half], marche_infos[half:], titulaires_lines
+
+
+@callback(
+    Output(component_id="marche_jsonld", component_property="children"),
+    Input("marche_data", "data"),
+    Input("titulaires_data", "data"),
+)
+def get_marche_jsonld(marche, titulaires) -> str:
+    acheteur_id = marche.get("acheteur_id")
+    type_order = (
+        "Service" if marche.get("categorie") in ["Services", "Travaux"] else "Product"
+    )
+    result = []
+
+    for titulaire in titulaires:
+        jsonld = {
+            "@context": "https://schema.org",
+            "@type": "Order",
+            "@id": f"https://decp.info/marches/{marche.get('uid')}",
+            "name": f"{marche.get('nature')} conclu par {marche.get('acheteur_nom')} le {marche.get('dateNotification')}",
+            "description": marche.get("objet"),
+            "orderNumber": marche.get("uid"),
+            "orderDate": marche.get("dateNotification"),
+            "price": unformat_montant(marche.get("montant")),
+            "priceCurrency": "EUR",
+            "customer": make_org_jsonld(
+                acheteur_id, org_name=marche.get("acheteur_nom"), org_type="acheteur"
+            ),
+            "seller": make_org_jsonld(
+                titulaire.get("titulaire_id"),
+                org_name=titulaire.get("titulaire_nom"),
+                org_type="titulaire",
+                type_org_id=titulaire.get("titulaire_typeIdentifiant", "SIRET"),
+            ),
+            "orderedItem": {
+                "@type": type_order,
+                "name": marche.get("objet"),
+                "category": {
+                    "@type": "CategoryCode",
+                    "propertyID": "cpv",
+                    "codeValue": marche.get("codeCPV"),
+                    # "description": "Description du code CPV"
+                },
+                # "serviceType": "Description du code CPV"
+            },
+        }
+        result.append(jsonld)
+    return json.dumps(result, indent=2)

src/pages/observatoire.py

@@ -0,0 +1,954 @@
+import urllib.parse
+from datetime import datetime
+
+import dash_bootstrap_components as dbc
+import polars as pl
+from dash import (
+    ALL,
+    Input,
+    Output,
+    State,
+    callback,
+    ctx,
+    dcc,
+    html,
+    no_update,
+    register_page,
+)
+
+from src.db import schema
+from src.figures import (
+    DataTable,
+    get_barchart_sources,
+    get_dashboard_summary_table,
+    get_distance_histogram,
+    get_duplicate_matrix,
+    get_geographic_maps,
+    get_top_org_table,
+    make_card,
+    make_column_picker,
+    make_donut,
+)
+from src.utils import logger
+from src.utils.cache import cache
+from src.utils.data import (
+    DEPARTEMENTS,
+    DF_ACHETEURS,
+    DF_TITULAIRES,
+    prepare_dashboard_data,
+)
+from src.utils.frontend import get_enum_values_as_dict
+from src.utils.seo import META_CONTENT
+from src.utils.table import COLUMNS, get_default_hidden_columns, prepare_table_data
+
+NAME = "Observatoire"
+
+register_page(
+    __name__,
+    path="/observatoire",
+    title="Observatoire | decp.info",
+    name=NAME,
+    description="Visualisez l'état de la publication des données essentielles des marchés publics en France.",
+    image_url=META_CONTENT["image_url"],
+    order=3,
+)
+OPTIONS_YEARS = []
+for year in reversed(range(2017, datetime.now().year + 1)):
+    option_year = {
+        "label": str(year),
+        "value": year,
+    }
+    OPTIONS_YEARS.append(option_year)
+
+OPTIONS_DEPARTEMENTS = []
+for code in DEPARTEMENTS.keys():
+    departement = {
+        "label": f"{DEPARTEMENTS[code]['departement']} ({code})",
+        "value": code,
+    }
+    OPTIONS_DEPARTEMENTS.append(departement)
+
+OBSERVATOIRE_COLUMNS = [
+    col
+    for col in schema.names()
+    if col.startswith("acheteur")
+    or col.startswith("titulaire")
+    or col
+    in [
+        "uid",
+        "dateNotification",
+        "montant",
+        "considerationsSociales",
+        "considerationsEnvironnementales",
+        "marcheInnovant",
+        "sousTraitanceDeclaree",
+        "techniques",
+        "sourceDataset",
+        "type",
+        "codeCPV",
+    ]
+]
+
+layout = [
+    dcc.Location(id="dashboard_url", refresh="callback-nav"),
+    dcc.Store(id="observatoire-filters", storage_type="local"),
+    dcc.Store(id="observatoire-hidden-columns", storage_type="local"),
+    dcc.Store(
+        id="filter-cleanup-trigger-observatoire-preview"
+    ),  # utilisé juste pour ne pas avoir à adapter les données retournées de prepare_table data
+    dbc.Modal(
+        [
+            dbc.ModalHeader(dbc.ModalTitle("Montants")),
+            dbc.ModalBody(
+                [
+                    dcc.Markdown(
+                        """
+Les données saisies et publiées par les acheteurs comportent de nombreux montants farfelus qui sabotent les statistiques, au lieu de montants estimés avec rigueur. On parle de montants atteignant parfois les millions de milliards. Certains réutilisateurs des données mettent de côté ces marchés ou bien modifient les montants selon des règles fatalement arbitraires. J'ai fait le choix de ne quasiment pas modifier les données* afin de visibiliser le problème.
+
+Alors, on fait comment ?
+
+\\* Les montants composés de plus de 11 chiffres, sans les décimales, [sont ramenés](https://github.com/ColinMaudry/decp-processing/blob/main/src/tasks/clean.py#L63-L71) à 12 311 111 111, un nombre qui reste très élevé et qui est facilement reconnaissable.
+"""
+                    ),
+                ]
+            ),
+            dbc.ModalFooter(
+                dbc.Button("Fermer", id="montant-modal-close", className="ms-auto")
+            ),
+        ],
+        id="montant-modal",
+        is_open=False,
+    ),
+    html.Div(
+        className="container-fluid",
+        children=[
+            html.H2(children=[NAME], id="page_title"),
+            dcc.Loading(
+                overlay_style={"visibility": "visible", "filter": "blur(2px)"},
+                id="loading-statistques",
+                type="default",
+                children=[
+                    dbc.Row(
+                        [
+                            dbc.Col(
+                                xl=3,
+                                lg=4,
+                                id="filters",
+                                children=[
+                                    html.H5("Période d'attribution"),
+                                    dbc.Row(
+                                        dbc.Col(
+                                            dcc.Dropdown(
+                                                id="dashboard_year",
+                                                options=OPTIONS_YEARS,
+                                                placeholder="12 derniers mois",
+                                                persistence=True,
+                                                persistence_type="local",
+                                            ),
+                                        ),
+                                    ),
+                                    html.H5("Acheteur"),
+                                    dbc.Row(
+                                        dbc.Col(
+                                            dcc.Input(
+                                                id="dashboard_acheteur_id",
+                                                placeholder="SIRET",
+                                                debounce=True,
+                                                style={"width": "100%"},
+                                                persistence=True,
+                                                persistence_type="local",
+                                            ),
+                                        ),
+                                    ),
+                                    dbc.Row(
+                                        dbc.Col(
+                                            dcc.Dropdown(
+                                                id="dashboard_acheteur_categorie",
+                                                options=get_enum_values_as_dict(
+                                                    "acheteur_categorie"
+                                                ),
+                                                placeholder="Catégorie",
+                                                persistence=True,
+                                                persistence_type="local",
+                                            )
+                                        ),
+                                    ),
+                                    dbc.Row(
+                                        dbc.Col(
+                                            dcc.Dropdown(
+                                                id="dashboard_acheteur_departement_code",
+                                                searchable=True,
+                                                multi=True,
+                                                placeholder="Département",
+                                                options=OPTIONS_DEPARTEMENTS,
+                                                persistence=True,
+                                                persistence_type="local",
+                                            ),
+                                        ),
+                                    ),
+                                    html.H5("Titulaire"),
+                                    dbc.Row(
+                                        dbc.Col(
+                                            dcc.Input(
+                                                id="dashboard_titulaire_id",
+                                                placeholder="SIRET",
+                                                debounce=True,
+                                                style={"width": "100%"},
+                                                persistence=True,
+                                                persistence_type="local",
+                                            ),
+                                        ),
+                                    ),
+                                    dbc.Row(
+                                        dbc.Col(
+                                            dcc.Dropdown(
+                                                id="dashboard_titulaire_categorie",
+                                                placeholder="Catégorie",
+                                                options=get_enum_values_as_dict(
+                                                    "titulaire_categorie"
+                                                ),
+                                                persistence=True,
+                                                persistence_type="local",
+                                            ),
+                                        ),
+                                    ),
+                                    dbc.Row(
+                                        dbc.Col(
+                                            dcc.Dropdown(
+                                                id="dashboard_titulaire_departement_code",
+                                                searchable=True,
+                                                multi=True,
+                                                placeholder="Département",
+                                                options=OPTIONS_DEPARTEMENTS,
+                                                persistence=True,
+                                                persistence_type="local",
+                                            ),
+                                        ),
+                                    ),
+                                    html.H5("Marché"),
+                                    dbc.Row(
+                                        dbc.Col(
+                                            dcc.Dropdown(
+                                                id="dashboard_marche_type",
+                                                placeholder="Type",
+                                                options=get_enum_values_as_dict("type"),
+                                                persistence=True,
+                                                persistence_type="local",
+                                            ),
+                                        ),
+                                    ),
+                                    dbc.Row(
+                                        dbc.Col(
+                                            dcc.Input(
+                                                id="dashboard_marche_objet",
+                                                placeholder="Objet",
+                                                debounce=True,
+                                                style={"width": "100%"},
+                                                persistence=True,
+                                                persistence_type="local",
+                                            ),
+                                        ),
+                                    ),
+                                    dbc.Row(
+                                        [
+                                            dbc.Col(
+                                                dcc.Input(
+                                                    id="dashboard_marche_code_cpv",
+                                                    placeholder="Code CPV (début)",
+                                                    debounce=True,
+                                                    style={"width": "100%"},
+                                                    persistence=True,
+                                                    persistence_type="local",
+                                                ),
+                                                lg=8,
+                                            ),
+                                            dbc.Col(
+                                                html.A(
+                                                    "liste des codes",
+                                                    href="https://cpvcodes.eu/fr",
+                                                    target="_blank",
+                                                ),
+                                                lg=4,
+                                            ),
+                                        ]
+                                    ),
+                                    dbc.Row(
+                                        [
+                                            dbc.Col(
+                                                dcc.Input(
+                                                    id="dashboard_montant_min",
+                                                    placeholder="Montant min.",
+                                                    type="number",
+                                                    min=0,
+                                                    debounce=True,
+                                                    style={"width": "100%"},
+                                                    persistence=True,
+                                                    persistence_type="local",
+                                                ),
+                                                width=6,
+                                            ),
+                                            dbc.Col(
+                                                dcc.Input(
+                                                    id="dashboard_montant_max",
+                                                    placeholder="Montant max.",
+                                                    type="number",
+                                                    min=0,
+                                                    debounce=True,
+                                                    style={"width": "100%"},
+                                                    persistence=True,
+                                                    persistence_type="local",
+                                                ),
+                                                width=6,
+                                            ),
+                                        ]
+                                    ),
+                                    dbc.Row(
+                                        dbc.Col(
+                                            dcc.Dropdown(
+                                                id="dashboard_marche_techniques",
+                                                placeholder="Techniques d'achat",
+                                                options=get_enum_values_as_dict(
+                                                    "techniques"
+                                                ),
+                                                multi=True,
+                                                persistence=True,
+                                                persistence_type="local",
+                                            ),
+                                        ),
+                                    ),
+                                    dbc.Row(
+                                        [
+                                            dbc.Col("Sous-traitance :", lg=5),
+                                            dbc.Col(
+                                                dbc.RadioItems(
+                                                    id="dashboard_marche_sous_traitance_declaree",
+                                                    options=[
+                                                        {
+                                                            "label": "Tous",
+                                                            "value": "all",
+                                                        },
+                                                        {
+                                                            "label": "Oui",
+                                                            "value": "oui",
+                                                        },
+                                                        {
+                                                            "label": "Non",
+                                                            "value": "non",
+                                                        },
+                                                    ],
+                                                    value="all",
+                                                    inline=True,
+                                                    persistence=True,
+                                                    persistence_type="local",
+                                                ),
+                                                lg=7,
+                                            ),
+                                        ]
+                                    ),
+                                    dbc.Row(
+                                        [
+                                            dbc.Col("Marché innovant :", lg=5),
+                                            dbc.Col(
+                                                dbc.RadioItems(
+                                                    id="dashboard_marche_innovant",
+                                                    options=[
+                                                        {
+                                                            "label": "Tous",
+                                                            "value": "all",
+                                                        },
+                                                        {
+                                                            "label": "Oui",
+                                                            "value": "oui",
+                                                        },
+                                                        {
+                                                            "label": "Non",
+                                                            "value": "non",
+                                                        },
+                                                    ],
+                                                    value="all",
+                                                    inline=True,
+                                                    persistence=True,
+                                                    persistence_type="local",
+                                                ),
+                                                lg=7,
+                                            ),
+                                        ]
+                                    ),
+                                    dbc.Row(
+                                        dbc.Col(
+                                            dcc.Dropdown(
+                                                id="dashboard_marche_considerations_sociales",
+                                                placeholder="Considérations sociales",
+                                                options=get_enum_values_as_dict(
+                                                    "considerationsSociales"
+                                                ),
+                                                multi=True,
+                                                persistence=True,
+                                                persistence_type="local",
+                                            ),
+                                        ),
+                                    ),
+                                    dbc.Row(
+                                        dbc.Col(
+                                            dcc.Dropdown(
+                                                id="dashboard_marche_considerations_environnementales",
+                                                placeholder="Considérations environnementales",
+                                                multi=True,
+                                                options=get_enum_values_as_dict(
+                                                    "considerationsEnvironnementales"
+                                                ),
+                                                persistence=True,
+                                                persistence_type="local",
+                                            ),
+                                        ),
+                                    ),
+                                    dbc.Row(
+                                        [
+                                            dbc.Col(
+                                                [
+                                                    dcc.Download(
+                                                        id="download-observatoire"
+                                                    ),
+                                                    dbc.Button(
+                                                        "Voir les données",
+                                                        id="btn-observatoire-preview",
+                                                        className="btn btn-primary mt-2",
+                                                        color="primary",
+                                                        outline=True,
+                                                    ),
+                                                    dcc.Input(
+                                                        id="observatoire-share-url",
+                                                        readOnly=True,
+                                                        style={"display": "none"},
+                                                    ),
+                                                ],
+                                                lg=12,
+                                                xl=6,
+                                            ),
+                                            dbc.Col(
+                                                id="observatoire-copy-container",
+                                                lg=12,
+                                                xl=6,
+                                            ),
+                                        ]
+                                    ),
+                                ],
+                            ),
+                            dbc.Col(
+                                width=12,
+                                lg=8,
+                                xl=9,
+                                id="cards",
+                                children=[],
+                            ),
+                        ]
+                    )
+                ],
+            ),
+        ],
+    ),
+    dbc.Offcanvas(
+        id="observatoire-preview",
+        title="Prévisualisation des données",
+        placement="bottom",
+        is_open=False,
+        scrollable=True,
+        style={"height": "75vh"},
+        children=[
+            # Header row: title + "Colonnes affichées" button
+            dbc.Row(
+                [
+                    dbc.Col(
+                        html.Div(
+                            className="table-menu",
+                            children=[
+                                dbc.Button(
+                                    "Choisir les colonnes",
+                                    id="observatoire-preview-columns-open",
+                                    className="btn btn-primary",
+                                ),
+                                html.P(id="nb_rows_observatoire"),
+                                dbc.Button(
+                                    "Télécharger au format Excel",
+                                    id="btn-download-observatoire",
+                                    disabled=True,
+                                    className="btn btn-primary",
+                                    outline=True,
+                                ),
+                            ],
+                        ),
+                        width="auto",
+                    ),
+                ],
+                className="mb-2 align-items-center",
+            ),
+            # Column picker modal
+            dbc.Modal(
+                [
+                    dbc.ModalHeader(
+                        dbc.ModalTitle("Colonnes affichées dans la prévisualisation")
+                    ),
+                    dbc.ModalBody(
+                        id="observatoire-preview-columns-body",
+                        children=make_column_picker("observatoire_preview"),
+                    ),
+                    dbc.ModalFooter(
+                        dbc.Button(
+                            "Fermer",
+                            id="observatoire-preview-columns-close",
+                            className="ms-auto",
+                            n_clicks=0,
+                        )
+                    ),
+                ],
+                id="observatoire-preview-columns-modal",
+                is_open=False,
+                fullscreen="md-down",
+                scrollable=True,
+                size="xl",
+            ),
+            # DataTable
+            dcc.Loading(
+                overlay_style={"visibility": "visible", "filter": "blur(2px)"},
+                id="loading-statistques",
+                type="default",
+                children=[
+                    html.Div(
+                        className="marches_table",
+                        children=DataTable(
+                            dtid="observatoire-preview-table",
+                            page_size=5,
+                            page_action="custom",
+                            sort_action="custom",
+                            filter_action="custom",
+                            hidden_columns=[],
+                            columns=[
+                                {"id": col, "name": col} for col in OBSERVATOIRE_COLUMNS
+                            ],
+                        ),
+                    )
+                ],
+            ),
+        ],
+    ),
+]
+
+
+FILTER_PARAMS = [
+    # (component_id, url_key, is_multi, default_value)
+    ("dashboard_year", "annee", False, None),
+    ("dashboard_acheteur_id", "acheteur_id", False, None),
+    ("dashboard_acheteur_categorie", "acheteur_cat", False, None),
+    ("dashboard_acheteur_departement_code", "acheteur_dept", True, None),
+    ("dashboard_titulaire_id", "titulaire_id", False, None),
+    ("dashboard_titulaire_categorie", "titulaire_cat", False, None),
+    ("dashboard_titulaire_departement_code", "titulaire_dept", True, None),
+    ("dashboard_marche_type", "type", False, None),
+    ("dashboard_marche_objet", "objet", False, None),
+    ("dashboard_marche_code_cpv", "cpv", False, None),
+    ("dashboard_montant_min", "montant_min", False, None),
+    ("dashboard_montant_max", "montant_max", False, None),
+    ("dashboard_marche_techniques", "techniques", True, None),
+    ("dashboard_marche_innovant", "innovant", False, "all"),
+    ("dashboard_marche_sous_traitance_declaree", "sous_traitance", False, "all"),
+    ("dashboard_marche_considerations_sociales", "social", True, None),
+    ("dashboard_marche_considerations_environnementales", "env", True, None),
+]
+
+
+@callback(
+    *[Output(fp[0], "value") for fp in FILTER_PARAMS],
+    Input("dashboard_url", "search"),
+    Input("dashboard_url", "pathname"),
+    State("observatoire-filters", "data"),
+)
+def restore_filters(search, _pathname, stored_filters):
+    if search:
+        params = urllib.parse.parse_qs(search.lstrip("?"))
+        known_keys = {fp[1] for fp in FILTER_PARAMS}
+        if any(k in params for k in known_keys):
+            values = []
+            for _comp_id, url_key, is_multi, default in FILTER_PARAMS:
+                if url_key in params:
+                    if is_multi:
+                        values.append(params[url_key])
+                    else:
+                        raw = params[url_key][0]
+                        if url_key in ("montant_min", "montant_max"):
+                            try:
+                                raw = float(raw)
+                            except (ValueError, TypeError):
+                                raw = None
+                        values.append(raw)
+                else:
+                    values.append(default)
+            return tuple(values)
+    return (no_update,) * 17
+
+
+@callback(
+    Output("observatoire-share-url", "value"),
+    Output("observatoire-copy-container", "children"),
+    *[Input(fp[0], "value") for fp in FILTER_PARAMS],
+    Input("dashboard_url", "href"),
+)
+def sync_observatoire_share_url(*args):
+    # Last arg is href (State), rest are filter values
+    filter_values = args[:-1]
+    href = args[-1]
+
+    if not href:
+        return no_update, no_update
+
+    base_url = href.split("?")[0]
+
+    params = []
+    for (_, url_key, is_multi, default), value in zip(FILTER_PARAMS, filter_values):
+        if value is None or value == default or value == [] or value == "":
+            continue
+        if is_multi and isinstance(value, list):
+            for v in value:
+                params.append((url_key, v))
+        else:
+            params.append((url_key, value))
+
+    query_string = urllib.parse.urlencode(params)
+    full_url = f"{base_url}?{query_string}" if query_string else base_url
+
+    if params:
+        copy_button = dcc.Clipboard(
+            id="btn-copy-observatoire-url",
+            target_id="observatoire-share-url",
+            title="Copier l'URL de cette vue",
+            style={
+                "display": "inline-block",
+                "fontSize": 20,
+                "verticalAlign": "top",
+                "cursor": "pointer",
+            },
+            className="fa fa-link",
+            children=[
+                dbc.Button(
+                    "Partager cette vue",
+                    id="btn-copy-observatoire",
+                    className="btn btn-primary mt-2",
+                    title="Copier l'adresse de cette vue filtrée pour la partager.",
+                )
+            ],
+        )
+    else:
+        copy_button = html.Div()
+
+    return full_url, copy_button
+
+
+@callback(
+    Output("observatoire-copy-container", "children", allow_duplicate=True),
+    Input("btn-copy-observatoire", "n_clicks", allow_optional=True),
+    prevent_initial_call=True,
+)
+def show_confirmation(n_clicks):
+    if n_clicks:
+        return html.Span(
+            "Adresse de la vue copiée",
+            style={"color": "green", "fontWeight": "bold", "marginLeft": "10px"},
+        )
+    return no_update
+
+
+def _normalize_filter_params(filter_params: dict) -> tuple:
+    """Produce a deterministic, hashable key for caching."""
+    return tuple(
+        sorted(
+            (k, tuple(v) if isinstance(v, list) else v)
+            for k, v in filter_params.items()
+        )
+    )
+
+
+@cache.memoize()
+def _compute_dashboard_children(filter_params_normalized: tuple):
+    logger.debug("Cache miss — computing dashboard")
+    filter_params = {
+        k: (list(v) if isinstance(v, tuple) else v) for k, v in filter_params_normalized
+    }
+
+    dff = prepare_dashboard_data(**filter_params)
+    lff = dff.lazy()
+
+    df_per_uid = (
+        dff.select("uid", "montant").group_by("uid").agg(pl.col("montant").first())
+    )
+    nb_marches = df_per_uid.height
+
+    cards = []
+    card_summary_table = get_dashboard_summary_table(dff, df_per_uid, nb_marches)
+    cards.append(make_card(title="Résumé", paragraphs=card_summary_table))
+
+    donut_acheteur_categorie, nb_acheteur_categories = make_donut(
+        lff,
+        "acheteur_categorie",
+        nulls="Autres",
+        per_uid=True,
+        potentially_many_names=True,
+    )
+    cards.append(
+        make_card(
+            title="Catégorie d'acheteur",
+            subtitle="en nombre de marchés attribués",
+            fig=donut_acheteur_categorie,
+            lg=12 if nb_acheteur_categories > 4 else 6,
+            xl=8 if nb_acheteur_categories > 4 else 4,
+        )
+    )
+
+    donut_titulaire_categorie = make_donut(
+        lff, "titulaire_categorie", per_uid=False, nulls="?"
+    )
+    cards.append(
+        make_card(
+            title="Catégorie d'entreprise",
+            subtitle="en nombre de titulaires",
+            fig=donut_titulaire_categorie,
+        )
+    )
+
+    donut_marche_type = make_donut(lff, "type", per_uid=True, nulls="?")
+    cards.append(
+        make_card(
+            title="Type d'achat",
+            subtitle="en nombre de marchés attribués",
+            fig=donut_marche_type,
+        )
+    )
+
+    distance_histogram = get_distance_histogram(lff)
+    cards.append(
+        make_card(
+            title="Distance acheteur–titulaire",
+            subtitle="en nombre de marchés, échelle logarithmique",
+            fig=distance_histogram,
+        )
+    )
+
+    top_acheteurs = get_top_org_table(
+        lff, org_type="acheteur", filters=False, extra_columns=[]
+    )
+    cards.append(make_card(title="Top acheteurs", fig=top_acheteurs, lg=12, xl=8))
+
+    top_titulaires = get_top_org_table(
+        lff, org_type="titulaire", filters=False, extra_columns=[]
+    )
+    cards.append(make_card(title="Top titulaires", fig=top_titulaires, lg=12, xl=8))
+
+    geographic_maps: list[dbc.Col] | None = get_geographic_maps(dff)
+
+    other_cards = []
+    sources_barchart = get_barchart_sources(lff, type_date="dateNotification")
+    other_cards.append(
+        make_card(
+            title="Sources de données",
+            subtitle="Nombre de marchés attribués par mois de notification et source de données",
+            fig=sources_barchart,
+            lg=12,
+            xl=8,
+        )
+    )
+
+    duplicate_matrix = get_duplicate_matrix()
+    other_cards.append(
+        make_card(
+            title="Matrice de doublons entre sources de données",
+            subtitle="Ce graphique illustre les doublons de marchés publics entre sources, c'est-à-dire la proportion de marchés publiés par plus d'une source.",
+            fig=duplicate_matrix,
+            lg=12,
+            xl=8,
+        )
+    )
+
+    return cards + geographic_maps + other_cards
+
+
+@callback(
+    Output("cards", "children"),
+    Output("observatoire-filters", "data"),
+    *[Input(fp[0], "value") for fp in FILTER_PARAMS],
+)
+def update_dashboard_cards(*filter_values):
+    filter_params = {}
+    for (input_id, _url_key, _is_multi, _default), value in zip(
+        FILTER_PARAMS, filter_values
+    ):
+        filter_params[input_id] = value
+
+    filter_params_normalized = _normalize_filter_params(filter_params)
+    children = _compute_dashboard_children(filter_params_normalized)
+
+    return dbc.Row(children=children), filter_params
+
+
+@callback(
+    Output("download-observatoire", "data"),
+    Input("btn-download-observatoire", "n_clicks"),
+    State("observatoire-filters", "data"),
+    State("observatoire-hidden-columns", "data"),
+    prevent_initial_call=True,
+)
+def download_observatoire(_n_clicks, filter_params, hidden_columns):
+    dff = prepare_dashboard_data(**(filter_params or {}))
+
+    if hidden_columns:
+        dff = dff.drop(hidden_columns)
+
+    def to_bytes(buffer):
+        dff.write_excel(buffer, worksheet="DECP")
+
+    date = datetime.now().strftime("%Y-%m-%d_%H:%M:%S")
+    return dcc.send_bytes(to_bytes, filename=f"decp_observatoire_{date}.xlsx")
+
+
+@callback(
+    Output("montant-modal", "is_open"),
+    Input({"type": "modal-trigger", "index": ALL}, "n_clicks"),
+    Input("montant-modal-close", "n_clicks"),
+    prevent_initial_call=True,
+)
+def toggle_montant_modal(n_triggers, _close):
+    return isinstance(ctx.triggered_id, dict) and any(n_triggers)
+
+
+@callback(
+    Output("page_title", "children"),
+    Input("dashboard_acheteur_id", "value"),
+    Input("dashboard_titulaire_id", "value"),
+    prevent_initial_call=False,
+)
+def add_organization_name_in_title(acheteur_id, titulaire_id):
+    acheteur_id = acheteur_id.replace(" ", "") if acheteur_id else None
+    titulaire_id = titulaire_id.replace(" ", "") if titulaire_id else None
+
+    def lookup_nom(df_org, id_col, nom_col, org_id):
+        match = df_org.filter(pl.col(id_col) == org_id)
+        return match[nom_col].item(0) if match.height >= 1 else None
+
+    if acheteur_id and len(acheteur_id) == 14:
+        if nom := lookup_nom(DF_ACHETEURS, "acheteur_id", "acheteur_nom", acheteur_id):
+            return [
+                NAME,
+                html.Small(nom, className="text-muted d-block fw-normal fs-5"),
+            ]
+    elif titulaire_id and len(titulaire_id) == 14:
+        if nom := lookup_nom(
+            DF_TITULAIRES, "titulaire_id", "titulaire_nom", titulaire_id
+        ):
+            return [
+                NAME,
+                html.Small(nom, className="text-muted d-block fw-normal fs-5"),
+            ]
+    return NAME
+
+
+@callback(
+    Output("observatoire-preview", "is_open"),
+    Input("btn-observatoire-preview", "n_clicks"),
+    State("observatoire-preview", "is_open"),
+    prevent_initial_call=True,
+)
+def toggle_observatoire_preview(n_clicks, is_open):
+    return not is_open
+
+
+@callback(
+    Output("observatoire-preview-table", "data"),
+    Output("observatoire-preview-table", "columns"),
+    Output("observatoire-preview-table", "tooltip_header"),
+    Output("observatoire-preview-table", "data_timestamp"),
+    Output("nb_rows_observatoire", "children"),
+    Output("btn-download-observatoire", "disabled"),
+    Output("btn-download-observatoire", "children"),
+    Output("btn-download-observatoire", "title"),
+    Output("filter-cleanup-trigger-observatoire-preview", "data", allow_duplicate=True),
+    Input("observatoire-preview", "is_open"),
+    Input("observatoire-preview-table", "filter_query"),
+    Input("observatoire-preview-table", "page_current"),
+    Input("observatoire-preview-table", "page_size"),
+    Input("observatoire-preview-table", "sort_by"),
+    State("observatoire-preview-table", "data_timestamp"),
+    State("observatoire-filters", "data"),
+    prevent_initial_call=True,
+)
+def populate_preview_table(
+    is_open,
+    filter_query,
+    page_current,
+    page_size,
+    sort_by,
+    data_timestamp,
+    filter_params,
+):
+    if not is_open:
+        return (no_update,) * 9
+
+    dff = prepare_dashboard_data(**(filter_params or {}))
+
+    return prepare_table_data(
+        dff.lazy(),
+        data_timestamp,
+        filter_query,
+        page_current,
+        page_size,
+        sort_by,
+        "observatoire-preview",
+    )
+
+
+@callback(
+    Output("observatoire-hidden-columns", "data", allow_duplicate=True),
+    Input("observatoire_preview_column_list", "selected_rows"),
+    prevent_initial_call=True,
+)
+def update_hidden_columns_from_checkboxes(selected_columns):
+    if selected_columns:
+        selected_columns = [COLUMNS[i] for i in selected_columns]
+        hidden_columns = [col for col in COLUMNS if col not in selected_columns]
+        return hidden_columns
+    else:
+        return []
+
+
+@callback(
+    Output("observatoire-preview-table", "hidden_columns"),
+    Input(
+        "observatoire-hidden-columns",
+        "data",
+    ),
+)
+def store_hidden_columns(hidden_columns):
+    return hidden_columns
+
+
+@callback(
+    Output("observatoire_preview_column_list", "selected_rows"),
+    Input("observatoire-preview-table", "hidden_columns"),
+    State(
+        "observatoire_preview_column_list", "selected_rows"
+    ),  # pour éviter la boucle infinie
+)
+def update_checkboxes_from_hidden_columns(hidden_cols, current_checkboxes):
+    hidden_cols = hidden_cols or get_default_hidden_columns("tableau")
+
+    # Show all columns that are NOT hidden
+    visible_cols = [COLUMNS.index(col) for col in COLUMNS if col not in hidden_cols]
+    return visible_cols
+
+
+@callback(
+    Output("observatoire-preview-columns-modal", "is_open"),
+    Input("observatoire-preview-columns-open", "n_clicks"),
+    Input("observatoire-preview-columns-close", "n_clicks"),
+    State("observatoire-preview-columns-modal", "is_open"),
+)
+def toggle_tableau_columns(click_open, click_close, is_open):
+    if click_open or click_close:
+        return not is_open
+    return is_open

src/pages/recherche.py

@@ -0,0 +1,132 @@
+import dash_bootstrap_components as dbc
+from dash import Input, Output, State, callback, dcc, html, register_page
+
+from src.figures import DataTable
+from src.utils.data import DF_ACHETEURS, DF_TITULAIRES
+from src.utils.search import search_org
+from src.utils.seo import META_CONTENT
+from src.utils.table import setup_table_columns
+
+NAME = "Recherche"
+
+register_page(
+    __name__,
+    path="/",
+    title="Recherche de marchés publics | decp.info",
+    name=NAME,
+    description="Explorez et analysez les données des marchés publics français avec cet outil libre et gratuit. Pour une commande publique accessible à toutes et tous.",
+    image_url=META_CONTENT["image_url"],
+    order=0,
+)
+
+layout = html.Div(
+    className="container",
+    children=[
+        html.Div(
+            className="tagline",
+            children=html.P("Recherchez un acheteur ou un titulaire de marché public"),
+        ),
+        html.Div(
+            style={
+                "display": "flex",
+                "justifyContent": "center",
+                "marginTop": "30px",
+                "marginBottom": "30px",
+            },
+            children=[
+                dcc.Input(
+                    id="search",
+                    type="text",
+                    placeholder="Nom d'acheteur/entreprise, SIREN/SIRET, code département",
+                    autoFocus=True,
+                    style={
+                        "margin": "0",
+                        "width": "500px",
+                        "border": "1px solid #ccc",
+                        "borderRight": "none",
+                        "borderRadius": "3px 0 0 3px",
+                        "padding": "5px 10px",
+                        "outline": "none",
+                        "height": "34px",
+                    },
+                ),
+                html.Button(
+                    "=>",
+                    id="search-button",
+                    className="btn btn-primary",
+                    style={
+                        "border": "1px solid #ccc",
+                        "borderRadius": "0 3px 3px 0",
+                        "marginLeft": "0",
+                        "height": "auto",  # Ensure it matches input height if necessary, often relying on padding/line-height
+                    },
+                ),
+            ],
+        ),
+        html.P(
+            [
+                "...ou bien filtrez les marchés publics dans la vue ",
+                dcc.Link("Tableau", href="/tableau"),
+            ],
+            style={"textAlign": "center"},
+            id="mention_tableau",
+        ),
+        # html.Div(
+        #     className="search_options",
+        #     children=[dcc.RadioItems(options=["Acheteur(s)"])],
+        # ),
+        dbc.Row(id="search_results"),
+    ],
+)
+
+
+@callback(
+    Output("search_results", "children"),
+    Output("mention_tableau", "style"),
+    Input("search", "n_submit"),
+    Input("search-button", "n_clicks"),
+    State("search", "value"),
+    prevent_initial_call=True,
+)
+def update_search_results(n_submit, n_clicks, query):
+    if query and len(query) >= 1:
+        cols = []
+
+        for org_type in ["acheteur", "titulaire"]:
+            if org_type == "acheteur":
+                dff = DF_ACHETEURS
+            elif org_type == "titulaire":
+                dff = DF_TITULAIRES
+            else:
+                raise ValueError(f"{org_type} is not supported")
+
+            # Search acheteurs and titulaires using the same function
+            results = search_org(dff, query, org_type=org_type)
+            count = results.height
+
+            # Format output
+            columns, tooltip = setup_table_columns(results, hideable=False)
+
+            col = (
+                dbc.Col(
+                    children=[
+                        html.H3(f"{org_type.title()}s : {count}"),
+                        DataTable(
+                            dtid=f"results_{org_type}_datatable",
+                            columns=columns,
+                            data=results.to_dicts(),
+                            page_size=10,
+                            sort_action="none",
+                            filter_action="none",
+                        ),
+                    ],
+                    md=6,
+                )
+                if count > 0
+                else html.P(f"Aucun {org_type} trouvé.")
+            )
+            cols.append(col)
+
+        style = {"textAlign": "center", "display": "none"}
+        return cols, style
+    return html.P(""), {"textAlign": "center"}

src/pages/statistiques.py

@@ -1,55 +0,0 @@
-from dash import dcc, html, register_page
-
-from src.figures import get_barchart_sources, get_map_count_marches
-from src.utils import df, meta_content
-
-name = "Statistiques"
-
-register_page(
-    __name__,
-    path="/statistiques",
-    title=meta_content["title"],
-    name=name,
-    description=meta_content["description"],
-    image_url=meta_content["image_url"],
-    order=3,
-)
-
-
-layout = [
-    html.Div(
-        className="container",
-        children=[
-            html.H2(name),
-            dcc.Loading(
-                overlay_style={"visibility": "visible", "filter": "blur(2px)"},
-                id="loading-statistques",
-                type="default",
-                children=[
-                    html.Div(
-                        children=[
-                            dcc.Markdown("""
-                            La publication de données essentielles de marchés publics (DECP) est souvent effectuée par
-                            les plateformes de marchés publics (profils d'acheteurs). Cependant, certaines plateformes ne publient pas,
-                            ou publient d'une manière qui rend la récupération des données compliquée. Les données présentées sur ce site
-                            ne représentent donc pas tous les marchés attribués en France, seulement une partie significative.
-
-                            L'ajout de nouvelles plateformes [est en cours](https://github.com/ColinMaudry/decp-processing/issues?q=is%3Aissue%20state%3Aopen%20label%3A%22source%20de%20donn%C3%A9es%22),
-                            toutes les [contributions](/a-propos#contribuer) sont les bienvenues pour atteindre l'exhaustivité.
-                            """),
-                            dcc.Graph(figure=get_map_count_marches(df)),
-                            dcc.Graph(
-                                figure=get_barchart_sources(df, "dateNotification")
-                            ),
-                            dcc.Graph(
-                                figure=get_barchart_sources(
-                                    df, "datePublicationDonnees"
-                                )
-                            ),
-                        ],
-                    )
-                ],
-            ),
-        ],
-    )
-]

src/pages/tableau.py

@@ -1,135 +1,140 @@
+import json
 import os
+import urllib.parse
+import uuid
 from datetime import datetime
 
+import dash_bootstrap_components as dbc
 import polars as pl
-from dash import Input, Output, State, callback, dash_table, dcc, html, register_page
+from dash import (
-
+    ClientsideFunction,
-from src.utils import (
+    Input,
-    add_links,
+    Output,
-    add_resource_link,
+    State,
-    df,
+    callback,
-    filter_table_data,
+    clientside_callback,
-    format_montant,
+    dcc,
-    format_number,
+    html,
-    meta_content,
+    no_update,
-    setup_table_columns,
+    register_page,
-    sort_table_data,
 )
 
-update_date = os.path.getmtime(os.getenv("DATA_FILE_PARQUET_PATH"))
+from src.db import query_marches, schema
-update_date = datetime.fromtimestamp(update_date).strftime("%d/%m/%Y")
+from src.figures import DataTable, make_column_picker
+from src.utils import logger
+from src.utils.seo import META_CONTENT
+from src.utils.table import (
+    COLUMNS,
+    filter_table_data,
+    get_default_hidden_columns,
+    invert_columns,
+    prepare_table_data,
+    sort_table_data,
+)
+from src.utils.tracking import track_search
 
-schema = df.collect_schema()
+update_date_timestamp = os.path.getmtime(os.getenv("DATA_FILE_PARQUET_PATH"))
+update_date = datetime.fromtimestamp(update_date_timestamp).strftime("%d/%m/%Y")
+update_date_iso = datetime.fromtimestamp(update_date_timestamp).isoformat()
 
-name = "Tableau"
+
+NAME = "Tableau"
 register_page(
     __name__,
-    path="/",
+    path="/tableau",
-    title=meta_content["title"],
+    title="Tableau des marchés publics | decp.info",
-    name=name,
+    name=NAME,
-    description=meta_content["description"],
+    description="Consultez, filtrez et exportez les données essentielles de la commande publique sous forme de tableau.",
-    image_url=meta_content["image_url"],
+    image_url=META_CONTENT["image_url"],
     order=1,
 )
 
-datatable = html.Div(
+DATATABLE = html.Div(
     className="marches_table",
-    children=dash_table.DataTable(
+    children=DataTable(
-        cell_selectable=False,
+        dtid="tableau_datatable",
-        id="table",
+        persisted_props=["filter_query", "sort_by"],
+        persistence_type="local",
+        persistence=True,
         page_size=20,
-        page_current=0,
         page_action="custom",
         filter_action="custom",
-        filter_options={"case": "insensitive", "placeholder_text": "Filtrer..."},
         sort_action="custom",
-        sort_mode="multi",
+        hidden_columns=[],
-        sort_by=[],
+        columns=[{"id": col, "name": col} for col in schema.names()],
-        row_deletable=False,
-        style_cell_conditional=[
-            {
-                "if": {"column_id": "objet"},
-                "minWidth": "350px",
-                "textAlign": "left",
-                "overflow": "hidden",
-                "lineHeight": "14px",
-                "whiteSpace": "normal",
-            },
-            {
-                "if": {"column_id": "acheteur_nom"},
-                "minWidth": "250px",
-                "textAlign": "left",
-                "overflow": "hidden",
-                "lineHeight": "14px",
-                "whiteSpace": "normal",
-            },
-            {
-                "if": {"column_id": "titulaire_nom"},
-                "minWidth": "250px",
-                "textAlign": "left",
-                "overflow": "hidden",
-                "lineHeight": "14px",
-                "whiteSpace": "normal",
-            },
-        ],
-        data_timestamp=0,
-        markdown_options={"html": True},
-        tooltip_duration=8000,
-        tooltip_delay=350,
     ),
 )
 
 layout = [
+    dcc.Location(id="tableau_url", refresh=False),
+    dcc.Store(id="filter-cleanup-trigger-tableau"),
+    dcc.Store(id="tableau-hidden-columns", storage_type="local"),
+    dcc.Store(id="tableau-table"),
+    html.Script(
+        type="application/ld+json",
+        id="dataset_jsonld",
+        children=[
+            json.dumps(
+                {
+                    "@context": "https://schema.org/",
+                    "@type": "Dataset",
+                    "name": "Données essentielles des marchés publics français (DECP)",
+                    "description": "Données de marchés publics exhaustives décrivant les marchés publics attribués en France depuis 2018.",
+                    "url": "https://decp.info",
+                    "sameAs": "https://www.data.gouv.fr/datasets/608c055b35eb4e6ee20eb325",
+                    "keywords": [
+                        "marchés publics",
+                        "commande publique",
+                        "decp",
+                        "public procurement",
+                    ],
+                    "license": "https://www.etalab.gouv.fr/licence-ouverte-open-licence",
+                    "isAccessibleForFree": True,
+                    "creator": {
+                        "@type": "Organization",
+                        "url": "https://colmo.tech",
+                        "name": "Colmo",
+                        "sameAs": "https://annuaire-entreprises.data.gouv.fr/entreprise/colmo-989393350",
+                        "contactPoint": {
+                            "@type": "ContactPoint",
+                            "contactType": "Support et contact commercial",
+                            "email": "colin@colmo.tech",
+                        },
+                    },
+                    "includedInDataCatalog": {
+                        "@type": "DataCatalog",
+                        "name": "data.gouv.fr",
+                    },
+                    "distribution": [
+                        {
+                            "@type": "DataDownload",
+                            "encodingFormat": "CSV",
+                            "contentUrl": "https://www.data.gouv.fr/api/1/datasets/r/22847056-61df-452d-837d-8b8ceadbfc52",
+                        },
+                        {
+                            "@type": "DataDownload",
+                            "encodingFormat": "Parquet",
+                            "contentUrl": "https://www.data.gouv.fr/api/1/datasets/r/11cea8e8-df3e-4ed1-932b-781e2635e432",
+                        },
+                    ],
+                    "temporalCoverage": f"2018-01-01/{update_date_iso[:10]}",
+                    "spatialCoverage": {
+                        "@type": "Place",
+                        "address": {"countryCode": "FR"},
+                    },
+                },
+                indent=2,
+            )
+        ],
+    ),
+    dcc.Markdown(
+        f"Ce tableau contient tous les marchés attribués en France. Il vous permet d'appliquer un filtre sur une ou plusieurs colonnes, et ainsi produire la liste de marchés dont vous avez besoin (exemples : [marchés de voirie < 40 k€ en 2025](/tableau?filtres=%7Bacheteur_id%7D+icontains+24350013900189+%26%26+%7BdateNotification%7D+icontains+2025%2A+%26%26+%7Bmontant%7D+i%3C+40000+%26%26+%7Bobjet%7D+icontains+voirie&colonnes=uid%2Cacheteur_id%2Cacheteur_nom%2Ctitulaire_id%2Ctitulaire_nom%2Cobjet%2Cmontant%2CdureeMois%2CdateNotification%2Cacheteur_departement_code%2CsourceDataset), [marchés > 500 k€ avec clause sociale attribués à des PME à plus de 100 km dans le Var](/tableau?filtres=%7Btitulaire_categorie%7D+icontains+PME+%26%26+%7Btitulaire_distance%7D+i%3E+100+%26%26+%7Bmontant%7D+i%3E+500000+%26%26+%7Bacheteur_departement_code%7D+icontains+83+%26%26+%7BconsiderationsSociales%7D+icontains+clause&colonnes=uid%2Cacheteur_id%2Cacheteur_nom%2Ctitulaire_id%2Ctitulaire_nom%2Cobjet%2Cmontant%2CdureeMois%2CdateNotification%2CconsiderationsSociales%2Ctitulaire_distance%2Cacheteur_departement_code%2Ctitulaire_categorie%2CsourceDataset)). Par défaut seules quelques colonnes sont affichées, mais vous pouvez en afficher jusqu'à {len(schema.names())} en cliquant sur le bouton **Choisir les colonnes**. Cet outil est assez puissant, je vous recommande de lire le mode d'emploi pour en tirer pleinement partie.",
+        style={"maxWidth": "1000px"},
+    ),
     html.Div(
-        html.Details(
+        [],
-            children=[
-                html.Summary(
-                    html.H3("Mode d'emploi", style={"text-decoration": "underline"}),
-                ),
-                dcc.Markdown(
-                    """
-    ##### Définition des colonnes
-
-    Pour voir la définition d'une colonne, passez votre souris sur son en-tête.
-
-    ##### Filtres
-
-    Vous pouvez appliquer un filtre pour chaque colonne en entrant du texte sous le nom de la colonne, puis en tapant sur `Entrée`.
-
-    - Champs textuels : la recherche est insensible à la casse (majuscules/minuscules) et retourne les valeurs qui contiennent
-    le texte recherché. Exemple : `rennes` retourne "RENNES METROPOLE".
-    - Champs numériques : vous pouvez soit taper un nombre pour trouver les valeurs égales, soit le précéder de **>** ou **<** pour filtrer les valeurs supérieures ou inférieures. Exemple pour les offres reçues : `> 4` retourne les marchés ayant reçu plus de 4 offres.
-    - Champs date : vous pouvez également utiliser **>** ou **<**. Exemples : `< 2024-01-31` pour "avant le 31 janvier 2024",
-    `2024` pour "en 2024", `> 2022` pour "à partir de 2022"
-
-    Vous pouvez filtrer plusieurs colonnes à la fois. Vos filtres sont remis à zéro quand vous rafraîchissez la page.
-
-    ##### Tri
-
-    Pour trier une colonne, utilisez les flèches grises à côté des noms de colonnes. Chaque clic change le tri dans cet ordre : tri ascendant, tri descendant, pas de tri.
-
-    ##### Télécharger le résultat
-
-    Vous pouvez télécharger le résultat de vos filtres et tris, pour les colonnes affichées, en cliquant sur **Télécharger au format Excel**.
-
-    ##### Liens
-
-    Les liens dans les colonnes Identifiant unique, Acheteur et Titulaire vous permettent de consulter une vue qui leur est dédiée
-    (informations, marchés attribués/remportés, etc.)
-
-    """
-                ),
-            ],
-            id="instructions",
-        ),
         id="header",
     ),
-    # html.Div(
-    #     [
-    #         "Recherche dans objet : ",
-    #         dcc.Input(id="search", value="", type="text"),
-    #     ]
-    # )]),
     dcc.Loading(
         overlay_style={"visibility": "visible", "filter": "blur(2px)"},
         id="loading-home",
@@ -137,8 +142,101 @@ layout = [
         children=[
             html.Div(
                 [
+                    # Modal du mode d'emploi
+                    dbc.Button("Mode d'emploi", id="tableau_help_open"),
+                    dbc.Modal(
+                        [
+                            dbc.ModalHeader(dbc.ModalTitle("Mode d'emploi")),
+                            dbc.ModalBody(
+                                dcc.Markdown(
+                                    dangerously_allow_html=True,
+                                    children=f"""
+            ##### Définition des colonnes
+
+            Pour voir la définition d'une colonne, passez votre souris sur son en-tête.
+
+            ##### Vos réglages sont persistents
+
+            Les filtres, les tris et le choix de colonnes sont automatiquement enregistrés dans votre navigateur et persistent même si vous changez de page ou si vous fermez votre navigateur. À votre retour, vous retrouverez cette page comme vous l'avez laissée.
+
+            ##### Appliquer des filtres
+
+            Vous pouvez appliquer un filtre pour chaque colonne en entrant du texte sous le nom de la colonne, puis en tapant sur `Entrée`.
+
+            - Champs textuels : la recherche retourne les valeurs qui contiennent le texte recherché, n'est pas sensible à la casse (majuscules/minuscules) et est sensbible à l'accentuation.
+                - `rennes` => le texte contient "rennes"
+                - `metro* *pole` => le texte contient un mot qui commence par "metro" et un mot qui finit par "pole"
+                - `metropole rennes` => le texte contient les mots "metropole" et "rennes", n'importe où dans le texte
+                - `metropole+rennes` => le texte contient "metropole rennes", collé et dans cet ordre
+                - `metropole+rennes travaux distri*` => le texte contient "metropole rennes", "travaux" et un mot qui commence par "distri"
+                - Les guillemets simples (apostrophe du 4) doivent être prédédées d'une barre oblique (AltGr + 8). Exemple : `services d\\\'assurances`
+            - Champs numériques (Durée en mois, Montant, ...) : vous pouvez...
+                - soit taper un nombre pour trouver les valeurs strictement égales. Exemple : `12` ne retourne que des 12
+                - soit le précéder de **>** ou **<** pour filtrer les valeurs supérieures ou inférieures. Exemple pour les offres reçues : `> 4` retourne les marchés ayant reçu plus de 4 offres.
+            - Champs date (Date de notification, ...) :
+                - `< 2024-01-31` pour "avant le 31 janvier 2024"
+                - `2024` pour "en 2024", `> 2022` pour "à partir de 2022"
+
+            Vous pouvez filtrer plusieurs colonnes à la fois.
+
+            ##### Trier les données
+
+            Pour trier une colonne, utilisez les flèches grises à côté des noms de colonnes. Chaque clic change le tri dans cet ordre :
+
+            1. tri croissant
+            2. tri décroissant
+            3. pas de tri
+
+            ##### Afficher plus de colonnes
+
+            Par défaut, un nombre réduit de colonnes est affiché pour ne pas surcharger la page. Mais vous avez le choix parmi {len(schema.names())} colonnes, ce serait dommage de vous limiter !
+
+            Pour afficher plus de colonnes, cliquez sur le bouton **Choisir les colonnes** et cochez les colonnes pour les afficher.
+
+            ##### Partager une vue
+
+            Une vue est un ensemble de filtres, de tris et de choix de colonnes que vous avez appliqués. Cliquez sur **Partager** pour copier une adresse Web qui reproduit la vue courante à l'identique : en la collant dans la barre d'adresse d'un navigateur, vous ouvrez la vue Tableau avec les mêmes paramètres.
+
+            Pratique pour partager une vue avec un·e collègue, sur les réseaux sociaux, ou la sauvegarder pour plus tard.
+
+            ##### Télécharger le résultat
+
+            Vous pouvez télécharger le résultat de vos filtres et tris, pour les colonnes affichées, en cliquant sur **Télécharger au format Excel**.
+
+            ##### Liens
+
+            Les liens dans les colonnes Identifiant unique, Acheteur et Titulaire vous permettent de consulter une vue qui leur est dédiée
+            (informations, marchés attribués/remportés, etc.)
+
+            """,
+                                ),
+                            ),
+                            dbc.ModalFooter(
+                                dbc.Button(
+                                    "Fermer",
+                                    id="tableau_help_close",
+                                    className="ms-auto",
+                                    n_clicks=0,
+                                )
+                            ),
+                        ],
+                        id="tableau_help",
+                        is_open=False,
+                        fullscreen="md-down",
+                        scrollable=True,
+                        size="lg",
+                    ),
+                    # Bouton modal des colonnes affichées
+                    dbc.Button(
+                        "Choisir les colonnes",
+                        id="tableau_columns_open",
+                        className="column_list",
+                        title="Choisir les colonnes à afficher et masquer",
+                    ),
                     html.P("lignes", id="nb_rows"),
-                    html.Button(
+                    html.Div(id="copy-container"),
+                    dcc.Input(id="share-url", readOnly=True, style={"display": "none"}),
+                    dbc.Button(
                         "Téléchargement désactivé au-delà de 65 000 lignes",
                         id="btn-download-data",
                         disabled=True,
@@ -146,109 +244,90 @@ layout = [
                     dcc.Download(id="download-data"),
                     dcc.Store(id="filtered_data", storage_type="memory"),
                     html.P("Données mises à jour le " + str(update_date)),
+                    dbc.Button(
+                        "Remettre à zéro",
+                        title="Supprime tous les filtres et les tris. Autrement ils sont conservés même si vous fermez la page.",
+                        id="btn-tableau-reset",
+                    ),
                 ],
                 className="table-menu",
             ),
-            datatable,
+            dbc.Modal(
+                [
+                    dbc.ModalHeader(dbc.ModalTitle("Choix des colonnes à afficher")),
+                    dbc.ModalBody(
+                        id="tableau_columns_body",
+                        children=make_column_picker("tableau"),
+                    ),
+                    dbc.ModalFooter(
+                        dbc.Button(
+                            "Fermer",
+                            id="tableau_columns_close",
+                            className="ms-auto",
+                            n_clicks=0,
+                        )
+                    ),
+                ],
+                id="tableau_columns",
+                is_open=False,
+                fullscreen="md-down",
+                scrollable=True,
+                size="xl",
+            ),
+            DATATABLE,
         ],
     ),
 ]
 
 
 @callback(
-    Output("table", "data"),
+    Output("tableau_datatable", "data"),
-    Output("table", "columns"),
+    Output("tableau_datatable", "columns"),
-    Output("table", "tooltip_header"),
+    Output("tableau_datatable", "tooltip_header"),
-    Output("table", "data_timestamp"),
+    Output("tableau_datatable", "data_timestamp"),
     Output("nb_rows", "children"),
     Output("btn-download-data", "disabled"),
     Output("btn-download-data", "children"),
     Output("btn-download-data", "title"),
-    Input("table", "page_current"),
+    Output("filter-cleanup-trigger-tableau", "data", allow_duplicate=True),
-    Input("table", "page_size"),
+    Input("tableau_url", "href"),
-    Input("table", "filter_query"),
+    Input("tableau_datatable", "page_current"),
-    Input("table", "sort_by"),
+    Input("tableau_datatable", "page_size"),
-    State("table", "data_timestamp"),
+    Input("tableau_datatable", "filter_query"),
+    Input("tableau_datatable", "sort_by"),
+    State("tableau_datatable", "data_timestamp"),
+    prevent_initial_call=True,
 )
-def update_table(page_current, page_size, filter_query, sort_by, data_timestamp):
+def update_table(href, page_current, page_size, filter_query, sort_by, data_timestamp):
-    if os.getenv("DEVELOPMENT").lower() == "true":
+    # if ctx.triggered_id != "url":
-        print(" + + + + + + + + + + + + + + + + + + ")
+    #     search_params = None
-
+    # else:
-    # Application des filtres
+    #     search_params = urllib.parse.parse_qs(search_params.lstrip("?"))
-    lff: pl.LazyFrame = df.lazy()  # start from the original data
+    return prepare_table_data(
-    if filter_query:
+        None, data_timestamp, filter_query, page_current, page_size, sort_by, "tableau"
-        lff = filter_table_data(lff, filter_query)
-
-    if len(sort_by) > 0:
-        lff = sort_table_data(lff, sort_by)
-
-    # Remplace les strings null par "", mais pas les numeric null
-    lff = lff.fill_null("")
-
-    # Matérialisation des filtres
-    dff: pl.DataFrame = lff.collect()
-
-    height = dff.height
-    nb_rows = f"{format_number(height)} lignes ({format_number(dff.select('uid').unique().height)} marchés)"
-
-    # Pagination des données
-    start_row = page_current * page_size
-    # end_row = (page_current + 1) * page_size
-    dff = dff.slice(start_row, page_size)
-
-    # Ajout des liens vers l'annuaire des entreprises
-    dff = add_links(dff)
-
-    # Ajout des liens vers les fichiers Open Data
-    dff = add_resource_link(dff)
-
-    # Formatage des montants
-    dff = format_montant(dff)
-
-    columns, tooltip = setup_table_columns(dff)
-
-    dicts = dff.to_dicts()
-
-    if height > 65000:
-        download_disabled = True
-        download_text = "Téléchargement désactivé au-delà de 65 000 lignes"
-        download_title = "Excel ne supporte pas d'avoir plus de 65 000 URLs dans une même feuille de calcul. Contactez-moi pour me présenter votre besoin en téléchargement afin que je puisse adapter la solution."
-    else:
-        download_disabled = False
-        download_text = "Télécharger au format Excel"
-        download_title = ""
-
-    return (
-        dicts,
-        columns,
-        tooltip,
-        data_timestamp + 1,
-        nb_rows,
-        download_disabled,
-        download_text,
-        download_title,
     )
 
 
 @callback(
     Output("download-data", "data"),
     Input("btn-download-data", "n_clicks"),
-    State("table", "filter_query"),
+    State("tableau_datatable", "filter_query"),
-    State("table", "sort_by"),
+    State("tableau_datatable", "sort_by"),
-    State("table", "hidden_columns"),
+    State("tableau_datatable", "hidden_columns"),
     prevent_initial_call=True,
 )
-def download_data(n_clicks, filter_query, sort_by, hidden_columns: list = None):
+def download_data(n_clicks, filter_query, sort_by, hidden_columns: list | None = None):
-    lff: pl.LazyFrame = df  # start from the original data
+    lff: pl.LazyFrame = query_marches().lazy()
 
     # Les colonnes masquées sont supprimées
     if hidden_columns:
         lff = lff.drop(hidden_columns)
 
     if filter_query:
+        track_search(filter_query, "tab download")
         lff = filter_table_data(lff, filter_query)
 
-    if len(sort_by) > 0:
+    if sort_by and len(sort_by) > 0:
         lff = sort_table_data(lff, sort_by)
 
     def to_bytes(buffer):
@@ -256,3 +335,204 @@ def download_data(n_clicks, filter_query, sort_by, hidden_columns: list = None):
 
     date = datetime.now().strftime("%Y-%m-%d_%H:%M:%S")
     return dcc.send_bytes(to_bytes, filename=f"decp_{date}.xlsx")
+
+
+@callback(
+    Output("tableau_datatable", "filter_query"),
+    Output("tableau_datatable", "sort_by"),
+    Output("tableau-hidden-columns", "data"),
+    Output("tableau_url", "search"),
+    Output("filter-cleanup-trigger-tableau", "data"),
+    Input("tableau_url", "search"),
+    State("tableau_datatable", "filter_query"),
+    State("tableau_datatable", "sort_by"),
+)
+def restore_view_from_url(search, stored_filters, stored_sort):
+    if not search and not stored_filters:
+        return no_update, no_update, no_update, no_update, no_update
+
+    params = urllib.parse.parse_qs(search.lstrip("?")) if search else {}
+    logger.debug("params " + json.dumps(params, indent=2))
+
+    filter_query = no_update
+    sort_by = no_update
+    hidden_columns = no_update
+    trigger_cleanup = no_update
+
+    if "filtres" in params:
+        filter_query = params["filtres"][0]
+        trigger_cleanup = str(uuid.uuid4())
+    elif stored_filters:
+        filter_query = stored_filters
+        trigger_cleanup = str(uuid.uuid4())
+
+    if "tris" in params:
+        try:
+            sort_by = json.loads(params["tris"][0])
+        except json.JSONDecodeError:
+            pass
+    elif stored_sort:
+        sort_by = stored_sort
+
+    if "colonnes" in params:
+        table_columns = params["colonnes"][0].split(",")
+        verified_columns = [
+            column for column in table_columns if column in schema.names()
+        ]
+        hidden_columns = invert_columns(verified_columns)
+
+    return filter_query, sort_by, hidden_columns, "", trigger_cleanup
+
+
+# Pour nettoyer les icontains et i< des filtres
+# voir aussi src/assets/dash_clientside.js
+clientside_callback(
+    ClientsideFunction(
+        namespace="clientside",
+        function_name="clean_filters",
+    ),
+    Output("filter-cleanup-trigger-tableau", "data", allow_duplicate=True),
+    Input("filter-cleanup-trigger-tableau", "data"),
+    prevent_initial_call=True,
+)
+
+
+@callback(
+    Output("share-url", "value"),
+    Output("copy-container", "children"),
+    Input("tableau_datatable", "filter_query"),
+    Input("tableau_datatable", "sort_by"),
+    Input("tableau_datatable", "hidden_columns"),
+    State("tableau_url", "href"),
+    prevent_initial_call=True,
+)
+def sync_url_and_reset_button(filter_query, sort_by, hidden_columns, href):
+    if not href:
+        return no_update, no_update
+
+    # Extract base URL (remove existing query params)
+    base_url = href.split("?")[0]
+
+    params = {}
+    if filter_query:
+        params["filtres"] = filter_query
+
+    if sort_by:
+        params["tris"] = json.dumps(sort_by)
+
+    if hidden_columns:
+        table_columns = invert_columns(hidden_columns)
+        table_columns = ",".join(table_columns)
+        params["colonnes"] = table_columns
+
+    query_string = urllib.parse.urlencode(params)
+    full_url = f"{base_url}?{query_string}" if query_string else base_url
+
+    copy_button = dcc.Clipboard(
+        id="btn-copy-url",
+        target_id="share-url",
+        title="Copier l'URL de cette vue",
+        style={
+            "display": "inline-block",
+            "fontSize": 20,
+            "verticalAlign": "top",
+            "cursor": "pointer",
+        },
+        className="fa fa-link",
+        children=[
+            dbc.Button(
+                "Partager la vue",
+                className="btn btn-primary",
+                title="Copier l'adresse de cette vue (filtres, tris, choix de colonnes) pour la partager.",
+            )
+        ],
+    )
+
+    return full_url, copy_button
+
+
+@callback(
+    Output("copy-container", "children", allow_duplicate=True),
+    Input("btn-copy-url", "n_clicks", allow_optional=True),
+    prevent_initial_call=True,
+)
+def show_confirmation(n_clicks):
+    if n_clicks:
+        return html.Span(
+            "Adresse de la vue copiée",
+            style={"color": "green", "fontWeight": "bold", "marginLeft": "10px"},
+        )
+    return no_update
+
+
+@callback(
+    Output("tableau_help", "is_open"),
+    [Input("tableau_help_open", "n_clicks"), Input("tableau_help_close", "n_clicks")],
+    [State("tableau_help", "is_open")],
+)
+def toggle_tableau_help(click_open, click_close, is_open):
+    if click_open or click_close:
+        return not is_open
+    return is_open
+
+
+@callback(
+    Output("tableau-hidden-columns", "data", allow_duplicate=True),
+    Input("tableau_column_list", "selected_rows"),
+    prevent_initial_call=True,
+)
+def update_hidden_columns_from_checkboxes(selected_columns):
+    if selected_columns:
+        selected_columns = [COLUMNS[i] for i in selected_columns]
+        hidden_columns = [col for col in COLUMNS if col not in selected_columns]
+        return hidden_columns
+    else:
+        return []
+
+
+@callback(
+    Output("tableau_datatable", "hidden_columns"),
+    Input(
+        "tableau-hidden-columns",
+        "data",
+    ),
+)
+def store_hidden_columns(hidden_columns):
+    if hidden_columns is None:
+        hidden_columns = get_default_hidden_columns("tableau")
+    return hidden_columns
+
+
+@callback(
+    Output("tableau_column_list", "selected_rows"),
+    Input("tableau_datatable", "hidden_columns"),
+    State("tableau_column_list", "selected_rows"),  # pour éviter la boucle infinie
+)
+def update_checkboxes_from_hidden_columns(hidden_cols, current_checkboxes):
+    hidden_cols = hidden_cols or get_default_hidden_columns("tableau")
+
+    # Show all columns that are NOT hidden
+    visible_cols = [COLUMNS.index(col) for col in COLUMNS if col not in hidden_cols]
+    return visible_cols
+
+
+@callback(
+    Output("tableau_columns", "is_open"),
+    Input("tableau_columns_open", "n_clicks"),
+    Input("tableau_columns_close", "n_clicks"),
+    State("tableau_columns", "is_open"),
+)
+def toggle_tableau_columns(click_open, click_close, is_open):
+    if click_open or click_close:
+        return not is_open
+    return is_open
+
+
+@callback(
+    Output("tableau_datatable", "filter_query", allow_duplicate=True),
+    Output("tableau_datatable", "sort_by", allow_duplicate=True),
+    Input("btn-tableau-reset", "n_clicks"),
+    prevent_initial_call=True,
+)
+def reset_view(n_clicks):
+    return "", []

src/pages/titulaire.py

@@ -1,101 +1,246 @@
 import datetime
+from typing import Any
 
+import dash_bootstrap_components as dbc
 import polars as pl
-from dash import Input, Output, State, callback, dash_table, dcc, html, register_page
+from dash import (
-
+    ClientsideFunction,
-from src.figures import point_on_map
+    Input,
-from src.utils import (
+    Output,
-    add_links_in_dict,
+    State,
-    df,
+    callback,
-    format_montant,
+    clientside_callback,
-    format_number,
+    dcc,
-    get_annuaire_data,
+    html,
-    get_departement_region,
+    register_page,
-    meta_content,
-    setup_table_columns,
 )
 
+from src.db import query_marches, schema
+from src.figures import (
+    DataTable,
+    get_distance_histogram,
+    get_top_org_table,
+    make_column_picker,
+    point_on_map,
+)
+from src.utils.data import DF_TITULAIRES, get_annuaire_data, get_departement_region
+from src.utils.frontend import get_button_properties
+from src.utils.seo import META_CONTENT
+from src.utils.table import (
+    COLUMNS,
+    filter_table_data,
+    format_number,
+    get_default_hidden_columns,
+    prepare_table_data,
+    sort_table_data,
+)
+from src.utils.tracking import track_search
+
+
+def get_title(titulaire_id: str = None) -> str:
+    titulaire_nom = DF_TITULAIRES.filter(pl.col("titulaire_id") == titulaire_id).select(
+        "titulaire_nom"
+    )
+    if titulaire_nom.height > 0:
+        return f"Marchés publics remportés par {titulaire_nom.item(0, 0)} | decp.info"
+    return "Marchés publics remportés | decp.info"
+
+
 register_page(
     __name__,
     path_template="/titulaires/<titulaire_id>",
-    title=meta_content["title"],
+    title=get_title,
     name="Titulaire",
-    description=meta_content["description"],
+    description="Consultez les marchés publics remportés par ce titulaire.",
-    image_url=meta_content["image_url"],
+    image_url=META_CONTENT["image_url"],
     order=5,
 )
 
-# 21690123100011
+DATATABLE = html.Div(
+    className="marches_table",
+    children=DataTable(
+        dtid="titulaire_datatable",
+        persistence=True,
+        persistence_type="local",
+        persisted_props=["filter_query", "sort_by"],
+        page_action="custom",
+        filter_action="custom",
+        sort_action="custom",
+        page_size=10,
+        hidden_columns=[],
+        columns=[{"id": col, "name": col} for col in schema.names()],
+    ),
+)
 
 layout = [
     dcc.Store(id="titulaire_data", storage_type="memory"),
-    dcc.Location(id="url", refresh="callback-nav"),
+    dcc.Store(id="titulaire-hidden-columns", storage_type="local"),
+    dcc.Store(id="filter-cleanup-trigger-titulaire"),
+    dcc.Location(id="titulaire_url", refresh="callback-nav"),
     html.Div(
-        className="container",
         children=[
             html.Div(
-                className="wrapper",
+                style={"marginBottom": "50px"},
                 children=[
-                    html.H2(
+                    dbc.Row(
-                        className="org_title",
+                        className="mb-2",
                         children=[
-                            html.Span(id="titulaire_siret"),
+                            dbc.Col(
-                            " - ",
+                                html.H2(
-                            html.Span(id="titulaire_nom"),
+                                    children=[
-                        ],
+                                        html.Span(id="titulaire_siret"),
-                    ),
+                                        " - ",
-                    html.Div(
+                                        html.Span(id="titulaire_nom"),
-                        className="org_year",
+                                    ],
-                        children=dcc.Dropdown(
+                                ),
-                            id="titulaire_year",
+                                width=8,
-                            options=["Toutes"]
-                            + [
-                                str(year)
-                                for year in range(
-                                    2018, int(datetime.date.today().year) + 1
-                                )
-                            ],
-                            placeholder="Année",
-                        ),
-                    ),
-                    html.Div(
-                        className="org_infos",
-                        children=[
-                            # TODO: ajouter le type d'acheteur : commune, CD, CR, etc.
-                            html.P(["Commune : ", html.Strong(id="titulaire_commune")]),
-                            html.P(
-                                [
-                                    "Département : ",
-                                    html.Strong(id="titulaire_departement"),
-                                ]
                             ),
-                            html.P(["Région : ", html.Strong(id="titulaire_region")]),
+                            dbc.Col(
-                            html.A(
+                                dcc.Dropdown(
-                                id="titulaire_lien_annuaire",
+                                    id="titulaire_year",
-                                children="Plus de détails sur l'Annuaire des entreprises",
+                                    options=["Toutes les années"]
-                                target="_blank",
+                                    + [
+                                        str(year)
+                                        for year in range(
+                                            2018, int(datetime.date.today().year) + 1
+                                        )
+                                    ],
+                                    placeholder="Année",
+                                ),
+                                width=4,
                             ),
                         ],
                     ),
-                    html.Div(
+                    dbc.Row(
-                        className="org_stats",
+                        className="mb-2",
                         children=[
-                            html.P(id="titulaire_titre_stats"),
+                            dbc.Col(
-                            html.P(id="titulaire_marches_remportes"),
+                                className="org_infos",
-                            html.P(id="titulaire_acheteurs_differents"),
+                                children=[
-                            html.Button(
+                                    # TODO: ajouter le type d'acheteur : commune, CD, CR, etc.
-                                "Téléchargement au format Excel",
+                                    html.P(
-                                id="btn-download-titulaire-data",
+                                        [
+                                            "Commune : ",
+                                            html.Strong(id="titulaire_commune"),
+                                        ]
+                                    ),
+                                    html.P(
+                                        [
+                                            "Département : ",
+                                            html.Strong(id="titulaire_departement"),
+                                        ]
+                                    ),
+                                    html.P(
+                                        [
+                                            "Région : ",
+                                            html.Strong(id="titulaire_region"),
+                                        ]
+                                    ),
+                                    html.A(
+                                        id="titulaire_lien_annuaire",
+                                        children="Plus de détails sur l'Annuaire des entreprises",
+                                    ),
+                                ],
+                                width=4,
+                            ),
+                            dbc.Col(
+                                children=[
+                                    html.P(id="titulaire_titre_stats"),
+                                    html.P(id="titulaire_marches_remportes"),
+                                    html.P(id="titulaire_acheteurs_differents"),
+                                    html.Button(
+                                        "Téléchargement au format Excel",
+                                        id="btn-download-data-titulaire",
+                                        className="btn btn-primary",
+                                    ),
+                                    dcc.Download(id="download-data-titulaire"),
+                                ],
+                                width=4,
+                            ),
+                            dbc.Col(
+                                id="titulaire_map",
+                                width=4,
                             ),
-                            dcc.Download(id="download-titulaire-data"),
                         ],
                     ),
-                    html.Div(className="org_map", id="titulaire_map"),
+                    dbc.Row(
+                        children=[
+                            dbc.Col(
+                                html.Div(
+                                    children=[
+                                        html.H3("Top acheteurs"),
+                                        html.Div(
+                                            className="marches_table",
+                                            id="top10_acheteurs",
+                                        ),
+                                    ],
+                                ),
+                                width=8,
+                            ),
+                            dbc.Col(id="titulaire-distance-histogram", width=4),
+                        ],
+                    ),
                 ],
             ),
             # récupérer les données de l'acheteur sur l'api annuaire
             html.H3("Derniers marchés publics remportés"),
-            html.Div(id="titulaire_last_marches", children=""),
+            dcc.Loading(
+                overlay_style={"visibility": "visible", "filter": "blur(2px)"},
+                id="loading-home",
+                type="default",
+                children=[
+                    html.Div(
+                        [
+                            # Bouton modal des colonnes affichées
+                            dbc.Button(
+                                "Colonnes affichées",
+                                id="titulaire_columns_open",
+                                className="column_list",
+                            ),
+                            html.P("lignes", id="titulaire_nb_rows"),
+                            html.Button(
+                                "Téléchargement désactivé au-delà de 65 000 lignes",
+                                id="btn-download-filtered-data-titulaire",
+                                disabled=True,
+                                className="btn btn-primary",
+                            ),
+                            dcc.Download(id="titulaire-download-filtered-data"),
+                            dbc.Button(
+                                "Remise à zéro",
+                                title="Supprime tous les filtres et les tris. Autrement ils sont conservés même si vous fermez la page.",
+                                id="btn-titulaire-reset",
+                                className="btn btn-primary",
+                            ),
+                        ],
+                        className="table-menu",
+                    ),
+                    dbc.Modal(
+                        [
+                            dbc.ModalHeader(
+                                dbc.ModalTitle("Choix des colonnes à afficher")
+                            ),
+                            dbc.ModalBody(
+                                id="titulaire_columns_body",
+                                children=make_column_picker("titulaire"),
+                            ),
+                            dbc.ModalFooter(
+                                dbc.Button(
+                                    "Fermer",
+                                    id="titulaire_columns_close",
+                                    className="ms-auto",
+                                    n_clicks=0,
+                                )
+                            ),
+                        ],
+                        id="titulaire_columns",
+                        is_open=False,
+                        fullscreen="md-down",
+                        scrollable=True,
+                        size="xl",
+                    ),
+                    DATATABLE,
+                ],
+            ),
         ],
     ),
 ]
@@ -109,30 +254,49 @@ layout = [
     Output(component_id="titulaire_departement", component_property="children"),
     Output(component_id="titulaire_region", component_property="children"),
     Output(component_id="titulaire_lien_annuaire", component_property="href"),
-    Input(component_id="url", component_property="pathname"),
+    Input(component_id="titulaire_url", component_property="pathname"),
 )
 def update_titulaire_infos(url):
     titulaire_siret = url.split("/")[-1]
-    if len(titulaire_siret) != 14:
-        titulaire_siret = (
-            f"Le SIRET renseigné doit faire 14 caractères ({titulaire_siret})"
-        )
     data = get_annuaire_data(titulaire_siret)
-    data_etablissement = data["matching_etablissements"][0]
+    data_etablissement = data.get("matching_etablissements") if data else None
-    titulaire_map = point_on_map(
+    if data_etablissement:
-        data_etablissement["latitude"], data_etablissement["longitude"]
+        data_etablissement = data_etablissement[0]
-    )
+
-    code_departement, nom_departement, nom_region = get_departement_region(
+        # Extraction du code département à partir du code postal
-        data_etablissement["code_postal"]
+        code_postal = data_etablissement.get("code_postal", "")
-    )
+        departement_code = code_postal[:2] if code_postal else None
-    departement = f"{nom_departement} ({code_departement})"
+
-    lien_annuaire = (
+        # Création de la carte avec le code département pour un centrage approprié
-        f"https://annuaire-entreprises.data.gouv.fr/etablissement/{titulaire_siret}"
+        titulaire_map = point_on_map(
-    )
+            data_etablissement["latitude"],
+            data_etablissement["longitude"],
+            departement_code,
+        )
+        code_departement, nom_departement, nom_region = get_departement_region(
+            data_etablissement["code_postal"]
+        )
+        departement = f"{nom_departement} ({code_departement})"
+        lien_annuaire = (
+            f"https://annuaire-entreprises.data.gouv.fr/etablissement/{titulaire_siret}"
+        )
+        raison_sociale = data["nom_raison_sociale"]
+        libelle_commune = data_etablissement["libelle_commune"]
+
+    else:
+        titulaire_map = html.Div()
+        code_departement, nom_departement, nom_region = "", "", ""
+        departement = ""
+        lien_annuaire = ""
+        raison_sociale = html.Span(
+            f"N° SIREN inconnu de l'INSEE ({titulaire_siret[:9]})"
+        )
+        libelle_commune = ""
+
     return (
         titulaire_siret,
-        data["nom_raison_sociale"],
+        raison_sociale,
-        data_etablissement["libelle_commune"],
+        libelle_commune,
         titulaire_map,
         departement,
         nom_region,
@@ -148,126 +312,98 @@ def update_titulaire_infos(url):
     Input(component_id="titulaire_data", component_property="data"),
 )
 def update_titulaire_stats(data):
-    dff = pl.DataFrame(data)
+    dff = pl.DataFrame(data, strict=False, infer_schema_length=5000)
     if dff.height == 0:
-        dff = pl.DataFrame(schema=dff.collect_schema())
+        nb_marches = 0
-    df_marches = dff.unique("uid")
+        nb_acheteurs = 0
-    nb_marches = format_number(df_marches.height)
+    else:
-    # somme_marches = format_number(int(df_marches.select(pl.sum("montant")).item()))
+        df_marches = dff.unique("uid")
-    marches_remportes = [html.Strong(nb_marches), " marchés et accord-cadres remportés"]
+        nb_marches = format_number(df_marches.height)
-    # + ", pour un total de ", html.Strong(somme_marches + " €")]
+        nb_acheteurs = dff.unique("acheteur_id").height
-    del df_marches
 
-    nb_acheteurs = dff.unique("acheteur_id").height
+    texte_marches_remportes = [
-    nb_acheteurs = [
+        html.Strong(nb_marches),
-        html.Strong(format_number(nb_acheteurs)),
+        " marchés et accord-cadres remportés",
-        " titulaires (SIRET) différents",
     ]
-    del dff
+    # + ", pour un total de ", html.Strong(somme_marches + " €")]
 
-    return marches_remportes, nb_acheteurs
+    texte_nb_acheteurs = [
+        html.Strong(format_number(nb_acheteurs)),
+        " acheteurs (SIRET) différents",
+    ]
+
+    return texte_marches_remportes, texte_nb_acheteurs
 
 
 @callback(
     Output(component_id="titulaire_data", component_property="data"),
-    Input(component_id="url", component_property="pathname"),
+    Output("btn-download-data-titulaire", "disabled"),
+    Output("btn-download-data-titulaire", "children"),
+    Output("btn-download-data-titulaire", "title"),
+    Input(component_id="titulaire_url", component_property="pathname"),
     Input(component_id="titulaire_year", component_property="value"),
 )
-def get_titulaire_marches_data(url, titulaire_year: str) -> list[dict]:
+def get_titulaire_marches_data(url, titulaire_year: str) -> tuple:
     titulaire_siret = url.split("/")[-1]
-    lff = df.lazy()
+    lff = query_marches(
-    lff = lff.filter(
+        "titulaire_id = ? AND titulaire_typeIdentifiant = 'SIRET'",
-        (pl.col("titulaire_id") == titulaire_siret)
+        (titulaire_siret,),
-        & (pl.col("titulaire_typeIdentifiant") == "SIRET")
+    ).lazy()
-    )
+    if titulaire_year and titulaire_year != "Toutes les années":
-    lff = lff.fill_null("")
-    lff = lff.select(
-        "id",
-        "uid",
-        "objet",
-        "dateNotification",
-        "acheteur_id",
-        "acheteur_nom",
-        "montant",
-        "codeCPV",
-        "dureeMois",
-    )
-    if titulaire_year and titulaire_year != "Toutes":
         lff = lff.filter(
             pl.col("dateNotification").cast(pl.String).str.starts_with(titulaire_year)
         )
     lff = lff.sort(["dateNotification", "uid"], descending=True, nulls_last=True)
-
+    lff = lff.fill_null("")
-    data = lff.collect(engine="streaming").to_dicts()
+    dff: pl.DataFrame = lff.collect(engine="streaming")
-    return data
+    download_disabled, download_text, download_title = get_button_properties(dff.height)
+    data = dff.to_dicts()
+    return data, download_disabled, download_text, download_title
 
 
 @callback(
-    Output(component_id="titulaire_last_marches", component_property="children"),
+    Output("titulaire_datatable", "data"),
+    Output("titulaire_datatable", "columns"),
+    Output("titulaire_datatable", "tooltip_header"),
+    Output("titulaire_datatable", "data_timestamp"),
+    Output("titulaire_nb_rows", "children"),
+    Output("btn-download-filtered-data-titulaire", "disabled"),
+    Output("btn-download-filtered-data-titulaire", "children"),
+    Output("btn-download-filtered-data-titulaire", "title"),
+    Output("filter-cleanup-trigger-titulaire", "data"),
+    Input(component_id="titulaire_url", component_property="href"),
+    Input("titulaire_data", "data"),
+    Input("titulaire_datatable", "page_current"),
+    Input("titulaire_datatable", "page_size"),
+    Input("titulaire_datatable", "filter_query"),
+    Input("titulaire_datatable", "sort_by"),
+    State("titulaire_datatable", "data_timestamp"),
+)
+def get_last_marches_data(
+    href, data, page_current, page_size, filter_query, sort_by, data_timestamp
+) -> list[dict]:
+    return prepare_table_data(
+        data,
+        data_timestamp,
+        filter_query,
+        page_current,
+        page_size,
+        sort_by,
+        "titulaire",
+    )
+
+
+@callback(
+    Output(component_id="top10_acheteurs", component_property="children"),
     Input(component_id="titulaire_data", component_property="data"),
 )
-def get_last_marches_table(data) -> html.Div:
+def get_top_acheteurs(data):
-    columns = [
+    return get_top_org_table(data, "acheteur", ["titulaire_distance"])
-        "uid",
-        "objet",
-        "dateNotification",
-        "acheteur_nom",
-        "montant",
-        "codeCPV",
-        "dureeMois",
-    ]
-
-    dff = pl.DataFrame(data)
-    dff = format_montant(dff)
-    columns, tooltip = setup_table_columns(
-        dff, hideable=False, exclude=["acheteur_id", "id"]
-    )
-    data = dff.to_dicts()
-    # Idéalement on utiliserait add_org_links(), mais le résultat attendu
-    # est différent de home.py (Tableau)
-    data = add_links_in_dict(data, "acheteur")
-
-    table = html.Div(
-        className="marches_table",
-        id="titulaire_datatable",
-        children=dash_table.DataTable(
-            data=data,
-            markdown_options={"html": True},
-            page_action="native",
-            filter_action="native",
-            filter_options={"case": "insensitive", "placeholder_text": "Filtrer..."},
-            columns=columns,
-            tooltip_header=tooltip,
-            tooltip_duration=8000,
-            tooltip_delay=350,
-            cell_selectable=False,
-            page_size=10,
-            style_cell_conditional=[
-                {
-                    "if": {"column_id": "objet"},
-                    "minWidth": "300px",
-                    "textAlign": "left",
-                    "overflow": "hidden",
-                    "lineHeight": "14px",
-                    "whiteSpace": "normal",
-                },
-                {
-                    "if": {"column_id": "acheteur_nom"},
-                    "maxWidth": "400px",
-                    "textAlign": "left",
-                    "overflow": "hidden",
-                    "lineHeight": "18px",
-                    "whiteSpace": "normal",
-                },
-            ],
-        ),
-    )
-    return table
 
 
 @callback(
-    Output("download-titulaire-data", "data"),
+    Output("download-data-titulaire", "data"),
-    Input("btn-download-titulaire-data", "n_clicks"),
+    Input("btn-download-data-titulaire", "n_clicks"),
     State(component_id="titulaire_data", component_property="data"),
     State(component_id="titulaire_nom", component_property="children"),
     State(component_id="titulaire_year", component_property="value"),
@@ -275,7 +411,7 @@ def get_last_marches_table(data) -> html.Div:
 )
 def download_titulaire_data(
     n_clicks,
-    data: [dict],
+    data: list[dict[str, Any]],
     titulaire_nom: str,
     annee: str,
 ):
@@ -283,8 +419,143 @@ def download_titulaire_data(
 
     def to_bytes(buffer):
         df_to_download.write_excel(
-            buffer, worksheet="DECP" if annee in ["Toutes", None] else annee
+            buffer, worksheet="DECP" if annee in ["Toutes les années", None] else annee
         )
 
     date = datetime.datetime.now().strftime("%Y-%m-%d_%H:%M:%S")
     return dcc.send_bytes(to_bytes, filename=f"decp_{titulaire_nom}_{date}.xlsx")
+
+
+@callback(
+    Output("titulaire-download-filtered-data", "data"),
+    State("titulaire_data", "data"),
+    Input("btn-download-filtered-data-titulaire", "n_clicks"),
+    State("titulaire_nom", "children"),
+    State("titulaire_datatable", "filter_query"),
+    State("titulaire_datatable", "sort_by"),
+    State("titulaire_datatable", "hidden_columns"),
+    prevent_initial_call=True,
+)
+def download_filtered_titulaire_data(
+    data,
+    n_clicks,
+    titulaire_nom,
+    filter_query,
+    sort_by,
+    hidden_columns: list | None = None,
+):
+    lff: pl.LazyFrame = pl.LazyFrame(
+        data
+    )  # start from the full titulaire data, not from paginated table data
+
+    # Les colonnes masquées sont supprimées
+    if hidden_columns:
+        lff = lff.drop(hidden_columns)
+
+    if filter_query:
+        track_search(filter_query, "titu download")
+        lff = filter_table_data(lff, filter_query)
+
+    if len(sort_by) > 0:
+        lff = sort_table_data(lff, sort_by)
+
+    def to_bytes(buffer):
+        lff.collect(engine="streaming").write_excel(buffer, worksheet="DECP")
+
+    date = datetime.datetime.now().strftime("%Y-%m-%d_%H:%M:%S")
+    return dcc.send_bytes(
+        to_bytes, filename=f"decp_filtrées_{titulaire_nom}_{date}.xlsx"
+    )
+
+
+# Pour nettoyer les icontains et i< des filtres
+# voir aussi src/assets/dash_clientside.js
+clientside_callback(
+    ClientsideFunction(
+        namespace="clientside",
+        function_name="clean_filters",
+    ),
+    Output("filter-cleanup-trigger-titulaire", "data", allow_duplicate=True),
+    Input("filter-cleanup-trigger-titulaire", "data"),
+    prevent_initial_call=True,
+)
+
+
+@callback(
+    Output("titulaire-hidden-columns", "data", allow_duplicate=True),
+    Input("titulaire_column_list", "selected_rows"),
+    prevent_initial_call=True,
+)
+def update_hidden_columns_from_checkboxes(selected_columns):
+    if selected_columns:
+        selected_columns = [COLUMNS[i] for i in selected_columns]
+        hidden_columns = [col for col in COLUMNS if col not in selected_columns]
+        return hidden_columns
+    else:
+        return []
+
+
+@callback(
+    Output("titulaire_datatable", "hidden_columns"),
+    Input(
+        "titulaire-hidden-columns",
+        "data",
+    ),
+)
+def store_hidden_columns(hidden_columns):
+    if hidden_columns is None:
+        hidden_columns = get_default_hidden_columns("titulaire")
+    return hidden_columns
+
+
+@callback(
+    Output("titulaire_column_list", "selected_rows"),
+    Input("titulaire_datatable", "hidden_columns"),
+    State("titulaire_column_list", "selected_rows"),  # pour éviter la boucle infinie
+)
+def update_checkboxes_from_hidden_columns(hidden_cols, current_checkboxes):
+    hidden_cols = hidden_cols or get_default_hidden_columns("titulaire")
+
+    # Show all columns that are NOT hidden
+    visible_cols = [COLUMNS.index(col) for col in COLUMNS if col not in hidden_cols]
+    return visible_cols
+
+
+@callback(
+    Output("titulaire_columns", "is_open"),
+    Input("titulaire_columns_open", "n_clicks"),
+    Input("titulaire_columns_close", "n_clicks"),
+    State("titulaire_columns", "is_open"),
+)
+def toggle_titulaire_columns(click_open, click_close, is_open):
+    if click_open or click_close:
+        return not is_open
+    return is_open
+
+
+@callback(
+    Output("titulaire_datatable", "filter_query", allow_duplicate=True),
+    Output("titulaire_datatable", "sort_by"),
+    Input("btn-titulaire-reset", "n_clicks"),
+    prevent_initial_call=True,
+)
+def reset_view(n_clicks):
+    return "", []
+
+
+@callback(
+    Output("titulaire-distance-histogram", "children"),
+    Input("titulaire_data", "data"),
+)
+def update_titulaire_distance_histogram(data):
+    lff = pl.LazyFrame(data)
+    if "titulaire_distance" in lff.collect_schema().names():
+        lff = lff.with_columns(
+            pl.col("titulaire_distance").cast(pl.Float64, strict=False)
+        )
+    fig = get_distance_histogram(lff)
+    return [
+        html.H3("Distance acheteur-titulaire"),
+        html.H6("par nombre de marchés", className="card-subtitle mb-2 text-muted"),
+        fig,
+    ]

src/utils.py

@@ -1,317 +0,0 @@
-import json
-import logging
-import os
-from time import sleep
-
-import polars as pl
-import polars.selectors as cs
-from httpx import get
-from polars.exceptions import ComputeError
-
-operators = [
-    ["s<", "<"],
-    ["s>", ">"],
-    ["i<", "<"],
-    ["i>", ">"],
-    ["icontains", "contains"],
-]
-
-logger = logging.getLogger("decp.info")
-logging.basicConfig(
-    format="%(asctime)s %(levelname)-8s %(message)s",
-    level=logging.INFO,
-    datefmt="%Y-%m-%d %H:%M:%S",
-)
-
-
-def split_filter_part(filter_part):
-    print("filter part", filter_part)
-    for operator_group in operators:
-        if operator_group[0] in filter_part:
-            name_part, value_part = filter_part.split(operator_group[0], 1)
-            name_part = name_part.strip()
-            value = value_part.strip()
-            name = name_part[name_part.find("{") + 1 : name_part.rfind("}")]
-            print("=>", name, operator_group[1], value)
-
-            return name, operator_group[1], value
-
-    return [None] * 3
-
-
-def add_resource_link(dff: pl.DataFrame) -> pl.DataFrame:
-    dff = dff.with_columns(
-        (
-            '<a href="' + pl.col("sourceFile") + '">' + pl.col("sourceDataset") + "</a>"
-        ).alias("source")
-    )
-    dff = dff.drop(["sourceFile", "sourceDataset"])
-    return dff
-
-
-def add_links(dff: pl.DataFrame):
-    dff = dff.with_columns(
-        pl.when(pl.col("titulaire_typeIdentifiant") == "SIRET")
-        .then(
-            '<a href = "/titulaires/'
-            + pl.col("titulaire_id")
-            + '">'
-            + pl.col("titulaire_id")
-            + "</a>"
-        )
-        .otherwise(pl.col("titulaire_id"))
-        .alias("titulaire_id")
-    )
-
-    for column, path in [("acheteur_id", "acheteurs"), ("uid", "marches")]:
-        dff = dff.with_columns(
-            (
-                f'<a href = "/{path}/'
-                + pl.col(column)
-                + '" target="_blank">'
-                + pl.col(column)
-                + "</a>"
-            ).alias(column)
-        )
-    return dff
-
-
-def add_links_in_dict(data: list, org_type: str) -> list:
-    new_data = []
-    for marche in data:
-        org_id = marche[org_type + "_id"]
-        marche[org_type + "_nom"] = (
-            f'<a href="/{org_type}s/{org_id}">{marche[org_type + "_nom"]}</a>'
-        )
-        marche["id"] = f'<a href="/marches/{marche["uid"]}">{marche["id"]}</a>'
-        marche["uid"] = f'<a href="/marches/{marche["uid"]}">{marche["uid"]}</a>'
-        new_data.append(marche)
-    return new_data
-
-
-def booleans_to_strings(lff: pl.LazyFrame) -> pl.LazyFrame:
-    """
-    Convert all boolean columns to string type.
-    """
-    lff = lff.with_columns(
-        pl.col(cs.Boolean)
-        .cast(pl.String)
-        .str.replace("true", "oui")
-        .str.replace("false", "non")
-    )
-    return lff
-
-
-def numbers_to_strings(lff: pl.LazyFrame) -> pl.LazyFrame:
-    """
-    Convert all numeric columns to string type.
-    """
-    lff = lff.with_columns(pl.col(pl.Float64, pl.Int16).cast(pl.String).fill_null(""))
-    return lff
-
-
-def dates_to_strings(lff: pl.LazyFrame, column: str) -> pl.LazyFrame:
-    """
-    Convert a date column to string type.
-    """
-    lff = lff.with_columns(pl.col(column).cast(pl.String).fill_null(""))
-    return lff
-
-
-def format_number(number) -> str:
-    number = "{:,}".format(number).replace(",", " ")
-    return number
-
-
-def format_montant(dff: pl.DataFrame) -> pl.DataFrame:
-    def format_function(expr, scale=None):
-        # https://stackoverflow.com/a/78636786
-        expr = expr.cast(pl.String).str.splitn(".", 2)
-
-        num = expr.struct[0]
-        frac = expr.struct[1]
-
-        # Ajout des espaces
-        num = (
-            num.str.reverse()
-            .str.replace_all(r"\d{3}", "$0 ")
-            .str.reverse()
-            .str.replace(r"^ ", "")
-        )
-
-        frac: pl.Expr = (
-            pl.when(frac.is_not_null() & ~frac.is_in(["0"]))
-            .then("," + frac)
-            .otherwise(pl.lit(""))
-        )
-
-        return num + frac + pl.lit(" €")
-
-    dff = dff.with_columns(pl.col("montant").pipe(format_function).alias("montant"))
-    return dff
-
-
-def get_annuaire_data(siret: str) -> dict:
-    url = f"https://recherche-entreprises.api.gouv.fr/search?q={siret}"
-    response = get(url)
-    return response.json()["results"][0]
-
-
-def get_decp_data() -> pl.DataFrame:
-    # Chargement du fichier parquet
-    # Le fichier est chargé en mémoire, ce qui est plus rapide qu'une base de données pour le moment.
-    # On utilise polars pour la rapidité et la facilité de manipulation des données.
-
-    try:
-        logger.info(
-            f"Lecture du fichier parquet ({os.getenv('DATA_FILE_PARQUET_PATH')})..."
-        )
-        lff: pl.LazyFrame = pl.scan_parquet(os.getenv("DATA_FILE_PARQUET_PATH"))
-    except ComputeError:
-        # Le fichier est probablement en cours de mise à jour
-        logger.info("Échec, nouvelle tentative dans 10s...")
-        sleep(10)
-        lff: pl.LazyFrame = pl.scan_parquet(os.getenv("DATA_FILE_PARQUET_PATH"))
-
-    # Tri des marchés par date de notification
-    lff = lff.sort(by=["dateNotification", "uid"], descending=True, nulls_last=True)
-
-    # Uniquement les données actuelles, pas les anciennes versions de marchés
-    lff = lff.filter(pl.col("donneesActuelles")).drop("donneesActuelles")
-
-    # Convertir les colonnes booléennes en chaînes de caractères
-    lff = booleans_to_strings(lff)
-
-    # Bizarrement je ne peux pas faire lff = lff.fill_null("") ici
-    # ça génère une erreur dans la page acheteur (acheteur_data.table) :
-    # AttributeError: partially initialized module 'pandas' has no attribute 'NaT' (most likely due to a circular import)
-
-    return lff.collect()
-
-
-def get_departements() -> dict:
-    with open("data/departements.json", "rb") as f:
-        data = json.load(f)
-        return data
-
-
-def get_departement_region(code_postal):
-    if code_postal > "97000":
-        code_departement = code_postal[:3]
-    else:
-        code_departement = code_postal[:2]
-    nom_departement = departements[code_departement]["departement"]
-    nom_region = departements[code_departement]["region"]
-    return code_departement, nom_departement, nom_region
-
-
-def filter_table_data(lff: pl.LazyFrame, filter_query: str) -> pl.LazyFrame:
-    debug = os.getenv("DEVELOPMENT", "False").lower() == "true"
-    schema = lff.collect_schema()
-    filtering_expressions = filter_query.split(" && ")
-    for filter_part in filtering_expressions:
-        col_name, operator, filter_value = split_filter_part(filter_part)
-        col_type = str(schema[col_name])
-        if debug:
-            print("filter_value:", filter_value)
-            print("filter_value_type:", type(filter_value))
-            print("col_type:", col_type)
-
-        if col_type == "Date":
-            # Convertir la colonne en chaînes de caractères
-            lff = dates_to_strings(lff, col_name)
-
-        if operator in ("<", "<=", ">", ">="):
-            lff = lff.filter(
-                pl.col(col_name).is_not_null() & (pl.col(col_name) != pl.lit(""))
-            )
-            if operator == "<":
-                lff = lff.filter(pl.col(col_name) < filter_value)
-            elif operator == ">":
-                lff = lff.filter(pl.col(col_name) > filter_value)
-            elif operator == ">=":
-                lff = lff.filter(pl.col(col_name) >= filter_value)
-            elif operator == "<=":
-                lff = lff.filter(pl.col(col_name) <= filter_value)
-
-        elif col_type.startswith("Int") or col_type.startswith("Float"):
-            try:
-                filter_value = int(filter_value)
-            except ValueError:
-                logger.error(f"Invalid numeric filter value: {filter_value}")
-                continue
-            lff = lff.filter(pl.col(col_name) == filter_value)
-
-        elif operator == "contains" and col_type in ["String", "Date"]:
-            lff = lff.filter(pl.col(col_name).str.contains("(?i)" + filter_value))
-
-        # elif operator == 'datestartswith':
-        # lff = lff.filter(pl.col(col_name).str.startswith(filter_value)")
-
-    return lff
-
-
-def sort_table_data(lff: pl.LazyFrame, sort_by: list) -> pl.LazyFrame:
-    lff = lff.sort(
-        [col["column_id"] for col in sort_by],
-        descending=[col["direction"] == "desc" for col in sort_by],
-        nulls_last=True,
-    )
-    print(sort_by)
-    return lff
-
-
-def setup_table_columns(dff, hideable: bool = True, exclude: list = None) -> tuple:
-    # Liste finale de colonnes
-    columns = []
-    tooltip = {}
-    for column_id in dff.columns:
-        if exclude and column_id in exclude:
-            continue
-        column_object = data_schema.get(column_id)
-        if column_object:
-            column_name = column_object.get("friendly_name", column_id)
-        else:
-            column_name = column_id
-
-        column = {
-            "name": column_name,
-            "id": column_id,
-            "presentation": "markdown",
-            "type": "text",
-            "format": {"nully": "N/A"},
-            "hideable": hideable,
-        }
-        columns.append(column)
-
-        if column_object:
-            tooltip[column_id] = {
-                "value": f"""**{column_object.get("friendly_name")}** ({column_id})
-
-    """
-                + column_object["description"],
-                "type": "markdown",
-            }
-    return columns, tooltip
-
-
-df: pl.DataFrame = get_decp_data()
-departements = get_departements()
-domain_name = (
-    "test.decp.info" if os.getenv("DEVELOPMENT").lower() == "true" else "decp.info"
-)
-meta_content = {
-    "image_url": f"https://{domain_name}/assets/decp.info.png",
-    "title": "decp.info - exploration des marchés publics français",
-    "description": (
-        "Explorez et analysez les données des marchés publics français avec cet outil libre et gratuit. "
-        "Pour une commande publique accessible à toutes et tous."
-    ),
-}
-
-# Récupération du schéma des données tabulaires
-data_schema: dict = get(os.getenv("DATA_SCHEMA_PATH"), follow_redirects=True).json()
-data_schema["source"] = {
-    "description": "Code de la source des données, avec un lien vers le fichier Open Data dont proviennent les données de ce marché public.",
-    "friendly_name": "Source des données",
-}

src/utils/__init__.py

@@ -0,0 +1,19 @@
+import logging
+import os
+
+logging.basicConfig(
+    format="%(asctime)s %(levelname)-8s %(message)s",
+    level=logging.INFO,
+    datefmt="%Y-%m-%d %H:%M:%S",
+)
+DEVELOPMENT = os.getenv("DEVELOPMENT", "False").lower() == "true"
+logger = logging.getLogger("decp.info")
+
+if DEVELOPMENT:
+    logger.setLevel(logging.DEBUG)
+
+DOMAIN_NAME = (
+    "test.decp.info"
+    if os.getenv("DEVELOPMENT", "False").lower() == "true"
+    else "decp.info"
+)

src/utils/cache.py

@@ -0,0 +1,4 @@
+from flask_caching import Cache
+
+# Isolé dans un fichier dédié pour éviter les imports circulaires
+cache = Cache()

src/utils/data.py

@@ -0,0 +1,115 @@
+import json
+import logging
+import os
+from collections import OrderedDict
+
+import polars as pl
+from httpx import HTTPError, get
+
+from src.db import get_cursor, query_marches, schema
+from src.utils import logger
+
+logging.getLogger("httpx").setLevel("WARNING")
+
+
+def get_annuaire_data(siret: str) -> dict | None:
+    url = f"https://recherche-entreprises.api.gouv.fr/search?q={siret}"
+    try:
+        response = get(url).raise_for_status()
+        response = response.json()["results"][0]
+    except (HTTPError, IndexError):
+        response = None
+        logger.warning("Could not fetch data from recherche-entreprises.api.")
+    return response
+
+
+def get_statistics() -> dict:
+    return (
+        get(
+            "https://www.data.gouv.fr/api/1/datasets/r/0ccf4a75-f3aa-4b46-8b6a-18aeb63e36df",
+            follow_redirects=True,
+        )
+        .raise_for_status()
+        .json()
+    )
+
+
+def get_departements() -> dict:
+    with open("data/departements.json", "rb") as f:
+        data = json.load(f)
+        return data
+
+
+def get_departements_geojson() -> dict:
+    with open("./data/departements-1000m.geojson") as f:
+        geojson = json.load(f)
+
+    # Ajout de feature.id
+    for f in geojson["features"]:
+        f["id"] = f["properties"]["code"]
+
+    return geojson
+
+
+def get_departement_region(code_postal):
+    if code_postal > "97000":
+        code_departement = code_postal[:3]
+    else:
+        code_departement = code_postal[:2]
+    nom_departement = DEPARTEMENTS[code_departement]["departement"]
+    nom_region = DEPARTEMENTS[code_departement]["region"]
+    return code_departement, nom_departement, nom_region
+
+
+def get_data_schema() -> dict:
+    # Récupération du schéma des données tabulaires
+    path = os.getenv("DATA_SCHEMA_PATH")
+    if path.startswith("http"):
+        original_schema: dict = get(
+            os.getenv("DATA_SCHEMA_PATH"), follow_redirects=True
+        ).json()
+    elif os.path.exists(path):
+        with open(path) as f:
+            original_schema: dict = json.load(f)
+    else:
+        raise Exception(f"Chemin vers le schéma invalide: {path}")
+
+    new_schema = OrderedDict()
+
+    for col in original_schema["fields"]:
+        new_schema[col["name"]] = col
+
+    return new_schema
+
+
+def prepare_dashboard_data(**filter_params) -> pl.DataFrame:
+    """Exécute la requête DuckDB filtrée pour le tableau de bord.
+
+    Retourne une pl.DataFrame matérialisée uniquement pour le sous-ensemble
+    correspondant aux filtres. Les appelants qui ont besoin d'une LazyFrame
+    appellent `.lazy()` sur le résultat.
+    """
+    from src.utils.table_sql import dashboard_filters_to_sql
+
+    where_sql, params = dashboard_filters_to_sql(**filter_params)
+    return query_marches(where_sql=where_sql, params=params)
+
+
+def build_org_frame(org_type: str) -> pl.DataFrame:
+    org_cols = [
+        c
+        for c in schema.names()
+        if c.startswith(f"{org_type}_")
+        and c not in (f"{org_type}_latitude", f"{org_type}_longitude")
+    ]
+    select_list = ", ".join(org_cols)
+    group_list = ", ".join(org_cols)
+    sql = f'SELECT {select_list}, COUNT(*) AS "Marchés" FROM decp GROUP BY {group_list}'
+    return get_cursor().execute(sql).pl()
+
+
+DF_ACHETEURS = build_org_frame("acheteur")
+DF_TITULAIRES = build_org_frame("titulaire")
+DEPARTEMENTS = get_departements()
+DEPARTEMENTS_GEOJSON = get_departements_geojson()
+DATA_SCHEMA = get_data_schema()

src/utils/frontend.py

@@ -0,0 +1,27 @@
+from src.utils.data import DATA_SCHEMA
+
+
+def get_button_properties(height):
+    if height > 65000:
+        download_disabled = True
+        download_text = "Téléchargement désactivé au-delà de 65 000 lignes"
+        download_title = " Ajoutez des filtres pour réduire le nombre de lignes, Excel ne supporte pas d'avoir plus de 65 000 URLs dans une même feuille de calcul."
+    elif height == 0:
+        download_disabled = True
+        download_text = "Pas de données à télécharger"
+        download_title = ""
+    else:
+        download_disabled = False
+        download_text = "Télécharger au format Excel"
+        download_title = "Télécharger les données telles qu'affichées au format Excel"
+    return download_disabled, download_text, download_title
+
+
+def get_enum_values_as_dict(column_name):
+    try:
+        options = {}
+        for value in DATA_SCHEMA[column_name]["enum"]:
+            options[value] = value
+        return options
+    except KeyError:
+        return {"not_found": "not found"}

src/utils/search.py

@@ -0,0 +1,84 @@
+import polars as pl
+from unidecode import unidecode
+
+from src.utils.table import add_links
+from src.utils.tracking import track_search
+
+
+def search_org(dff: pl.DataFrame, query: str, org_type: str) -> pl.DataFrame:
+    """
+    Search in either 'acheteur' or 'titulaire' DataFrame.
+
+    :param dff: Polars DataFrame with acheteur or titulaire columns
+    :param query: User search string
+    :param org_type: 'acheteur' or 'titulaire'
+    :return: Filtered DataFrame with 'matches' column
+    """
+    if not query.strip():
+        return dff.select(pl.lit(False).alias("matches"))
+
+    # Enregistrement des recherche dans Matomo
+    track_search(query, "home_page_search")
+
+    # Normalize query
+    normalized_query = unidecode(query.strip()).upper()
+    tokens = [" " + t.strip() for t in normalized_query.split() if t.strip()]
+
+    # Define columns based on entity type
+    cols = [
+        f"{org_type}_id",
+        f"{org_type}_nom",
+        f"{org_type}_departement_nom",
+        f"{org_type}_departement_code",
+        f"{org_type}_commune_nom",
+    ]
+
+    # Concatenate all fields into one string per row
+    org_str = pl.concat_str(pl.lit(" "), pl.col(cols), separator=" ").str.replace(
+        "-", " "
+    )
+
+    # For each token, create a boolean column: True if token is found
+    token_matches = []
+    for token in tokens:
+        token_match = org_str.str.contains(token).alias(f"token_{token}")
+        token_matches.append(token_match)
+
+    # Count how many tokens match per row
+    match_score = pl.sum_horizontal(token_matches).alias("match_score")
+
+    # For each token, create a boolean column: True if token is found
+    token_matches = []
+    for token in tokens:
+        token_match = org_str.str.contains(token).alias(f"token_{token}")
+        token_matches.append(token_match)
+
+    # Sélection des colonnes
+    if org_type == "acheteur":
+        dff = dff.select(cols + ["Marchés"])
+    if org_type == "titulaire":
+        dff = dff.select(cols + ["Marchés", "titulaire_typeIdentifiant"])
+
+    # Apply and filter
+    dff = (
+        dff.with_columns(token_matches + [match_score])
+        .filter(pl.col("match_score") == len(tokens))
+        .drop([f"token_{token}" for token in tokens])
+    )
+
+    # Format result
+    dff = add_links(dff)
+    dff = dff.with_columns(
+        pl.concat_str(
+            pl.col(f"{org_type}_departement_nom"),
+            pl.lit(" ("),
+            pl.col(f"{org_type}_departement_code"),
+            pl.lit(")"),
+        ).alias("Département")
+    )
+
+    dff = dff.select(f"{org_type}_id", f"{org_type}_nom", "Département", "Marchés")
+    dff = dff.group_by(f"{org_type}_id", f"{org_type}_nom", "Département").sum()
+    dff = dff.sort("Marchés", descending=True)
+
+    return dff

src/utils/seo.py

@@ -0,0 +1,54 @@
+from src.utils import DOMAIN_NAME
+from src.utils.data import get_annuaire_data
+
+
+def make_org_jsonld(org_id, org_type, org_name=None, type_org_id="SIRET") -> dict:
+    org_types = {"acheteur": "GovernmentOrganization", "titulaire": "Organization"}
+    address = None
+    if type_org_id.lower() == "siret" and len(org_id) == 14:
+        annuaire_data = get_annuaire_data(org_id)
+        if not annuaire_data:
+            return {}
+        annuaire_address = annuaire_data["matching_etablissements"][0]
+        code_postal = annuaire_address["code_postal"]
+        commune = annuaire_address["libelle_commune"]
+
+        address = (
+            {
+                "@type": "PostalAddress",
+                "streetAddress": annuaire_address.get("adresse", "")
+                .replace(code_postal, "")
+                .replace(commune, "")
+                .strip(),
+                "addressLocality": commune,
+                "postalCode": code_postal,
+                "addressCountry": "FR",
+            },
+        )
+
+    jsonld = {
+        "@type": org_types[org_type],
+        "name": org_name,
+        "url": f"https://decp.info/{org_type}s/{org_id}",
+        "sameAs": f"https://annuaire-entreprises.data.gouv.fr/etablissement/{org_id}",
+        "identifier": {
+            "@type": "PropertyValue",
+            "propertyID": type_org_id.lower(),
+            "value": org_id,
+        },
+    }
+
+    if address:
+        jsonld["address"] = address
+
+    return jsonld
+
+
+META_CONTENT = {
+    "image_url": f"https://{DOMAIN_NAME}/assets/decp.info.png",
+    "title": "decp.info - exploration des marchés publics français",
+    "description": (
+        "Explorez et analysez les données des marchés publics français avec cet outil libre et gratuit. "
+        "Pour une commande publique accessible à toutes et tous."
+    ),
+}

src/utils/table.py

@@ -0,0 +1,533 @@
+import os
+import uuid
+
+import polars as pl
+from dash import no_update
+from polars import selectors as cs
+
+from src.db import count_marches, count_unique_marches, query_marches, schema
+from src.utils import logger
+from src.utils.cache import cache
+from src.utils.data import DATA_SCHEMA
+from src.utils.frontend import get_button_properties
+from src.utils.tracking import track_search
+
+
+def split_filter_part(filter_part):
+    operators = [
+        ["s<", "<"],
+        ["s>", ">"],
+        ["i<", "<"],
+        ["i>", ">"],
+        ["icontains", "contains"],
+        # [" ", "contains"]
+    ]
+    logger.debug("filter part " + filter_part)
+    for operator_group in operators:
+        if operator_group[0] in filter_part:
+            name_part, value_part = filter_part.split(operator_group[0], 1)
+            name_part = name_part.strip()
+            value = value_part.strip()
+            name = name_part[name_part.find("{") + 1 : name_part.rfind("}")]
+            logger.debug("=> " + " ".join([name, operator_group[1], value]))
+
+            return name, operator_group[1], value
+
+    return [None] * 3
+
+
+def add_resource_link(dff: pl.DataFrame) -> pl.DataFrame:
+    dff = dff.with_columns(
+        (
+            '<a href="' + pl.col("sourceFile") + '">' + pl.col("sourceDataset") + "</a>"
+        ).alias("sourceDataset")
+    )
+    dff = dff.drop(["sourceFile"])
+    return dff
+
+
+def add_links(dff: pl.DataFrame):
+    for col in ["uid", "acheteur_nom", "titulaire_nom", "acheteur_id", "titulaire_id"]:
+        if col in dff.columns:
+            if col.startswith("titulaire_"):
+                detail_link = (
+                    '<a href = "/titulaires/'
+                    + pl.col("titulaire_id")
+                    + '">'
+                    + pl.col(col)
+                    + "</a>"
+                )
+                if col == "titulaire_nom":
+                    detail_link = (
+                        detail_link
+                        + ' <a href="/observatoire?titulaire_id='
+                        + pl.col("titulaire_id")
+                        + '" title="Voir dans l\'observatoire">📊</a>'
+                    )
+                dff = dff.with_columns(
+                    pl.when(
+                        pl.Expr.or_(
+                            pl.col("titulaire_typeIdentifiant").is_null(),
+                            pl.col("titulaire_typeIdentifiant") == "SIRET",
+                        )
+                    )
+                    .then(detail_link)
+                    .otherwise(pl.col(col))
+                    .alias(col)
+                )
+            if col.startswith("acheteur_"):
+                detail_link = (
+                    '<a href = "/acheteurs/'
+                    + pl.col("acheteur_id")
+                    + '">'
+                    + pl.col(col)
+                    + "</a>"
+                )
+                if col == "acheteur_nom":
+                    detail_link = (
+                        detail_link
+                        + ' <a href="/observatoire?acheteur_id='
+                        + pl.col("acheteur_id")
+                        + '" title="Voir dans l\'observatoire">📊</a>'
+                    )
+                dff = dff.with_columns(detail_link.alias(col))
+            if col == "uid":
+                dff = dff.with_columns(
+                    (
+                        '<a href = "/marches/'
+                        + pl.col("uid")
+                        + '">'
+                        + pl.col("uid")
+                        + "</a>"
+                    ).alias("uid")
+                )
+    return dff
+
+
+def add_links_in_dict(data: list[dict], org_type: str) -> list:
+    new_data = []
+    for marche in data:
+        org_id = marche[org_type + "_id"]
+        marche[org_type + "_nom"] = (
+            f'<a href="/{org_type}s/{org_id}">{marche[org_type + "_nom"]}</a>'
+        )
+        if marche.get("uid"):
+            marche["id"] = f'<a href="/marches/{marche["uid"]}">{marche["id"]}</a>'
+            marche["uid"] = f'<a href="/marches/{marche["uid"]}">{marche["uid"]}</a>'
+        new_data.append(marche)
+    return new_data
+
+
+def booleans_to_strings(lff: pl.LazyFrame) -> pl.LazyFrame:
+    """
+    Convert all boolean columns to string type.
+    """
+    lff = lff.with_columns(
+        pl.col(cs.Boolean)
+        .cast(pl.String)
+        .str.replace("true", "oui")
+        .str.replace("false", "non")
+    )
+    return lff
+
+
+def numbers_to_strings(lff: pl.LazyFrame) -> pl.LazyFrame:
+    """
+    Convert all numeric columns to string type.
+    """
+    lff = lff.with_columns(pl.col(pl.Float64, pl.Int16).cast(pl.String).fill_null(""))
+    return lff
+
+
+def dates_to_strings(lff: pl.LazyFrame, column: str) -> pl.LazyFrame:
+    """
+    Convert a date column to string type.
+    """
+    lff = lff.with_columns(pl.col(column).cast(pl.String).fill_null(""))
+    return lff
+
+
+def normalize_sort_by(sort_by) -> tuple:
+    if not sort_by:
+        return ()
+    return tuple((entry["column_id"], entry["direction"]) for entry in sort_by)
+
+
+def format_number(number) -> str:
+    if not number:
+        return ""
+    number = "{:,}".format(number).replace(",", " ")
+    return number
+
+
+def unformat_montant(number: str) -> float:
+    number = number.replace(" €", "")
+    number = number.replace(" €", "").replace(" ", "")
+    number = number.replace(",", ".")
+    number = number.strip()
+    return float(number)
+
+
+def format_values(dff: pl.DataFrame) -> pl.DataFrame:
+    def format_montant(expr):
+        # https://stackoverflow.com/a/78636786
+        expr = expr.cast(pl.String)
+        expr = expr.str.splitn(".", 2)
+
+        num = expr.struct[0]
+        frac = expr.struct[1]
+
+        # Ajout des espaces
+        num = (
+            num.str.reverse()
+            .str.replace_all(r"\d{3}", "$0 ")
+            .str.reverse()
+            .str.replace(r"^ ", "")
+        )
+
+        frac: pl.Expr = (
+            pl.when(frac.is_not_null() & ~frac.is_in(["0"]))
+            .then("," + frac.str.head(2))
+            .otherwise(pl.lit(""))
+        )
+
+        montant: pl.Expr = (
+            pl.when((num + frac) == pl.lit(""))
+            .then(pl.lit(""))
+            .otherwise(num + frac + pl.lit(" €"))
+        )
+
+        return montant
+
+    def format_distance(expr):
+        expr = expr.cast(pl.String)
+        return pl.concat_str(expr, pl.lit(" km"))
+
+    if "montant" in dff.columns:
+        dff = dff.with_columns(pl.col("montant").pipe(format_montant).alias("montant"))
+    if "titulaire_distance" in dff.columns:
+        dff = dff.with_columns(
+            pl.col("titulaire_distance")
+            .pipe(format_distance)
+            .alias("titulaire_distance")
+        )
+
+    return dff
+
+
+def filter_table_data(lff: pl.LazyFrame, filter_query: str) -> pl.LazyFrame:
+    _schema = lff.collect_schema()
+    filtering_expressions = filter_query.split(" && ")
+    for filter_part in filtering_expressions:
+        col_name, operator, filter_value = split_filter_part(filter_part)
+        if not isinstance(col_name, str) or not isinstance(filter_value, str):
+            continue
+        col_type = str(_schema[col_name])
+        # logger.debug("filter_value:", filter_value)
+        # logger.debug("filter_value_type:", type(filter_value))
+        # logger.debug("operator:", operator)
+        # logger.debug("col_type:", col_type)
+
+        lff = lff.filter(pl.col(col_name).is_not_null())
+
+        if col_type == "Date":
+            # Convertir la colonne date en chaînes de caractères
+            lff = dates_to_strings(lff, col_name)
+            col_type = "String"
+        if col_type == "String":
+            lff = lff.filter(pl.col(col_name) != pl.lit(""))
+
+        elif col_type.startswith("Int") or col_type.startswith("Float"):
+            try:
+                filter_value = int(filter_value)
+            except ValueError:
+                logger.error(f"Invalid numeric filter value: {filter_value}")
+                continue
+
+        if operator in ("contains", "<", "<=", ">", ">="):
+            if operator == "<":
+                lff = lff.filter(pl.col(col_name) < filter_value)
+            elif operator == ">":
+                lff = lff.filter(pl.col(col_name) > filter_value)
+            elif operator == ">=":
+                lff = lff.filter(pl.col(col_name) >= filter_value)
+            elif operator == "<=":
+                lff = lff.filter(pl.col(col_name) <= filter_value)
+            elif operator == "contains":
+                if col_type in ["String", "Date"] and isinstance(filter_value, str):
+                    filter_value = filter_value.strip('"')
+                    if filter_value.endswith("*"):
+                        lff = lff.filter(
+                            pl.col(col_name).str.starts_with(filter_value[:-1])
+                        )
+                    elif filter_value.startswith("*"):
+                        lff = lff.filter(
+                            pl.col(col_name).str.ends_with(filter_value[1:])
+                        )
+                    else:
+                        lff = lff.filter(
+                            pl.col(col_name).str.contains("(?i)" + filter_value)
+                        )
+                elif col_type.startswith("Int") or col_type.startswith("Float"):
+                    lff = lff.filter(pl.col(col_name) == filter_value)
+                else:
+                    logger.error(f"Invalid column type: {col_type}")
+        else:
+            logger.error(f"Invalid operator: {operator}")
+
+        # elif operator == 'datestartswith':
+        # lff = lff.filter(pl.col(col_name).str.startswith(filter_value)")
+
+    return lff
+
+
+def sort_table_data(lff: pl.LazyFrame, sort_by: list) -> pl.LazyFrame:
+    lff = lff.sort(
+        [col["column_id"] for col in sort_by],
+        descending=[col["direction"] == "desc" for col in sort_by],
+        nulls_last=True,
+    )
+    logger.debug(sort_by)
+    return lff
+
+
+def setup_table_columns(
+    dff,
+    hideable: bool = True,
+    exclude: list | None = None,
+) -> tuple:
+    # Liste finale de colonnes
+    markdown_exceptions = ["montant", "titulaire_distance", "distance", "dureeMois"]
+    columns = []
+    tooltip = {}
+    for column_id in dff.columns:
+        if exclude and column_id in exclude:
+            continue
+        column_object = DATA_SCHEMA.get(column_id)
+        if column_object:
+            column_name = column_object.get("title")
+        else:
+            # Si le champ est un champ créé par erreur lors d'une jointure, on le skip
+            if column_id.endswith("_left") or column_id.endswith("_right"):
+                logger.warning(f"Champ innatendu : {column_id}")
+                continue
+            column_name = column_id
+            column_object = {"title": column_name, "description": ""}
+
+        presentation = "input" if column_id in markdown_exceptions else "markdown"
+
+        column = {
+            "name": column_name,
+            "id": column_id,
+            "presentation": presentation,
+            "type": "text",
+            "format": {"nully": "N/A"},
+            "hideable": hideable,
+        }
+        columns.append(column)
+        if column_object:
+            tooltip[column_id] = {
+                "value": f"""**{column_object.get("title")}** ({column_id})
+
+    """
+                + column_object.get("description", ""),
+                "type": "markdown",
+            }
+    return columns, tooltip
+
+
+def get_default_hidden_columns(page):
+    if page == "acheteur":
+        displayed_columns = [
+            "uid",
+            "objet",
+            "dateNotification",
+            "titulaire_id",
+            "titulaire_typeIdentifiant",
+            "titulaire_nom",
+            "titulaire_distance",
+            "montant",
+            "codeCPV",
+            "dureeRestanteMois",
+        ]
+    elif page == "titulaire":
+        displayed_columns = [
+            "uid",
+            "objet",
+            "dateNotification",
+            "acheteur_id",
+            "acheteur_nom",
+            "titulaire_distance",
+            "montant",
+            "codeCPV",
+            "dureeRestanteMois",
+        ]
+    elif page == "tableau":
+        displayed_columns = os.getenv("DISPLAYED_COLUMNS")
+    else:
+        displayed_columns = os.getenv("DISPLAYED_COLUMNS")
+        logger.warning(f"Invalid page: {page}")
+
+    hidden_columns = []
+
+    for col in schema.names():
+        if col in displayed_columns:
+            continue
+        else:
+            hidden_columns.append(col)
+    return hidden_columns
+
+
+def postprocess_page(dff: pl.DataFrame) -> pl.DataFrame:
+    """Post-traitement à appliquer sur une page déjà paginée.
+
+    À appeler après la pagination.
+    """
+    dff = dff.with_columns(pl.all().cast(pl.String).fill_null(""))
+    dff = add_links(dff)
+    if "sourceFile" in dff.columns:
+        dff = add_resource_link(dff)
+    if dff.height > 0:
+        dff = format_values(dff)
+    return dff
+
+
+@cache.memoize()
+def _fetch_page_sql(
+    filter_query: str | None,
+    sort_by_key: tuple,
+    page_current: int,
+    page_size: int,
+) -> tuple[pl.DataFrame, int, int]:
+    """Chemin rapide : filtre/tri/pagine dans DuckDB, post-traite la page seule.
+
+    Retourne (page_dataframe_post_traitée, total_count, total_unique_count).
+    """
+    # Import local pour éviter une dépendance circulaire
+    # (src.utils.table_sql importe split_filter_part depuis src.utils.table).
+    from src.utils.table_sql import filter_query_to_sql, sort_by_to_sql
+
+    logger.debug(
+        f"Cache miss SQL — filter={filter_query!r} sort={sort_by_key!r} "
+        f"page={page_current} size={page_size}"
+    )
+
+    where_sql, params = filter_query_to_sql(filter_query or "", schema)
+
+    sort_by_dash = [
+        {"column_id": col, "direction": direction} for col, direction in sort_by_key
+    ]
+    order_by = sort_by_to_sql(sort_by_dash, schema) or None
+
+    total = count_marches(where_sql, params)
+    total_unique = count_unique_marches(where_sql, params)
+
+    page = query_marches(
+        where_sql=where_sql,
+        params=params,
+        order_by=order_by,
+        limit=page_size,
+        offset=page_current * page_size,
+    )
+
+    page = postprocess_page(page)
+    return page, total, total_unique
+
+
+def prepare_table_data(
+    data, data_timestamp, filter_query, page_current, page_size, sort_by, source_table
+):
+    """
+    Fonction de préparation des données pour les datatables, afin de permettre une gestion fine des logiques,
+    notamment pour les filtres et les tris.
+    :param data
+    :param data_timestamp:
+    :param filter_query:
+    :param page_current:
+    :param page_size:
+    :param sort_by:
+    :param source_table:
+    :return:
+    """
+    logger.debug(" + + + + + + + + + + + + + + + + + + ")
+
+    if filter_query:
+        track_search(filter_query, source_table)
+
+    trigger_cleanup = no_update if source_table == "tableau" else str(uuid.uuid4())
+
+    if data is None:
+        # Probablement car il s'agit de la page Tableau
+        sort_by_key = normalize_sort_by(sort_by)
+        dff, height, total_unique = _fetch_page_sql(
+            filter_query=filter_query,
+            sort_by_key=sort_by_key,
+            page_current=page_current,
+            page_size=page_size,
+        )
+    else:
+        if isinstance(data, list):
+            lff: pl.LazyFrame = pl.LazyFrame(
+                data, strict=False, infer_schema_length=5000
+            )
+        elif isinstance(data, pl.LazyFrame):
+            lff = data
+        else:
+            lff = query_marches().lazy()
+
+        if filter_query:
+            lff = filter_table_data(lff, filter_query)
+
+        df_height = lff.select("uid").collect(engine="streaming")
+        height = df_height.height
+        total_unique = df_height["uid"].n_unique()
+
+        if sort_by and len(sort_by) > 0:
+            lff = sort_table_data(lff, sort_by)
+
+        start_row = page_current * page_size
+        lff = lff.slice(start_row, page_size)
+        dff = lff.collect(engine="streaming")
+        dff: pl.DataFrame = postprocess_page(dff)
+
+    if height > 0:
+        nb_rows = (
+            f"{format_number(height)} lignes ({format_number(total_unique)} marchés)"
+        )
+    else:
+        nb_rows = "0 lignes (0 marchés)"
+
+    table_columns, tooltip = setup_table_columns(dff)
+
+    dicts = dff.to_dicts()
+
+    download_disabled, download_text, download_title = get_button_properties(height)
+
+    return (
+        dicts,
+        table_columns,
+        tooltip,
+        data_timestamp + 1,
+        nb_rows,
+        download_disabled,
+        download_text,
+        download_title,
+        trigger_cleanup,
+    )
+
+
+def invert_columns(columns):
+    """
+    Renvoie les colonnes du schéma non spécifiées en paramètre. Utile pour passer d'une colonnes masquées à une liste de colonnes affichées, et vice versa.
+
+    :param columns:
+    :return:
+    """
+    inverted_columns = []
+    for column in schema.names():
+        if column not in columns:
+            inverted_columns.append(column)
+    return inverted_columns
+
+
+COLUMNS = schema.names()

src/utils/table_sql.py

@@ -0,0 +1,241 @@
+from datetime import datetime, timedelta
+
+import polars as pl
+
+from src.utils import logger
+from src.utils.table import split_filter_part
+
+
+def filter_query_to_sql(filter_query: str, schema: pl.Schema) -> tuple[str, list]:
+    """Traduit le DSL de filtres de dash_table.DataTable en fragment SQL DuckDB.
+
+    Retourne (where_clause, params) où where_clause est un fragment à injecter
+    après WHERE et params est la liste des valeurs à passer à
+    cursor.execute(sql, params). Les identifiants de colonnes sont validés
+    contre le schéma fourni ; jamais concaténés avec des valeurs utilisateur.
+    """
+    if not filter_query:
+        return "TRUE", []
+
+    clauses: list[str] = []
+    params: list = []
+
+    for part in filter_query.split(" && "):
+        col_name, operator, raw_value = split_filter_part(part)
+        if not isinstance(col_name, str) or not isinstance(raw_value, str):
+            continue
+
+        if col_name not in schema.names():
+            logger.warning(f"Colonne inconnue ignorée : {col_name!r}")
+            continue
+
+        col_type = schema[col_name]
+        is_numeric = col_type.is_numeric()
+        col_is_date = col_type == pl.Date
+        quoted_col = f'"{col_name}"'
+
+        if is_numeric:
+            try:
+                value = int(raw_value) if col_type.is_integer() else float(raw_value)
+            except ValueError:
+                logger.warning(f"Valeur numérique invalide ignorée : {raw_value!r}")
+                continue
+
+            if operator == "contains":
+                clauses.append(f"{quoted_col} IS NOT NULL AND {quoted_col} = ?")
+            elif operator == ">":
+                clauses.append(f"{quoted_col} IS NOT NULL AND {quoted_col} > ?")
+            elif operator == "<":
+                clauses.append(f"{quoted_col} IS NOT NULL AND {quoted_col} < ?")
+            else:
+                logger.warning(f"Opérateur invalide pour numérique : {operator!r}")
+                continue
+            params.append(value)
+            continue
+
+        # String / Date : toujours traité comme texte (parité avec Polars)
+        value = raw_value.strip('"')
+
+        if operator == "contains":
+            if col_is_date:
+                target = f"CAST({quoted_col} AS VARCHAR)"
+
+            if col_name in ("acheteur_id", "titulaire_id"):
+                value = value.replace(" ", "")
+            where_clause, param_list = tokenize_text_filter(
+                col_name, value, col_is_date
+            )
+            clauses.append(where_clause)
+            params.extend(param_list)
+            logger.debug(params)
+            continue
+
+        elif operator in (">", "<"):
+            target = f"CAST({quoted_col} AS VARCHAR)" if col_is_date else quoted_col
+            clauses.append(f"{quoted_col} IS NOT NULL AND {target} {operator} ?")
+            params.append(value)
+        else:
+            logger.warning(f"Opérateur invalide pour chaîne : {operator!r}")
+            continue
+
+    if not clauses:
+        return "TRUE", []
+    return " AND ".join(clauses), params
+
+
+def sort_by_to_sql(sort_by: list[dict] | None, schema: pl.Schema) -> str:
+    """Traduit sort_by (format Dash) en clause ORDER BY DuckDB.
+
+    Retourne '' si pas de tri (aucun ORDER BY à ajouter).
+    """
+    if not sort_by:
+        return ""
+
+    fragments: list[str] = []
+    for entry in sort_by:
+        col = entry.get("column_id")
+        direction = entry.get("direction")
+        if col not in schema.names():
+            logger.warning(f"Tri sur colonne inconnue ignoré : {col!r}")
+            continue
+        if direction not in ("asc", "desc"):
+            logger.warning(f"Tri sur direction inconnue ignoré : {direction!r}")
+            continue
+        fragments.append(f'"{col}" {direction.upper()} NULLS LAST')
+
+    return ", ".join(fragments)
+
+
+def dashboard_filters_to_sql(
+    dashboard_year=None,
+    dashboard_acheteur_id=None,
+    dashboard_acheteur_categorie=None,
+    dashboard_acheteur_departement_code=None,
+    dashboard_titulaire_id=None,
+    dashboard_titulaire_categorie=None,
+    dashboard_titulaire_departement_code=None,
+    dashboard_marche_type=None,
+    dashboard_marche_objet=None,
+    dashboard_marche_code_cpv=None,
+    dashboard_marche_considerations_sociales=None,
+    dashboard_marche_considerations_environnementales=None,
+    dashboard_marche_techniques=None,
+    dashboard_marche_innovant=None,
+    dashboard_marche_sous_traitance_declaree=None,
+    dashboard_montant_min=None,
+    dashboard_montant_max=None,
+) -> tuple[str, list]:
+    """Traduit les filtres du tableau de bord en (where_clause, params) DuckDB."""
+    clauses: list[str] = []
+    params: list = []
+
+    if dashboard_year:
+        clauses.append('YEAR("dateNotification") = ?')
+        params.append(int(dashboard_year))
+    else:
+        clauses.append('"dateNotification" > ?')
+        params.append(datetime.now() - timedelta(days=365))
+
+    if dashboard_acheteur_id:
+        dashboard_acheteur_id = dashboard_acheteur_id.replace(" ", "")
+        clauses.append('"acheteur_id" LIKE ?')
+        params.append(f"%{dashboard_acheteur_id}%")
+    else:
+        if dashboard_acheteur_categorie:
+            clauses.append('"acheteur_categorie" = ?')
+            params.append(dashboard_acheteur_categorie)
+        if dashboard_acheteur_departement_code:
+            placeholders = ", ".join(["?"] * len(dashboard_acheteur_departement_code))
+            clauses.append(f'"acheteur_departement_code" IN ({placeholders})')
+            params.extend(dashboard_acheteur_departement_code)
+
+    if dashboard_titulaire_id:
+        dashboard_titulaire_id = dashboard_titulaire_id.replace(" ", "")
+        clauses.append('"titulaire_id" LIKE ?')
+        params.append(f"%{dashboard_titulaire_id}%")
+    else:
+        if dashboard_titulaire_categorie:
+            clauses.append('"titulaire_categorie" = ?')
+            params.append(dashboard_titulaire_categorie)
+        if dashboard_titulaire_departement_code:
+            placeholders = ", ".join(["?"] * len(dashboard_titulaire_departement_code))
+            clauses.append(f'"titulaire_departement_code" IN ({placeholders})')
+            params.extend(dashboard_titulaire_departement_code)
+
+    if dashboard_marche_type:
+        clauses.append('"type" = ?')
+        params.append(dashboard_marche_type)
+
+    if dashboard_marche_objet:
+        where_clause, param_list = tokenize_text_filter("objet", dashboard_marche_objet)
+        clauses.append(where_clause)
+        params.extend(param_list)
+
+    if dashboard_marche_code_cpv:
+        clauses.append('"codeCPV" LIKE ?')
+        params.append(f"{dashboard_marche_code_cpv}%")
+
+    if dashboard_marche_innovant and dashboard_marche_innovant != "all":
+        clauses.append('"marcheInnovant" = ?')
+        params.append(dashboard_marche_innovant)
+
+    if (
+        dashboard_marche_sous_traitance_declaree
+        and dashboard_marche_sous_traitance_declaree != "all"
+    ):
+        clauses.append('"sousTraitanceDeclaree" = ?')
+        params.append(dashboard_marche_sous_traitance_declaree)
+
+    if dashboard_marche_techniques:
+        clauses.append("list_has_any(string_split(\"techniques\", ', '), ?::VARCHAR[])")
+        params.append(list(dashboard_marche_techniques))
+
+    if dashboard_marche_considerations_sociales:
+        clauses.append(
+            "list_has_any(string_split(\"considerationsSociales\", ', '), ?::VARCHAR[])"
+        )
+        params.append(list(dashboard_marche_considerations_sociales))
+
+    if dashboard_marche_considerations_environnementales:
+        clauses.append(
+            "list_has_any(string_split(\"considerationsEnvironnementales\", ', '), ?::VARCHAR[])"
+        )
+        params.append(list(dashboard_marche_considerations_environnementales))
+
+    if dashboard_montant_min is not None:
+        clauses.append('"montant" >= ?')
+        params.append(dashboard_montant_min)
+
+    if dashboard_montant_max is not None:
+        clauses.append('"montant" <= ?')
+        params.append(dashboard_montant_max)
+
+    return " AND ".join(clauses), params
+
+
+def tokenize_text_filter(
+    column: str, text: str, col_is_date: bool = False
+) -> tuple[str, list]:
+    terms = text.split()
+    # si col_is_date alors le deuxième doit être casté en VARCHAR
+    if col_is_date:
+        quoted_col = f'CAST("{column}" AS VARCHAR)'
+    else:
+        quoted_col = f'"{column}"'
+
+    conditions = [f'"{column}" IS NOT NULL', f"{quoted_col} <> ''"]
+
+    params = []
+
+    for term in terms:
+        conditions.append(f"{quoted_col} ILIKE ?")
+
+        if term.startswith("*") or term.endswith("*"):
+            params.append(term.replace("*", "%"))
+        elif "+" in term:
+            params.append(f"%{term.replace('+', ' ')}%")
+        else:
+            params.append(f"%{term}%")
+
+    where_clause = " AND ".join(conditions)
+    return where_clause, params

src/utils/tracking.py

@@ -0,0 +1,30 @@
+import os
+import uuid
+from time import localtime
+
+from httpx import post
+
+from src.utils import DEVELOPMENT
+
+
+def track_search(query, category):
+    if len(query) >= 4 and not DEVELOPMENT and os.getenv("MATOMO_DOMAIN"):
+        url = "https://decp.info"
+        params = {
+            "idsite": os.getenv("MATOMO_ID_SITE"),
+            "url": url,
+            "rec": "1",
+            "action_name": "search" if category == "home_page_search" else "filter",
+            "search_cat": category,
+            "rand": uuid.uuid4().hex,
+            "apiv": "1",
+            "h": localtime().tm_hour,
+            "m": localtime().tm_min,
+            "s": localtime().tm_sec,
+            "search": query,
+            "token_auth": os.getenv("MATOMO_TOKEN"),
+        }
+        post(
+            url=f"https://{os.getenv('MATOMO_DOMAIN')}/matomo.php",
+            params=params,
+        ).raise_for_status()

tests/conftest.py

@@ -0,0 +1,83 @@
+import datetime
+import os
+from pathlib import Path
+
+import polars as pl
+import pytest
+from selenium.webdriver.chrome.options import Options
+
+_TEST_DATA = [
+    {
+        "uid": "1",
+        "id": "1",
+        "acheteur_nom": "ACHETEUR 1",
+        "acheteur_id": "123",
+        "titulaire_nom": "TITULAIRE 1",
+        "titulaire_id": "345",
+        "montant": 10,
+        "dateNotification": datetime.date(2025, 1, 1),
+        "codeCPV": "71600000",
+        "donneesActuelles": True,
+        "acheteur_departement_code": "75",
+        "acheteur_departement_nom": "Paris",
+        "acheteur_commune_nom": "Paris",
+        "titulaire_departement_code": "35",
+        "titulaire_departement_nom": "Ille-et-Vilaine",
+        "titulaire_commune_nom": "Rennes",
+        "titulaire_distance": 10,
+        "titulaire_typeIdentifiant": "SIRET",
+        "objet": "Objet test",
+        "dureeRestanteMois": 12,
+        "lieuExecution_code": "75001",
+        "sourceFile": "test.xml",
+        "sourceDataset": "test_dataset",
+        "datePublicationDonnees": datetime.date(2025, 1, 1),
+        "considerationsSociales": "",
+        "considerationsEnvironnementales": "",
+        "type": "Marché",
+        "acheteur_categorie": "Collectivité",
+        "titulaire_categorie": "PME",
+    }
+]
+_PARQUET_PATH = Path(os.path.abspath("tests/test.parquet"))
+_DB_PATH = Path(os.path.abspath("decp.duckdb"))
+
+
+def _cleanup_db_artifacts() -> None:
+    for artifact in (
+        _DB_PATH,
+        _DB_PATH.with_suffix(".duckdb.tmp"),
+        _DB_PATH.with_suffix(".duckdb.lock"),
+    ):
+        if artifact.exists():
+            artifact.unlink()
+
+
+# Runs at conftest import, before test modules import src.db (which builds the
+# DuckDB at import time). Guarantees the test parquet exists and the stale DB
+# from a previous `python run.py` is wiped so src.db rebuilds from test data.
+pl.DataFrame(_TEST_DATA).write_parquet(_PARQUET_PATH)
+_cleanup_db_artifacts()
+
+
+@pytest.fixture(scope="session", autouse=True)
+def test_data():
+    yield str(_PARQUET_PATH)
+    # Teardown: remove the test DuckDB so the next `python run.py` rebuilds
+    # from decp_prod.parquet.
+    _cleanup_db_artifacts()
+
+
+def pytest_setup_options():
+    options = Options()
+    options.add_argument("--window-size=1200,1200 ")
+    options.add_experimental_option(
+        "prefs",
+        {
+            "download.default_directory": "/home/colin/git/decp.info",
+            "download.prompt_for_download": False,
+            "download.directory_upgrade": True,
+            "safebrowsing.enabled": True,
+        },
+    )
+    return options

tests/test_dashboard_filters_to_sql.py

@@ -0,0 +1,225 @@
+from datetime import datetime, timedelta
+
+from src.utils.table_sql import dashboard_filters_to_sql
+
+
+def test_no_filters_uses_default_365_day_window():
+    where_sql, params = dashboard_filters_to_sql()
+    assert where_sql == '"dateNotification" > ?'
+    assert len(params) == 1
+    assert isinstance(params[0], datetime)
+    expected = datetime.now() - timedelta(days=365)
+    assert abs((params[0] - expected).total_seconds()) < 2
+
+
+def test_year_filter_overrides_default_window():
+    where_sql, params = dashboard_filters_to_sql(dashboard_year="2025")
+    assert where_sql == 'YEAR("dateNotification") = ?'
+    assert params == [2025]
+
+
+def test_marche_type_equality():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_type="Marché",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "type" = ?'
+    assert params == [2025, "Marché"]
+
+
+def test_innovant_value_all_is_skipped():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_innovant="all",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ?'
+    assert params == [2025]
+
+
+def test_innovant_value_oui_adds_clause():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_innovant="oui",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "marcheInnovant" = ?'
+    assert params == [2025, "oui"]
+
+
+def test_sous_traitance_value_non_adds_clause():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_sous_traitance_declaree="non",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "sousTraitanceDeclaree" = ?'
+    assert params == [2025, "non"]
+
+
+def test_acheteur_id_uses_like_wildcards():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_acheteur_id="12345678900010",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "acheteur_id" LIKE ?'
+    assert params == [2025, "%12345678900010%"]
+
+
+def test_titulaire_id_uses_like_wildcards():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_titulaire_id="999",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "titulaire_id" LIKE ?'
+    assert params == [2025, "%999%"]
+
+
+def test_marche_objet_uses_case_insensitive_ilike():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_objet="travaux",
+    )
+    assert (
+        where_sql
+        == 'YEAR("dateNotification") = ? AND "objet" IS NOT NULL AND "objet" <> \'\' AND "objet" ILIKE ?'
+    )
+    assert params == [2025, "%travaux%"]
+
+
+def test_code_cpv_uses_prefix_like():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_code_cpv="4521",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "codeCPV" LIKE ?'
+    assert params == [2025, "4521%"]
+
+
+def test_acheteur_departement_multiple_uses_in_clause():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_acheteur_departement_code=["75", "92", "93"],
+    )
+    assert where_sql == (
+        'YEAR("dateNotification") = ? AND "acheteur_departement_code" IN (?, ?, ?)'
+    )
+    assert params == [2025, "75", "92", "93"]
+
+
+def test_acheteur_categorie_adds_clause():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_acheteur_categorie="Commune",
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "acheteur_categorie" = ?'
+    assert params == [2025, "Commune"]
+
+
+def test_titulaire_categorie_and_departement():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_titulaire_categorie="PME",
+        dashboard_titulaire_departement_code=["35"],
+    )
+    assert where_sql == (
+        'YEAR("dateNotification") = ? '
+        'AND "titulaire_categorie" = ? '
+        'AND "titulaire_departement_code" IN (?)'
+    )
+    assert params == [2025, "PME", "35"]
+
+
+def test_acheteur_id_present_skips_categorie_and_departement():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_acheteur_id="123",
+        dashboard_acheteur_categorie="Commune",
+        dashboard_acheteur_departement_code=["75"],
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "acheteur_id" LIKE ?'
+    assert params == [2025, "%123%"]
+
+
+def test_titulaire_id_present_skips_categorie_and_departement():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_titulaire_id="999",
+        dashboard_titulaire_categorie="PME",
+        dashboard_titulaire_departement_code=["35"],
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "titulaire_id" LIKE ?'
+    assert params == [2025, "%999%"]
+
+
+def test_marche_techniques_uses_list_has_any():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_techniques=["Enchère", "Accord-cadre"],
+    )
+    assert where_sql == (
+        'YEAR("dateNotification") = ? '
+        "AND list_has_any(string_split(\"techniques\", ', '), ?::VARCHAR[])"
+    )
+    assert params == [2025, ["Enchère", "Accord-cadre"]]
+
+
+def test_considerations_sociales_uses_list_has_any():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_considerations_sociales=["Clause sociale"],
+    )
+    assert where_sql == (
+        'YEAR("dateNotification") = ? '
+        "AND list_has_any(string_split(\"considerationsSociales\", ', '), ?::VARCHAR[])"
+    )
+    assert params == [2025, ["Clause sociale"]]
+
+
+def test_considerations_environnementales_uses_list_has_any():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_marche_considerations_environnementales=["Clause env."],
+    )
+    assert where_sql == (
+        'YEAR("dateNotification") = ? '
+        "AND list_has_any(string_split(\"considerationsEnvironnementales\", ', '), ?::VARCHAR[])"
+    )
+    assert params == [2025, ["Clause env."]]
+
+
+def test_montant_min_only():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_montant_min=1000,
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "montant" >= ?'
+    assert params == [2025, 1000]
+
+
+def test_montant_max_only():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_montant_max=500,
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "montant" <= ?'
+    assert params == [2025, 500]
+
+
+def test_montant_zero_is_a_valid_lower_bound():
+    # 0 est falsy mais reste un filtre valide (distinct de None)
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_montant_min=0,
+    )
+    assert where_sql == 'YEAR("dateNotification") = ? AND "montant" >= ?'
+    assert params == [2025, 0]
+
+
+def test_montant_min_and_max_combined():
+    where_sql, params = dashboard_filters_to_sql(
+        dashboard_year="2025",
+        dashboard_montant_min=100,
+        dashboard_montant_max=1000,
+    )
+    assert where_sql == (
+        'YEAR("dateNotification") = ? AND "montant" >= ? AND "montant" <= ?'
+    )
+    assert params == [2025, 100, 1000]

tests/test_db.py

@@ -0,0 +1,300 @@
+import datetime
+import os
+import time
+
+import polars as pl
+import pytest
+
+from src.db import should_rebuild
+
+
+@pytest.fixture
+def parquet_and_db(tmp_path, monkeypatch):
+    parquet = tmp_path / "source.parquet"
+    db = tmp_path / "decp.duckdb"
+    parquet.write_bytes(b"fake parquet content")
+    monkeypatch.delenv("REBUILD_DUCKDB", raising=False)
+    monkeypatch.delenv("DEVELOPMENT", raising=False)
+    return parquet, db
+
+
+def test_should_rebuild_when_db_missing(parquet_and_db):
+    parquet, db = parquet_and_db
+    assert should_rebuild(db, parquet) is True
+
+
+def test_should_rebuild_prod_when_parquet_newer(parquet_and_db, monkeypatch):
+    parquet, db = parquet_and_db
+    db.write_bytes(b"x")
+    parquet.touch()
+    now = time.time()
+    os.utime(db, (now, now))
+    os.utime(parquet, (now + 10, now + 10))
+    monkeypatch.setenv("DEVELOPMENT", "false")
+    assert should_rebuild(db, parquet) is True
+
+
+def test_should_not_rebuild_prod_when_parquet_older(parquet_and_db, monkeypatch):
+    parquet, db = parquet_and_db
+    parquet.touch()
+    db.write_bytes(b"x")
+    now = time.time()
+    os.utime(parquet, (now, now))
+    os.utime(db, (now + 10, now + 10))
+    monkeypatch.setenv("DEVELOPMENT", "false")
+    assert should_rebuild(db, parquet) is False
+
+
+def test_should_not_rebuild_dev_even_when_parquet_newer(parquet_and_db, monkeypatch):
+    parquet, db = parquet_and_db
+    db.write_bytes(b"x")
+    parquet.touch()
+    now = time.time()
+    os.utime(db, (now, now))
+    os.utime(parquet, (now + 10, now + 10))
+    monkeypatch.setenv("DEVELOPMENT", "true")
+    monkeypatch.delenv("REBUILD_DUCKDB", raising=False)
+    assert should_rebuild(db, parquet) is False
+
+
+def test_should_rebuild_dev_when_rebuild_forced(parquet_and_db, monkeypatch):
+    parquet, db = parquet_and_db
+    db.write_bytes(b"x")
+    parquet.touch()
+    now = time.time()
+    os.utime(db, (now, now))
+    os.utime(parquet, (now + 10, now + 10))
+    monkeypatch.setenv("DEVELOPMENT", "true")
+    monkeypatch.setenv("REBUILD_DUCKDB", "true")
+    assert should_rebuild(db, parquet) is True
+
+
+@pytest.fixture
+def built_db(tmp_path, monkeypatch):
+    """Build a DuckDB from a small Polars frame written as parquet."""
+    parquet_path = tmp_path / "source.parquet"
+    db_path = tmp_path / "decp.duckdb"
+
+    data = pl.DataFrame(
+        [
+            {
+                "uid": "1",
+                "id": "1",
+                "objet": "Travaux",
+                "acheteur_id": "123",
+                "acheteur_nom": "ACHETEUR 1",
+                "acheteur_departement_code": "75",
+                "acheteur_departement_nom": "Paris",
+                "acheteur_commune_nom": "Paris",
+                "titulaire_commune_nom": "Paris",
+                "titulaire_departement_nom": "Paris",
+                "titulaire_id": "345",
+                "titulaire_nom": "TITULAIRE 1",
+                "titulaire_departement_code": "35",
+                "titulaire_typeIdentifiant": "SIRET",
+                "montant": 1000.0,
+                "dateNotification": datetime.date(2025, 1, 1),
+                "donneesActuelles": True,
+                "marcheInnovant": True,
+            },
+            {
+                "uid": "2",
+                "id": "2",
+                "objet": "Études",
+                "acheteur_id": "123",
+                "acheteur_nom": "ACHETEUR 1",
+                "acheteur_departement_code": "75",
+                "acheteur_departement_nom": "Paris",
+                "acheteur_commune_nom": "Paris",
+                "titulaire_commune_nom": "Paris",
+                "titulaire_departement_nom": "Paris",
+                "titulaire_id": "567",
+                "titulaire_nom": None,
+                "titulaire_departement_code": "75",
+                "titulaire_typeIdentifiant": "SIRET",
+                "montant": 500.0,
+                "dateNotification": datetime.date(2024, 6, 1),
+                "donneesActuelles": True,
+                "marcheInnovant": False,
+            },
+            {
+                "uid": "3",
+                "id": "3",
+                "objet": "Ancien",
+                "acheteur_id": "A2",
+                "acheteur_nom": None,
+                "acheteur_departement_code": "13",
+                "acheteur_departement_nom": "Paris",
+                "acheteur_commune_nom": "Paris",
+                "titulaire_commune_nom": "Paris",
+                "titulaire_departement_nom": "Paris",
+                "titulaire_id": "T3",
+                "titulaire_nom": "Autre",
+                "titulaire_departement_code": "13",
+                "titulaire_typeIdentifiant": "SIRET",
+                "montant": 100.0,
+                "dateNotification": datetime.date(2023, 1, 1),
+                "donneesActuelles": False,  # must be filtered out
+                "marcheInnovant": False,
+            },
+        ]
+    )
+    data.write_parquet(parquet_path)
+    monkeypatch.setenv("DATA_FILE_PARQUET_PATH", str(parquet_path))
+    monkeypatch.setenv("DUCKDB_PATH", str(db_path))
+
+    from src.db import build_database
+
+    build_database(db_path, parquet_path)
+    return db_path
+
+
+def test_build_filters_donnees_actuelles(built_db):
+    import duckdb
+
+    with duckdb.connect(str(built_db), read_only=True) as c:
+        rows = c.execute("SELECT uid FROM decp ORDER BY uid").fetchall()
+    assert [r[0] for r in rows] == ["1", "2"]
+
+
+def test_build_converts_booleans_to_oui_non(built_db):
+    import duckdb
+
+    with duckdb.connect(str(built_db), read_only=True) as c:
+        values = c.execute("SELECT marcheInnovant FROM decp ORDER BY uid").fetchall()
+    assert [v[0] for v in values] == ["oui", "non"]
+
+
+def test_build_replaces_null_org_names(built_db):
+    import duckdb
+
+    with duckdb.connect(str(built_db), read_only=True) as c:
+        titulaire_2 = c.execute(
+            "SELECT titulaire_nom FROM decp WHERE uid = '2'"
+        ).fetchone()
+    assert titulaire_2[0] == "[Identifiant non reconnu dans la base INSEE]"
+
+
+def test_build_creates_derived_tables(built_db):
+    import duckdb
+
+    with duckdb.connect(str(built_db), read_only=True) as c:
+        tables = {r[0] for r in c.execute("SHOW TABLES").fetchall()}
+    assert {
+        "decp",
+        "acheteurs_marches",
+        "titulaires_marches",
+        "acheteurs_departement",
+        "titulaires_departement",
+    } <= tables
+
+
+def test_query_marches_returns_polars_frame(built_db, monkeypatch):
+    monkeypatch.setenv(
+        "DATA_FILE_PARQUET_PATH", str(built_db.parent / "source.parquet")
+    )
+    # Force src.db to load pointing at this test DB.
+    import importlib
+
+    import src.db
+
+    importlib.reload(src.db)
+    from src.db import query_marches
+
+    frame = query_marches("acheteur_id = ?", ("123",))
+    assert isinstance(frame, pl.DataFrame)
+    assert frame.height == 2
+    assert set(frame["uid"].to_list()) == {"1", "2"}
+
+
+def test_count_marches_returns_total_without_filter():
+    from src.db import count_marches
+
+    n = count_marches()
+    assert isinstance(n, int)
+    assert n > 0
+
+
+def test_count_marches_with_filter():
+    from src.db import count_marches
+
+    n = count_marches('"uid" = ?', ["__nonexistent__"])
+    assert n == 0
+
+
+def test_count_unique_marches_respects_distinct():
+    from src.db import count_unique_marches
+
+    n = count_unique_marches()
+    assert isinstance(n, int)
+    assert n > 0
+
+
+def test_query_marches_with_offset():
+    from src.db import query_marches
+
+    page_0 = query_marches(limit=2, offset=0)
+    page_1 = query_marches(limit=2, offset=2)
+    if page_0.height == 2 and page_1.height >= 1:
+        assert set(page_0["uid"].to_list()).isdisjoint(set(page_1["uid"].to_list()))
+
+
+def test_concurrent_build_serialized(tmp_path):
+    """Multiple threads calling _ensure_database must serialize via flock.
+
+    Only one should actually build; others wait, see the fresh DB, and skip.
+    No tmp file should leak. No exceptions should occur.
+    """
+    import fcntl
+    import threading
+
+    import src.db as db
+
+    # Set up source parquet
+    parquet_path = tmp_path / "src.parquet"
+    df = pl.DataFrame(
+        {
+            "uid": ["A"],
+            "donneesActuelles": [True],
+            "dateNotification": ["2024-01-01"],
+            "objet": ["Test"],
+            "acheteur_id": ["a1"],
+            "acheteur_nom": ["A1"],
+            "titulaire_id": ["t1"],
+            "titulaire_nom": ["T1"],
+            "acheteur_departement_code": ["75"],
+            "titulaire_departement_code": ["75"],
+            "montant": [1000.0],
+            "dureeMois": [12],
+        }
+    )
+    df.write_parquet(parquet_path)
+
+    db_path = tmp_path / "decp.duckdb"
+    lock_path = db_path.with_suffix(".duckdb.lock")
+    tmp_path_artifact = db_path.with_suffix(".duckdb.tmp")
+
+    errors: list[BaseException] = []
+
+    def worker():
+        try:
+            # Mirror the locking logic in _ensure_database
+            with open(lock_path, "w") as lf:
+                fcntl.flock(lf.fileno(), fcntl.LOCK_EX)
+                try:
+                    if db.should_rebuild(db_path, parquet_path):
+                        db.build_database(db_path, parquet_path)
+                finally:
+                    fcntl.flock(lf.fileno(), fcntl.LOCK_UN)
+        except BaseException as exc:
+            errors.append(exc)
+
+    threads = [threading.Thread(target=worker) for _ in range(3)]
+    for t in threads:
+        t.start()
+    for t in threads:
+        t.join()
+
+    assert errors == []
+    assert db_path.exists()
+    assert not tmp_path_artifact.exists()

tests/test_main.py

@@ -0,0 +1,344 @@
+import polars as pl
+from dash.testing.composite import DashComposite
+from selenium.webdriver import Keys
+from selenium.webdriver.common.by import By
+from selenium.webdriver.remote.webelement import WebElement
+
+
+def test_001_logo_and_search(dash_duo: DashComposite):
+    from src.app import app
+
+    dash_duo.start_server(app)
+    dash_duo.wait_for_text_to_equal(".logo > h1", "decp.info", timeout=4)
+    assert dash_duo.find_element(".logo > h1").text == "decp.info"
+
+    for org_type in ["acheteur", "titulaire"]:
+        name = f"{org_type.upper()} 1"
+        search_bar: WebElement = dash_duo.find_element("#search")
+
+        dash_duo.clear_input(search_bar)
+
+        search_bar.send_keys(name)
+        search_bar.send_keys(Keys.ENTER)
+
+        dash_duo.wait_for_element(f"#results_{org_type}_datatable", timeout=2)
+        result_table: WebElement = dash_duo.find_element(
+            f"#results_{org_type}_datatable tbody"
+        )
+
+        assert len(result_table.find_elements(by=By.TAG_NAME, value="tr")) == 2, (
+            "The search should return only one result"
+        )  # header row + 1 result
+        assert result_table.find_element(
+            by=By.CSS_SELECTOR, value=f'td[data-dash-column="{org_type}_nom"]'
+        ).text.startswith(name), (
+            f"The search result should have the right {org_type} name"
+        )
+
+
+def test_002_filter_persistence(dash_duo: DashComposite):
+    from src.app import app
+
+    dash_duo.start_server(app)
+    dash_duo.wait_for_text_to_equal(".logo > h1", "decp.info", timeout=4)
+
+    def open_page_and_check_filter_input():
+        dash_duo.wait_for_page(f"{dash_duo.server_url}/{page}")
+        filter_input_selector = (
+            '.marches_table th[data-dash-column="uid"] input[type="text"]'
+        )
+        dash_duo.wait_for_element(filter_input_selector, timeout=2)
+        _filter_input: WebElement = dash_duo.find_element(filter_input_selector)
+        return _filter_input
+
+    for page in ["tableau", "acheteurs/123", "titulaires/345"]:
+        filter_input = open_page_and_check_filter_input()
+        filter_input.send_keys("11")  # a UID that doesn't exist
+        filter_input.send_keys(Keys.ENTER)
+        filter_input = open_page_and_check_filter_input()
+        assert filter_input.get_attribute("value") == "11"
+
+
+def test_003_tableau_download(dash_duo: DashComposite):
+    from pages.acheteur import download_acheteur_data
+    from pages.tableau import download_data
+    from pages.titulaire import download_titulaire_data
+    from src.app import app
+
+    # Juste pour instancier l'app
+    print(app.server.name)
+
+    dicts = pl.read_parquet("tests/test.parquet").to_dicts()
+
+    outputs = [
+        download_data(1, "", [], None),
+        download_acheteur_data(1, dicts, "123", "2025"),
+        download_titulaire_data(1, dicts, "345", "2025"),
+    ]
+    for output in outputs:
+        assert isinstance(output, dict)
+        for f in ["content", "filename", "type", "base64"]:
+            assert f in output
+        assert isinstance(output["content"], str) and len(output["content"]) > 100
+        assert isinstance(output["filename"], str) and output["filename"].startswith(
+            "decp_"
+        )
+        assert output["type"] is None
+        assert output["base64"] is True
+
+
+def test_004_add_links_observatoire_acheteur():
+    import polars as pl
+
+    from src.utils.table import add_links
+
+    dff = pl.DataFrame(
+        {
+            "acheteur_id": ["123"],
+            "acheteur_nom": ["ACHETEUR 1"],
+        }
+    )
+    result = add_links(dff)
+    nom_value = result["acheteur_nom"][0]
+    id_value = result["acheteur_id"][0]
+
+    # acheteur_nom should contain detail link + observatoire link
+    assert "/acheteurs/123" in nom_value
+    assert "ACHETEUR 1" in nom_value
+    assert "/observatoire?acheteur_id=123" in nom_value
+    assert "📊" in nom_value
+
+    # acheteur_id should NOT contain observatoire link
+    assert "/observatoire" not in id_value
+
+
+def test_005_add_links_observatoire_titulaire():
+    import polars as pl
+
+    from src.utils.table import add_links
+
+    dff = pl.DataFrame(
+        {
+            "titulaire_id": ["345"],
+            "titulaire_nom": ["TITULAIRE 1"],
+            "titulaire_typeIdentifiant": ["SIRET"],
+        }
+    )
+    result = add_links(dff)
+    nom_value = result["titulaire_nom"][0]
+    id_value = result["titulaire_id"][0]
+
+    # titulaire_nom should contain detail link + observatoire link
+    assert "/titulaires/345" in nom_value
+    assert "TITULAIRE 1" in nom_value
+    assert "/observatoire?titulaire_id=345" in nom_value
+    assert "📊" in nom_value
+
+    # titulaire_id should NOT contain observatoire link
+    assert "/observatoire" not in id_value
+
+
+def test_006_observatoire_url_to_input(dash_duo: DashComposite):
+    from src.app import app
+
+    dash_duo.start_server(app)
+    dash_duo.wait_for_text_to_equal(".logo > h1", "decp.info", timeout=4)
+
+    # Navigate to observatoire with acheteur_id query param
+    dash_duo.wait_for_page(f"{dash_duo.server_url}/observatoire?acheteur_id=123")
+    dash_duo.wait_for_element("#dashboard_acheteur_id", timeout=4)
+
+    import time
+
+    time.sleep(1)  # Allow callback chain to complete
+
+    acheteur_input = dash_duo.find_element("#dashboard_acheteur_id")
+    assert acheteur_input.get_attribute("value") == "123", (
+        "acheteur_id input should be populated from URL param"
+    )
+
+
+def test_007_observatoire_share_url(dash_duo: DashComposite):
+    from src.app import app
+
+    dash_duo.start_server(app)
+    dash_duo.wait_for_text_to_equal(".logo > h1", "decp.info", timeout=4)
+
+    # Navigate to observatoire with acheteur_id query param
+    dash_duo.wait_for_page(f"{dash_duo.server_url}/observatoire?acheteur_id=123")
+    dash_duo.wait_for_element("#observatoire-share-url", timeout=4)
+
+    import time
+
+    time.sleep(1)  # Allow callback chain to complete
+
+    share_url_input = dash_duo.find_element("#observatoire-share-url")
+    share_url_value = share_url_input.get_attribute("value")
+
+    assert "acheteur_id=123" in share_url_value, (
+        f"Share URL should contain acheteur_id param, got: {share_url_value}"
+    )
+
+
+def test_008_search_to_observatoire(dash_duo: DashComposite):
+    from src.app import app
+
+    dash_duo.start_server(app)
+    dash_duo.wait_for_text_to_equal(".logo > h1", "decp.info", timeout=4)
+
+    # Search for an acheteur
+    search_bar = dash_duo.find_element("#search")
+    search_bar.send_keys("ACHETEUR 1")
+    search_bar.send_keys(Keys.ENTER)
+
+    dash_duo.wait_for_element("#results_acheteur_datatable", timeout=2)
+
+    # Find the observatoire link in acheteur_nom column
+    observatoire_link = dash_duo.find_element(
+        '#results_acheteur_datatable td[data-dash-column="acheteur_nom"] a[href*="observatoire"]'
+    )
+    assert "📊" in observatoire_link.text
+
+    # Click the observatoire link
+    observatoire_link.click()
+
+    # Wait for observatoire page to load
+    dash_duo.wait_for_element("#dashboard_acheteur_id", timeout=4)
+
+    import time
+
+    time.sleep(1)  # Allow callback chain to complete
+
+    acheteur_input = dash_duo.find_element("#dashboard_acheteur_id")
+    assert acheteur_input.get_attribute("value") == "123", (
+        "acheteur_id input should be populated after navigating from search"
+    )
+
+
+def test_009_observatoire_filter_persistence(dash_duo: DashComposite):
+    import time
+
+    from src.app import app
+
+    dash_duo.start_server(app)
+    dash_duo.wait_for_text_to_equal(".logo > h1", "decp.info", timeout=4)
+
+    # Clear localStorage to start from a clean state
+    dash_duo.driver.execute_script("localStorage.clear()")
+
+    # Navigate to observatoire without URL params
+    dash_duo.wait_for_page(f"{dash_duo.server_url}/observatoire")
+    dash_duo.wait_for_element("#dashboard_acheteur_id", timeout=4)
+
+    # Set the acheteur_id text input; press Enter to trigger the debounced save callback
+    acheteur_input = dash_duo.find_element("#dashboard_acheteur_id")
+    dash_duo.clear_input(acheteur_input)
+    acheteur_input.send_keys("123")
+    acheteur_input.send_keys(Keys.ENTER)
+
+    time.sleep(0.3)  # allow the save callback to write to localStorage
+
+    # Navigate away
+    dash_duo.wait_for_page(f"{dash_duo.server_url}/")
+
+    # Navigate back without URL params
+    dash_duo.wait_for_page(f"{dash_duo.server_url}/observatoire")
+    dash_duo.wait_for_element("#dashboard_acheteur_id", timeout=4)
+    time.sleep(0.5)  # allow restore callback chain to complete
+
+    acheteur_input = dash_duo.find_element("#dashboard_acheteur_id")
+    assert acheteur_input.get_attribute("value") == "123", (
+        "acheteur_id should be restored from localStorage after navigating back"
+    )
+
+    # Also verify URL params still override localStorage
+    dash_duo.wait_for_page(f"{dash_duo.server_url}/observatoire?acheteur_id=123")
+    dash_duo.wait_for_element("#dashboard_acheteur_id", timeout=4)
+    time.sleep(0.5)
+
+    acheteur_input = dash_duo.find_element("#dashboard_acheteur_id")
+    assert acheteur_input.get_attribute("value") == "123", (
+        "URL param acheteur_id should override the value stored in localStorage"
+    )
+
+
+def test_011_observatoire_multi_param_url(dash_duo: DashComposite):
+    import time
+
+    from src.app import app
+
+    dash_duo.start_server(app)
+    dash_duo.wait_for_text_to_equal(".logo > h1", "decp.info", timeout=4)
+
+    # Navigate with multiple filter params
+    dash_duo.wait_for_page(
+        f"{dash_duo.server_url}/observatoire?annee=2024&acheteur_id=12345678901234&montant_min=10000"
+    )
+    dash_duo.wait_for_element("#dashboard_acheteur_id", timeout=4)
+
+    time.sleep(1)  # Allow callback chain to complete
+
+    # Verify acheteur_id input
+    acheteur_input = dash_duo.find_element("#dashboard_acheteur_id")
+    assert acheteur_input.get_attribute("value") == "12345678901234", (
+        "acheteur_id input should be populated from URL param"
+    )
+
+    # Verify montant_min input
+    montant_input = dash_duo.find_element("#dashboard_montant_min")
+    montant_value = montant_input.get_attribute("value")
+    assert montant_value in ("10000", "10000.0"), (
+        f"montant_min input should be populated from URL param, got: {montant_value}"
+    )
+
+
+def test_012_get_distance_histogram_returns_graph():
+    import polars as pl
+    from dash import dcc
+
+    from src.figures import get_distance_histogram
+
+    lff = pl.LazyFrame({"titulaire_distance": [1, 10, 100, 500, 1000]})
+    result = get_distance_histogram(lff)
+    assert isinstance(result, dcc.Graph)
+
+
+def test_013_get_distance_histogram_handles_nulls():
+    import polars as pl
+    from dash import dcc
+
+    from src.figures import get_distance_histogram
+
+    lff = pl.LazyFrame({"titulaire_distance": [None, None, 50]})
+    result = get_distance_histogram(lff)
+    assert isinstance(result, dcc.Graph)
+
+
+def test_014_get_distance_histogram_all_nulls():
+    import polars as pl
+    from dash import dcc
+
+    from src.figures import get_distance_histogram
+
+    lff = pl.LazyFrame({"titulaire_distance": pl.Series([], dtype=pl.Int64)})
+    result = get_distance_histogram(lff)
+
+    assert isinstance(result, dcc.Graph)
+
+
+def test_015_tableau_filter_date(dash_duo: DashComposite):
+    from src.app import app
+
+    dash_duo.start_server(app)
+    dash_duo.wait_for_text_to_equal(".logo > h1", "decp.info", timeout=4)
+
+    for page in ["tableau", "acheteurs/123", "titulaires/345"]:
+        dash_duo.wait_for_page(f"{dash_duo.server_url}/{page}")
+        filter_input = '.marches_table th[data-dash-column="dateNotification"] input'
+        filter_cell_result = '.marches_table td[data-dash-column="dateNotification"] p'
+        dash_duo.wait_for_element(filter_input, timeout=2)
+        _filter_input: WebElement = dash_duo.find_element(filter_input)
+        _filter_input.send_keys("3333")  # a dateNotification that doesn't exist
+        _filter_input.send_keys(Keys.ENTER)
+        _filter_result: list[WebElement] = dash_duo.find_elements(filter_cell_result)
+        assert len(_filter_result) == 0, f"Page : {page}"

tests/test_table.py

@@ -0,0 +1,288 @@
+import polars as pl
+import pytest
+
+
+@pytest.fixture
+def sample_lff():
+    """Small LazyFrame with the columns needed by add_links / format_values."""
+    return pl.LazyFrame(
+        [
+            {
+                "uid": "u1",
+                "id": "u1",
+                "acheteur_id": "12345678900011",
+                "acheteur_nom": "Mairie de Test",
+                "titulaire_id": "98765432100022",
+                "titulaire_nom": "Entreprise Test",
+                "titulaire_typeIdentifiant": "SIRET",
+                "objet": "Travaux divers",
+                "montant": 12500.0,
+                "dateNotification": "2025-03-15",
+                "codeCPV": "45000000",
+                "dureeRestanteMois": 6,
+                "titulaire_distance": 42.0,
+            }
+        ]
+    )
+
+
+def test_table_module_imports():
+    from src.utils import table
+
+    assert hasattr(table, "prepare_table_data")
+
+
+def test_filter_table_data_does_not_call_track_search(monkeypatch, sample_lff):
+    from src.utils import table
+
+    calls = []
+    monkeypatch.setattr(table, "track_search", lambda *a, **kw: calls.append(a))
+
+    result = table.filter_table_data(sample_lff, "{objet} icontains travaux").collect()
+
+    assert calls == []
+    assert result.height == 1
+
+
+def test_normalize_sort_by_handles_empty():
+    from src.utils.table import normalize_sort_by
+
+    assert normalize_sort_by(None) == ()
+    assert normalize_sort_by([]) == ()
+
+
+def test_normalize_sort_by_returns_hashable_tuple():
+    from src.utils.table import normalize_sort_by
+
+    sort_by = [
+        {"column_id": "montant", "direction": "desc"},
+        {"column_id": "dateNotification", "direction": "asc"},
+    ]
+    key = normalize_sort_by(sort_by)
+
+    assert key == (("montant", "desc"), ("dateNotification", "asc"))
+    # Must be hashable so that flask-caching can build a cache key from it
+    hash(key)
+
+
+def test_normalize_sort_by_preserves_order():
+    """Order matters for sort: [A, B] != [B, A]."""
+    from src.utils.table import normalize_sort_by
+
+    a_then_b = normalize_sort_by(
+        [{"column_id": "a", "direction": "asc"}, {"column_id": "b", "direction": "asc"}]
+    )
+    b_then_a = normalize_sort_by(
+        [{"column_id": "b", "direction": "asc"}, {"column_id": "a", "direction": "asc"}]
+    )
+    assert a_then_b != b_then_a
+
+
+@pytest.fixture(scope="module")
+def flask_app():
+    """Minimal Flask app with SimpleCache so @cache.memoize() works in tests."""
+    from flask import Flask
+
+    from src.utils.cache import cache
+
+    app = Flask(__name__)
+    cache.init_app(app, config={"CACHE_TYPE": "SimpleCache"})
+    return app
+
+
+@pytest.fixture(autouse=True)
+def reset_cache(flask_app):
+    """Ensure the flask-caching backend is empty between tests so that
+    cache-hit assertions are meaningful. Falls back to no-op when no
+    Flask app context is active (NullCache)."""
+    from src.utils.cache import cache
+
+    with flask_app.app_context():
+        try:
+            cache.clear()
+        except (RuntimeError, AttributeError):
+            # No app context — cache is NullCache, nothing to clear
+            pass
+        yield
+
+
+def test_prepare_table_data_returns_expected_tuple(flask_app):
+    from src.utils import table
+
+    with flask_app.app_context():
+        result = table.prepare_table_data(
+            data=None,
+            data_timestamp=5,
+            filter_query=None,
+            page_current=0,
+            page_size=20,
+            sort_by=[],
+            source_table="tableau",
+        )
+
+    # Same arity as before: 9 outputs
+    assert len(result) == 9
+    dicts, columns, tooltip, ts, nb_rows, dl_disabled, dl_text, dl_title, cleanup = (
+        result
+    )
+    assert isinstance(dicts, list)
+    assert ts == 6  # data_timestamp + 1 must still increment
+    assert "lignes" in nb_rows
+
+
+def test_prepare_table_data_calls_track_search_on_filter(monkeypatch, flask_app):
+    from src.utils import table
+
+    calls = []
+    monkeypatch.setattr(table, "track_search", lambda *a, **kw: calls.append(a))
+
+    with flask_app.app_context():
+        table.prepare_table_data(
+            data=None,
+            data_timestamp=0,
+            filter_query="{objet} icontains travaux",
+            page_current=0,
+            page_size=20,
+            sort_by=[],
+            source_table="tableau",
+        )
+
+    assert calls == [("{objet} icontains travaux", "tableau")]
+
+
+def test_prepare_table_data_same_page_uses_cache(monkeypatch, flask_app):
+    """Two calls with exactly the same (filter, sort, page, size)
+    must call _fetch_page_sql at least once."""
+    from src.utils import table
+
+    call_count = {"n": 0}
+
+    def counting_fetch(*args, **kwargs):
+        call_count["n"] += 1
+        import polars as pl
+
+        return (
+            pl.DataFrame(
+                {
+                    "uid": [],
+                    "acheteur_id": [],
+                    "titulaire_id": [],
+                    "titulaire_typeIdentifiant": [],
+                }
+            ),
+            0,
+            0,
+        )
+
+    monkeypatch.setattr(table, "_fetch_page_sql", counting_fetch)
+
+    with flask_app.app_context():
+        table.prepare_table_data(
+            data=None,
+            data_timestamp=0,
+            filter_query=None,
+            page_current=0,
+            page_size=10,
+            sort_by=[],
+            source_table="tableau",
+        )
+        table.prepare_table_data(
+            data=None,
+            data_timestamp=0,
+            filter_query=None,
+            page_current=0,
+            page_size=10,
+            sort_by=[],
+            source_table="tableau",
+        )
+    assert call_count["n"] >= 1
+
+
+def test_prepare_table_data_cleanup_trigger_for_non_tableau(flask_app):
+    """Non-tableau pages still get a fresh uuid trigger, not no_update."""
+    from dash import no_update
+
+    from src.utils import table
+
+    with flask_app.app_context():
+        result = table.prepare_table_data(
+            data=None,
+            data_timestamp=0,
+            filter_query="{objet} icontains travaux",
+            page_current=0,
+            page_size=20,
+            sort_by=[],
+            source_table="acheteur",
+        )
+
+    cleanup = result[8]
+    assert cleanup is not no_update
+    assert isinstance(cleanup, str)
+    assert len(cleanup) >= 32  # uuid4 hex string
+
+
+def test_prepare_table_data_with_external_data_does_not_use_cache(
+    monkeypatch, flask_app, sample_lff
+):
+    """When a caller passes data (acheteur/titulaire/observatoire path),
+    bypass the memoized helper entirely."""
+    from src.utils import table
+
+    sentinel = {"called": False}
+
+    def should_not_be_called(*a, **kw):
+        sentinel["called"] = True
+        raise AssertionError("Memoized helper must not be called when data is provided")
+
+    monkeypatch.setattr(table, "_fetch_page_sql", should_not_be_called)
+
+    with flask_app.app_context():
+        table.prepare_table_data(
+            data=sample_lff,
+            data_timestamp=0,
+            filter_query=None,
+            page_current=0,
+            page_size=20,
+            sort_by=[],
+            source_table="acheteur",
+        )
+
+    assert sentinel["called"] is False
+
+
+def test_fetch_page_sql_respects_pagination(flask_app):
+    """New path: returns (page_dff, total_count, total_unique) via DuckDB."""
+    from src.utils import table
+
+    with flask_app.app_context():
+        page, total, total_unique = table._fetch_page_sql(
+            filter_query=None, sort_by_key=(), page_current=0, page_size=5
+        )
+    assert page.height <= 5
+    assert total >= page.height
+    assert isinstance(total_unique, int)
+
+
+def test_fetch_page_sql_applies_filter(flask_app):
+    from src.utils import table
+
+    with flask_app.app_context():
+        page, total, total_unique = table._fetch_page_sql(
+            filter_query="{uid} icontains __ne_matche_rien__",
+            sort_by_key=(),
+            page_current=0,
+            page_size=20,
+        )
+    assert total == 0
+    assert page.height == 0
+
+
+def test_fetch_page_sql_post_processes_links(flask_app):
+    from src.utils import table
+
+    with flask_app.app_context():
+        page, _, _ = table._fetch_page_sql(
+            filter_query=None, sort_by_key=(), page_current=0, page_size=1
+        )
+    if page.height > 0:
+        assert "<a href" in page["uid"][0]

tests/test_table_sql.py

@@ -0,0 +1,140 @@
+import polars as pl
+
+SCHEMA = pl.Schema(
+    {
+        "uid": pl.String,
+        "objet": pl.String,
+        "acheteur_id": pl.String,
+        "montant": pl.Float64,
+        "dureeMois": pl.Int64,
+        "dateNotification": pl.Date,
+    }
+)
+
+
+def test_empty_filter_returns_true():
+    from src.utils.table_sql import filter_query_to_sql
+
+    where, params = filter_query_to_sql("", SCHEMA)
+    assert where == "TRUE"
+    assert params == []
+
+
+def test_icontains_string_is_case_insensitive_like():
+    from src.utils.table_sql import filter_query_to_sql
+
+    where, params = filter_query_to_sql("{objet} icontains travaux", SCHEMA)
+    assert where == '"objet" IS NOT NULL AND "objet" <> \'\' AND "objet" ILIKE ?'
+    assert params == ["%travaux%"]
+
+
+def test_icontains_with_trailing_wildcard_is_starts_with():
+    from src.utils.table_sql import filter_query_to_sql
+
+    where, params = filter_query_to_sql(
+        "{acheteur_id} icontains 24350013900189*", SCHEMA
+    )
+    assert (
+        where
+        == '"acheteur_id" IS NOT NULL AND "acheteur_id" <> \'\' AND "acheteur_id" ILIKE ?'
+    )
+    assert params == ["24350013900189%"]
+
+
+def test_icontains_with_leading_wildcard_is_ends_with():
+    from src.utils.table_sql import filter_query_to_sql
+
+    where, params = filter_query_to_sql("{uid} icontains *2024", SCHEMA)
+    assert where == '"uid" IS NOT NULL AND "uid" <> \'\' AND "uid" ILIKE ?'
+    assert params == ["%2024"]
+
+
+def test_numeric_greater_than():
+    from src.utils.table_sql import filter_query_to_sql
+
+    where, params = filter_query_to_sql("{montant} i> 40000", SCHEMA)
+    assert where == '"montant" IS NOT NULL AND "montant" > ?'
+    assert params == [40000.0]
+
+
+def test_numeric_less_than():
+    from src.utils.table_sql import filter_query_to_sql
+
+    where, params = filter_query_to_sql("{montant} i< 1000", SCHEMA)
+    assert where == '"montant" IS NOT NULL AND "montant" < ?'
+    assert params == [1000.0]
+
+
+def test_numeric_equality_via_icontains():
+    from src.utils.table_sql import filter_query_to_sql
+
+    where, params = filter_query_to_sql("{dureeMois} icontains 12", SCHEMA)
+    assert where == '"dureeMois" IS NOT NULL AND "dureeMois" = ?'
+    assert params == [12]
+
+
+def test_date_column_treated_as_string_ilike():
+    from src.utils.table_sql import filter_query_to_sql
+
+    where, params = filter_query_to_sql("{dateNotification} icontains 2024*", SCHEMA)
+    assert "ILIKE" in where
+    assert params == ["2024%"]
+
+
+def test_multiple_filters_joined_by_and():
+    from src.utils.table_sql import filter_query_to_sql
+
+    filter_query = "{objet} icontains voirie && {montant} i> 40000"
+    where, params = filter_query_to_sql(filter_query, SCHEMA)
+    assert " AND " in where
+    assert params == ["%voirie%", 40000.0]
+
+
+def test_invalid_numeric_value_is_skipped():
+    from src.utils.table_sql import filter_query_to_sql
+
+    where, params = filter_query_to_sql("{montant} i> notanumber", SCHEMA)
+    assert where == "TRUE"
+    assert params == []
+
+
+def test_unknown_column_is_skipped():
+    from src.utils.table_sql import filter_query_to_sql
+
+    where, params = filter_query_to_sql("{inexistant} icontains foo", SCHEMA)
+    assert where == "TRUE"
+    assert params == []
+
+
+def test_sort_by_empty():
+    from src.utils.table_sql import sort_by_to_sql
+
+    assert sort_by_to_sql([], SCHEMA) == ""
+    assert sort_by_to_sql(None, SCHEMA) == ""
+
+
+def test_sort_by_single_column_desc():
+    from src.utils.table_sql import sort_by_to_sql
+
+    result = sort_by_to_sql([{"column_id": "montant", "direction": "desc"}], SCHEMA)
+    assert result == '"montant" DESC NULLS LAST'
+
+
+def test_sort_by_multiple_columns_preserves_order():
+    from src.utils.table_sql import sort_by_to_sql
+
+    result = sort_by_to_sql(
+        [
+            {"column_id": "dateNotification", "direction": "desc"},
+            {"column_id": "montant", "direction": "asc"},
+        ],
+        SCHEMA,
+    )
+    assert result == '"dateNotification" DESC NULLS LAST, "montant" ASC NULLS LAST'
+
+
+def test_sort_by_ignores_unknown_column():
+    from src.utils.table_sql import sort_by_to_sql
+
+    result = sort_by_to_sql([{"column_id": "fake", "direction": "asc"}], SCHEMA)
+    assert result == ""