Colin Maudry
9b124deeea
- Rename project from decp.info to colibre across all codebase - Update domain from https://decp.info to https://colibre.fr - Update GitHub repo references to ColinMaudry/colibre - Rename deployment files: decpinfo-backup.* → colibre-backup.* - Update project configuration and documentation - Rename project assets: decp.info.png → colibre.png - Update environment variables and constants (DOMAIN_NAME, TOKEN_PREFIX, GITHUB_REPO, etc.) - Update URLs in all pages, tests, and configuration files - Keep DECP acronym in text (unchanged per requirements) - Add rebrand note to README.md Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
140 lines
5.1 KiB
Markdown
140 lines
5.1 KiB
Markdown
# CLAUDE.md
|
|
|
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
|
|
## Project Overview
|
|
|
|
**colibre** is a French public procurement data explorer — a Dash (Python) web app for browsing, filtering, and visualizing _Données Essentielles de la Commande Publique_ (DECP). The UI is in French.
|
|
|
|
## Commands
|
|
|
|
### Setup
|
|
|
|
Setting up the virtual environment:
|
|
|
|
```bash
|
|
python -m venv .venv # s'il n'existe pas déjà
|
|
source .venv/bin/activate
|
|
rtk pip install -U pip > /dev/null 2>&1
|
|
rtk pip install -e . --group=dev
|
|
```
|
|
|
|
Environment variables:
|
|
|
|
```bash
|
|
cp .template.env .env # then customize .env
|
|
```
|
|
|
|
### Development
|
|
|
|
```bash
|
|
python run.py # starts Dash app
|
|
```
|
|
|
|
### Production
|
|
|
|
```bash
|
|
gunicorn app:server
|
|
```
|
|
|
|
### Tests
|
|
|
|
```bash
|
|
rtk pytest # run all tests (some are Selenium-based integration tests)
|
|
rtk pytest tests/test_main.py::test_001_logo_and_search # run a single test
|
|
```
|
|
|
|
Tests require a running Chrome/Chromium browser. They use `DashComposite` from `dash[testing]` with Selenium WebDriver.
|
|
|
|
## Architecture
|
|
|
|
### Multi-page Dash app
|
|
|
|
- `src/app.py` — creates the Dash app instance, navbar, SEO endpoints (robots.txt, sitemap.xml), Matomo analytics
|
|
- `src/pages/*.py` — each page registers itself with `@register_page()` and o.wns its own layout and callbacks
|
|
- `run.py` — dev entry point; exports `server` (Flask) for gunicorn
|
|
|
|
### Module imports
|
|
|
|
- always import modules from the app starting with `src.` (e.g. `src.utils.`, `src.pages.recherche`, etc.), NOT `utils.cache` or `pages.observatoire`.
|
|
|
|
### Key pages
|
|
|
|
| Page | URL | Purpose |
|
|
| ----------------- | --------------- | -------------------------------------- |
|
|
| `recherche.py` | `/` | Search homepage for buyers/contractors |
|
|
| `acheteur.py` | `/acheteur` | Buyer detail with stats, charts, maps |
|
|
| `titulaire.py` | `/titulaire` | Contractor detail |
|
|
| `tableau.py` | `/tableau` | Filterable data table with exports |
|
|
| `marche.py` | `/marche` | Individual contract detail |
|
|
| `observatoire.py` | `/observatoire` | An interactive analytics dashboard |
|
|
|
|
### Data layer
|
|
|
|
- Data is stored as **Parquet** at rest, possibly in DuckDB, loaded in DuckDB, served from DuckDB for big queries and manipulated with **Polars** for the remaining steps
|
|
- Path set via `DATA_FILE_PARQUET_PATH` env var; tests use `tests/test.parquet`
|
|
- `src/util/*.py` — helpers shared by other modules, search (`search_org`), link generation, geographic data loading
|
|
- `src/callbacks.py` — shared Dash callbacks (e.g. `get_top_org_table`)
|
|
- `src/figures.py` — chart and map components (Plotly Express, Dash Leaflet with marker clustering)
|
|
- a Parquet file with production data is located at `../decp-processing/decp_prod.parquet` (~ 1,5 million records)
|
|
- the TableSchema of the dataset with the list of field and their definition is located at `../decp-processing/reference/base_schema.json`
|
|
- `tests/test.parquet` is very small and may not contain all possible columns, only those necessary for testing
|
|
|
|
### UI stack
|
|
|
|
- **Dash 3.4** + **Dash Bootstrap Components** for layout
|
|
- **Plotly Express** for charts
|
|
- **Dash Leaflet** + **Dash Extensions** for interactive maps with clustering
|
|
- Custom CSS in `src/assets/css/`
|
|
|
|
### Environment
|
|
|
|
- `DEVELOPMENT=true` enables debug logging and is set automatically during tests
|
|
- `.env` file is required at runtime (copy from `template.env`)
|
|
|
|
### Migrations de schéma SQLite
|
|
|
|
Les migrations sont gérées dans `src/migrations.py` via une liste `_MIGRATIONS` de tuples `(id, sql)`. Elles sont appliquées automatiquement au démarrage de l'app (via `init_subscriptions`).
|
|
|
|
Pour ajouter une migration :
|
|
|
|
```python
|
|
# src/migrations.py
|
|
_MIGRATIONS = [
|
|
("0001_add_prix_ht_to_subscriptions", "ALTER TABLE subscriptions ADD COLUMN prix_ht REAL"),
|
|
("0002_ma_nouvelle_migration", "ALTER TABLE ... "), # ajouter ici
|
|
]
|
|
```
|
|
|
|
- L'ID doit être unique et croissant (convention `NNNN_description`)
|
|
- Les migrations appliquées sont tracées dans la table `schema_migrations`
|
|
- `apply_pending()` est idempotent : sans effet si la migration est déjà enregistrée, et tolère le cas où la colonne existe déjà dans le schéma (DB fraîche)
|
|
|
|
### Deployment
|
|
|
|
- `main` branch → manual deploy to colibre.fr via GitHub Actions
|
|
- `dev` branch → auto-deploy to test.colibre.fr via GitHub Actions
|
|
|
|
#### Sauvegarde de la base utilisateurs
|
|
|
|
`users.sqlite` est sauvegardée toutes les heures sur S3 par un timer systemd
|
|
(voir `deploy/colibre-backup.{service,timer}`). Installation initiale (une fois,
|
|
sur le serveur, en root) :
|
|
|
|
```bash
|
|
cp deploy/colibre-backup.service deploy/colibre-backup.timer /etc/systemd/system/
|
|
systemctl daemon-reload
|
|
systemctl enable --now colibre-backup.timer
|
|
systemctl list-timers colibre-backup.timer # vérifier le prochain déclenchement
|
|
```
|
|
|
|
Restauration manuelle :
|
|
|
|
```bash
|
|
cd /var/www/colibre && source .venv/bin/activate
|
|
python -m src.backup list
|
|
systemctl stop colibre
|
|
python -m src.backup restore backups/users-YYYYMMDDTHHMMSSZ.sqlite.gz.enc
|
|
systemctl start colibre
|
|
```
|