feat: release v1.8.0 — connections dashboard, VK OAuth, sync config, catalog browser
- Connections dashboard with add/remove flow and background health checks - VK OAuth connection with profile info and health monitoring - Sync configuration page with master toggle and filter summary - Catalog browser with store/group/product tables, filter management, CSV export - Alembic migrations for all new tables - run/read_config.py for shell sync script DB integration - CHANGELOG.md updated for v1.8.0 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
mguschin
262
docs/plans/catalog-browser.md
Normal file
262
docs/plans/catalog-browser.md
Normal file
@@ -0,0 +1,262 @@
|
||||
# Catalog Browser with Filter Management & CSV Export
|
||||
|
||||
## Context
|
||||
|
||||
Users need to browse their Evotor catalog (stores, groups, products) in a table view, manage sync whitelist/blacklist rules inline, and export data to CSV.
|
||||
|
||||
This feature **replaces** the separate `/sync/stores`, `/sync/groups`, `/sync/products` pages from the sync-configuration plan. The catalog browser becomes the unified place for both viewing data and managing filter rules.
|
||||
|
||||
Data is cached in DB with a refresh mechanism — not fetched live on every page load.
|
||||
|
||||
## Data Model
|
||||
|
||||
### Catalog Cache Tables
|
||||
|
||||
```
|
||||
tablename: "cached_stores"
|
||||
- id (Integer, PK)
|
||||
- user_id (Integer, FK users.id CASCADE)
|
||||
- evotor_id (String 255) # Evotor UUID
|
||||
- name (String 255)
|
||||
- address (String 500, nullable)
|
||||
- fetched_at (DateTime) # when this snapshot was taken
|
||||
|
||||
UniqueConstraint: (user_id, evotor_id)
|
||||
Index: user_id
|
||||
```
|
||||
|
||||
```
|
||||
tablename: "cached_groups"
|
||||
- id (Integer, PK)
|
||||
- user_id (Integer, FK users.id CASCADE)
|
||||
- evotor_id (String 255) # Evotor UUID
|
||||
- store_evotor_id (String 255) # parent store UUID
|
||||
- name (String 255)
|
||||
- fetched_at (DateTime)
|
||||
|
||||
UniqueConstraint: (user_id, evotor_id)
|
||||
Index: (user_id, store_evotor_id)
|
||||
```
|
||||
|
||||
```
|
||||
tablename: "cached_products"
|
||||
- id (Integer, PK)
|
||||
- user_id (Integer, FK users.id CASCADE)
|
||||
- evotor_id (String 255) # Evotor UUID
|
||||
- store_evotor_id (String 255) # parent store UUID
|
||||
- group_evotor_id (String 255, nullable) # parent group UUID
|
||||
- name (String 255)
|
||||
- price (Numeric(12,2), nullable)
|
||||
- quantity (Numeric(12,3), nullable)
|
||||
- measure_name (String 20, nullable)
|
||||
- article_number (String 100, nullable)
|
||||
- allow_to_sell (Boolean, nullable)
|
||||
- fetched_at (DateTime)
|
||||
|
||||
UniqueConstraint: (user_id, evotor_id)
|
||||
Index: (user_id, store_evotor_id, group_evotor_id)
|
||||
```
|
||||
|
||||
### `SyncFilter` (from sync-configuration plan, unchanged)
|
||||
|
||||
```
|
||||
tablename: "sync_filters"
|
||||
- sync_config_id, entity_type, entity_id, entity_name, filter_mode, parent_entity_id
|
||||
```
|
||||
|
||||
The catalog browser reads from cache tables for display and from `sync_filters` for the current filter state of each entity.
|
||||
|
||||
### Cache Refresh
|
||||
|
||||
`web/evotor_api.py` gets a new function:
|
||||
|
||||
```python
|
||||
async def refresh_catalog_cache(user_id: int, access_token: str, db: Session):
|
||||
"""Fetch all stores, groups, products from Evotor API and upsert into cache tables."""
|
||||
```
|
||||
|
||||
Triggered by:
|
||||
- Manual "Обновить" button on the catalog page
|
||||
- Background job (optional, can reuse health_checker interval or separate setting)
|
||||
- First visit to catalog if cache is empty
|
||||
|
||||
## Plan
|
||||
|
||||
### 1. New Models — `web/models.py`
|
||||
|
||||
Add `CachedStore`, `CachedGroup`, `CachedProduct` models as described above.
|
||||
|
||||
### 2. Alembic Migration
|
||||
|
||||
Create `cached_stores`, `cached_groups`, `cached_products` tables.
|
||||
|
||||
### 3. Evotor API Helper — `web/evotor_api.py`
|
||||
|
||||
Extend with:
|
||||
|
||||
```python
|
||||
async def fetch_stores(access_token: str) -> list[dict]
|
||||
async def fetch_groups(access_token: str, store_id: str) -> list[dict]
|
||||
async def fetch_products(access_token: str, store_id: str) -> list[dict]
|
||||
async def refresh_catalog_cache(user_id: int, access_token: str, db: Session)
|
||||
```
|
||||
|
||||
`refresh_catalog_cache` does:
|
||||
1. Fetch all stores
|
||||
2. For each store, fetch groups and products
|
||||
3. Upsert into cache tables (delete old rows for user, insert fresh)
|
||||
4. Update `fetched_at` timestamps
|
||||
|
||||
### 4. Catalog Route — `web/routes/catalog.py` (new)
|
||||
|
||||
**`GET /catalog`** — Stores table. Requires auth + Evotor connection.
|
||||
- Reads `cached_stores` for user
|
||||
- If cache is empty, triggers refresh
|
||||
- Shows table with columns: Название, Адрес, Статус фильтра, Действия
|
||||
- Each row shows the store's current `SyncFilter` state (included/excluded/no rule)
|
||||
- Link to drill into groups for each store
|
||||
- "Обновить каталог" button, "Экспорт CSV" button, back link
|
||||
|
||||
**`GET /catalog/groups?store_id=UUID`** — Groups table for a store.
|
||||
- Reads `cached_groups` filtered by `store_evotor_id`
|
||||
- Table columns: Название, Статус фильтра, Кол-во товаров, Действия
|
||||
- Each row shows group's `SyncFilter` state
|
||||
- Link to drill into products for each group
|
||||
- "Экспорт CSV" button, back to stores
|
||||
|
||||
**`GET /catalog/products?store_id=UUID&group_id=UUID`** — Products table for a group.
|
||||
- Reads `cached_products` filtered by `store_evotor_id` and `group_evotor_id`
|
||||
- Table columns: Название, Артикул, Цена, Кол-во, Ед. изм., В продаже, Статус фильтра, Действия
|
||||
- Each row shows product's `SyncFilter` state
|
||||
- "Экспорт CSV" button, back to groups
|
||||
|
||||
**`GET /catalog/products?store_id=UUID`** — All products for a store (no group filter).
|
||||
- Same table, but shows all products in the store with a "Группа" column added
|
||||
|
||||
**`POST /catalog/filter`** — Toggle filter for an entity.
|
||||
- Body: `entity_type`, `entity_id`, `entity_name`, `filter_mode` (include/exclude/none), `parent_entity_id`
|
||||
- Creates, updates, or deletes the `SyncFilter` row
|
||||
- Redirects back to the referring page
|
||||
|
||||
**`POST /catalog/refresh`** — Manual cache refresh.
|
||||
- Calls `refresh_catalog_cache()`
|
||||
- Redirects back to `/catalog`
|
||||
|
||||
**`GET /catalog/export?type=stores|groups|products&store_id=UUID&group_id=UUID`** — CSV export.
|
||||
- Reads from cache tables
|
||||
- Returns `StreamingResponse` with `text/csv` content type and `Content-Disposition: attachment`
|
||||
- Filename: `{type}_{date}.csv`
|
||||
|
||||
### 5. Templates
|
||||
|
||||
**`web/templates/catalog_stores.html`** — Stores table:
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────┐
|
||||
│ Каталог [Обновить] [Экспорт CSV] │
|
||||
│ Последнее обновление: 06.03.2026 14:30 │
|
||||
├──────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ Магазины │
|
||||
│ ┌──────────────────────────────────────────────────────┐ │
|
||||
│ │ Название │ Адрес │ Фильтр │ │ │
|
||||
│ ├──────────────────────────────────────────────────────┤ │
|
||||
│ │ Чайная │ ул. Мира, 1 │ ✓ Вкл │ [→][▼] │ │
|
||||
│ │ Склад │ — │ ✗ Выкл │ [→][▼] │ │
|
||||
│ │ Точка 2 │ ул. Мира, 5 │ — Нет │ [→][▼] │ │
|
||||
│ └──────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ [→] = перейти к группам │
|
||||
│ [▼] = dropdown: Включить / Исключить / Убрать правило │
|
||||
│ │
|
||||
└──────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Filter status column** shows:
|
||||
- `✓ Включено` (green badge) — entity has an "include" rule
|
||||
- `✗ Исключено` (red badge) — entity has an "exclude" rule
|
||||
- `— Нет правила` (grey badge) — no filter rule (follows default behavior)
|
||||
|
||||
**Actions column** per row:
|
||||
- Link icon → drill into children (groups for stores, products for groups)
|
||||
- Dropdown button with filter actions: "Включить в синхронизацию" / "Исключить из синхронизации" / "Убрать правило". Each is a small POST form to `/catalog/filter`.
|
||||
|
||||
**`web/templates/catalog_groups.html`** — Groups table:
|
||||
- Breadcrumb: Каталог > {Store name} > Группы
|
||||
- Same table pattern, columns: Название, Кол-во товаров, Фильтр, Действия
|
||||
- Drill-down link to products per group
|
||||
|
||||
**`web/templates/catalog_products.html`** — Products table:
|
||||
- Breadcrumb: Каталог > {Store name} > {Group name} > Товары
|
||||
- Columns: Название, Артикул, Цена, Кол-во, Ед. изм., В продаже, Фильтр, Действия
|
||||
- "В продаже" column: green check / red cross based on `allow_to_sell`
|
||||
|
||||
All tables use Bootstrap table styling (`table table-striped table-hover`) with responsive wrapper.
|
||||
|
||||
### 6. CSV Export Format
|
||||
|
||||
**Stores CSV:**
|
||||
```
|
||||
Название,Адрес,ID,Фильтр
|
||||
Чайная,"ул. Мира, 1",uuid-123,Включено
|
||||
```
|
||||
|
||||
**Groups CSV:**
|
||||
```
|
||||
Магазин,Название,ID,Фильтр
|
||||
Чайная,Белый чай,uuid-456,Включено
|
||||
```
|
||||
|
||||
**Products CSV:**
|
||||
```
|
||||
Магазин,Группа,Название,Артикул,Цена,Количество,Ед. измерения,В продаже,ID,Фильтр
|
||||
Чайная,Белый чай,Бай Му Дань,1005,350.00,180.0,г,Да,uuid-789,Включено
|
||||
```
|
||||
|
||||
UTF-8 with BOM (`\ufeff`) for Excel compatibility. Delimiter: comma.
|
||||
|
||||
### 7. Update Sync Configuration Plan
|
||||
|
||||
The `/sync` page links to `/catalog` instead of separate filter pages:
|
||||
- "Настроить фильтры" button → `/catalog`
|
||||
- Filter summary on `/sync` reads from `SyncFilter` table (unchanged)
|
||||
- Remove `/sync/stores`, `/sync/groups`, `/sync/products` routes from sync-configuration plan — replaced by catalog browser
|
||||
|
||||
### 8. Navbar / Navigation
|
||||
|
||||
Add "Каталог" link to navbar for logged-in users. Order: Подключения → Каталог → Синхронизация → Личный кабинет → Выход.
|
||||
|
||||
### 9. Register Route — `web/main.py`
|
||||
|
||||
```python
|
||||
from web.routes import catalog
|
||||
app.include_router(catalog.router)
|
||||
```
|
||||
|
||||
## Files Summary
|
||||
|
||||
| File | Action |
|
||||
|------|--------|
|
||||
| `web/models.py` | Modify — add `CachedStore`, `CachedGroup`, `CachedProduct` |
|
||||
| `web/evotor_api.py` | Create — API fetch + cache refresh functions |
|
||||
| `web/routes/catalog.py` | Create — catalog routes (tables, filter toggle, refresh, CSV export) |
|
||||
| `web/templates/catalog_stores.html` | Create — stores table |
|
||||
| `web/templates/catalog_groups.html` | Create — groups table |
|
||||
| `web/templates/catalog_products.html` | Create — products table |
|
||||
| `web/templates/base.html` | Modify — add "Каталог" nav link |
|
||||
| `web/main.py` | Modify — register catalog router |
|
||||
| `docs/plans/sync-configuration.md` | Update — remove /sync/stores,groups,products; link to /catalog |
|
||||
| Alembic migration | Create — cache tables |
|
||||
|
||||
## Verification
|
||||
|
||||
1. Run `alembic upgrade head`
|
||||
2. Visit `/catalog` without Evotor connection → warning to connect first
|
||||
3. Connect Evotor, visit `/catalog` → triggers first cache refresh, shows stores table
|
||||
4. Click store → shows groups table with group names from Evotor
|
||||
5. Click group → shows products table with full product details
|
||||
6. Toggle filter on a product → badge changes, `SyncFilter` row created in DB
|
||||
7. Go to `/sync` → filter summary reflects the change
|
||||
8. Click "Экспорт CSV" on products page → downloads CSV, opens correctly in Excel
|
||||
9. Click "Обновить каталог" → re-fetches from Evotor API, updates cache
|
||||
10. Verify breadcrumb navigation works correctly through the hierarchy
|
||||
@@ -2,19 +2,33 @@
|
||||
|
||||
## Context
|
||||
|
||||
Users currently access Evotor connection via a dedicated `/evotor` page linked from the navbar. As more integrations are planned, we need a unified **Connections** page where users can see all their connections at a glance, with real-time status indicators powered by background health checks.
|
||||
Users currently access Evotor connection via a dedicated `/evotor` page linked from the navbar. As more integrations are planned, we need a unified **Connections** page where users can manage all their connections: add new ones, view status, edit, and delete. The dashboard starts empty — users explicitly add each connection they need.
|
||||
|
||||
Supported connection types: **Evotor**, **VK** (one per type per user).
|
||||
|
||||
## Data Design
|
||||
|
||||
### Current state (separate models)
|
||||
|
||||
`EvotorConnection` and `VkConnection` remain as-is — they hold service-specific fields (store_id/store_name for Evotor, vk_user_id/first_name/last_name for VK). The connections dashboard reads from both tables.
|
||||
|
||||
No new unified "connection" table needed. The dashboard builds a virtual list by querying both tables. The "add" flow is just a gateway to the existing per-service OAuth pages.
|
||||
|
||||
### Model additions (both `EvotorConnection` and `VkConnection`)
|
||||
|
||||
Already planned:
|
||||
- `is_online` (Boolean, default=False, server_default="0")
|
||||
- `last_checked_at` (DateTime, nullable)
|
||||
|
||||
## Plan
|
||||
|
||||
### 1. Model Changes — `web/models.py`
|
||||
|
||||
Add to `EvotorConnection`:
|
||||
- `is_online` (Boolean, default=False, server_default="0")
|
||||
- `last_checked_at` (DateTime, nullable)
|
||||
Add `is_online` and `last_checked_at` to both `EvotorConnection` and `VkConnection`.
|
||||
|
||||
### 2. Alembic Migration
|
||||
|
||||
Generate migration for the two new columns on `evotor_connections` table.
|
||||
Add health check fields to both connection tables.
|
||||
|
||||
### 3. Config Addition — `web/config.py`
|
||||
|
||||
@@ -22,74 +36,157 @@ Add `HEALTH_CHECK_INTERVAL_SECONDS: int = 600` (10 minutes default).
|
||||
|
||||
### 4. Background Health Checker — `web/health_checker.py` (new)
|
||||
|
||||
- `check_evotor_connection(access_token) -> bool` — async, calls `GET https://api.evotor.ru/stores` with Bearer token, returns True if 200
|
||||
- `run_health_checks()` — queries all `EvotorConnection` rows using its own `SessionLocal()`, checks each, updates `is_online` and `last_checked_at`
|
||||
- `health_check_loop(interval)` — infinite loop with `asyncio.sleep`, calls `run_health_checks()`
|
||||
- `check_evotor_connection(access_token) -> bool` — async, `GET https://api.evotor.ru/stores` with Bearer token
|
||||
- `check_vk_connection(access_token) -> bool` — async, `GET https://api.vk.com/method/users.get` with token
|
||||
- `run_health_checks()` — queries all connection rows, checks each, updates `is_online` and `last_checked_at`
|
||||
- `health_check_loop(interval)` — infinite loop with `asyncio.sleep`
|
||||
|
||||
### 5. Wire Background Task — `web/main.py`
|
||||
|
||||
Add FastAPI lifespan context manager:
|
||||
- On startup: `asyncio.create_task(health_check_loop(...))`
|
||||
- On shutdown: cancel the task
|
||||
- Register new connections router
|
||||
- Register connections router
|
||||
|
||||
### 6. Connections Route — `web/routes/connections.py` (new)
|
||||
|
||||
`GET /connections` — requires auth, builds a list of connection descriptors:
|
||||
**`GET /connections`** — Main dashboard. Requires auth.
|
||||
|
||||
Queries both `EvotorConnection` and `VkConnection` for the current user. Builds a list of available service types and their connection state:
|
||||
|
||||
```python
|
||||
connections = [{
|
||||
"name": "Эвотор",
|
||||
"icon": "bi-shop",
|
||||
"connected": bool,
|
||||
"is_online": bool,
|
||||
"last_checked_at": datetime | None,
|
||||
"details": store_name,
|
||||
"connect_url": "/evotor",
|
||||
"disconnect_url": "/evotor/disconnect",
|
||||
}]
|
||||
SERVICE_TYPES = [
|
||||
{"type": "evotor", "name": "Эвотор", "icon": "bi-shop", "connect_url": "/evotor", "disconnect_url": "/evotor/disconnect"},
|
||||
{"type": "vk", "name": "ВКонтакте", "icon": "bi-chat-dots", "connect_url": "/vk", "disconnect_url": "/vk/disconnect"},
|
||||
]
|
||||
```
|
||||
Future connections just append another dict — template stays generic.
|
||||
|
||||
### 7. Connections Template — `web/templates/connections.html` (new)
|
||||
For each type, attach the connection record (or None). Template renders based on state.
|
||||
|
||||
Card per connection showing:
|
||||
- Icon + name + optional details (store name)
|
||||
- Status: green `bi-circle-fill` (online), red `bi-circle-fill` (offline), grey `bi-circle` (not connected)
|
||||
- Action button: "Подключить" (link to connect_url) or "Отключить" (POST form to disconnect_url)
|
||||
- Card footer with last check timestamp
|
||||
**`GET /connections/add`** — "Add connection" page.
|
||||
|
||||
Shows only service types the user has NOT yet connected:
|
||||
- Card per available type with service name, icon, short description
|
||||
- "Подключить" button linking to the service's OAuth page (`/evotor` or `/vk`)
|
||||
- If all types already connected — message "Все доступные сервисы подключены"
|
||||
- Back link to `/connections`
|
||||
|
||||
**`POST /connections/delete?type=evotor|vk`** — Delete a connection.
|
||||
|
||||
Same as existing disconnect endpoints but accessed from the dashboard. Deletes the connection record, redirects to `/connections`.
|
||||
|
||||
(The existing `/evotor/disconnect` and `/vk/disconnect` routes remain as aliases.)
|
||||
|
||||
### 7. Templates
|
||||
|
||||
**`web/templates/connections.html`** — Dashboard:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────┐
|
||||
│ Подключения [+ Добавить] │
|
||||
├─────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌─ Card ─────────────────────────────────────┐ │
|
||||
│ │ 🏪 Эвотор ● (green) │ │
|
||||
│ │ Магазин "Чайная" │ │
|
||||
│ │ Последняя проверка: 06.03.2026 14:30 │ │
|
||||
│ │ │ │
|
||||
│ │ [Настроить] [Отключить] │ │
|
||||
│ └────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌─ Card ─────────────────────────────────────┐ │
|
||||
│ │ 💬 ВКонтакте ● (green) │ │
|
||||
│ │ Иван Иванов │ │
|
||||
│ │ Последняя проверка: 06.03.2026 14:30 │ │
|
||||
│ │ │ │
|
||||
│ │ [Настроить] [Отключить] │ │
|
||||
│ └────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ (Нет подключений — нажмите «Добавить») │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
Each connection card:
|
||||
- Icon + service name + status indicator (green/red/grey)
|
||||
- Details line (store name for Evotor, profile name for VK)
|
||||
- Last checked timestamp in card footer
|
||||
- "Настроить" button → links to service page (`/evotor` or `/vk`) for reconnect/details
|
||||
- "Отключить" button → POST to `/connections/delete?type=...` with confirmation
|
||||
|
||||
Empty state: message prompting user to add their first connection.
|
||||
|
||||
**`web/templates/connections_add.html`** — Add connection page:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────┐
|
||||
│ Добавить подключение │
|
||||
├─────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌─ Card ─────────────────────────────────────┐ │
|
||||
│ │ 🏪 Эвотор │ │
|
||||
│ │ Подключите кассу Эвотор для синхронизации │ │
|
||||
│ │ каталога товаров. │ │
|
||||
│ │ [Подключить →] │ │
|
||||
│ └────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌─ Card ─────────────────────────────────────┐ │
|
||||
│ │ 💬 ВКонтакте │ │
|
||||
│ │ Подключите аккаунт ВКонтакте для │ │
|
||||
│ │ публикации товаров в вашу группу. │ │
|
||||
│ │ [Подключить →] │ │
|
||||
│ └────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ← Вернуться к подключениям │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### 8. Navbar Update — `web/templates/base.html`
|
||||
|
||||
Replace "Эвотор" link with "Подключения" → `/connections`.
|
||||
|
||||
### 9. Evotor Callback Update — `web/routes/evotor.py`
|
||||
### 9. Evotor/VK Callback Updates
|
||||
|
||||
On successful OAuth callback, set `is_online=True` and `last_checked_at=func.now()`.
|
||||
On successful OAuth callback in both `/evotor/callback` and `/vk/callback`:
|
||||
- Set `is_online=True` and `last_checked_at=now()`
|
||||
- Redirect to `/connections` (already done for Evotor)
|
||||
|
||||
### 10. Evotor Template Back Link — `web/templates/evotor.html`
|
||||
### 10. Evotor/VK Template Back Links
|
||||
|
||||
Change "Вернуться в личный кабинет" → "Вернуться к подключениям" linking to `/connections`.
|
||||
Change back links on `/evotor` and `/vk` pages: "Вернуться к подключениям" → `/connections`.
|
||||
|
||||
### 11. Delete Confirmation
|
||||
|
||||
The "Отключить" button on the dashboard uses a simple JS `confirm()` dialog: "Вы уверены, что хотите отключить {service name}?" before submitting the POST form.
|
||||
|
||||
## Files Summary
|
||||
|
||||
| File | Action |
|
||||
|------|--------|
|
||||
| `web/models.py` | Modify — add 2 fields |
|
||||
| `web/config.py` | Modify — add interval setting |
|
||||
| `web/main.py` | Modify — lifespan + router |
|
||||
| `web/routes/evotor.py` | Modify — set online on callback |
|
||||
| `web/routes/connections.py` | Create |
|
||||
| `web/health_checker.py` | Create |
|
||||
| `web/templates/connections.html` | Create |
|
||||
| `web/templates/base.html` | Modify — navbar |
|
||||
| `web/templates/evotor.html` | Modify — back link |
|
||||
| `web/models.py` | Modify — add `is_online`, `last_checked_at` to both connection models |
|
||||
| `web/config.py` | Modify — add `HEALTH_CHECK_INTERVAL_SECONDS` |
|
||||
| `web/main.py` | Modify — lifespan + register connections router |
|
||||
| `web/routes/evotor.py` | Modify — set online on callback, redirect to /connections |
|
||||
| `web/routes/vk.py` | Modify — set online on callback, redirect to /connections |
|
||||
| `web/routes/connections.py` | Create — dashboard, add page, delete endpoint |
|
||||
| `web/health_checker.py` | Create — background checks for both Evotor and VK |
|
||||
| `web/templates/connections.html` | Create — dashboard with cards |
|
||||
| `web/templates/connections_add.html` | Create — add connection page |
|
||||
| `web/templates/base.html` | Modify — navbar link |
|
||||
| `web/templates/evotor.html` | Modify — back link to /connections |
|
||||
| `web/templates/vk.html` | Modify — back link to /connections |
|
||||
| Alembic migration | Create |
|
||||
|
||||
## Verification
|
||||
|
||||
1. Run `alembic upgrade head` to apply migration
|
||||
1. Run `alembic upgrade head`
|
||||
2. Start the app, verify background task logs appear
|
||||
3. Visit `/connections` — should show Evotor as disconnected (grey)
|
||||
4. Connect Evotor via `/evotor` — should redirect back, connections page shows green status
|
||||
5. Disconnect — status returns to grey
|
||||
6. Wait for health check interval or trigger manually — verify `is_online` and `last_checked_at` update
|
||||
3. Visit `/connections` — empty state, "Добавить" button visible
|
||||
4. Click "Добавить" → shows Evotor and VK as available services
|
||||
5. Add Evotor → goes through OAuth → returns to `/connections` with green status card
|
||||
6. Add VK → same flow → both connections visible
|
||||
7. Click "Добавить" again → shows "Все доступные сервисы подключены"
|
||||
8. Click "Отключить" on Evotor → confirmation dialog → connection removed → card disappears
|
||||
9. Click "Добавить" → Evotor is available again
|
||||
10. Wait for health check cycle → verify `is_online` and `last_checked_at` update on remaining connections
|
||||
|
||||
151
docs/plans/sync-configuration.md
Normal file
151
docs/plans/sync-configuration.md
Normal file
@@ -0,0 +1,151 @@
|
||||
# Sync Configuration Feature
|
||||
|
||||
## Context
|
||||
|
||||
EvoSync syncs product catalogs from Evotor → VK. Currently sync runs as a shell-based cron service with a hardcoded store ID and a flat-file whitelist of group names (`vk/whitelist`). This doesn't support multi-user or per-user configuration.
|
||||
|
||||
Users need a web UI to:
|
||||
- Enable/disable the whole sync process
|
||||
- Configure which stores, groups, and products to sync (whitelist/blacklist)
|
||||
- Explicitly confirm before sync starts
|
||||
|
||||
The web app will store config in DB; the shell sync service will read from DB instead of flat files.
|
||||
|
||||
## Data Model
|
||||
|
||||
### `SyncConfig` — per-user master switch
|
||||
|
||||
```
|
||||
tablename: "sync_configs"
|
||||
- id (Integer, PK)
|
||||
- user_id (Integer, FK users.id CASCADE, unique)
|
||||
- is_enabled (Boolean, default=False) # master on/off
|
||||
- confirmed_at (DateTime, nullable) # NULL = never confirmed/started
|
||||
- created_at (DateTime, server_default=now)
|
||||
- updated_at (DateTime, server_default=now, onupdate=now)
|
||||
|
||||
Relationship: User.sync_config (one-to-one)
|
||||
```
|
||||
|
||||
### `SyncFilter` — stores, groups, products filter rules
|
||||
|
||||
```
|
||||
tablename: "sync_filters"
|
||||
- id (Integer, PK)
|
||||
- sync_config_id (Integer, FK sync_configs.id CASCADE)
|
||||
- entity_type (String, enum: "store", "group", "product")
|
||||
- entity_id (String 255) # Evotor UUID
|
||||
- entity_name (String 255) # human-readable, cached
|
||||
- filter_mode (String, enum: "include", "exclude")
|
||||
- parent_entity_id (String 255, nullable) # store_id for groups, group_id for products
|
||||
- created_at (DateTime, server_default=now)
|
||||
|
||||
UniqueConstraint: (sync_config_id, entity_type, entity_id)
|
||||
Relationship: SyncConfig.filters (one-to-many)
|
||||
```
|
||||
|
||||
### Filter Logic
|
||||
|
||||
The filter model uses **explicit include/exclude rules** with these semantics:
|
||||
- **No rules for an entity type** = sync everything of that type (default permissive)
|
||||
- **Any "include" rule exists for a type** = ONLY sync included entities (whitelist mode)
|
||||
- **Only "exclude" rules for a type** = sync everything EXCEPT excluded (blacklist mode)
|
||||
- Hierarchy: store filters → group filters → product filters. If a store is excluded, all its groups/products are excluded regardless of their individual rules.
|
||||
|
||||
## Plan
|
||||
|
||||
### 1. New Models — `web/models.py`
|
||||
|
||||
Add `SyncConfig` and `SyncFilter` as described above. Add `sync_config` relationship to `User`.
|
||||
|
||||
### 2. Alembic Migration
|
||||
|
||||
Create `sync_configs` and `sync_filters` tables.
|
||||
|
||||
### 3. Evotor API Helper — `web/evotor_api.py` (new)
|
||||
|
||||
Async functions to fetch data from Evotor API using a user's stored access token:
|
||||
|
||||
```python
|
||||
async def fetch_stores(access_token: str) -> list[dict]:
|
||||
"""GET https://api.evotor.ru/stores → [{"id": "uuid", "name": "..."}]"""
|
||||
|
||||
async def fetch_groups(access_token: str, store_id: str) -> list[dict]:
|
||||
"""GET https://api.evotor.ru/stores/{store_id}/product-groups → [{"id": "uuid", "name": "..."}]"""
|
||||
|
||||
async def fetch_products(access_token: str, store_id: str) -> list[dict]:
|
||||
"""GET https://api.evotor.ru/stores/{store_id}/products → [{"id": "uuid", "name": "...", "parent_id": "..."}]"""
|
||||
```
|
||||
|
||||
Uses `httpx.AsyncClient`. Returns simplified dicts. Raises on auth failure.
|
||||
|
||||
### 4. Sync Config Route — `web/routes/sync.py` (new)
|
||||
|
||||
**`GET /sync`** — Main sync configuration page.
|
||||
- Requires auth + active Evotor connection
|
||||
- Loads `SyncConfig` (creates default if missing)
|
||||
- Shows: master enable/disable toggle, confirm button, link to filter config
|
||||
|
||||
**`POST /sync/toggle`** — Enable/disable sync.
|
||||
- Toggles `is_enabled`. If enabling for the first time and no filters configured, stays on page with message to configure filters first.
|
||||
|
||||
**`POST /sync/confirm`** — Confirm and start sync.
|
||||
- Sets `confirmed_at = now()`. Only works if `is_enabled=True` and at least one store is configured.
|
||||
|
||||
**Filter management is handled by the Catalog Browser** (see `docs/plans/catalog-browser.md`).
|
||||
The `/catalog` page provides table views of stores, groups, and products with inline filter toggle actions. No separate `/sync/stores`, `/sync/groups`, `/sync/products` routes needed.
|
||||
|
||||
### 5. Templates
|
||||
|
||||
**`web/templates/sync.html`** — Main sync page:
|
||||
- Card with master toggle (on/off switch)
|
||||
- Status: "Не настроено" / "Настроено, ожидает подтверждения" / "Активна"
|
||||
- Warning if Evotor not connected (link to /evotor)
|
||||
- Warning if VK not connected (link to /vk)
|
||||
- "Настроить фильтры" button → `/catalog` (catalog browser)
|
||||
- "Подтвердить и запустить" button (disabled until filters configured)
|
||||
- Summary of current filter rules (X stores, Y groups, Z products)
|
||||
|
||||
### 6. Navbar / Navigation
|
||||
|
||||
Add "Синхронизация" link to navbar (for logged-in users), or add it as a card on the `/connections` page since sync depends on connections.
|
||||
|
||||
### 7. Register Route — `web/main.py`
|
||||
|
||||
```python
|
||||
from web.routes import sync
|
||||
app.include_router(sync.router)
|
||||
```
|
||||
|
||||
### 8. Shell Script DB Integration
|
||||
|
||||
Modify the sync service to read configuration from DB instead of flat files:
|
||||
|
||||
- Add a Python helper script `run/read_config.py` that queries `sync_configs` + `sync_filters` for a given user and outputs JSON config
|
||||
- Shell scripts call this helper to get: enabled flag, store IDs, whitelisted/blacklisted group names, product exclusions
|
||||
- The sync service only runs for users where `is_enabled=True` AND `confirmed_at IS NOT NULL`
|
||||
- Replaces the flat `vk/whitelist` file
|
||||
|
||||
## Files Summary
|
||||
|
||||
| File | Action |
|
||||
|------|--------|
|
||||
| `web/models.py` | Modify — add `SyncConfig`, `SyncFilter` + User relationship |
|
||||
| `web/routes/sync.py` | Create — sync config routes (toggle, confirm) |
|
||||
| `web/templates/sync.html` | Create — main sync config page |
|
||||
| `web/templates/base.html` | Modify — add sync nav link |
|
||||
| `web/main.py` | Modify — register sync router |
|
||||
| `run/read_config.py` | Create — DB config reader for shell scripts |
|
||||
| Alembic migration | Create — sync_configs + sync_filters tables |
|
||||
|
||||
## Verification
|
||||
|
||||
1. Run `alembic upgrade head`
|
||||
2. Visit `/sync` without Evotor connection → shows warning to connect first
|
||||
3. Connect Evotor, visit `/sync` → shows disabled state, "Настроить фильтры" button
|
||||
4. Go to `/sync/stores` → fetches live stores from Evotor API, shows checkboxes
|
||||
5. Select stores, save → drill into groups, select groups, save → drill into products
|
||||
6. Back to `/sync` → shows summary of configured filters
|
||||
7. Enable sync toggle → confirm → `confirmed_at` set
|
||||
8. Verify `run/read_config.py` outputs correct JSON for the user's config
|
||||
9. Disable sync → `is_enabled=False`, sync service stops processing this user
|
||||
Reference in New Issue
Block a user