9aeef73b10
- Connections dashboard with add/remove flow and background health checks - VK OAuth connection with profile info and health monitoring - Sync configuration page with master toggle and filter summary - Catalog browser with store/group/product tables, filter management, CSV export - Alembic migrations for all new tables - run/read_config.py for shell sync script DB integration - CHANGELOG.md updated for v1.8.0 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
152 lines
6.5 KiB
Markdown
152 lines
6.5 KiB
Markdown
# Sync Configuration Feature
|
||
|
||
## Context
|
||
|
||
EvoSync syncs product catalogs from Evotor → VK. Currently sync runs as a shell-based cron service with a hardcoded store ID and a flat-file whitelist of group names (`vk/whitelist`). This doesn't support multi-user or per-user configuration.
|
||
|
||
Users need a web UI to:
|
||
- Enable/disable the whole sync process
|
||
- Configure which stores, groups, and products to sync (whitelist/blacklist)
|
||
- Explicitly confirm before sync starts
|
||
|
||
The web app will store config in DB; the shell sync service will read from DB instead of flat files.
|
||
|
||
## Data Model
|
||
|
||
### `SyncConfig` — per-user master switch
|
||
|
||
```
|
||
tablename: "sync_configs"
|
||
- id (Integer, PK)
|
||
- user_id (Integer, FK users.id CASCADE, unique)
|
||
- is_enabled (Boolean, default=False) # master on/off
|
||
- confirmed_at (DateTime, nullable) # NULL = never confirmed/started
|
||
- created_at (DateTime, server_default=now)
|
||
- updated_at (DateTime, server_default=now, onupdate=now)
|
||
|
||
Relationship: User.sync_config (one-to-one)
|
||
```
|
||
|
||
### `SyncFilter` — stores, groups, products filter rules
|
||
|
||
```
|
||
tablename: "sync_filters"
|
||
- id (Integer, PK)
|
||
- sync_config_id (Integer, FK sync_configs.id CASCADE)
|
||
- entity_type (String, enum: "store", "group", "product")
|
||
- entity_id (String 255) # Evotor UUID
|
||
- entity_name (String 255) # human-readable, cached
|
||
- filter_mode (String, enum: "include", "exclude")
|
||
- parent_entity_id (String 255, nullable) # store_id for groups, group_id for products
|
||
- created_at (DateTime, server_default=now)
|
||
|
||
UniqueConstraint: (sync_config_id, entity_type, entity_id)
|
||
Relationship: SyncConfig.filters (one-to-many)
|
||
```
|
||
|
||
### Filter Logic
|
||
|
||
The filter model uses **explicit include/exclude rules** with these semantics:
|
||
- **No rules for an entity type** = sync everything of that type (default permissive)
|
||
- **Any "include" rule exists for a type** = ONLY sync included entities (whitelist mode)
|
||
- **Only "exclude" rules for a type** = sync everything EXCEPT excluded (blacklist mode)
|
||
- Hierarchy: store filters → group filters → product filters. If a store is excluded, all its groups/products are excluded regardless of their individual rules.
|
||
|
||
## Plan
|
||
|
||
### 1. New Models — `web/models.py`
|
||
|
||
Add `SyncConfig` and `SyncFilter` as described above. Add `sync_config` relationship to `User`.
|
||
|
||
### 2. Alembic Migration
|
||
|
||
Create `sync_configs` and `sync_filters` tables.
|
||
|
||
### 3. Evotor API Helper — `web/evotor_api.py` (new)
|
||
|
||
Async functions to fetch data from Evotor API using a user's stored access token:
|
||
|
||
```python
|
||
async def fetch_stores(access_token: str) -> list[dict]:
|
||
"""GET https://api.evotor.ru/stores → [{"id": "uuid", "name": "..."}]"""
|
||
|
||
async def fetch_groups(access_token: str, store_id: str) -> list[dict]:
|
||
"""GET https://api.evotor.ru/stores/{store_id}/product-groups → [{"id": "uuid", "name": "..."}]"""
|
||
|
||
async def fetch_products(access_token: str, store_id: str) -> list[dict]:
|
||
"""GET https://api.evotor.ru/stores/{store_id}/products → [{"id": "uuid", "name": "...", "parent_id": "..."}]"""
|
||
```
|
||
|
||
Uses `httpx.AsyncClient`. Returns simplified dicts. Raises on auth failure.
|
||
|
||
### 4. Sync Config Route — `web/routes/sync.py` (new)
|
||
|
||
**`GET /sync`** — Main sync configuration page.
|
||
- Requires auth + active Evotor connection
|
||
- Loads `SyncConfig` (creates default if missing)
|
||
- Shows: master enable/disable toggle, confirm button, link to filter config
|
||
|
||
**`POST /sync/toggle`** — Enable/disable sync.
|
||
- Toggles `is_enabled`. If enabling for the first time and no filters configured, stays on page with message to configure filters first.
|
||
|
||
**`POST /sync/confirm`** — Confirm and start sync.
|
||
- Sets `confirmed_at = now()`. Only works if `is_enabled=True` and at least one store is configured.
|
||
|
||
**Filter management is handled by the Catalog Browser** (see `docs/plans/catalog-browser.md`).
|
||
The `/catalog` page provides table views of stores, groups, and products with inline filter toggle actions. No separate `/sync/stores`, `/sync/groups`, `/sync/products` routes needed.
|
||
|
||
### 5. Templates
|
||
|
||
**`web/templates/sync.html`** — Main sync page:
|
||
- Card with master toggle (on/off switch)
|
||
- Status: "Не настроено" / "Настроено, ожидает подтверждения" / "Активна"
|
||
- Warning if Evotor not connected (link to /evotor)
|
||
- Warning if VK not connected (link to /vk)
|
||
- "Настроить фильтры" button → `/catalog` (catalog browser)
|
||
- "Подтвердить и запустить" button (disabled until filters configured)
|
||
- Summary of current filter rules (X stores, Y groups, Z products)
|
||
|
||
### 6. Navbar / Navigation
|
||
|
||
Add "Синхронизация" link to navbar (for logged-in users), or add it as a card on the `/connections` page since sync depends on connections.
|
||
|
||
### 7. Register Route — `web/main.py`
|
||
|
||
```python
|
||
from web.routes import sync
|
||
app.include_router(sync.router)
|
||
```
|
||
|
||
### 8. Shell Script DB Integration
|
||
|
||
Modify the sync service to read configuration from DB instead of flat files:
|
||
|
||
- Add a Python helper script `run/read_config.py` that queries `sync_configs` + `sync_filters` for a given user and outputs JSON config
|
||
- Shell scripts call this helper to get: enabled flag, store IDs, whitelisted/blacklisted group names, product exclusions
|
||
- The sync service only runs for users where `is_enabled=True` AND `confirmed_at IS NOT NULL`
|
||
- Replaces the flat `vk/whitelist` file
|
||
|
||
## Files Summary
|
||
|
||
| File | Action |
|
||
|------|--------|
|
||
| `web/models.py` | Modify — add `SyncConfig`, `SyncFilter` + User relationship |
|
||
| `web/routes/sync.py` | Create — sync config routes (toggle, confirm) |
|
||
| `web/templates/sync.html` | Create — main sync config page |
|
||
| `web/templates/base.html` | Modify — add sync nav link |
|
||
| `web/main.py` | Modify — register sync router |
|
||
| `run/read_config.py` | Create — DB config reader for shell scripts |
|
||
| Alembic migration | Create — sync_configs + sync_filters tables |
|
||
|
||
## Verification
|
||
|
||
1. Run `alembic upgrade head`
|
||
2. Visit `/sync` without Evotor connection → shows warning to connect first
|
||
3. Connect Evotor, visit `/sync` → shows disabled state, "Настроить фильтры" button
|
||
4. Go to `/sync/stores` → fetches live stores from Evotor API, shows checkboxes
|
||
5. Select stores, save → drill into groups, select groups, save → drill into products
|
||
6. Back to `/sync` → shows summary of configured filters
|
||
7. Enable sync toggle → confirm → `confirmed_at` set
|
||
8. Verify `run/read_config.py` outputs correct JSON for the user's config
|
||
9. Disable sync → `is_enabled=False`, sync service stops processing this user
|