feat: release v1.8.0 — connections dashboard, VK OAuth, sync config, catalog browser
- Connections dashboard with add/remove flow and background health checks - VK OAuth connection with profile info and health monitoring - Sync configuration page with master toggle and filter summary - Catalog browser with store/group/product tables, filter management, CSV export - Alembic migrations for all new tables - run/read_config.py for shell sync script DB integration - CHANGELOG.md updated for v1.8.0 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
mguschin
151
docs/plans/sync-configuration.md
Normal file
151
docs/plans/sync-configuration.md
Normal file
@@ -0,0 +1,151 @@
|
||||
# Sync Configuration Feature
|
||||
|
||||
## Context
|
||||
|
||||
EvoSync syncs product catalogs from Evotor → VK. Currently sync runs as a shell-based cron service with a hardcoded store ID and a flat-file whitelist of group names (`vk/whitelist`). This doesn't support multi-user or per-user configuration.
|
||||
|
||||
Users need a web UI to:
|
||||
- Enable/disable the whole sync process
|
||||
- Configure which stores, groups, and products to sync (whitelist/blacklist)
|
||||
- Explicitly confirm before sync starts
|
||||
|
||||
The web app will store config in DB; the shell sync service will read from DB instead of flat files.
|
||||
|
||||
## Data Model
|
||||
|
||||
### `SyncConfig` — per-user master switch
|
||||
|
||||
```
|
||||
tablename: "sync_configs"
|
||||
- id (Integer, PK)
|
||||
- user_id (Integer, FK users.id CASCADE, unique)
|
||||
- is_enabled (Boolean, default=False) # master on/off
|
||||
- confirmed_at (DateTime, nullable) # NULL = never confirmed/started
|
||||
- created_at (DateTime, server_default=now)
|
||||
- updated_at (DateTime, server_default=now, onupdate=now)
|
||||
|
||||
Relationship: User.sync_config (one-to-one)
|
||||
```
|
||||
|
||||
### `SyncFilter` — stores, groups, products filter rules
|
||||
|
||||
```
|
||||
tablename: "sync_filters"
|
||||
- id (Integer, PK)
|
||||
- sync_config_id (Integer, FK sync_configs.id CASCADE)
|
||||
- entity_type (String, enum: "store", "group", "product")
|
||||
- entity_id (String 255) # Evotor UUID
|
||||
- entity_name (String 255) # human-readable, cached
|
||||
- filter_mode (String, enum: "include", "exclude")
|
||||
- parent_entity_id (String 255, nullable) # store_id for groups, group_id for products
|
||||
- created_at (DateTime, server_default=now)
|
||||
|
||||
UniqueConstraint: (sync_config_id, entity_type, entity_id)
|
||||
Relationship: SyncConfig.filters (one-to-many)
|
||||
```
|
||||
|
||||
### Filter Logic
|
||||
|
||||
The filter model uses **explicit include/exclude rules** with these semantics:
|
||||
- **No rules for an entity type** = sync everything of that type (default permissive)
|
||||
- **Any "include" rule exists for a type** = ONLY sync included entities (whitelist mode)
|
||||
- **Only "exclude" rules for a type** = sync everything EXCEPT excluded (blacklist mode)
|
||||
- Hierarchy: store filters → group filters → product filters. If a store is excluded, all its groups/products are excluded regardless of their individual rules.
|
||||
|
||||
## Plan
|
||||
|
||||
### 1. New Models — `web/models.py`
|
||||
|
||||
Add `SyncConfig` and `SyncFilter` as described above. Add `sync_config` relationship to `User`.
|
||||
|
||||
### 2. Alembic Migration
|
||||
|
||||
Create `sync_configs` and `sync_filters` tables.
|
||||
|
||||
### 3. Evotor API Helper — `web/evotor_api.py` (new)
|
||||
|
||||
Async functions to fetch data from Evotor API using a user's stored access token:
|
||||
|
||||
```python
|
||||
async def fetch_stores(access_token: str) -> list[dict]:
|
||||
"""GET https://api.evotor.ru/stores → [{"id": "uuid", "name": "..."}]"""
|
||||
|
||||
async def fetch_groups(access_token: str, store_id: str) -> list[dict]:
|
||||
"""GET https://api.evotor.ru/stores/{store_id}/product-groups → [{"id": "uuid", "name": "..."}]"""
|
||||
|
||||
async def fetch_products(access_token: str, store_id: str) -> list[dict]:
|
||||
"""GET https://api.evotor.ru/stores/{store_id}/products → [{"id": "uuid", "name": "...", "parent_id": "..."}]"""
|
||||
```
|
||||
|
||||
Uses `httpx.AsyncClient`. Returns simplified dicts. Raises on auth failure.
|
||||
|
||||
### 4. Sync Config Route — `web/routes/sync.py` (new)
|
||||
|
||||
**`GET /sync`** — Main sync configuration page.
|
||||
- Requires auth + active Evotor connection
|
||||
- Loads `SyncConfig` (creates default if missing)
|
||||
- Shows: master enable/disable toggle, confirm button, link to filter config
|
||||
|
||||
**`POST /sync/toggle`** — Enable/disable sync.
|
||||
- Toggles `is_enabled`. If enabling for the first time and no filters configured, stays on page with message to configure filters first.
|
||||
|
||||
**`POST /sync/confirm`** — Confirm and start sync.
|
||||
- Sets `confirmed_at = now()`. Only works if `is_enabled=True` and at least one store is configured.
|
||||
|
||||
**Filter management is handled by the Catalog Browser** (see `docs/plans/catalog-browser.md`).
|
||||
The `/catalog` page provides table views of stores, groups, and products with inline filter toggle actions. No separate `/sync/stores`, `/sync/groups`, `/sync/products` routes needed.
|
||||
|
||||
### 5. Templates
|
||||
|
||||
**`web/templates/sync.html`** — Main sync page:
|
||||
- Card with master toggle (on/off switch)
|
||||
- Status: "Не настроено" / "Настроено, ожидает подтверждения" / "Активна"
|
||||
- Warning if Evotor not connected (link to /evotor)
|
||||
- Warning if VK not connected (link to /vk)
|
||||
- "Настроить фильтры" button → `/catalog` (catalog browser)
|
||||
- "Подтвердить и запустить" button (disabled until filters configured)
|
||||
- Summary of current filter rules (X stores, Y groups, Z products)
|
||||
|
||||
### 6. Navbar / Navigation
|
||||
|
||||
Add "Синхронизация" link to navbar (for logged-in users), or add it as a card on the `/connections` page since sync depends on connections.
|
||||
|
||||
### 7. Register Route — `web/main.py`
|
||||
|
||||
```python
|
||||
from web.routes import sync
|
||||
app.include_router(sync.router)
|
||||
```
|
||||
|
||||
### 8. Shell Script DB Integration
|
||||
|
||||
Modify the sync service to read configuration from DB instead of flat files:
|
||||
|
||||
- Add a Python helper script `run/read_config.py` that queries `sync_configs` + `sync_filters` for a given user and outputs JSON config
|
||||
- Shell scripts call this helper to get: enabled flag, store IDs, whitelisted/blacklisted group names, product exclusions
|
||||
- The sync service only runs for users where `is_enabled=True` AND `confirmed_at IS NOT NULL`
|
||||
- Replaces the flat `vk/whitelist` file
|
||||
|
||||
## Files Summary
|
||||
|
||||
| File | Action |
|
||||
|------|--------|
|
||||
| `web/models.py` | Modify — add `SyncConfig`, `SyncFilter` + User relationship |
|
||||
| `web/routes/sync.py` | Create — sync config routes (toggle, confirm) |
|
||||
| `web/templates/sync.html` | Create — main sync config page |
|
||||
| `web/templates/base.html` | Modify — add sync nav link |
|
||||
| `web/main.py` | Modify — register sync router |
|
||||
| `run/read_config.py` | Create — DB config reader for shell scripts |
|
||||
| Alembic migration | Create — sync_configs + sync_filters tables |
|
||||
|
||||
## Verification
|
||||
|
||||
1. Run `alembic upgrade head`
|
||||
2. Visit `/sync` without Evotor connection → shows warning to connect first
|
||||
3. Connect Evotor, visit `/sync` → shows disabled state, "Настроить фильтры" button
|
||||
4. Go to `/sync/stores` → fetches live stores from Evotor API, shows checkboxes
|
||||
5. Select stores, save → drill into groups, select groups, save → drill into products
|
||||
6. Back to `/sync` → shows summary of configured filters
|
||||
7. Enable sync toggle → confirm → `confirmed_at` set
|
||||
8. Verify `run/read_config.py` outputs correct JSON for the user's config
|
||||
9. Disable sync → `is_enabled=False`, sync service stops processing this user
|
||||
Reference in New Issue
Block a user