mish/evo-sync
1
0
Fork 0
Code Issues 3 Pull Requests Actions Packages Projects Releases 5 Wiki Activity
Files
master
evo-sync/README.md
mguschin 796cf49ff9 feat: Evotor + VK catalog sync, connections, and store/group filters 
- Evotor catalog: background Celery task syncing stores/groups/products
  from Evotor API; UI pages with per-store and per-group sync toggles
- VK connection: manual token + group ID entry with inline test button
- Evotor connection: inline test button (calls /stores)
- VK catalog: background task syncing VK Market albums and products;
  separate catalog UI at /vk-catalog/albums
- SyncFilter extended to support entity_type=group with parent_entity_id
- Migration 0004: vk_cached_albums + vk_cached_products tables
- Beat schedule updated to run both refresh_catalog and refresh_vk_catalog
- README updated with new schema, routes, tasks, and config

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-01 18:09:11 +03:00

227 lines
13 KiB
Markdown
Raw Permalink Blame History

# EvoSync
Web service for syncing a product catalog from Evotor POS → VK Market. Users connect their Evotor account and a VK community; products from the cash register then appear automatically in the VK store.
---
## Architecture
```
┌─────────────────────────────────────────┐
│ Docker Compose │
│ │
Browser / Evotor ─────► web :8000 (FastAPI + Uvicorn) │
:8080 │ │ │
│ ├── MariaDB :3306 (primary DB) │
│ ├── Redis :6379 (Celery broker) │
│ │ │
│ ├── worker (Celery worker) │
│ ├── beat (Celery beat scheduler) │
│ └── flower :5555 (queue monitor) │
└─────────────────────────────────────────┘
```
### Services
| Service | Image / Dockerfile | Purpose | External port |
|----------|---------------------|-----------------------------------------------|---------------|
| `web` | `Dockerfile.web` | FastAPI app, runs Alembic migrations on start | 8080 → 8000 |
| `worker` | `Dockerfile.web` | Celery worker (sync, health, notifications…) | — |
| `beat` | `Dockerfile.web` | Celery Beat — periodic task scheduler | — |
| `flower` | `Dockerfile.web` | Celery queue monitoring UI | 5555 |
| `db` | `mariadb:11.4` | Primary relational database | — |
| `redis` | `redis:7-alpine` | Celery broker and result backend | — |
### Stack
- **Python 3.12**, FastAPI 0.115, Uvicorn
- **SQLAlchemy 2** + Alembic, MariaDB (PyMySQL)
- **Celery 5** + Redis — background tasks, periodic catalog sync
- **Jinja2** — server-side HTML rendering (`web/templates/`)
- **Pydantic Settings** — configuration from env vars / `.env`
- bcrypt — password hashing
- python-json-logger — structured JSON logs to stdout
---
## Database Schema
| Table | Purpose |
|-----------------------|--------------------------------------------------------------------------------------|
| `users` | User accounts (roles: system / admin / user; statuses: pending / active / suspended) |
| `evotor_connections` | User ↔ Evotor link (access_token, api_token returned to Evotor webhooks) |
| `vk_connections` | User ↔ VK link (user access token + VK community ID) |
| `sync_configs` | Per-user sync settings |
| `sync_filters` | Store / group inclusion filters (entity_type: store / group) |
| `cached_stores` | Cached list of Evotor stores |
| `cached_groups` | Cached Evotor product groups |
| `cached_products` | Cached Evotor product catalog |
| `vk_cached_albums` | Cached VK Market albums (product groups) |
| `vk_cached_products` | Cached VK Market products |
| `roles` | RBAC roles |
| `permissions` | RBAC permissions |
| `role_permissions` | M2M: role ↔ permission |
| `user_roles` | M2M: user ↔ role |
---
## Background Tasks
Periodic tasks run via **Celery Beat** and are executed by the **worker** service.
| Task | Schedule | Description |
|------|----------|-------------|
| `web.tasks.catalog.refresh_catalog` | Every `CATALOG_REFRESH_INTERVAL_SECONDS` | Fetches stores, product groups, and products from the Evotor API for every connected user; upserts into `cached_stores`, `cached_groups`, `cached_products` |
| `web.tasks.vk_catalog.refresh_vk_catalog` | Every `CATALOG_REFRESH_INTERVAL_SECONDS` | Fetches Market albums and products from VK API for every connected user; upserts into `vk_cached_albums`, `vk_cached_products` |
**Evotor sync sequence per user:**
1. `GET /stores` → upsert `cached_stores`
2. For each store: `GET /stores/{id}/product-groups` → upsert `cached_groups`
3. For each store: `GET /stores/{id}/products` → upsert `cached_products`
**VK sync sequence per user:**
1. `market.getAlbums` → upsert `vk_cached_albums`
2. `market.get` (extended=1, paginated) → upsert `vk_cached_products` with album membership
Per-user failures are logged and skipped — one broken token does not block other users.
Evotor stores that return `402 Payment Required` (subscription limit) are silently skipped at debug log level.
---
## Routes
### Connections
| Method | Path | Description |
|--------|-----------------------------------|------------------------------------------|
| GET | `/connections` | View Evotor and VK connection status |
| POST | `/connections/evotor` | Save / update Evotor API token manually |
| POST | `/connections/evotor/disconnect` | Remove Evotor connection |
| POST | `/connections/evotor/test` | Test Evotor connection (JSON) |
| POST | `/connections/vk` | Save / update VK token and group ID |
| POST | `/connections/vk/disconnect` | Remove VK connection |
| POST | `/connections/vk/test` | Test VK connection (JSON) |
### Public / Authentication
| Method | Path | Description |
|----------|--------------------|----------------------------------------|
| GET | `/` | Redirects to `/profile` or `/login` |
| GET | `/health` | Health check (JSON) |
| GET/POST | `/register` | User registration |
| GET | `/confirm-email` | Email confirmation via token |
| GET | `/resend-confirm` | Resend confirmation email |
| GET/POST | `/login` | Login |
| GET | `/logout` | Logout |
| GET/POST | `/forgot-password` | Request password reset |
| GET/POST | `/reset-password` | Reset password via token |
| GET/POST | `/invite` | Complete registration via invite link |
### Profile (requires session)
| Method | Path | Description |
|----------|----------------------------|----------------------|
| GET | `/profile` | View profile |
| GET/POST | `/profile/edit` | Edit profile |
| GET/POST | `/profile/change-password` | Change password |
| GET/POST | `/profile/delete` | Delete account |
### Admin panel (`/admin`, roles: admin / system)
| Method | Path | Description |
|--------|------------------------------------|---------------------------|
| GET | `/admin/users` | User list |
| GET | `/admin/users/{id}` | User detail |
| POST | `/admin/users/{id}/activate` | Activate user |
| POST | `/admin/users/{id}/suspend` | Suspend user |
| POST | `/admin/users/{id}/reset-password` | Reset user password |
| POST | `/admin/users/{id}/send-invite` | Send invite email |
| POST | `/admin/users/{id}/edit` | Edit user data |
| POST | `/admin/users/{id}/delete` | Delete user |
| GET | `/admin/roles` | Roles and permissions |
| POST | `/admin/roles/{id}/permissions` | Update role permissions |
### Evotor Webhooks (Bearer `EVOTOR_WEBHOOK_SECRET`)
| Method | Path | Description |
|--------|----------------|-----------------------------------------------------------|
| POST | `/user/create` | Evotor creates/links a user; returns api_token |
| POST | `/user/verify` | Evotor verifies user credentials; returns api_token |
| POST | `/user/token` | Evotor delivers its own access_token for a user |
### Evotor Catalog (requires session)
| Method | Path | Description |
|--------|---------------------------------------------------|--------------------------------------------|
| GET | `/catalog` | Redirects to `/catalog/stores` |
| GET | `/catalog/stores` | Evotor stores with per-store sync toggle |
| GET | `/catalog/stores/{id}/groups` | Product groups with per-group sync toggle |
| GET | `/catalog/stores/{id}/products` | Products (filterable by group) |
| POST | `/catalog/stores/{id}/toggle` | Enable / disable store sync |
| POST | `/catalog/stores/{id}/groups/{gid}/toggle` | Enable / disable group sync |
### VK Catalog (requires session)
| Method | Path | Description |
|--------|---------------------------------------|------------------------------------|
| GET | `/vk-catalog/albums` | VK Market albums (product groups) |
| GET | `/vk-catalog/albums/{id}/products` | Products in a VK album |
### API Docs
| Path | Description |
|----------|-------------|
| `/docs` | Swagger UI |
| `/redoc` | ReDoc |
---
## Configuration
All settings are read from environment variables or a `.env` file:
| Variable | Default | Description |
|------------------------------------|-------------------------------------|---------------------------------------|
| `DATABASE_URL` | `mysql+pymysql://…@db:3306/evosync` | MariaDB connection string |
| `REDIS_URL` | `redis://redis:6379/0` | Redis connection string |
| `SECRET_KEY` | `change-me-in-production` | Session signing key |
| `BASE_URL` | `http://localhost:8000` | Public URL of the service |
| `EVOTOR_APP_ID` | — | Evotor application ID |
| `EVOTOR_WEBHOOK_SECRET` | — | Bearer secret for webhook endpoints |
| `JIVOSITE_WIDGET_ID` | — | JivoSite widget ID |
| `VK_DEFAULT_PHOTO_PATH` | `/app/default_product.png` | Fallback image path for VK products |
| `VK_API_VERSION` | `5.199` | VK API version |
| `CATALOG_REFRESH_INTERVAL_SECONDS` | `3600` | Evotor + VK catalog sync interval (s) |
| `INVITE_EXPIRE_HOURS` | `48` | Invite link TTL in hours |
| `EMAIL_PROVIDER` | `console` | Email provider (console / smtp / …) |
| `SMS_PROVIDER` | `console` | SMS provider |
| `FLOWER_USER` / `FLOWER_PASSWORD` | `admin` / `changeme` | Basic Auth credentials for Flower |
---
## Running
```bash
cp .env.example .env # fill in your values
docker compose up -d --build
```
App is available at `http://localhost:8080`.
Flower (queue monitor) at `http://localhost:5555`.
### Development
```bash
pip install -r requirements.txt
alembic upgrade head
uvicorn web.main:app --reload --port 8000
```
---
## Tests
```bash
pytest --cov=web
```
Reference in New Issue View Git Blame Copy Permalink