7df5da76d7
- nginx/nginx.conf: pre-generated config for both domains (IDN punycode for .рф) - scripts/generate-nginx-conf.sh: generates sites-available config from template per domain - scripts/init-letsencrypt.sh: accepts domain as arg (falls back to .env) - README.md: updated deploy section, removed stale VK_WEIGHT_PRICE_MULTIPLIER, added sync/logs routes Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
289 lines
16 KiB
Markdown
289 lines
16 KiB
Markdown
# EvoSync
|
|
|
|
Web service for syncing a product catalog from Evotor POS → VK Market. Users connect their Evotor account and a VK community; products from the cash register then appear automatically in the VK store.
|
|
|
|
---
|
|
|
|
## Architecture
|
|
|
|
```
|
|
┌─────────────────────────────────────────┐
|
|
│ Docker Compose │
|
|
│ │
|
|
Browser / Evotor ─────► web :8000 (FastAPI + Uvicorn) │
|
|
:8080 │ │ │
|
|
│ ├── MariaDB :3306 (primary DB) │
|
|
│ ├── Redis :6379 (Celery broker) │
|
|
│ │ │
|
|
│ ├── worker (Celery worker) │
|
|
│ ├── beat (Celery beat scheduler) │
|
|
│ └── flower :5555 (queue monitor) │
|
|
└─────────────────────────────────────────┘
|
|
```
|
|
|
|
### Services
|
|
|
|
| Service | Image / Dockerfile | Purpose | External port |
|
|
|----------|---------------------|-----------------------------------------------|---------------|
|
|
| `web` | `Dockerfile.web` | FastAPI app, runs Alembic migrations on start | 8080 → 8000 |
|
|
| `worker` | `Dockerfile.web` | Celery worker (sync, health, notifications…) | — |
|
|
| `beat` | `Dockerfile.web` | Celery Beat — periodic task scheduler | — |
|
|
| `flower` | `Dockerfile.web` | Celery queue monitoring UI | 5555 |
|
|
| `db` | `mariadb:11.4` | Primary relational database | — |
|
|
| `redis` | `redis:7-alpine` | Celery broker and result backend | — |
|
|
|
|
### Stack
|
|
|
|
- **Python 3.12**, FastAPI 0.115, Uvicorn
|
|
- **SQLAlchemy 2** + Alembic, MariaDB (PyMySQL)
|
|
- **Celery 5** + Redis — background tasks, periodic catalog sync
|
|
- **Jinja2** — server-side HTML rendering (`web/templates/`)
|
|
- **Pydantic Settings** — configuration from env vars / `.env`
|
|
- bcrypt — password hashing
|
|
- python-json-logger — structured JSON logs to stdout
|
|
|
|
---
|
|
|
|
## Database Schema
|
|
|
|
| Table | Purpose |
|
|
|-----------------------|--------------------------------------------------------------------------------------|
|
|
| `users` | User accounts (roles: system / admin / user; statuses: pending / active / suspended) |
|
|
| `evotor_connections` | User ↔ Evotor link (access_token, api_token returned to Evotor webhooks) |
|
|
| `vk_connections` | User ↔ VK link (user access token + VK community ID) |
|
|
| `sync_configs` | Per-user sync settings |
|
|
| `sync_filters` | Store / group inclusion filters (entity_type: store / group) |
|
|
| `cached_stores` | Cached list of Evotor stores |
|
|
| `cached_groups` | Cached Evotor product groups |
|
|
| `cached_products` | Cached Evotor products; `vk_product_id` stores the VK market item ID after first push |
|
|
| `vk_cached_albums` | Cached VK Market albums (product groups) |
|
|
| `vk_cached_products` | Cached VK Market products |
|
|
| `roles` | RBAC roles |
|
|
| `permissions` | RBAC permissions |
|
|
| `role_permissions` | M2M: role ↔ permission |
|
|
| `user_roles` | M2M: user ↔ role |
|
|
|
|
---
|
|
|
|
## Background Tasks
|
|
|
|
Periodic tasks run via **Celery Beat** and are executed by the **worker** service.
|
|
|
|
Beat fires a single **sync pipeline** every `CATALOG_REFRESH_INTERVAL_SECONDS`. The three tasks run as a Celery chain — each step starts only after the previous one completes:
|
|
|
|
```
|
|
run_sync_pipeline (beat entry)
|
|
└─► refresh_catalog — fetch Evotor stores / groups / products
|
|
└─► refresh_vk_catalog — fetch VK Market albums / products
|
|
└─► mirror_to_vk — push Evotor → VK
|
|
```
|
|
|
|
| Task | Queue | Description |
|
|
|------|-------|-------------|
|
|
| `web.tasks.celery_app.run_sync_pipeline` | default | Beat entry point; dispatches the chain |
|
|
| `web.tasks.catalog.refresh_catalog` | default | Fetches Evotor catalog for all connected users; upserts `cached_stores`, `cached_groups`, `cached_products` |
|
|
| `web.tasks.vk_catalog.refresh_vk_catalog` | default | Fetches VK Market albums and products for all connected users; upserts `vk_cached_albums`, `vk_cached_products` |
|
|
| `web.tasks.vk_sync.mirror_to_vk` | sync | Mirrors enabled Evotor groups/products → VK Market (create or conditional update) |
|
|
|
|
**Evotor fetch sequence per user:**
|
|
1. `GET /stores` → upsert `cached_stores`
|
|
2. For each store: `GET /stores/{id}/product-groups` → upsert `cached_groups`
|
|
3. For each store: `GET /stores/{id}/products` → upsert `cached_products`
|
|
|
|
**VK fetch sequence per user:**
|
|
1. `market.getAlbums` → upsert `vk_cached_albums`
|
|
2. `market.get` (extended=1, paginated) → upsert `vk_cached_products` with album membership
|
|
|
|
**Mirror logic per user (`mirror_to_vk`):**
|
|
- Skips stores not in enabled store filters; skips groups not in enabled group filters
|
|
- For each enabled group: ensures a matching VK album exists (creates via `market.addAlbum` if missing)
|
|
- For each product in the group:
|
|
- **Create** (`allow_to_sell=true`, no `vk_product_id` yet): uploads default photo once per run, calls `market.add`, assigns to album, stores returned `vk_product_id`
|
|
- **Update** (has `vk_product_id`): calls `market.edit` only if name, price, description, or stock_amount changed vs the cached VK state
|
|
- **Skip**: product disabled and never pushed, or nothing changed
|
|
- Price is multiplied by the per-user `price_multiplier` from `sync_configs` (configurable on the `/sync` page)
|
|
- Description is built as `"Name (цена за N unit.)"` when the product has a `measure_name`
|
|
|
|
Per-user failures are logged and skipped — one broken token does not block other users.
|
|
Evotor stores that return `402 Payment Required` (subscription limit) are silently skipped at debug log level.
|
|
|
|
---
|
|
|
|
## Routes
|
|
|
|
### Connections
|
|
|
|
| Method | Path | Description |
|
|
|--------|-----------------------------------|------------------------------------------|
|
|
| GET | `/connections` | View Evotor and VK connection status |
|
|
| POST | `/connections/evotor` | Save / update Evotor API token manually |
|
|
| POST | `/connections/evotor/disconnect` | Remove Evotor connection |
|
|
| POST | `/connections/evotor/test` | Test Evotor connection (JSON) |
|
|
| POST | `/connections/vk` | Save / update VK token and group ID |
|
|
| POST | `/connections/vk/disconnect` | Remove VK connection |
|
|
| POST | `/connections/vk/test` | Test VK connection (JSON) |
|
|
|
|
### Public / Authentication
|
|
|
|
| Method | Path | Description |
|
|
|----------|--------------------|----------------------------------------|
|
|
| GET | `/` | Redirects to `/profile` or `/login` |
|
|
| GET | `/health` | Health check (JSON) |
|
|
| GET/POST | `/register` | User registration |
|
|
| GET | `/confirm-email` | Email confirmation via token |
|
|
| GET | `/resend-confirm` | Resend confirmation email |
|
|
| GET/POST | `/login` | Login |
|
|
| GET | `/logout` | Logout |
|
|
| GET/POST | `/forgot-password` | Request password reset |
|
|
| GET/POST | `/reset-password` | Reset password via token |
|
|
| GET/POST | `/invite` | Complete registration via invite link |
|
|
|
|
### Profile (requires session)
|
|
|
|
| Method | Path | Description |
|
|
|----------|----------------------------|----------------------|
|
|
| GET | `/profile` | View profile |
|
|
| GET/POST | `/profile/edit` | Edit profile |
|
|
| GET/POST | `/profile/change-password` | Change password |
|
|
| GET/POST | `/profile/delete` | Delete account |
|
|
|
|
### Admin panel (`/admin`, roles: admin / system)
|
|
|
|
| Method | Path | Description |
|
|
|--------|------------------------------------|---------------------------|
|
|
| GET | `/admin/users` | User list |
|
|
| GET | `/admin/users/{id}` | User detail |
|
|
| POST | `/admin/users/{id}/activate` | Activate user |
|
|
| POST | `/admin/users/{id}/suspend` | Suspend user |
|
|
| POST | `/admin/users/{id}/reset-password` | Reset user password |
|
|
| POST | `/admin/users/{id}/send-invite` | Send invite email |
|
|
| POST | `/admin/users/{id}/edit` | Edit user data |
|
|
| POST | `/admin/users/{id}/delete` | Delete user |
|
|
| GET | `/admin/roles` | Roles and permissions |
|
|
| POST | `/admin/roles/{id}/permissions` | Update role permissions |
|
|
|
|
### Evotor Webhooks (Bearer `EVOTOR_WEBHOOK_SECRET`)
|
|
|
|
| Method | Path | Description |
|
|
|--------|----------------|-----------------------------------------------------------|
|
|
| POST | `/user/create` | Evotor creates/links a user; returns api_token |
|
|
| POST | `/user/verify` | Evotor verifies user credentials; returns api_token |
|
|
| POST | `/user/token` | Evotor delivers its own access_token for a user |
|
|
|
|
### Evotor Catalog (requires session)
|
|
|
|
| Method | Path | Description |
|
|
|--------|---------------------------------------------------|--------------------------------------------|
|
|
| GET | `/catalog` | Redirects to `/catalog/stores` |
|
|
| GET | `/catalog/stores` | Evotor stores with per-store sync toggle |
|
|
| GET | `/catalog/stores/{id}/groups` | Product groups with per-group sync toggle |
|
|
| GET | `/catalog/stores/{id}/products` | Products (filterable by group) |
|
|
| POST | `/catalog/stores/{id}/toggle` | Enable / disable store sync |
|
|
| POST | `/catalog/stores/{id}/groups/{gid}/toggle` | Enable / disable group sync |
|
|
|
|
### Sync Settings (requires session)
|
|
|
|
| Method | Path | Description |
|
|
|----------|------------------|------------------------------------------------------|
|
|
| GET | `/sync` | Sync settings: task on/off switches, price multiplier |
|
|
| POST | `/sync/settings` | Save sync settings |
|
|
|
|
### VK Catalog (requires session)
|
|
|
|
| Method | Path | Description |
|
|
|--------|---------------------------------------|------------------------------------|
|
|
| GET | `/vk-catalog/albums` | VK Market albums (product groups) |
|
|
| GET | `/vk-catalog/albums/{id}/products` | Products in a VK album |
|
|
|
|
### Admin Logs (`/admin`, roles: admin / system)
|
|
|
|
| Method | Path | Description |
|
|
|--------|---------------|-------------------------------------------------------|
|
|
| GET | `/admin/logs` | API request/response log viewer with filters and pagination |
|
|
|
|
### API Docs
|
|
|
|
| Path | Description |
|
|
|----------|-------------|
|
|
| `/docs` | Swagger UI |
|
|
| `/redoc` | ReDoc |
|
|
|
|
---
|
|
|
|
## Configuration
|
|
|
|
All settings are read from environment variables or a `.env` file:
|
|
|
|
| Variable | Default | Description |
|
|
|------------------------------------|-------------------------------------|---------------------------------------|
|
|
| `DATABASE_URL` | `mysql+pymysql://…@db:3306/evosync` | MariaDB connection string |
|
|
| `REDIS_URL` | `redis://redis:6379/0` | Redis connection string |
|
|
| `SECRET_KEY` | `change-me-in-production` | Session signing key |
|
|
| `DOMAIN` | — | Public domain name (e.g. `example.com`); used to derive `BASE_URL` and nginx config |
|
|
| `BASE_URL` | `https://${DOMAIN}` | Public URL of the service |
|
|
| `EVOTOR_APP_ID` | — | Evotor application ID |
|
|
| `EVOTOR_WEBHOOK_SECRET` | — | Bearer secret for webhook endpoints |
|
|
| `JIVOSITE_WIDGET_ID` | — | JivoSite widget ID |
|
|
| `VK_DEFAULT_PHOTO_PATH` | `/app/default_product.png` | Fallback image path for VK products |
|
|
| `VK_API_VERSION` | `5.199` | VK API version |
|
|
| `CATALOG_REFRESH_INTERVAL_SECONDS` | `3600` | Sync pipeline interval in seconds |
|
|
| `VK_CATEGORY_ID` | `40932` | VK Market category ID for all products |
|
|
| `VK_STOCK_AMOUNT` | `1000` | Stock amount set for in-sale products |
|
|
| `INVITE_EXPIRE_HOURS` | `48` | Invite link TTL in hours |
|
|
| `EMAIL_PROVIDER` | `console` | Email provider (console / smtp / …) |
|
|
| `SMS_PROVIDER` | `console` | SMS provider |
|
|
| `FLOWER_USER` / `FLOWER_PASSWORD` | `admin` / `changeme` | Basic Auth credentials for Flower |
|
|
|
|
---
|
|
|
|
## Running
|
|
|
|
```bash
|
|
cp .env.example .env # set DOMAIN and other values
|
|
docker compose up -d --build
|
|
```
|
|
|
|
App is available at `http://localhost:8080`.
|
|
Flower (queue monitor) at `http://localhost:5555`.
|
|
|
|
### First deploy (TLS)
|
|
|
|
Run once per domain (repeat for every domain pointing to this server):
|
|
|
|
```bash
|
|
# 1. Obtain TLS certificate
|
|
sudo ./scripts/init-letsencrypt.sh my-products.ru
|
|
sudo ./scripts/init-letsencrypt.sh мои-товары.рф
|
|
|
|
# 2. Generate nginx site config and symlink it into sites-enabled
|
|
sudo ./scripts/generate-nginx-conf.sh my-products.ru
|
|
sudo ./scripts/generate-nginx-conf.sh мои-товары.рф
|
|
|
|
# 3. Reload nginx
|
|
sudo systemctl reload nginx
|
|
```
|
|
|
|
`generate-nginx-conf.sh` expands `nginx/nginx.conf.template` with the given domain and writes the result to `/etc/nginx/sites-available/<domain>.conf`, then creates a symlink in `sites-enabled`.
|
|
|
|
Set up auto-renewal (if not already configured by certbot):
|
|
|
|
```
|
|
0 3 * * * root certbot renew --quiet && systemctl reload nginx
|
|
```
|
|
|
|
### Development
|
|
|
|
```bash
|
|
pip install -r requirements.txt
|
|
alembic upgrade head
|
|
uvicorn web.main:app --reload --port 8000
|
|
```
|
|
|
|
---
|
|
|
|
## Tests
|
|
|
|
```bash
|
|
pytest --cov=web
|
|
```
|