mish/evo-sync
1
0
Fork 0
Code Issues 3 Pull Requests Actions Packages Projects 1 Releases 5 Wiki Activity
Files
7b4f52b00526749bd6f5fb66803bd8f8c3216998
evo-sync/README.md
mguschin 4f4081c54c config: make domain configurable via DOMAIN env var 
Replace hardcoded evosync.ru with a DOMAIN variable read from .env.
nginx.conf is now generated from nginx.conf.template via envsubst;
init-letsencrypt.sh reads DOMAIN from .env and fails loudly if unset.
README documents the new variable and first-deploy TLS workflow.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-12 14:01:38 +03:00

266 lines
15 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# EvoSync
Web service for syncing a product catalog from Evotor POS → VK Market. Users connect their Evotor account and a VK community; products from the cash register then appear automatically in the VK store.
---
## Architecture
```
┌─────────────────────────────────────────┐
│ Docker Compose │
│ │
Browser / Evotor ─────► web :8000 (FastAPI + Uvicorn) │
:8080 │ │ │
│ ├── MariaDB :3306 (primary DB) │
│ ├── Redis :6379 (Celery broker) │
│ │ │
│ ├── worker (Celery worker) │
│ ├── beat (Celery beat scheduler) │
│ └── flower :5555 (queue monitor) │
└─────────────────────────────────────────┘
```
### Services
| Service | Image / Dockerfile | Purpose | External port |
|----------|---------------------|-----------------------------------------------|---------------|
| `web` | `Dockerfile.web` | FastAPI app, runs Alembic migrations on start | 8080 → 8000 |
| `worker` | `Dockerfile.web` | Celery worker (sync, health, notifications…) | — |
| `beat` | `Dockerfile.web` | Celery Beat — periodic task scheduler | — |
| `flower` | `Dockerfile.web` | Celery queue monitoring UI | 5555 |
| `db` | `mariadb:11.4` | Primary relational database | — |
| `redis` | `redis:7-alpine` | Celery broker and result backend | — |
### Stack
- **Python 3.12**, FastAPI 0.115, Uvicorn
- **SQLAlchemy 2** + Alembic, MariaDB (PyMySQL)
- **Celery 5** + Redis — background tasks, periodic catalog sync
- **Jinja2** — server-side HTML rendering (`web/templates/`)
- **Pydantic Settings** — configuration from env vars / `.env`
- bcrypt — password hashing
- python-json-logger — structured JSON logs to stdout
---
## Database Schema
| Table | Purpose |
|-----------------------|--------------------------------------------------------------------------------------|
| `users` | User accounts (roles: system / admin / user; statuses: pending / active / suspended) |
| `evotor_connections` | User ↔ Evotor link (access_token, api_token returned to Evotor webhooks) |
| `vk_connections` | User ↔ VK link (user access token + VK community ID) |
| `sync_configs` | Per-user sync settings |
| `sync_filters` | Store / group inclusion filters (entity_type: store / group) |
| `cached_stores` | Cached list of Evotor stores |
| `cached_groups` | Cached Evotor product groups |
| `cached_products` | Cached Evotor products; `vk_product_id` stores the VK market item ID after first push |
| `vk_cached_albums` | Cached VK Market albums (product groups) |
| `vk_cached_products` | Cached VK Market products |
| `roles` | RBAC roles |
| `permissions` | RBAC permissions |
| `role_permissions` | M2M: role ↔ permission |
| `user_roles` | M2M: user ↔ role |
---
## Background Tasks
Periodic tasks run via **Celery Beat** and are executed by the **worker** service.
Beat fires a single **sync pipeline** every `CATALOG_REFRESH_INTERVAL_SECONDS`. The three tasks run as a Celery chain — each step starts only after the previous one completes:
```
run_sync_pipeline (beat entry)
└─► refresh_catalog — fetch Evotor stores / groups / products
└─► refresh_vk_catalog — fetch VK Market albums / products
└─► mirror_to_vk — push Evotor → VK
```
| Task | Queue | Description |
|------|-------|-------------|
| `web.tasks.celery_app.run_sync_pipeline` | default | Beat entry point; dispatches the chain |
| `web.tasks.catalog.refresh_catalog` | default | Fetches Evotor catalog for all connected users; upserts `cached_stores`, `cached_groups`, `cached_products` |
| `web.tasks.vk_catalog.refresh_vk_catalog` | default | Fetches VK Market albums and products for all connected users; upserts `vk_cached_albums`, `vk_cached_products` |
| `web.tasks.vk_sync.mirror_to_vk` | sync | Mirrors enabled Evotor groups/products → VK Market (create or conditional update) |
**Evotor fetch sequence per user:**
1. `GET /stores` → upsert `cached_stores`
2. For each store: `GET /stores/{id}/product-groups` → upsert `cached_groups`
3. For each store: `GET /stores/{id}/products` → upsert `cached_products`
**VK fetch sequence per user:**
1. `market.getAlbums` → upsert `vk_cached_albums`
2. `market.get` (extended=1, paginated) → upsert `vk_cached_products` with album membership
**Mirror logic per user (`mirror_to_vk`):**
- Skips stores not in enabled store filters; skips groups not in enabled group filters
- For each enabled group: ensures a matching VK album exists (creates via `market.addAlbum` if missing)
- For each product in the group:
- **Create** (`allow_to_sell=true`, no `vk_product_id` yet): uploads default photo once per run, calls `market.add`, assigns to album, stores returned `vk_product_id`
- **Update** (has `vk_product_id`): calls `market.edit` only if name, price, description, or stock_amount changed vs the cached VK state
- **Skip**: product disabled and never pushed, or nothing changed
- Price for weight measures (`г`, `гр`, etc.) is multiplied by `VK_WEIGHT_PRICE_MULTIPLIER` before conversion to kopecks
Per-user failures are logged and skipped — one broken token does not block other users.
Evotor stores that return `402 Payment Required` (subscription limit) are silently skipped at debug log level.
---
## Routes
### Connections
| Method | Path | Description |
|--------|-----------------------------------|------------------------------------------|
| GET | `/connections` | View Evotor and VK connection status |
| POST | `/connections/evotor` | Save / update Evotor API token manually |
| POST | `/connections/evotor/disconnect` | Remove Evotor connection |
| POST | `/connections/evotor/test` | Test Evotor connection (JSON) |
| POST | `/connections/vk` | Save / update VK token and group ID |
| POST | `/connections/vk/disconnect` | Remove VK connection |
| POST | `/connections/vk/test` | Test VK connection (JSON) |
### Public / Authentication
| Method | Path | Description |
|----------|--------------------|----------------------------------------|
| GET | `/` | Redirects to `/profile` or `/login` |
| GET | `/health` | Health check (JSON) |
| GET/POST | `/register` | User registration |
| GET | `/confirm-email` | Email confirmation via token |
| GET | `/resend-confirm` | Resend confirmation email |
| GET/POST | `/login` | Login |
| GET | `/logout` | Logout |
| GET/POST | `/forgot-password` | Request password reset |
| GET/POST | `/reset-password` | Reset password via token |
| GET/POST | `/invite` | Complete registration via invite link |
### Profile (requires session)
| Method | Path | Description |
|----------|----------------------------|----------------------|
| GET | `/profile` | View profile |
| GET/POST | `/profile/edit` | Edit profile |
| GET/POST | `/profile/change-password` | Change password |
| GET/POST | `/profile/delete` | Delete account |
### Admin panel (`/admin`, roles: admin / system)
| Method | Path | Description |
|--------|------------------------------------|---------------------------|
| GET | `/admin/users` | User list |
| GET | `/admin/users/{id}` | User detail |
| POST | `/admin/users/{id}/activate` | Activate user |
| POST | `/admin/users/{id}/suspend` | Suspend user |
| POST | `/admin/users/{id}/reset-password` | Reset user password |
| POST | `/admin/users/{id}/send-invite` | Send invite email |
| POST | `/admin/users/{id}/edit` | Edit user data |
| POST | `/admin/users/{id}/delete` | Delete user |
| GET | `/admin/roles` | Roles and permissions |
| POST | `/admin/roles/{id}/permissions` | Update role permissions |
### Evotor Webhooks (Bearer `EVOTOR_WEBHOOK_SECRET`)
| Method | Path | Description |
|--------|----------------|-----------------------------------------------------------|
| POST | `/user/create` | Evotor creates/links a user; returns api_token |
| POST | `/user/verify` | Evotor verifies user credentials; returns api_token |
| POST | `/user/token` | Evotor delivers its own access_token for a user |
### Evotor Catalog (requires session)
| Method | Path | Description |
|--------|---------------------------------------------------|--------------------------------------------|
| GET | `/catalog` | Redirects to `/catalog/stores` |
| GET | `/catalog/stores` | Evotor stores with per-store sync toggle |
| GET | `/catalog/stores/{id}/groups` | Product groups with per-group sync toggle |
| GET | `/catalog/stores/{id}/products` | Products (filterable by group) |
| POST | `/catalog/stores/{id}/toggle` | Enable / disable store sync |
| POST | `/catalog/stores/{id}/groups/{gid}/toggle` | Enable / disable group sync |
### VK Catalog (requires session)
| Method | Path | Description |
|--------|---------------------------------------|------------------------------------|
| GET | `/vk-catalog/albums` | VK Market albums (product groups) |
| GET | `/vk-catalog/albums/{id}/products` | Products in a VK album |
### API Docs
| Path | Description |
|----------|-------------|
| `/docs` | Swagger UI |
| `/redoc` | ReDoc |
---
## Configuration
All settings are read from environment variables or a `.env` file:
| Variable | Default | Description |
|------------------------------------|-------------------------------------|---------------------------------------|
| `DATABASE_URL` | `mysql+pymysql://…@db:3306/evosync` | MariaDB connection string |
| `REDIS_URL` | `redis://redis:6379/0` | Redis connection string |
| `SECRET_KEY` | `change-me-in-production` | Session signing key |
| `DOMAIN` | — | Public domain name (e.g. `example.com`); used to derive `BASE_URL` and nginx config |
| `BASE_URL` | `https://${DOMAIN}` | Public URL of the service |
| `EVOTOR_APP_ID` | — | Evotor application ID |
| `EVOTOR_WEBHOOK_SECRET` | — | Bearer secret for webhook endpoints |
| `JIVOSITE_WIDGET_ID` | — | JivoSite widget ID |
| `VK_DEFAULT_PHOTO_PATH` | `/app/default_product.png` | Fallback image path for VK products |
| `VK_API_VERSION` | `5.199` | VK API version |
| `CATALOG_REFRESH_INTERVAL_SECONDS` | `3600` | Sync pipeline interval in seconds |
| `VK_CATEGORY_ID` | `40932` | VK Market category ID for all products |
| `VK_STOCK_AMOUNT` | `1000` | Stock amount set for in-sale products |
| `VK_WEIGHT_PRICE_MULTIPLIER` | `10` | Price multiplier for weight-unit products (г, гр, …) |
| `INVITE_EXPIRE_HOURS` | `48` | Invite link TTL in hours |
| `EMAIL_PROVIDER` | `console` | Email provider (console / smtp / …) |
| `SMS_PROVIDER` | `console` | SMS provider |
| `FLOWER_USER` / `FLOWER_PASSWORD` | `admin` / `changeme` | Basic Auth credentials for Flower |
---
## Running
```bash
cp .env.example .env # set DOMAIN and other values
docker compose up -d --build
```
App is available at `http://localhost:8080`.
Flower (queue monitor) at `http://localhost:5555`.
### First deploy (TLS)
```bash
# 1. Generate nginx config from template
DOMAIN=yourdomain.com envsubst '${DOMAIN}' < nginx/nginx.conf.template > nginx/nginx.conf
# 2. Obtain TLS certificate (reads DOMAIN from .env automatically)
sudo ./scripts/init-letsencrypt.sh
# 3. Reload nginx
sudo systemctl reload nginx
```
To change the domain later, repeat all three steps with the new domain.
### Development
```bash
pip install -r requirements.txt
alembic upgrade head
uvicorn web.main:app --reload --port 8000
```
---
## Tests
```bash
pytest --cov=web
```
Reference in New Issue View Git Blame Copy Permalink