chore: update architecture and documentation to reflect removal of MongoDB and transition to NAS storage for chapter content
Build and Push Reader API Image / docker (push) Successful in 1m3s
Build and Push Reader API Image / docker (push) Successful in 1m3s
- Updated `.env.example` to remove MongoDB configuration. - Revised `ARCHITECTURE.md` to reflect changes in data orchestration and storage strategy. - Removed `CHAPTER_SAVE_DEBUG.md` and `FIXES_APPLIED.md` as they are no longer relevant. - Updated `CONTRACT.md` to include new error codes. - Adjusted endpoint documentation in `CROSS_REPO_ENDPOINT_MATRIX.md` and `README.md` to align with the new import flow. - Removed legacy import scripts and tables from the codebase.
This commit is contained in:
+1
-2
@@ -1,6 +1,5 @@
|
||||
# PostgreSQL + MongoDB
|
||||
# PostgreSQL
|
||||
DATABASE_URL=postgresql://reader:reader@localhost:5432/reader
|
||||
MONGODB_URI=mongodb://localhost:27017/reader
|
||||
|
||||
# Auth / OAuth
|
||||
NEXTAUTH_SECRET=replace-with-strong-secret
|
||||
|
||||
+8
-5
@@ -8,7 +8,7 @@ Tai lieu nay mo ta `reader-api` la backend dung chung cho Web (`reader`) va Andr
|
||||
- API contract
|
||||
- domain rule
|
||||
- auth mapping web/mobile
|
||||
- data orchestration PostgreSQL + MongoDB
|
||||
- data orchestration: PostgreSQL + file/NAS chapter storage (khong con MongoDB)
|
||||
- Moi thay doi contract phai uu tien backward-compatible cho 2 client.
|
||||
|
||||
## Domain ownership
|
||||
@@ -20,10 +20,9 @@ Tai lieu nay mo ta `reader-api` la backend dung chung cho Web (`reader`) va Andr
|
||||
## Data strategy
|
||||
|
||||
- PostgreSQL:
|
||||
- user, novel metadata, genres, comments, ratings, bookmarks, progress.
|
||||
- MongoDB:
|
||||
- chapter text content lon.
|
||||
- recommendation document payload (neu can rich document).
|
||||
- user, novel metadata, genres, comments, ratings, bookmarks, progress, chapter metadata (`ChapterMeta`), refs noi dung (`ChapterContentRef`).
|
||||
- Chapter body:
|
||||
- file/NAS (qua `app.storage`) — txt/html theo href trong `ChapterContentRef`.
|
||||
|
||||
## Auth and identity
|
||||
|
||||
@@ -44,3 +43,7 @@ Tai lieu nay mo ta `reader-api` la backend dung chung cho Web (`reader`) va Andr
|
||||
- Da verify voi web + mobile happy path va auth edge cases.
|
||||
- Healthcheck va monitoring khong bi anh huong.
|
||||
- Docker/local dev van chay voi huong dan README.
|
||||
|
||||
## Import (MOD)
|
||||
|
||||
Chi con endpoint import preview `POST /api/import/uploads/preview` va nhom `/api/mod/epub*`. Cac endpoint SourceAsset/job legacy da go khoi service.
|
||||
|
||||
@@ -1,256 +0,0 @@
|
||||
# 🐛 Chapter Save Debugging Guide
|
||||
|
||||
## Vấn Đề Đã Fix
|
||||
|
||||
### ✅ Fix 1: Ownership Check (Line 932)
|
||||
- **Vấn đề:** MOD không thể tạo chapter cho truyện mặc định (uploaderId = NULL)
|
||||
- **Fix:** Thêm `OR uploaderId IS NULL` vào WHERE clause
|
||||
- **Dòng:** 932, 1002
|
||||
|
||||
### ✅ Fix 2: Input Validation
|
||||
- **Vấn đề:** Content có thể trống, number có thể âm
|
||||
- **Fix:** Thêm validation trước insert
|
||||
- **Dòng:** 920-927, 997-1004
|
||||
|
||||
### ✅ Fix 3: Data Consistency Logging
|
||||
- **Vấn đề:** Nếu MongoDB insert succeed nhưng PostgreSQL fail → dữ liệu inconsistent
|
||||
- **Fix:** Thêm separate error handling và logging [CRITICAL]
|
||||
- **Dòng:** 956-974
|
||||
|
||||
---
|
||||
|
||||
## 🔍 Testing Checklist
|
||||
|
||||
### A. Network Debugging (Browser DevTools)
|
||||
|
||||
1. **Mở DevTools:** F12 → Network tab
|
||||
2. **Ấn "Lưu chương"**
|
||||
3. **Kiểm tra request `POST /api/mod/chuong`:**
|
||||
- ✅ Status code: `201` = success, `4xx`/`5xx` = error
|
||||
- ✅ Response body: Phần lấy `id`, `number`, `title`
|
||||
- ❌ Status 403: Ownership issue
|
||||
- ❌ Status 400: Duplicate or validation error
|
||||
- ❌ Status 500: Server error (check [CRITICAL] logs)
|
||||
|
||||
### B. MongoDB Verification
|
||||
|
||||
```bash
|
||||
# Access MongoDB
|
||||
mongosh # hoặc mongo
|
||||
|
||||
# Switch to database
|
||||
use reader_db # (thay bằng tên DB thực tế)
|
||||
|
||||
# List recent chapters
|
||||
db.chapters.find({}, {novelId: 1, number: 1, title: 1, createdAt: 1})
|
||||
.sort({_id: -1})
|
||||
.limit(5)
|
||||
|
||||
# Check specific novel
|
||||
db.chapters.countDocuments({novelId: "YOUR_NOVEL_ID"})
|
||||
|
||||
# Check for duplicates (race condition)
|
||||
db.chapters.find({novelId: "YOUR_NOVEL_ID", number: 1})
|
||||
```
|
||||
|
||||
### C. PostgreSQL Verification
|
||||
|
||||
```bash
|
||||
# Access PostgreSQL
|
||||
psql # hoặc your database client
|
||||
|
||||
# Check novel total chapters count
|
||||
SELECT id, title, "totalChapters" FROM "Novel" WHERE id = 'YOUR_NOVEL_ID';
|
||||
|
||||
# Verify it matches MongoDB count
|
||||
-- MongoDB should have same count as "totalChapters"
|
||||
```
|
||||
|
||||
### D. Server Log Analysis
|
||||
|
||||
Look for these patterns in backend logs:
|
||||
|
||||
```
|
||||
✅ Success:
|
||||
[timestamp] POST /mod/chuong - Status 201
|
||||
[timestamp] Inserted chapter id: xxx
|
||||
|
||||
❌ Issues:
|
||||
[CRITICAL] ⚠️ INCONSISTENT STATE: Chapter inserted in MongoDB...
|
||||
[timestamp] Lỗi MongoDB: [error message]
|
||||
[timestamp] Ownership check failed: 403
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Common Scenarios & Solutions
|
||||
|
||||
### Scenario 1: Network Shows 201 But Chapter Not Visible
|
||||
|
||||
**Cause:** Chapter saved but not refreshed in UI
|
||||
**Solution:**
|
||||
- Press F5 to refresh page
|
||||
- Check MongoDB to confirm chapter exists
|
||||
- Check if `fetchChapters()` was called after save
|
||||
|
||||
### Scenario 2: Network Shows 403 Forbidden
|
||||
|
||||
**Cause:** Novel ownership check failed
|
||||
**Solution:**
|
||||
- Verify you are MOD or ADMIN user
|
||||
- Verify novel exists in PostgreSQL:
|
||||
```sql
|
||||
SELECT id, title, "uploaderId" FROM "Novel" WHERE id = 'YOUR_ID';
|
||||
```
|
||||
- If uploaderId is NULL (default), ensure you're MOD user
|
||||
|
||||
### Scenario 3: Network Shows 400 Bad Request
|
||||
|
||||
**Causes:**
|
||||
- Chapter number already exists
|
||||
- Title or content empty
|
||||
- Chapter number ≤ 0
|
||||
|
||||
**Solution:** Check response detail message and fix input
|
||||
|
||||
### Scenario 4: Network Shows 500 Server Error
|
||||
|
||||
**Cause:** MongoDB or PostgreSQL failure
|
||||
**Solution:**
|
||||
- Check server logs for [CRITICAL] message
|
||||
- If MongoDB failed: Check MongoDB connection
|
||||
- If PostgreSQL failed: Check PostgreSQL connection
|
||||
- Contact admin with error message
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Advanced Debug Commands
|
||||
|
||||
### Check MongoDB Connection Status
|
||||
|
||||
```bash
|
||||
# From backend terminal
|
||||
python3 -c "
|
||||
import asyncio
|
||||
from app.database import mongo_db
|
||||
|
||||
async def check():
|
||||
await mongo_db.command('ping')
|
||||
print('✓ MongoDB Connected!')
|
||||
|
||||
asyncio.run(check())
|
||||
"
|
||||
```
|
||||
|
||||
### Check PostgreSQL Connection Status
|
||||
|
||||
```bash
|
||||
# From backend terminal
|
||||
python3 -c "
|
||||
import asyncio
|
||||
from sqlalchemy.ext.asyncio import create_async_engine
|
||||
from app.database import SessionLocal
|
||||
|
||||
async def check():
|
||||
async with SessionLocal() as session:
|
||||
result = await session.execute('SELECT 1')
|
||||
print('✓ PostgreSQL Connected!')
|
||||
|
||||
asyncio.run(check())
|
||||
"
|
||||
```
|
||||
|
||||
### Manually Sync Total Chapters
|
||||
|
||||
```bash
|
||||
# If totalChapters is out of sync
|
||||
mongosh
|
||||
use reader_db
|
||||
|
||||
# Get count
|
||||
db.chapters.countDocuments({novelId: "YOUR_NOVEL_ID"})
|
||||
|
||||
# Then update PostgreSQL manually:
|
||||
# psql
|
||||
UPDATE "Novel" SET "totalChapters" = 123 WHERE id = 'YOUR_NOVEL_ID';
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📋 Test Cases
|
||||
|
||||
### Test 1: Basic Chapter Save
|
||||
```
|
||||
1. Create novel
|
||||
2. Save chapter #1
|
||||
3. ✅ Should appear in chapter list
|
||||
4. ✅ totalChapters should be 1
|
||||
```
|
||||
|
||||
### Test 2: Sequential Chapters
|
||||
```
|
||||
1. Save chapters 1, 2, 3
|
||||
2. ✅ All should appear with correct numbers
|
||||
3. ✅ Next chapter field should suggest 4
|
||||
```
|
||||
|
||||
### Test 3: Duplicate Prevention
|
||||
```
|
||||
1. Save chapter #5
|
||||
2. Try to save chapter #5 again
|
||||
3. ✅ Should show "Chương 5 đã tồn tại"
|
||||
```
|
||||
|
||||
### Test 4: Default Novel (MOD Permission)
|
||||
```
|
||||
1. Verify a novel with uploaderId = NULL exists
|
||||
2. As MOD user, save chapter to that novel
|
||||
3. ✅ Should succeed (not 403 Forbidden)
|
||||
```
|
||||
|
||||
### Test 5: No Empty Content
|
||||
```
|
||||
1. Try to save chapter with empty title
|
||||
2. ✅ Should show "Tiêu đề chương không được trống"
|
||||
3. Try to save chapter with empty content
|
||||
4. ✅ Should show "Nội dung chương không được trống"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🆘 Still Having Issues?
|
||||
|
||||
1. **Run checklist A, B, C above** and collect outputs
|
||||
2. **Screenshot of Network tab response**
|
||||
3. **MongoDB output from `find()`**
|
||||
4. **Server log output (especially [CRITICAL] lines)**
|
||||
5. Share these with: [your-dev-contact]
|
||||
|
||||
---
|
||||
|
||||
## 📊 Monitoring
|
||||
|
||||
### Health Check Script
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# save as monitor-save.sh
|
||||
|
||||
echo "=== Chapter Save System Health Check ==="
|
||||
echo ""
|
||||
echo "1. MongoDB Connection:"
|
||||
# mongosh check here
|
||||
|
||||
echo ""
|
||||
echo "2. PostgreSQL Connection:"
|
||||
# psql check here
|
||||
|
||||
echo ""
|
||||
echo "3. Backend API:"
|
||||
curl -s http://localhost:8000/docs | head -20
|
||||
|
||||
echo ""
|
||||
echo "=== Done ==="
|
||||
```
|
||||
|
||||
Run: `bash monitor-save.sh`
|
||||
@@ -32,6 +32,7 @@ Tai lieu contract chung cho `reader`, `reader-app`, `reader-api`.
|
||||
- `409`: xung dot du lieu.
|
||||
- `422`: payload format dung JSON nhung khong dat rule nghiep vu.
|
||||
- `500`: loi he thong.
|
||||
- `410`: (du tru) tai nguyen da go bo hoac khong con ho tro.
|
||||
|
||||
## Pagination Convention
|
||||
|
||||
|
||||
@@ -26,17 +26,12 @@ Legend:
|
||||
| Comment | `GET/POST /api/truyen/{id}/comments` | Y | Y | Y | |
|
||||
| Rating | `POST /api/truyen/{id}/rate` | Y | Y | N | Mobile chua thay rating flow |
|
||||
| Search | `GET /api/truyen/suggest` | Y | Y | N | Mobile search suggest can bo sung |
|
||||
| Import | `GET /api/import/assets/search` | Y | Y | N | Web MOD import wizard step 1 |
|
||||
| Import | `GET /api/import/assets/{id}/preview-metadata` | Y | Y | N | Web MOD import wizard step 2 |
|
||||
| Import | `POST /api/import/assets/{id}/ai-suggest` | Y | Y | N | Gen toi da 6 genres + short description |
|
||||
| Import | `POST /api/import/assets/{id}/review` | Y | Y | N | Save reviewed metadata before import |
|
||||
| Import | `POST /api/import/assets/{id}/parse-preview` | Y | Y | N | TOC/regex-start preview (10 head/mid/tail samples) |
|
||||
| Import | `POST /api/import/assets/{id}/start-import` | Y | Y | N | Start import session |
|
||||
| Import | `GET /api/import/sessions/{sessionId}` | Y | Y | N | Poll import progress |
|
||||
| Import | `POST /api/import/uploads/preview` | Y | Y | N | Upload EPUB multipart (preview) |
|
||||
| Import | `POST /api/mod/epub`, `POST /api/mod/epub/ai-suggest` | Y | Y | N | Luong `/mod/import` |
|
||||
| Import | `GET/POST/PUT/DELETE /api/mod/the-loai` | Y | Y | N | MOD quan ly the loai trong wizard |
|
||||
|
||||
## Priority gaps de dong bo tiep
|
||||
|
||||
1. Mobile: `user/settings`, `recommendations`, `rate`, `suggest`.
|
||||
2. Web/Mobile chapter-read strategy can unify (`chapters/{id}` vs `by-number`).
|
||||
3. Chuan hoa error contract implementation theo `CONTRACT.md`.
|
||||
4. Mobile import flow currently not planned (MOD-only on web).
|
||||
|
||||
@@ -1,186 +0,0 @@
|
||||
# ✅ Chapter Save System - Fixed Issues Summary
|
||||
|
||||
**Date:** 2026-03-24
|
||||
**Status:** All HIGH priority issues fixed ✅
|
||||
|
||||
---
|
||||
|
||||
## 🐛 Issues Identified & Fixed
|
||||
|
||||
### 1️⃣ **Ownership Check Bypass (HIGH)**
|
||||
- **File:** `app/routers/mod.py`
|
||||
- **Lines:** 932-936 (POST), 1000-1007 (PUT)
|
||||
- **Issue:** MOD couldn't create chapters for default novels (uploaderId = NULL)
|
||||
- **Fix:** Changed query from:
|
||||
```python
|
||||
WHERE id = :nid AND "uploaderId" = :uid
|
||||
```
|
||||
To:
|
||||
```python
|
||||
WHERE id = :nid AND ("uploaderId" = :uid OR "uploaderId" IS NULL)
|
||||
```
|
||||
- **Impact:** ✅ Fixed - MOD can now manage default novels
|
||||
|
||||
### 2️⃣ **Missing Input Validation (HIGH)**
|
||||
- **File:** `app/routers/mod.py`
|
||||
- **Lines:** 920-927 (POST), 997-1004 (PUT)
|
||||
- **Issue:** Could save empty title/content, negative chapter numbers
|
||||
- **Fix:** Added validation:
|
||||
```python
|
||||
if not body.title or not body.title.strip():
|
||||
raise HTTPException(400, "Tiêu đề chương không được trống")
|
||||
if body.number <= 0:
|
||||
raise HTTPException(400, "Số chương phải > 0")
|
||||
```
|
||||
- **Impact:** ✅ Fixed - Invalid data rejected at API level
|
||||
|
||||
### 3️⃣ **Data Inconsistency on PostgreSQL Failure (HIGH)**
|
||||
- **File:** `app/routers/mod.py`
|
||||
- **Lines:** 956-974 (POST)
|
||||
- **Issue:** If MongoDB insert succeeds but PostgreSQL sync fails → inconsistent state
|
||||
- **Fix:** Added separate error handling:
|
||||
```python
|
||||
try:
|
||||
result = await mongo_db.chapters.insert_one(doc)
|
||||
except Exception as mongo_err:
|
||||
raise HTTPException(500, f"Lỗi MongoDB: {str(mongo_err)}")
|
||||
|
||||
try:
|
||||
total = await _sync_total_chapters(db, body.novelId)
|
||||
except Exception as pg_err:
|
||||
# Log [CRITICAL] and alert user
|
||||
raise HTTPException(500, "Dữ liệu has được lưu nhưng...")
|
||||
```
|
||||
- **Impact:** ✅ Fixed - Clear error messages identify MongoDB vs PostgreSQL failures
|
||||
|
||||
### 4️⃣ **Frontend Error Display (MEDIUM)**
|
||||
- **File:** `reader/app/mod/chuong/chapter-client.tsx`
|
||||
- **Line:** 356
|
||||
- **Issue:** Only checked `error` field, not FastAPI's `detail` field
|
||||
- **Fix:** Changed:
|
||||
```javascript
|
||||
if (!res.ok) throw new Error(resData.error || resData.detail || "...")
|
||||
```
|
||||
- **Impact:** ✅ Fixed - Users see actual backend error messages
|
||||
|
||||
### 5️⃣ **Missing Novel Existence Check (MEDIUM)**
|
||||
- **File:** `app/routers/mod.py`
|
||||
- **Lines:** 948-951 (POST)
|
||||
- **Issue:** Could try to save chapter for non-existent novel
|
||||
- **Fix:** Added explicit check:
|
||||
```python
|
||||
novel_check = await db.execute(
|
||||
text('SELECT id FROM "Novel" WHERE id = :nid'),
|
||||
{"nid": body.novelId},
|
||||
)
|
||||
if not novel_check.first():
|
||||
raise HTTPException(404, "Truyện không tồn tại")
|
||||
```
|
||||
- **Impact:** ✅ Fixed - Better error message (404 instead of 500)
|
||||
|
||||
---
|
||||
|
||||
## 📊 Testing Results
|
||||
|
||||
### Build Verification
|
||||
```
|
||||
✅ Python syntax: OK (py_compile passed)
|
||||
✅ Next.js build: OK (all 11 routes successfully compiled)
|
||||
✅ No type errors in modified files
|
||||
```
|
||||
|
||||
### Modified Files
|
||||
| File | Changes | Status |
|
||||
|------|---------|--------|
|
||||
| `app/routers/mod.py` | POST/PUT endpoints refactored with validation | ✅ Fixed |
|
||||
| `reader/app/mod/chuong/chapter-client.tsx` | Error handling improved | ✅ Fixed |
|
||||
| `CHAPTER_SAVE_DEBUG.md` | Debug guide created | ✅ New |
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Next Steps for User
|
||||
|
||||
### ⚠️ IMPORTANT: Test the following scenarios
|
||||
|
||||
1. **Basic Save:** Create new novel → save Chapter 1 → verify appears in list
|
||||
2. **Ownership:** Try saving to default novel as MOD user → should succeed now
|
||||
3. **Duplicate:** Try saving same chapter twice → should show "đã tồn tại"
|
||||
4. **Empty Content:** Try saving without title → should show validation error
|
||||
5. **Negative Number:** Try chapter #-1 → should reject
|
||||
|
||||
### If Still Failing:
|
||||
|
||||
1. Open **DevTools Network tab** → F12 → Network
|
||||
2. Try to save chapter
|
||||
3. Look for **POST /api/mod/chuong** request
|
||||
4. Check **Status code** and **Response body**
|
||||
5. Use guide in `CHAPTER_SAVE_DEBUG.md` to troubleshoot
|
||||
|
||||
### Possible Remaining Issues
|
||||
|
||||
⚠️ **Not yet fixed (MEDIUM/LOW priority):**
|
||||
- [ ] Race condition on duplicate chapter check (add MongoDB unique index)
|
||||
- [ ] No MongoDB/PostgreSQL timeout configuration
|
||||
- [ ] Generic exception handler logging (uses traceback.print_exc)
|
||||
- [ ] Missing structured logging system
|
||||
|
||||
These are less critical but could cause issues under high load.
|
||||
|
||||
---
|
||||
|
||||
## 📝 Code Changes Summary
|
||||
|
||||
**Total lines modified:** ~150
|
||||
**Files affected:** 2
|
||||
**Breaking changes:** None (backward compatible)
|
||||
**Rollback difficulty:** Low (simple validation additions)
|
||||
|
||||
---
|
||||
|
||||
## ✨ What Changed
|
||||
|
||||
```diff
|
||||
# POST /mod/chuong
|
||||
- Missing input validation
|
||||
- Missing novel existence check
|
||||
- Ownership query doesn't allow NULL uploaderId
|
||||
- No separation of MongoDB vs PostgreSQL error handling
|
||||
|
||||
+ Full input validation (title, content, number)
|
||||
+ Novel existence check with clear 404 error
|
||||
+ Ownership check allows both user-owned and default novels
|
||||
+ Separate error handling for MongoDB and PostgreSQL
|
||||
+ Better error messages for debugging
|
||||
+ Data consistency logging ([CRITICAL] alerts)
|
||||
|
||||
# PUT /mod/chuong
|
||||
- Same issues as POST
|
||||
|
||||
+ Same fixes applied
|
||||
|
||||
# Frontend error handling
|
||||
- Ignored FastAPI's 'detail' field
|
||||
|
||||
+ Now checks both 'error' and 'detail' fields
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔍 Monitoring Recommendations
|
||||
|
||||
1. **Set up log monitoring** for `[CRITICAL]` messages
|
||||
2. **Verify MongoDB connection** on startup
|
||||
3. **Verify PostgreSQL connection** on startup
|
||||
4. **Add request logging** to track save operations
|
||||
5. **Monitor totalChapters sync** for discrepancies
|
||||
|
||||
---
|
||||
|
||||
## 📞 Support
|
||||
|
||||
If issues persist after testing:
|
||||
1. Follow debugging guide in `CHAPTER_SAVE_DEBUG.md`
|
||||
2. Check Network tab for response codes
|
||||
3. Verify MongoDB and PostgreSQL connectivity
|
||||
4. Look for [CRITICAL] messages in server logs
|
||||
5. Check browser console for JavaScript errors
|
||||
@@ -39,20 +39,13 @@ Backend flow theo domain, de web/mobile follow giong nhau.
|
||||
|
||||
## Flow E: EPUB Import (MOD/ADMIN)
|
||||
|
||||
- Step 1 search source:
|
||||
- `/api/import/assets/search`
|
||||
- Step 2 review metadata:
|
||||
- `/api/import/assets/{id}/preview-metadata`
|
||||
- `/api/import/assets/{id}/ai-suggest`
|
||||
- `/api/import/assets/{id}/review`
|
||||
- Step 3 chapter split preview:
|
||||
- `/api/import/assets/{id}/parse-preview`
|
||||
- split mode: `toc` or `regex` (chapter-start pattern only)
|
||||
- Step 4 start import + progress:
|
||||
- `/api/import/assets/{id}/start-import`
|
||||
- `/api/import/sessions/{sessionId}`
|
||||
Luồng trên web (`reader` `/mod/import`):
|
||||
|
||||
- Thể loại: `GET/POST/PUT/DELETE /api/mod/the-loai`
|
||||
- Upload EPUB preview: `POST /api/import/uploads/preview` (multipart)
|
||||
- Gợi ý metadata: `POST /api/mod/epub/ai-suggest`
|
||||
- Import EPUB: `POST /api/mod/epub`
|
||||
|
||||
Rules:
|
||||
- No filesystem scan in search request path (scan by cron/incremental).
|
||||
- Reviewer confirms metadata before import.
|
||||
- Import writes NAS content + chapter refs, then updates novel counters.
|
||||
|
||||
- Import ghi nội dung NAS / refs chương và cập nhật metadata novel khi hoàn tất.
|
||||
|
||||
@@ -11,7 +11,7 @@ This project is Python-first (FastAPI), with production-focused Docker setup and
|
||||
- Python 3.11+
|
||||
- FastAPI
|
||||
- UV (package manager / runner)
|
||||
- PostgreSQL (structured data)
|
||||
- PostgreSQL (metadata, user data, chapter refs; chapter text stored via NAS/files — see `NAS_CONTENT_ROOT` / storage layer)
|
||||
|
||||
## API Base URL
|
||||
|
||||
@@ -133,47 +133,40 @@ volumes:
|
||||
|
||||
For your EPUB structure (folder per novel, multiple `.epub` parts inside), mount the parent folder to `/data/epub-source`.
|
||||
|
||||
## Implemented Endpoints
|
||||
## Implemented Endpoints (snapshot)
|
||||
|
||||
- GET /api/health
|
||||
- POST /api/auth/mobile-login
|
||||
- GET /api/user/profile
|
||||
- GET/POST /api/user/bookmarks
|
||||
- DELETE /api/user/bookmarks/{novelId}
|
||||
- POST /api/user/reading-progress
|
||||
- GET/POST /api/user/settings
|
||||
- GET/POST/DELETE /api/user/recommendations
|
||||
- GET /api/genres
|
||||
- GET /api/novels/browse
|
||||
- GET /api/novels/{idOrSlug}
|
||||
- GET /api/truyen/{id}/chapters
|
||||
- GET/POST /api/truyen/{id}/comments
|
||||
- POST /api/truyen/{id}/rate
|
||||
- GET /api/truyen/suggest
|
||||
- GET /api/chapters/{chapterId}
|
||||
- GET /api/import/assets/search
|
||||
- GET /api/import/assets/{assetId}/preview-metadata
|
||||
- POST /api/import/assets/{assetId}/ai-suggest
|
||||
- POST /api/import/assets/{assetId}/review
|
||||
- POST /api/import/assets/{assetId}/parse-preview
|
||||
- POST /api/import/assets/{assetId}/start-import
|
||||
- GET /api/import/sessions/{sessionId}
|
||||
**Public / user**
|
||||
|
||||
## Simple EPUB Import Flow (Review-first)
|
||||
- `GET /api/health`
|
||||
- `GET /api/genres`, `GET /api/genres/{slug}`
|
||||
- `GET /api/novels/browse`, `GET /api/novels/{idOrSlug}`
|
||||
- `GET /api/truyen` (query `slug`), `GET /api/truyen/{novel_id}/chapters`, `GET /api/truyen/{novel_id}/chapters/by-number/{n}`
|
||||
- `GET /api/chapters/{chapter_id}`
|
||||
- `GET /api/truyen/suggest`
|
||||
- `GET/POST /api/truyen/{novel_id}/comments`, `POST /api/truyen/{novel_id}/rate`
|
||||
|
||||
MOD/ADMIN flow on new import wizard:
|
||||
**Auth**
|
||||
|
||||
1. Search source EPUB by name (DB index): `GET /api/import/assets/search`
|
||||
2. Review/edit metadata: `GET /api/import/assets/{id}/preview-metadata` + `POST /api/import/assets/{id}/review`
|
||||
3. Preview chapter split (TOC or regex-start): `POST /api/import/assets/{id}/parse-preview`
|
||||
4. Start import and poll progress:
|
||||
- `POST /api/import/assets/{id}/start-import`
|
||||
- `GET /api/import/sessions/{sessionId}`
|
||||
- `POST /api/auth/mobile-login` (JWT cho mobile)
|
||||
- `GET /api/auth/session` (session bridge cho web — xem handler trong `main.py`)
|
||||
|
||||
AI assist in step 2:
|
||||
- `POST /api/import/assets/{id}/ai-suggest`
|
||||
- Returns up to 6 genres + 1 short description.
|
||||
- New genres are allowed and created immediately on apply.
|
||||
**User (login)**
|
||||
|
||||
- `GET /api/user/profile`
|
||||
- `GET/POST /api/user/bookmarks`, `DELETE /api/user/bookmarks/{novel_id}`
|
||||
- `POST /api/user/reading-progress`
|
||||
- `GET/POST /api/user/settings`
|
||||
- `GET/POST/DELETE /api/user/recommendations`
|
||||
|
||||
**MOD / ADMIN** — prefix `/api/mod/*` (thể loại, truyện, chương, overview, đề cử, upload bìa, EPUB…). Liệt kê đầy đủ trong `app/main.py`.
|
||||
|
||||
**Import (MOD — web)**
|
||||
|
||||
- `POST /api/import/uploads/preview` — upload EPUB multipart để lấy preview metadata/cover gợi ý.
|
||||
- `POST /api/mod/epub`, `POST /api/mod/epub/ai-suggest` — luồng import EPUB chính.
|
||||
- `GET/POST/PUT/DELETE /api/mod/the-loai` — quản lý thể loại trong wizard.
|
||||
|
||||
Luồng SourceAsset / `/api/import/assets/*` và job pipeline cũ đã **gỡ khỏi codebase** (không còn endpoint).
|
||||
|
||||
## NAS Migration Ops
|
||||
|
||||
@@ -203,12 +196,6 @@ Checkpoint/resume mode:
|
||||
python scripts/backfill_chapter_content_refs.py --limit 1000 --state-file .backfill_state.json
|
||||
```
|
||||
|
||||
Or continue from a known ObjectId:
|
||||
|
||||
```bash
|
||||
python scripts/backfill_chapter_content_refs.py --limit 1000 --after-id 680f7f3a2f0d53f4f2b7a123
|
||||
```
|
||||
|
||||
## Chapter Read Cutover Flag
|
||||
|
||||
Set in `.env`:
|
||||
|
||||
@@ -1,30 +0,0 @@
|
||||
# Rollout Checklist - NAS Chapter Storage
|
||||
|
||||
## Pre-Deploy
|
||||
- [ ] Backup PostgreSQL schema + critical tables
|
||||
- [ ] Verify NAS mount/access permissions in API runtime
|
||||
- [ ] Enable feature flags (default: Mongo fallback on)
|
||||
|
||||
## Deploy Order
|
||||
1. Deploy DB migrations
|
||||
2. Deploy API with dual-read disabled by default
|
||||
3. Enable discover/approve/convert job APIs
|
||||
4. Run pilot import set (small curated EPUB batch)
|
||||
5. Enable NAS-first for pilot users/env
|
||||
6. Gradually ramp NAS-first traffic
|
||||
|
||||
## Runtime Verification
|
||||
- [ ] `/api/health` stable
|
||||
- [ ] Chapter read success rate >= target
|
||||
- [ ] NAS read timeout/error rate below threshold
|
||||
- [ ] Mongo fallback rate trending down
|
||||
|
||||
## Rollback
|
||||
- [ ] Switch feature flag to Mongo-first immediately
|
||||
- [ ] Stop import jobs
|
||||
- [ ] Keep imported refs for investigation (no destructive cleanup)
|
||||
|
||||
## Post-Deploy
|
||||
- [ ] Compare chapter counts and random content samples
|
||||
- [ ] Review failed/review_required import queue
|
||||
- [ ] Publish release notes for web/mobile teams
|
||||
+115
-1811
File diff suppressed because it is too large
Load Diff
@@ -161,7 +161,7 @@ model Comment {
|
||||
content String @db.Text
|
||||
userId String
|
||||
novelId String
|
||||
chapterId String? // Có thể bình luận riêng tư cho từng chương (Lưu chapterId từ MongoDB)
|
||||
chapterId String? // Bình luận theo chương (id chương từ backend / ChapterMeta)
|
||||
|
||||
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
|
||||
novel Novel @relation(fields: [novelId], references: [id], onDelete: Cascade)
|
||||
|
||||
@@ -1,60 +0,0 @@
|
||||
const { PrismaClient } = require('@prisma/client')
|
||||
const mongoose = require('mongoose')
|
||||
require('dotenv').config({ path: '.env.local' })
|
||||
require('dotenv').config()
|
||||
|
||||
const prisma = new PrismaClient()
|
||||
|
||||
async function main() {
|
||||
console.log('Connecting to MongoDB...')
|
||||
// Connect to MongoDB using MONGODB_URI
|
||||
const mongoUri = process.env.MONGODB_URI
|
||||
if (!mongoUri) {
|
||||
throw new Error('MONGODB_URI is not defined in env')
|
||||
}
|
||||
await mongoose.connect(mongoUri)
|
||||
|
||||
// Wipe MongoDB Chapters
|
||||
console.log('Wiping chapters from MongoDB...')
|
||||
try {
|
||||
const chapterSchema = new mongoose.Schema({}, { strict: false })
|
||||
const Chapter = mongoose.models.Chapter || mongoose.model('Chapter', chapterSchema, 'chapters')
|
||||
const res = await Chapter.deleteMany({})
|
||||
console.log(`Deleted ${res.deletedCount} chapters.`)
|
||||
} catch (e) {
|
||||
console.error('Error wiping mongo chapters', e)
|
||||
}
|
||||
|
||||
// Wipe PostgreSQL Content
|
||||
console.log('Wiping Novels, Genres, Comments, Bookmarks from PostgreSQL...')
|
||||
try {
|
||||
// Delete in order to respect foreign keys if Cascade isn't perfect, but Cascade is set on most.
|
||||
await prisma.comment.deleteMany({})
|
||||
console.log('Deleted all comments.')
|
||||
|
||||
await prisma.bookmark.deleteMany({})
|
||||
console.log('Deleted all bookmarks.')
|
||||
|
||||
await prisma.novelGenre.deleteMany({})
|
||||
console.log('Deleted all novel_genres.')
|
||||
|
||||
await prisma.genre.deleteMany({})
|
||||
console.log('Deleted all genres.')
|
||||
|
||||
await prisma.novel.deleteMany({})
|
||||
console.log('Deleted all novels.')
|
||||
|
||||
} catch (error) {
|
||||
console.error('Error wiping postgres', error)
|
||||
}
|
||||
|
||||
console.log('Cleanup complete.')
|
||||
}
|
||||
|
||||
main()
|
||||
.catch(console.error)
|
||||
.finally(async () => {
|
||||
await prisma.$disconnect()
|
||||
await mongoose.disconnect()
|
||||
process.exit(0)
|
||||
})
|
||||
Reference in New Issue
Block a user