chore: update architecture and documentation to reflect removal of MongoDB and transition to NAS storage for chapter content
Build and Push Reader API Image / docker (push) Successful in 1m3s
Build and Push Reader API Image / docker (push) Successful in 1m3s
- Updated `.env.example` to remove MongoDB configuration. - Revised `ARCHITECTURE.md` to reflect changes in data orchestration and storage strategy. - Removed `CHAPTER_SAVE_DEBUG.md` and `FIXES_APPLIED.md` as they are no longer relevant. - Updated `CONTRACT.md` to include new error codes. - Adjusted endpoint documentation in `CROSS_REPO_ENDPOINT_MATRIX.md` and `README.md` to align with the new import flow. - Removed legacy import scripts and tables from the codebase.
This commit is contained in:
+1
-2
@@ -1,6 +1,5 @@
|
|||||||
# PostgreSQL + MongoDB
|
# PostgreSQL
|
||||||
DATABASE_URL=postgresql://reader:reader@localhost:5432/reader
|
DATABASE_URL=postgresql://reader:reader@localhost:5432/reader
|
||||||
MONGODB_URI=mongodb://localhost:27017/reader
|
|
||||||
|
|
||||||
# Auth / OAuth
|
# Auth / OAuth
|
||||||
NEXTAUTH_SECRET=replace-with-strong-secret
|
NEXTAUTH_SECRET=replace-with-strong-secret
|
||||||
|
|||||||
+8
-5
@@ -8,7 +8,7 @@ Tai lieu nay mo ta `reader-api` la backend dung chung cho Web (`reader`) va Andr
|
|||||||
- API contract
|
- API contract
|
||||||
- domain rule
|
- domain rule
|
||||||
- auth mapping web/mobile
|
- auth mapping web/mobile
|
||||||
- data orchestration PostgreSQL + MongoDB
|
- data orchestration: PostgreSQL + file/NAS chapter storage (khong con MongoDB)
|
||||||
- Moi thay doi contract phai uu tien backward-compatible cho 2 client.
|
- Moi thay doi contract phai uu tien backward-compatible cho 2 client.
|
||||||
|
|
||||||
## Domain ownership
|
## Domain ownership
|
||||||
@@ -20,10 +20,9 @@ Tai lieu nay mo ta `reader-api` la backend dung chung cho Web (`reader`) va Andr
|
|||||||
## Data strategy
|
## Data strategy
|
||||||
|
|
||||||
- PostgreSQL:
|
- PostgreSQL:
|
||||||
- user, novel metadata, genres, comments, ratings, bookmarks, progress.
|
- user, novel metadata, genres, comments, ratings, bookmarks, progress, chapter metadata (`ChapterMeta`), refs noi dung (`ChapterContentRef`).
|
||||||
- MongoDB:
|
- Chapter body:
|
||||||
- chapter text content lon.
|
- file/NAS (qua `app.storage`) — txt/html theo href trong `ChapterContentRef`.
|
||||||
- recommendation document payload (neu can rich document).
|
|
||||||
|
|
||||||
## Auth and identity
|
## Auth and identity
|
||||||
|
|
||||||
@@ -44,3 +43,7 @@ Tai lieu nay mo ta `reader-api` la backend dung chung cho Web (`reader`) va Andr
|
|||||||
- Da verify voi web + mobile happy path va auth edge cases.
|
- Da verify voi web + mobile happy path va auth edge cases.
|
||||||
- Healthcheck va monitoring khong bi anh huong.
|
- Healthcheck va monitoring khong bi anh huong.
|
||||||
- Docker/local dev van chay voi huong dan README.
|
- Docker/local dev van chay voi huong dan README.
|
||||||
|
|
||||||
|
## Import (MOD)
|
||||||
|
|
||||||
|
Chi con endpoint import preview `POST /api/import/uploads/preview` va nhom `/api/mod/epub*`. Cac endpoint SourceAsset/job legacy da go khoi service.
|
||||||
|
|||||||
@@ -1,256 +0,0 @@
|
|||||||
# 🐛 Chapter Save Debugging Guide
|
|
||||||
|
|
||||||
## Vấn Đề Đã Fix
|
|
||||||
|
|
||||||
### ✅ Fix 1: Ownership Check (Line 932)
|
|
||||||
- **Vấn đề:** MOD không thể tạo chapter cho truyện mặc định (uploaderId = NULL)
|
|
||||||
- **Fix:** Thêm `OR uploaderId IS NULL` vào WHERE clause
|
|
||||||
- **Dòng:** 932, 1002
|
|
||||||
|
|
||||||
### ✅ Fix 2: Input Validation
|
|
||||||
- **Vấn đề:** Content có thể trống, number có thể âm
|
|
||||||
- **Fix:** Thêm validation trước insert
|
|
||||||
- **Dòng:** 920-927, 997-1004
|
|
||||||
|
|
||||||
### ✅ Fix 3: Data Consistency Logging
|
|
||||||
- **Vấn đề:** Nếu MongoDB insert succeed nhưng PostgreSQL fail → dữ liệu inconsistent
|
|
||||||
- **Fix:** Thêm separate error handling và logging [CRITICAL]
|
|
||||||
- **Dòng:** 956-974
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🔍 Testing Checklist
|
|
||||||
|
|
||||||
### A. Network Debugging (Browser DevTools)
|
|
||||||
|
|
||||||
1. **Mở DevTools:** F12 → Network tab
|
|
||||||
2. **Ấn "Lưu chương"**
|
|
||||||
3. **Kiểm tra request `POST /api/mod/chuong`:**
|
|
||||||
- ✅ Status code: `201` = success, `4xx`/`5xx` = error
|
|
||||||
- ✅ Response body: Phần lấy `id`, `number`, `title`
|
|
||||||
- ❌ Status 403: Ownership issue
|
|
||||||
- ❌ Status 400: Duplicate or validation error
|
|
||||||
- ❌ Status 500: Server error (check [CRITICAL] logs)
|
|
||||||
|
|
||||||
### B. MongoDB Verification
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Access MongoDB
|
|
||||||
mongosh # hoặc mongo
|
|
||||||
|
|
||||||
# Switch to database
|
|
||||||
use reader_db # (thay bằng tên DB thực tế)
|
|
||||||
|
|
||||||
# List recent chapters
|
|
||||||
db.chapters.find({}, {novelId: 1, number: 1, title: 1, createdAt: 1})
|
|
||||||
.sort({_id: -1})
|
|
||||||
.limit(5)
|
|
||||||
|
|
||||||
# Check specific novel
|
|
||||||
db.chapters.countDocuments({novelId: "YOUR_NOVEL_ID"})
|
|
||||||
|
|
||||||
# Check for duplicates (race condition)
|
|
||||||
db.chapters.find({novelId: "YOUR_NOVEL_ID", number: 1})
|
|
||||||
```
|
|
||||||
|
|
||||||
### C. PostgreSQL Verification
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Access PostgreSQL
|
|
||||||
psql # hoặc your database client
|
|
||||||
|
|
||||||
# Check novel total chapters count
|
|
||||||
SELECT id, title, "totalChapters" FROM "Novel" WHERE id = 'YOUR_NOVEL_ID';
|
|
||||||
|
|
||||||
# Verify it matches MongoDB count
|
|
||||||
-- MongoDB should have same count as "totalChapters"
|
|
||||||
```
|
|
||||||
|
|
||||||
### D. Server Log Analysis
|
|
||||||
|
|
||||||
Look for these patterns in backend logs:
|
|
||||||
|
|
||||||
```
|
|
||||||
✅ Success:
|
|
||||||
[timestamp] POST /mod/chuong - Status 201
|
|
||||||
[timestamp] Inserted chapter id: xxx
|
|
||||||
|
|
||||||
❌ Issues:
|
|
||||||
[CRITICAL] ⚠️ INCONSISTENT STATE: Chapter inserted in MongoDB...
|
|
||||||
[timestamp] Lỗi MongoDB: [error message]
|
|
||||||
[timestamp] Ownership check failed: 403
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🚀 Common Scenarios & Solutions
|
|
||||||
|
|
||||||
### Scenario 1: Network Shows 201 But Chapter Not Visible
|
|
||||||
|
|
||||||
**Cause:** Chapter saved but not refreshed in UI
|
|
||||||
**Solution:**
|
|
||||||
- Press F5 to refresh page
|
|
||||||
- Check MongoDB to confirm chapter exists
|
|
||||||
- Check if `fetchChapters()` was called after save
|
|
||||||
|
|
||||||
### Scenario 2: Network Shows 403 Forbidden
|
|
||||||
|
|
||||||
**Cause:** Novel ownership check failed
|
|
||||||
**Solution:**
|
|
||||||
- Verify you are MOD or ADMIN user
|
|
||||||
- Verify novel exists in PostgreSQL:
|
|
||||||
```sql
|
|
||||||
SELECT id, title, "uploaderId" FROM "Novel" WHERE id = 'YOUR_ID';
|
|
||||||
```
|
|
||||||
- If uploaderId is NULL (default), ensure you're MOD user
|
|
||||||
|
|
||||||
### Scenario 3: Network Shows 400 Bad Request
|
|
||||||
|
|
||||||
**Causes:**
|
|
||||||
- Chapter number already exists
|
|
||||||
- Title or content empty
|
|
||||||
- Chapter number ≤ 0
|
|
||||||
|
|
||||||
**Solution:** Check response detail message and fix input
|
|
||||||
|
|
||||||
### Scenario 4: Network Shows 500 Server Error
|
|
||||||
|
|
||||||
**Cause:** MongoDB or PostgreSQL failure
|
|
||||||
**Solution:**
|
|
||||||
- Check server logs for [CRITICAL] message
|
|
||||||
- If MongoDB failed: Check MongoDB connection
|
|
||||||
- If PostgreSQL failed: Check PostgreSQL connection
|
|
||||||
- Contact admin with error message
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🔧 Advanced Debug Commands
|
|
||||||
|
|
||||||
### Check MongoDB Connection Status
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# From backend terminal
|
|
||||||
python3 -c "
|
|
||||||
import asyncio
|
|
||||||
from app.database import mongo_db
|
|
||||||
|
|
||||||
async def check():
|
|
||||||
await mongo_db.command('ping')
|
|
||||||
print('✓ MongoDB Connected!')
|
|
||||||
|
|
||||||
asyncio.run(check())
|
|
||||||
"
|
|
||||||
```
|
|
||||||
|
|
||||||
### Check PostgreSQL Connection Status
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# From backend terminal
|
|
||||||
python3 -c "
|
|
||||||
import asyncio
|
|
||||||
from sqlalchemy.ext.asyncio import create_async_engine
|
|
||||||
from app.database import SessionLocal
|
|
||||||
|
|
||||||
async def check():
|
|
||||||
async with SessionLocal() as session:
|
|
||||||
result = await session.execute('SELECT 1')
|
|
||||||
print('✓ PostgreSQL Connected!')
|
|
||||||
|
|
||||||
asyncio.run(check())
|
|
||||||
"
|
|
||||||
```
|
|
||||||
|
|
||||||
### Manually Sync Total Chapters
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# If totalChapters is out of sync
|
|
||||||
mongosh
|
|
||||||
use reader_db
|
|
||||||
|
|
||||||
# Get count
|
|
||||||
db.chapters.countDocuments({novelId: "YOUR_NOVEL_ID"})
|
|
||||||
|
|
||||||
# Then update PostgreSQL manually:
|
|
||||||
# psql
|
|
||||||
UPDATE "Novel" SET "totalChapters" = 123 WHERE id = 'YOUR_NOVEL_ID';
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📋 Test Cases
|
|
||||||
|
|
||||||
### Test 1: Basic Chapter Save
|
|
||||||
```
|
|
||||||
1. Create novel
|
|
||||||
2. Save chapter #1
|
|
||||||
3. ✅ Should appear in chapter list
|
|
||||||
4. ✅ totalChapters should be 1
|
|
||||||
```
|
|
||||||
|
|
||||||
### Test 2: Sequential Chapters
|
|
||||||
```
|
|
||||||
1. Save chapters 1, 2, 3
|
|
||||||
2. ✅ All should appear with correct numbers
|
|
||||||
3. ✅ Next chapter field should suggest 4
|
|
||||||
```
|
|
||||||
|
|
||||||
### Test 3: Duplicate Prevention
|
|
||||||
```
|
|
||||||
1. Save chapter #5
|
|
||||||
2. Try to save chapter #5 again
|
|
||||||
3. ✅ Should show "Chương 5 đã tồn tại"
|
|
||||||
```
|
|
||||||
|
|
||||||
### Test 4: Default Novel (MOD Permission)
|
|
||||||
```
|
|
||||||
1. Verify a novel with uploaderId = NULL exists
|
|
||||||
2. As MOD user, save chapter to that novel
|
|
||||||
3. ✅ Should succeed (not 403 Forbidden)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Test 5: No Empty Content
|
|
||||||
```
|
|
||||||
1. Try to save chapter with empty title
|
|
||||||
2. ✅ Should show "Tiêu đề chương không được trống"
|
|
||||||
3. Try to save chapter with empty content
|
|
||||||
4. ✅ Should show "Nội dung chương không được trống"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🆘 Still Having Issues?
|
|
||||||
|
|
||||||
1. **Run checklist A, B, C above** and collect outputs
|
|
||||||
2. **Screenshot of Network tab response**
|
|
||||||
3. **MongoDB output from `find()`**
|
|
||||||
4. **Server log output (especially [CRITICAL] lines)**
|
|
||||||
5. Share these with: [your-dev-contact]
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📊 Monitoring
|
|
||||||
|
|
||||||
### Health Check Script
|
|
||||||
|
|
||||||
```bash
|
|
||||||
#!/bin/bash
|
|
||||||
# save as monitor-save.sh
|
|
||||||
|
|
||||||
echo "=== Chapter Save System Health Check ==="
|
|
||||||
echo ""
|
|
||||||
echo "1. MongoDB Connection:"
|
|
||||||
# mongosh check here
|
|
||||||
|
|
||||||
echo ""
|
|
||||||
echo "2. PostgreSQL Connection:"
|
|
||||||
# psql check here
|
|
||||||
|
|
||||||
echo ""
|
|
||||||
echo "3. Backend API:"
|
|
||||||
curl -s http://localhost:8000/docs | head -20
|
|
||||||
|
|
||||||
echo ""
|
|
||||||
echo "=== Done ==="
|
|
||||||
```
|
|
||||||
|
|
||||||
Run: `bash monitor-save.sh`
|
|
||||||
@@ -32,6 +32,7 @@ Tai lieu contract chung cho `reader`, `reader-app`, `reader-api`.
|
|||||||
- `409`: xung dot du lieu.
|
- `409`: xung dot du lieu.
|
||||||
- `422`: payload format dung JSON nhung khong dat rule nghiep vu.
|
- `422`: payload format dung JSON nhung khong dat rule nghiep vu.
|
||||||
- `500`: loi he thong.
|
- `500`: loi he thong.
|
||||||
|
- `410`: (du tru) tai nguyen da go bo hoac khong con ho tro.
|
||||||
|
|
||||||
## Pagination Convention
|
## Pagination Convention
|
||||||
|
|
||||||
|
|||||||
@@ -26,17 +26,12 @@ Legend:
|
|||||||
| Comment | `GET/POST /api/truyen/{id}/comments` | Y | Y | Y | |
|
| Comment | `GET/POST /api/truyen/{id}/comments` | Y | Y | Y | |
|
||||||
| Rating | `POST /api/truyen/{id}/rate` | Y | Y | N | Mobile chua thay rating flow |
|
| Rating | `POST /api/truyen/{id}/rate` | Y | Y | N | Mobile chua thay rating flow |
|
||||||
| Search | `GET /api/truyen/suggest` | Y | Y | N | Mobile search suggest can bo sung |
|
| Search | `GET /api/truyen/suggest` | Y | Y | N | Mobile search suggest can bo sung |
|
||||||
| Import | `GET /api/import/assets/search` | Y | Y | N | Web MOD import wizard step 1 |
|
| Import | `POST /api/import/uploads/preview` | Y | Y | N | Upload EPUB multipart (preview) |
|
||||||
| Import | `GET /api/import/assets/{id}/preview-metadata` | Y | Y | N | Web MOD import wizard step 2 |
|
| Import | `POST /api/mod/epub`, `POST /api/mod/epub/ai-suggest` | Y | Y | N | Luong `/mod/import` |
|
||||||
| Import | `POST /api/import/assets/{id}/ai-suggest` | Y | Y | N | Gen toi da 6 genres + short description |
|
| Import | `GET/POST/PUT/DELETE /api/mod/the-loai` | Y | Y | N | MOD quan ly the loai trong wizard |
|
||||||
| Import | `POST /api/import/assets/{id}/review` | Y | Y | N | Save reviewed metadata before import |
|
|
||||||
| Import | `POST /api/import/assets/{id}/parse-preview` | Y | Y | N | TOC/regex-start preview (10 head/mid/tail samples) |
|
|
||||||
| Import | `POST /api/import/assets/{id}/start-import` | Y | Y | N | Start import session |
|
|
||||||
| Import | `GET /api/import/sessions/{sessionId}` | Y | Y | N | Poll import progress |
|
|
||||||
|
|
||||||
## Priority gaps de dong bo tiep
|
## Priority gaps de dong bo tiep
|
||||||
|
|
||||||
1. Mobile: `user/settings`, `recommendations`, `rate`, `suggest`.
|
1. Mobile: `user/settings`, `recommendations`, `rate`, `suggest`.
|
||||||
2. Web/Mobile chapter-read strategy can unify (`chapters/{id}` vs `by-number`).
|
2. Web/Mobile chapter-read strategy can unify (`chapters/{id}` vs `by-number`).
|
||||||
3. Chuan hoa error contract implementation theo `CONTRACT.md`.
|
3. Chuan hoa error contract implementation theo `CONTRACT.md`.
|
||||||
4. Mobile import flow currently not planned (MOD-only on web).
|
|
||||||
|
|||||||
@@ -1,186 +0,0 @@
|
|||||||
# ✅ Chapter Save System - Fixed Issues Summary
|
|
||||||
|
|
||||||
**Date:** 2026-03-24
|
|
||||||
**Status:** All HIGH priority issues fixed ✅
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🐛 Issues Identified & Fixed
|
|
||||||
|
|
||||||
### 1️⃣ **Ownership Check Bypass (HIGH)**
|
|
||||||
- **File:** `app/routers/mod.py`
|
|
||||||
- **Lines:** 932-936 (POST), 1000-1007 (PUT)
|
|
||||||
- **Issue:** MOD couldn't create chapters for default novels (uploaderId = NULL)
|
|
||||||
- **Fix:** Changed query from:
|
|
||||||
```python
|
|
||||||
WHERE id = :nid AND "uploaderId" = :uid
|
|
||||||
```
|
|
||||||
To:
|
|
||||||
```python
|
|
||||||
WHERE id = :nid AND ("uploaderId" = :uid OR "uploaderId" IS NULL)
|
|
||||||
```
|
|
||||||
- **Impact:** ✅ Fixed - MOD can now manage default novels
|
|
||||||
|
|
||||||
### 2️⃣ **Missing Input Validation (HIGH)**
|
|
||||||
- **File:** `app/routers/mod.py`
|
|
||||||
- **Lines:** 920-927 (POST), 997-1004 (PUT)
|
|
||||||
- **Issue:** Could save empty title/content, negative chapter numbers
|
|
||||||
- **Fix:** Added validation:
|
|
||||||
```python
|
|
||||||
if not body.title or not body.title.strip():
|
|
||||||
raise HTTPException(400, "Tiêu đề chương không được trống")
|
|
||||||
if body.number <= 0:
|
|
||||||
raise HTTPException(400, "Số chương phải > 0")
|
|
||||||
```
|
|
||||||
- **Impact:** ✅ Fixed - Invalid data rejected at API level
|
|
||||||
|
|
||||||
### 3️⃣ **Data Inconsistency on PostgreSQL Failure (HIGH)**
|
|
||||||
- **File:** `app/routers/mod.py`
|
|
||||||
- **Lines:** 956-974 (POST)
|
|
||||||
- **Issue:** If MongoDB insert succeeds but PostgreSQL sync fails → inconsistent state
|
|
||||||
- **Fix:** Added separate error handling:
|
|
||||||
```python
|
|
||||||
try:
|
|
||||||
result = await mongo_db.chapters.insert_one(doc)
|
|
||||||
except Exception as mongo_err:
|
|
||||||
raise HTTPException(500, f"Lỗi MongoDB: {str(mongo_err)}")
|
|
||||||
|
|
||||||
try:
|
|
||||||
total = await _sync_total_chapters(db, body.novelId)
|
|
||||||
except Exception as pg_err:
|
|
||||||
# Log [CRITICAL] and alert user
|
|
||||||
raise HTTPException(500, "Dữ liệu has được lưu nhưng...")
|
|
||||||
```
|
|
||||||
- **Impact:** ✅ Fixed - Clear error messages identify MongoDB vs PostgreSQL failures
|
|
||||||
|
|
||||||
### 4️⃣ **Frontend Error Display (MEDIUM)**
|
|
||||||
- **File:** `reader/app/mod/chuong/chapter-client.tsx`
|
|
||||||
- **Line:** 356
|
|
||||||
- **Issue:** Only checked `error` field, not FastAPI's `detail` field
|
|
||||||
- **Fix:** Changed:
|
|
||||||
```javascript
|
|
||||||
if (!res.ok) throw new Error(resData.error || resData.detail || "...")
|
|
||||||
```
|
|
||||||
- **Impact:** ✅ Fixed - Users see actual backend error messages
|
|
||||||
|
|
||||||
### 5️⃣ **Missing Novel Existence Check (MEDIUM)**
|
|
||||||
- **File:** `app/routers/mod.py`
|
|
||||||
- **Lines:** 948-951 (POST)
|
|
||||||
- **Issue:** Could try to save chapter for non-existent novel
|
|
||||||
- **Fix:** Added explicit check:
|
|
||||||
```python
|
|
||||||
novel_check = await db.execute(
|
|
||||||
text('SELECT id FROM "Novel" WHERE id = :nid'),
|
|
||||||
{"nid": body.novelId},
|
|
||||||
)
|
|
||||||
if not novel_check.first():
|
|
||||||
raise HTTPException(404, "Truyện không tồn tại")
|
|
||||||
```
|
|
||||||
- **Impact:** ✅ Fixed - Better error message (404 instead of 500)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📊 Testing Results
|
|
||||||
|
|
||||||
### Build Verification
|
|
||||||
```
|
|
||||||
✅ Python syntax: OK (py_compile passed)
|
|
||||||
✅ Next.js build: OK (all 11 routes successfully compiled)
|
|
||||||
✅ No type errors in modified files
|
|
||||||
```
|
|
||||||
|
|
||||||
### Modified Files
|
|
||||||
| File | Changes | Status |
|
|
||||||
|------|---------|--------|
|
|
||||||
| `app/routers/mod.py` | POST/PUT endpoints refactored with validation | ✅ Fixed |
|
|
||||||
| `reader/app/mod/chuong/chapter-client.tsx` | Error handling improved | ✅ Fixed |
|
|
||||||
| `CHAPTER_SAVE_DEBUG.md` | Debug guide created | ✅ New |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🚀 Next Steps for User
|
|
||||||
|
|
||||||
### ⚠️ IMPORTANT: Test the following scenarios
|
|
||||||
|
|
||||||
1. **Basic Save:** Create new novel → save Chapter 1 → verify appears in list
|
|
||||||
2. **Ownership:** Try saving to default novel as MOD user → should succeed now
|
|
||||||
3. **Duplicate:** Try saving same chapter twice → should show "đã tồn tại"
|
|
||||||
4. **Empty Content:** Try saving without title → should show validation error
|
|
||||||
5. **Negative Number:** Try chapter #-1 → should reject
|
|
||||||
|
|
||||||
### If Still Failing:
|
|
||||||
|
|
||||||
1. Open **DevTools Network tab** → F12 → Network
|
|
||||||
2. Try to save chapter
|
|
||||||
3. Look for **POST /api/mod/chuong** request
|
|
||||||
4. Check **Status code** and **Response body**
|
|
||||||
5. Use guide in `CHAPTER_SAVE_DEBUG.md` to troubleshoot
|
|
||||||
|
|
||||||
### Possible Remaining Issues
|
|
||||||
|
|
||||||
⚠️ **Not yet fixed (MEDIUM/LOW priority):**
|
|
||||||
- [ ] Race condition on duplicate chapter check (add MongoDB unique index)
|
|
||||||
- [ ] No MongoDB/PostgreSQL timeout configuration
|
|
||||||
- [ ] Generic exception handler logging (uses traceback.print_exc)
|
|
||||||
- [ ] Missing structured logging system
|
|
||||||
|
|
||||||
These are less critical but could cause issues under high load.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📝 Code Changes Summary
|
|
||||||
|
|
||||||
**Total lines modified:** ~150
|
|
||||||
**Files affected:** 2
|
|
||||||
**Breaking changes:** None (backward compatible)
|
|
||||||
**Rollback difficulty:** Low (simple validation additions)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## ✨ What Changed
|
|
||||||
|
|
||||||
```diff
|
|
||||||
# POST /mod/chuong
|
|
||||||
- Missing input validation
|
|
||||||
- Missing novel existence check
|
|
||||||
- Ownership query doesn't allow NULL uploaderId
|
|
||||||
- No separation of MongoDB vs PostgreSQL error handling
|
|
||||||
|
|
||||||
+ Full input validation (title, content, number)
|
|
||||||
+ Novel existence check with clear 404 error
|
|
||||||
+ Ownership check allows both user-owned and default novels
|
|
||||||
+ Separate error handling for MongoDB and PostgreSQL
|
|
||||||
+ Better error messages for debugging
|
|
||||||
+ Data consistency logging ([CRITICAL] alerts)
|
|
||||||
|
|
||||||
# PUT /mod/chuong
|
|
||||||
- Same issues as POST
|
|
||||||
|
|
||||||
+ Same fixes applied
|
|
||||||
|
|
||||||
# Frontend error handling
|
|
||||||
- Ignored FastAPI's 'detail' field
|
|
||||||
|
|
||||||
+ Now checks both 'error' and 'detail' fields
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🔍 Monitoring Recommendations
|
|
||||||
|
|
||||||
1. **Set up log monitoring** for `[CRITICAL]` messages
|
|
||||||
2. **Verify MongoDB connection** on startup
|
|
||||||
3. **Verify PostgreSQL connection** on startup
|
|
||||||
4. **Add request logging** to track save operations
|
|
||||||
5. **Monitor totalChapters sync** for discrepancies
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📞 Support
|
|
||||||
|
|
||||||
If issues persist after testing:
|
|
||||||
1. Follow debugging guide in `CHAPTER_SAVE_DEBUG.md`
|
|
||||||
2. Check Network tab for response codes
|
|
||||||
3. Verify MongoDB and PostgreSQL connectivity
|
|
||||||
4. Look for [CRITICAL] messages in server logs
|
|
||||||
5. Check browser console for JavaScript errors
|
|
||||||
@@ -39,20 +39,13 @@ Backend flow theo domain, de web/mobile follow giong nhau.
|
|||||||
|
|
||||||
## Flow E: EPUB Import (MOD/ADMIN)
|
## Flow E: EPUB Import (MOD/ADMIN)
|
||||||
|
|
||||||
- Step 1 search source:
|
Luồng trên web (`reader` `/mod/import`):
|
||||||
- `/api/import/assets/search`
|
|
||||||
- Step 2 review metadata:
|
- Thể loại: `GET/POST/PUT/DELETE /api/mod/the-loai`
|
||||||
- `/api/import/assets/{id}/preview-metadata`
|
- Upload EPUB preview: `POST /api/import/uploads/preview` (multipart)
|
||||||
- `/api/import/assets/{id}/ai-suggest`
|
- Gợi ý metadata: `POST /api/mod/epub/ai-suggest`
|
||||||
- `/api/import/assets/{id}/review`
|
- Import EPUB: `POST /api/mod/epub`
|
||||||
- Step 3 chapter split preview:
|
|
||||||
- `/api/import/assets/{id}/parse-preview`
|
|
||||||
- split mode: `toc` or `regex` (chapter-start pattern only)
|
|
||||||
- Step 4 start import + progress:
|
|
||||||
- `/api/import/assets/{id}/start-import`
|
|
||||||
- `/api/import/sessions/{sessionId}`
|
|
||||||
|
|
||||||
Rules:
|
Rules:
|
||||||
- No filesystem scan in search request path (scan by cron/incremental).
|
|
||||||
- Reviewer confirms metadata before import.
|
- Import ghi nội dung NAS / refs chương và cập nhật metadata novel khi hoàn tất.
|
||||||
- Import writes NAS content + chapter refs, then updates novel counters.
|
|
||||||
|
|||||||
@@ -11,7 +11,7 @@ This project is Python-first (FastAPI), with production-focused Docker setup and
|
|||||||
- Python 3.11+
|
- Python 3.11+
|
||||||
- FastAPI
|
- FastAPI
|
||||||
- UV (package manager / runner)
|
- UV (package manager / runner)
|
||||||
- PostgreSQL (structured data)
|
- PostgreSQL (metadata, user data, chapter refs; chapter text stored via NAS/files — see `NAS_CONTENT_ROOT` / storage layer)
|
||||||
|
|
||||||
## API Base URL
|
## API Base URL
|
||||||
|
|
||||||
@@ -133,47 +133,40 @@ volumes:
|
|||||||
|
|
||||||
For your EPUB structure (folder per novel, multiple `.epub` parts inside), mount the parent folder to `/data/epub-source`.
|
For your EPUB structure (folder per novel, multiple `.epub` parts inside), mount the parent folder to `/data/epub-source`.
|
||||||
|
|
||||||
## Implemented Endpoints
|
## Implemented Endpoints (snapshot)
|
||||||
|
|
||||||
- GET /api/health
|
**Public / user**
|
||||||
- POST /api/auth/mobile-login
|
|
||||||
- GET /api/user/profile
|
|
||||||
- GET/POST /api/user/bookmarks
|
|
||||||
- DELETE /api/user/bookmarks/{novelId}
|
|
||||||
- POST /api/user/reading-progress
|
|
||||||
- GET/POST /api/user/settings
|
|
||||||
- GET/POST/DELETE /api/user/recommendations
|
|
||||||
- GET /api/genres
|
|
||||||
- GET /api/novels/browse
|
|
||||||
- GET /api/novels/{idOrSlug}
|
|
||||||
- GET /api/truyen/{id}/chapters
|
|
||||||
- GET/POST /api/truyen/{id}/comments
|
|
||||||
- POST /api/truyen/{id}/rate
|
|
||||||
- GET /api/truyen/suggest
|
|
||||||
- GET /api/chapters/{chapterId}
|
|
||||||
- GET /api/import/assets/search
|
|
||||||
- GET /api/import/assets/{assetId}/preview-metadata
|
|
||||||
- POST /api/import/assets/{assetId}/ai-suggest
|
|
||||||
- POST /api/import/assets/{assetId}/review
|
|
||||||
- POST /api/import/assets/{assetId}/parse-preview
|
|
||||||
- POST /api/import/assets/{assetId}/start-import
|
|
||||||
- GET /api/import/sessions/{sessionId}
|
|
||||||
|
|
||||||
## Simple EPUB Import Flow (Review-first)
|
- `GET /api/health`
|
||||||
|
- `GET /api/genres`, `GET /api/genres/{slug}`
|
||||||
|
- `GET /api/novels/browse`, `GET /api/novels/{idOrSlug}`
|
||||||
|
- `GET /api/truyen` (query `slug`), `GET /api/truyen/{novel_id}/chapters`, `GET /api/truyen/{novel_id}/chapters/by-number/{n}`
|
||||||
|
- `GET /api/chapters/{chapter_id}`
|
||||||
|
- `GET /api/truyen/suggest`
|
||||||
|
- `GET/POST /api/truyen/{novel_id}/comments`, `POST /api/truyen/{novel_id}/rate`
|
||||||
|
|
||||||
MOD/ADMIN flow on new import wizard:
|
**Auth**
|
||||||
|
|
||||||
1. Search source EPUB by name (DB index): `GET /api/import/assets/search`
|
- `POST /api/auth/mobile-login` (JWT cho mobile)
|
||||||
2. Review/edit metadata: `GET /api/import/assets/{id}/preview-metadata` + `POST /api/import/assets/{id}/review`
|
- `GET /api/auth/session` (session bridge cho web — xem handler trong `main.py`)
|
||||||
3. Preview chapter split (TOC or regex-start): `POST /api/import/assets/{id}/parse-preview`
|
|
||||||
4. Start import and poll progress:
|
|
||||||
- `POST /api/import/assets/{id}/start-import`
|
|
||||||
- `GET /api/import/sessions/{sessionId}`
|
|
||||||
|
|
||||||
AI assist in step 2:
|
**User (login)**
|
||||||
- `POST /api/import/assets/{id}/ai-suggest`
|
|
||||||
- Returns up to 6 genres + 1 short description.
|
- `GET /api/user/profile`
|
||||||
- New genres are allowed and created immediately on apply.
|
- `GET/POST /api/user/bookmarks`, `DELETE /api/user/bookmarks/{novel_id}`
|
||||||
|
- `POST /api/user/reading-progress`
|
||||||
|
- `GET/POST /api/user/settings`
|
||||||
|
- `GET/POST/DELETE /api/user/recommendations`
|
||||||
|
|
||||||
|
**MOD / ADMIN** — prefix `/api/mod/*` (thể loại, truyện, chương, overview, đề cử, upload bìa, EPUB…). Liệt kê đầy đủ trong `app/main.py`.
|
||||||
|
|
||||||
|
**Import (MOD — web)**
|
||||||
|
|
||||||
|
- `POST /api/import/uploads/preview` — upload EPUB multipart để lấy preview metadata/cover gợi ý.
|
||||||
|
- `POST /api/mod/epub`, `POST /api/mod/epub/ai-suggest` — luồng import EPUB chính.
|
||||||
|
- `GET/POST/PUT/DELETE /api/mod/the-loai` — quản lý thể loại trong wizard.
|
||||||
|
|
||||||
|
Luồng SourceAsset / `/api/import/assets/*` và job pipeline cũ đã **gỡ khỏi codebase** (không còn endpoint).
|
||||||
|
|
||||||
## NAS Migration Ops
|
## NAS Migration Ops
|
||||||
|
|
||||||
@@ -203,12 +196,6 @@ Checkpoint/resume mode:
|
|||||||
python scripts/backfill_chapter_content_refs.py --limit 1000 --state-file .backfill_state.json
|
python scripts/backfill_chapter_content_refs.py --limit 1000 --state-file .backfill_state.json
|
||||||
```
|
```
|
||||||
|
|
||||||
Or continue from a known ObjectId:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python scripts/backfill_chapter_content_refs.py --limit 1000 --after-id 680f7f3a2f0d53f4f2b7a123
|
|
||||||
```
|
|
||||||
|
|
||||||
## Chapter Read Cutover Flag
|
## Chapter Read Cutover Flag
|
||||||
|
|
||||||
Set in `.env`:
|
Set in `.env`:
|
||||||
|
|||||||
@@ -1,30 +0,0 @@
|
|||||||
# Rollout Checklist - NAS Chapter Storage
|
|
||||||
|
|
||||||
## Pre-Deploy
|
|
||||||
- [ ] Backup PostgreSQL schema + critical tables
|
|
||||||
- [ ] Verify NAS mount/access permissions in API runtime
|
|
||||||
- [ ] Enable feature flags (default: Mongo fallback on)
|
|
||||||
|
|
||||||
## Deploy Order
|
|
||||||
1. Deploy DB migrations
|
|
||||||
2. Deploy API with dual-read disabled by default
|
|
||||||
3. Enable discover/approve/convert job APIs
|
|
||||||
4. Run pilot import set (small curated EPUB batch)
|
|
||||||
5. Enable NAS-first for pilot users/env
|
|
||||||
6. Gradually ramp NAS-first traffic
|
|
||||||
|
|
||||||
## Runtime Verification
|
|
||||||
- [ ] `/api/health` stable
|
|
||||||
- [ ] Chapter read success rate >= target
|
|
||||||
- [ ] NAS read timeout/error rate below threshold
|
|
||||||
- [ ] Mongo fallback rate trending down
|
|
||||||
|
|
||||||
## Rollback
|
|
||||||
- [ ] Switch feature flag to Mongo-first immediately
|
|
||||||
- [ ] Stop import jobs
|
|
||||||
- [ ] Keep imported refs for investigation (no destructive cleanup)
|
|
||||||
|
|
||||||
## Post-Deploy
|
|
||||||
- [ ] Compare chapter counts and random content samples
|
|
||||||
- [ ] Review failed/review_required import queue
|
|
||||||
- [ ] Publish release notes for web/mobile teams
|
|
||||||
+115
-1811
File diff suppressed because it is too large
Load Diff
@@ -161,7 +161,7 @@ model Comment {
|
|||||||
content String @db.Text
|
content String @db.Text
|
||||||
userId String
|
userId String
|
||||||
novelId String
|
novelId String
|
||||||
chapterId String? // Có thể bình luận riêng tư cho từng chương (Lưu chapterId từ MongoDB)
|
chapterId String? // Bình luận theo chương (id chương từ backend / ChapterMeta)
|
||||||
|
|
||||||
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
|
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
|
||||||
novel Novel @relation(fields: [novelId], references: [id], onDelete: Cascade)
|
novel Novel @relation(fields: [novelId], references: [id], onDelete: Cascade)
|
||||||
|
|||||||
@@ -1,60 +0,0 @@
|
|||||||
const { PrismaClient } = require('@prisma/client')
|
|
||||||
const mongoose = require('mongoose')
|
|
||||||
require('dotenv').config({ path: '.env.local' })
|
|
||||||
require('dotenv').config()
|
|
||||||
|
|
||||||
const prisma = new PrismaClient()
|
|
||||||
|
|
||||||
async function main() {
|
|
||||||
console.log('Connecting to MongoDB...')
|
|
||||||
// Connect to MongoDB using MONGODB_URI
|
|
||||||
const mongoUri = process.env.MONGODB_URI
|
|
||||||
if (!mongoUri) {
|
|
||||||
throw new Error('MONGODB_URI is not defined in env')
|
|
||||||
}
|
|
||||||
await mongoose.connect(mongoUri)
|
|
||||||
|
|
||||||
// Wipe MongoDB Chapters
|
|
||||||
console.log('Wiping chapters from MongoDB...')
|
|
||||||
try {
|
|
||||||
const chapterSchema = new mongoose.Schema({}, { strict: false })
|
|
||||||
const Chapter = mongoose.models.Chapter || mongoose.model('Chapter', chapterSchema, 'chapters')
|
|
||||||
const res = await Chapter.deleteMany({})
|
|
||||||
console.log(`Deleted ${res.deletedCount} chapters.`)
|
|
||||||
} catch (e) {
|
|
||||||
console.error('Error wiping mongo chapters', e)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Wipe PostgreSQL Content
|
|
||||||
console.log('Wiping Novels, Genres, Comments, Bookmarks from PostgreSQL...')
|
|
||||||
try {
|
|
||||||
// Delete in order to respect foreign keys if Cascade isn't perfect, but Cascade is set on most.
|
|
||||||
await prisma.comment.deleteMany({})
|
|
||||||
console.log('Deleted all comments.')
|
|
||||||
|
|
||||||
await prisma.bookmark.deleteMany({})
|
|
||||||
console.log('Deleted all bookmarks.')
|
|
||||||
|
|
||||||
await prisma.novelGenre.deleteMany({})
|
|
||||||
console.log('Deleted all novel_genres.')
|
|
||||||
|
|
||||||
await prisma.genre.deleteMany({})
|
|
||||||
console.log('Deleted all genres.')
|
|
||||||
|
|
||||||
await prisma.novel.deleteMany({})
|
|
||||||
console.log('Deleted all novels.')
|
|
||||||
|
|
||||||
} catch (error) {
|
|
||||||
console.error('Error wiping postgres', error)
|
|
||||||
}
|
|
||||||
|
|
||||||
console.log('Cleanup complete.')
|
|
||||||
}
|
|
||||||
|
|
||||||
main()
|
|
||||||
.catch(console.error)
|
|
||||||
.finally(async () => {
|
|
||||||
await prisma.$disconnect()
|
|
||||||
await mongoose.disconnect()
|
|
||||||
process.exit(0)
|
|
||||||
})
|
|
||||||
Reference in New Issue
Block a user