virtus/reader
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Revamp EPUB import process with batch upload support and enhanced API integration
Build and Push Reader Image / docker (push) Failing after 22s
Details

Browse Source 
- Introduced a new batch import client for handling multiple EPUB files simultaneously.
- Updated API routes for previewing and importing EPUB files, improving error handling and response management.
- Enhanced genre management during import, allowing for dynamic creation and association of genres.
- Implemented long-fetch handling to accommodate lengthy processing times for large EPUB files.
- Refined UI components for better user experience in the import workflow.
This commit is contained in:
virtus
2026-05-11 15:27:13 +07:00
parent 3cc0ea1b9f
commit 669addf799
20 changed files with 605 additions and 112 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ARCHITECTURE.md
+3 -3
View File
@@ -17,14 +17,14 @@ Tai lieu nay mo ta vai tro cua `reader` (web app) trong bo 3 he thong: Web + And
- Mobile: Bearer JWT. - Mobile: Bearer JWT.
- Ca hai deu map vao cung user identity trong backend. - Ca hai deu map vao cung user identity trong backend.
- Data ownership: - Data ownership:
- PostgreSQL: metadata co cau truc (user, novel, genre, comment, bookmark...). - PostgreSQL (web): metadata co cau truc (user, novel, genre, comment, bookmark...).
- MongoDB: chapter content, recommendation payload lon. - Chapter body / file lon: do reader-api orchestrate (NAS storage refs, khong con MongoDB trong stack).
## Kien truc module web ## Kien truc module web
- App layer (`app/*`): route, rendering, page composition. - App layer (`app/*`): route, rendering, page composition.
- UI layer (`components/*`): reusable components, khong chua business rule quan trong. - UI layer (`components/*`): reusable components, khong chua business rule quan trong.
- Data access layer: goi REST API qua `READER_API_ORIGIN` cho endpoint da migrate. - Data access layer: goi REST API qua `READER_API_ORIGIN` (rewrite trong `next.config.mjs` va/hoac proxy trong `app/api/*/route.ts`).
- Auth adapter: dong bo session NextAuth va profile API. - Auth adapter: dong bo session NextAuth va profile API.
## Quy uoc tich hop API ## Quy uoc tich hop API
CONTRACT.md
+1
View File
@@ -36,6 +36,7 @@ Tai lieu contract chung cho `reader`, `reader-app`, `reader-api`.
- `409`: xung dot du lieu. - `409`: xung dot du lieu.
- `422`: payload format dung JSON nhung khong dat rule nghiep vu. - `422`: payload format dung JSON nhung khong dat rule nghiep vu.
- `500`: loi he thong. - `500`: loi he thong.
- `410`: (du tru) tai nguyen da go bo hoac khong con ho tro.
## Pagination Convention ## Pagination Convention
CROSS_REPO_ENDPOINT_MATRIX.md
+3 -8
View File
@@ -26,17 +26,12 @@ Legend:
| Comment | `GET/POST /api/truyen/{id}/comments` | Y | Y | Y | | | Comment | `GET/POST /api/truyen/{id}/comments` | Y | Y | Y | |
| Rating | `POST /api/truyen/{id}/rate` | Y | Y | N | Mobile chua thay rating flow | | Rating | `POST /api/truyen/{id}/rate` | Y | Y | N | Mobile chua thay rating flow |
| Search | `GET /api/truyen/suggest` | Y | Y | N | Mobile search suggest can bo sung | | Search | `GET /api/truyen/suggest` | Y | Y | N | Mobile search suggest can bo sung |
| Import | `GET /api/import/assets/search` | Y | Y | N | Web MOD import wizard step 1 | | Import | `POST /api/import/uploads/preview` | Y | Y | N | Upload EPUB multipart (preview) |
| Import | `GET /api/import/assets/{id}/preview-metadata` | Y | Y | N | Web MOD import wizard step 2 | | Import | `POST /api/mod/epub`, `POST /api/mod/epub/ai-suggest` | Y | Y | N | Luong `/mod/import` |
| Import | `POST /api/import/assets/{id}/ai-suggest` | Y | Y | N | AI metadata suggestion | | Import | `GET/POST/PUT/DELETE /api/mod/the-loai` | Y | Y | N | MOD quan ly the loai trong wizard |
| Import | `POST /api/import/assets/{id}/review` | Y | Y | N | Save reviewed metadata |
| Import | `POST /api/import/assets/{id}/parse-preview` | Y | Y | N | TOC/regex-start preview |
| Import | `POST /api/import/assets/{id}/start-import` | Y | Y | N | Start import session |
| Import | `GET /api/import/sessions/{sessionId}` | Y | Y | N | Poll progress |
## Priority gaps de dong bo tiep ## Priority gaps de dong bo tiep
1. Mobile: `user/settings`, `recommendations`, `rate`, `suggest`. 1. Mobile: `user/settings`, `recommendations`, `rate`, `suggest`.
2. Web/Mobile chapter-read strategy can unify (`chapters/{id}` vs `by-number`). 2. Web/Mobile chapter-read strategy can unify (`chapters/{id}` vs `by-number`).
3. Chuan hoa error contract implementation theo `CONTRACT.md`. 3. Chuan hoa error contract implementation theo `CONTRACT.md`.
4. Mobile import flow currently not planned.
FEATURES.md
+1 -1
View File
@@ -28,7 +28,7 @@ Trang thai tinh nang cho web app `reader`.
| Feature | Status | Notes | | Feature | Status | Notes |
|---|---|---| |---|---|---|
| MOD dashboard/workflows | partial | Mot phan route mod da co, tiep tuc bo sung | | MOD dashboard/workflows | partial | Mot phan route mod da co, tiep tuc bo sung |
| EPUB import wizard (4 step) | done | `/mod/import` + `/api/import/*` review-first flow | | EPUB import wizard | done | `/mod/import`: `/api/mod/the-loai`, `POST /api/import/uploads/preview`, `/api/mod/epub`, `/api/mod/epub/ai-suggest` |
## Dependencies ## Dependencies
FLOWS.md
+6 -6
View File
@@ -42,9 +42,9 @@ Luon xem `reader-api` la canonical behavior.
## Flow 5: MOD EPUB Import Wizard ## Flow 5: MOD EPUB Import Wizard
- Route: `/mod/import` - Route: `/mod/import`
- Steps: - Steps (khớp `app/mod/import/import-client.tsx`):
1. Search source asset by name (`GET /api/import/assets/search`) 1. Chọn thể loại / tạo thể loại qua `/api/mod/the-loai`.
2. Review metadata + AI suggestion (`preview-metadata`, `ai-suggest`, `review`) 2. Upload EPUB để preview: `POST /api/import/uploads/preview` (multipart).
3. Parse preview with TOC/regex-start (`POST /api/import/assets/{id}/parse-preview`) 3. Gợi ý metadata (tuỳ bước): `POST /api/mod/epub/ai-suggest`.
4. Start import and poll progress (`start-import`, `GET /api/import/sessions/{sessionId}`) 4. Import/xử lý EPUB: `POST /api/mod/epub` (multipart; có luồng preview và apply).
- Rule: reviewer confirms metadata before import starts.
README.md
+11 -10
View File
@@ -1,21 +1,20 @@
# Reader Project # Reader Project
Đây là dự án nền tảng đọc truyện (Web Application) được xây dựng với kiến trúc hiện đại, kết hợp cơ sở dữ liệu quan hệ (PostgreSQL) và NoSQL (MongoDB) để tối ưu hóa việc lưu trữ và truy xuất nội dung văn bản lớn. Đây là dự án nền tảng đọc truyện (Web Application) được xây dựng với kiến trúc hiện đại; dữ liệu cấu trúc và metadata người dùng lưu trên PostgreSQL (Prisma), nội dung chương và file quản lý qua API backend (`reader-api`).
## 🚀 Tính năng nổi bật ## 🚀 Tính năng nổi bật
- **Xác thực & Phân quyền**: Đăng nhập bằng Google Authentication (NextAuth). Hỗ trợ phân quyền người dùng (USER, MOD, ADMIN). - **Xác thực & Phân quyền**: Đăng nhập bằng Google Authentication (NextAuth). Hỗ trợ phân quyền người dùng (USER, MOD, ADMIN).
- **Quản lý nội dung (Dành cho MOD/ADMIN)**: Dashboard quản lý truyện, tải lên chương mới, quản lý trạng thái truyện (Đang ra, Hoàn thành, Tạm ngưng). - **Quản lý nội dung (Dành cho MOD/ADMIN)**: Dashboard quản lý truyện, tải lên chương mới, quản lý trạng thái truyện (Đang ra, Hoàn thành, Tạm ngưng).
- **Trải nghiệm đọc**: Khám phá truyện theo thể loại, tìm kiếm truyện, đọc chương truyện với hiệu suất cao (nội dung lưu ở MongoDB). - **Trải nghiệm đọc**: Khám phá truyện theo thể loại, tìm kiếm truyện, đọc chương qua API backend.
- **Tương tác người dùng**: Tính năng tủ sách (bookmark) giúp lưu lại tiến độ đọc, hỗ trợ bình luận ở truyện và từng chương. - **Tương tác người dùng**: Tính năng tủ sách (bookmark) giúp lưu lại tiến độ đọc, hỗ trợ bình luận ở truyện và từng chương.
## 🛠 Tech Stack ## 🛠 Tech Stack
- **Framework**: [Next.js](https://nextjs.org/) (App Router), React 19 - **Framework**: [Next.js](https://nextjs.org/) (App Router), React 19
- **Styling**: [TailwindCSS v4](https://tailwindcss.com/) & [Radix UI](https://www.radix-ui.com/) (shadcn/ui) - **Styling**: [TailwindCSS v4](https://tailwindcss.com/) & [Radix UI](https://www.radix-ui.com/) (shadcn/ui)
- **Database Hybrid**: - **Database**:
- **PostgreSQL**: Lưu trữ dữ liệu cấu trúc (Tài khoản, Truyện, Thể loại, Bình luận, Tủ sách) thông qua **Prisma ORM**. - **PostgreSQL**: Metadata và dữ liệu người dùng trên web (Prisma). Nội dung chương và file do **reader-api** phục vụ (NAS/R2 tùy cấu hình backend).
- **MongoDB**: Lưu trữ nội dung lớn (Chương truyện) thông qua **Mongoose**.
- **Auth**: [NextAuth.js](https://next-auth.js.org/) - **Auth**: [NextAuth.js](https://next-auth.js.org/)
--- ---
@@ -25,7 +24,7 @@
### 1. Yêu cầu cài đặt ### 1. Yêu cầu cài đặt
- [Node.js](https://nodejs.org/) (Khuyến nghị bản LTS) - [Node.js](https://nodejs.org/) (Khuyến nghị bản LTS)
- [pnpm](https://pnpm.io/) (Tool quản lý package) - [pnpm](https://pnpm.io/) (Tool quản lý package)
- Database: PostgreSQL và MongoDB đang chạy cục bộ hoặc trên máy chủ. - Database: PostgreSQL (local hoặc máy chủ). Backend `reader-api` dùng chung hoặc riêng tùy triển khai.
### 2. Cấu hình môi trường ### 2. Cấu hình môi trường
Tạo file `.env` ở thư mục gốc dựa trên `.env.example` (nếu có) hoặc điền các thông tin sau: Tạo file `.env` ở thư mục gốc dựa trên `.env.example` (nếu có) hoặc điền các thông tin sau:
@@ -34,9 +33,6 @@ Tạo file `.env` ở thư mục gốc dựa trên `.env.example` (nếu có) ho
# URL kết nối PostgreSQL # URL kết nối PostgreSQL
DATABASE_URL="postgresql://user:password@localhost:5432/reader?schema=public" DATABASE_URL="postgresql://user:password@localhost:5432/reader?schema=public"
# URL kết nối MongoDB
MONGODB_URI="mongodb://user:password@localhost:27017/reader?authSource=admin"
# Cấu hình NextAuth # Cấu hình NextAuth
NEXTAUTH_SECRET="your-super-secret-key" NEXTAUTH_SECRET="your-super-secret-key"
NEXTAUTH_URL="http://localhost:3000" NEXTAUTH_URL="http://localhost:3000"
@@ -83,7 +79,12 @@ pnpm dev
``` ```
Truy cập vào [http://localhost:3000](http://localhost:3000) để xem ứng dụng. Truy cập vào [http://localhost:3000](http://localhost:3000) để xem ứng dụng.
Lưu ý: các endpoint user-facing đã migrate (`/api/genres`, `/api/novels/*`, `/api/truyen/*`, `/api/chapters/*`, `/api/user/*`, `/api/auth/mobile-login`) sẽ được proxy sang `READER_API_ORIGIN`. Lưu ý: traffic API user-facing và MOD đi qua `READER_API_ORIGIN` theo hai cách:
- **Rewrites** trong `next.config.mjs`: `/api/genres`, `/api/novels/*`, `/api/chapters/*`, `/api/auth/mobile-login`, `/api/health`, `/api/dev/*`.
- **Route handlers** proxy trong `app/api/*/route.ts`: `/api/truyen/*`, `/api/user/*`, `/api/mod/*`, và `POST /api/import/uploads/preview` (forward request kèm cookie/session).
Một số chỗ server-side gọi API trực tiếp qua `lib/server-api.ts` / `lib/server-auth.ts` (không đi qua rewrite ở trên).
--- ---
app/api/import/[...path]/route.ts
-61
View File
@@ -1,61 +0,0 @@
import { NextRequest, NextResponse } from "next/server"
import { AUTH_COOKIE_NAME } from "@/lib/auth-cookie"
export const runtime = "nodejs"
export const dynamic = "force-dynamic"
const readerApiOrigin = (process.env.READER_API_ORIGIN || "http://localhost:8000").replace(/\/+$/, "")
async function proxyToReaderApi(req: NextRequest, path: string[]) {
const accessToken = req.cookies.get(AUTH_COOKIE_NAME)?.value || null
const url = new URL(req.url)
const query = url.search || ""
const targetUrl = `${readerApiOrigin}/api/import/${path.join("/")}${query}`
const headers = new Headers(req.headers)
headers.delete("host")
headers.delete("cookie")
if (accessToken) {
headers.set("authorization", `Bearer ${accessToken}`)
}
const isBodyMethod = req.method !== "GET" && req.method !== "HEAD"
const upstream = await fetch(targetUrl, {
method: req.method,
headers,
body: isBodyMethod ? req.body : undefined,
cache: "no-store",
duplex: "half",
} as any)
return new NextResponse(upstream.body, {
status: upstream.status,
headers: upstream.headers,
})
}
export async function GET(req: NextRequest, ctx: { params: Promise<{ path: string[] }> }) {
const { path } = await ctx.params
return proxyToReaderApi(req, path)
}
export async function POST(req: NextRequest, ctx: { params: Promise<{ path: string[] }> }) {
const { path } = await ctx.params
return proxyToReaderApi(req, path)
}
export async function PUT(req: NextRequest, ctx: { params: Promise<{ path: string[] }> }) {
const { path } = await ctx.params
return proxyToReaderApi(req, path)
}
export async function PATCH(req: NextRequest, ctx: { params: Promise<{ path: string[] }> }) {
const { path } = await ctx.params
return proxyToReaderApi(req, path)
}
export async function DELETE(req: NextRequest, ctx: { params: Promise<{ path: string[] }> }) {
const { path } = await ctx.params
return proxyToReaderApi(req, path)
}
app/api/import/uploads/preview/route.ts
+41
View File
@@ -0,0 +1,41 @@
import { NextRequest, NextResponse } from "next/server"
import { AUTH_COOKIE_NAME } from "@/lib/auth-cookie"
import { readerApiLongFetch } from "@/lib/reader-api-long-fetch"
export const runtime = "nodejs"
export const dynamic = "force-dynamic"
export const maxDuration = 900
const readerApiOrigin = (process.env.READER_API_ORIGIN || "http://localhost:8000").replace(/\/+$/, "")
async function proxyUploadPreview(req: NextRequest) {
const accessToken = req.cookies.get(AUTH_COOKIE_NAME)?.value || null
const url = new URL(req.url)
const query = url.search || ""
const targetUrl = `${readerApiOrigin}/api/import/uploads/preview${query}`
const headers = new Headers(req.headers)
headers.delete("host")
headers.delete("cookie")
if (accessToken) {
headers.set("authorization", `Bearer ${accessToken}`)
}
const upstream = await readerApiLongFetch(targetUrl, {
method: req.method,
headers,
body: req.method !== "GET" && req.method !== "HEAD" ? req.body : undefined,
cache: "no-store",
duplex: "half",
})
return new NextResponse(upstream.body, {
status: upstream.status,
headers: upstream.headers,
})
}
export async function POST(req: NextRequest) {
return proxyUploadPreview(req)
}
app/api/mod/[...path]/route.ts
+5 -2
View File
@@ -1,8 +1,11 @@
import { NextRequest, NextResponse } from "next/server" import { NextRequest, NextResponse } from "next/server"
import { AUTH_COOKIE_NAME } from "@/lib/auth-cookie" import { AUTH_COOKIE_NAME } from "@/lib/auth-cookie"
import { readerApiLongFetch } from "@/lib/reader-api-long-fetch"
export const runtime = "nodejs" export const runtime = "nodejs"
export const dynamic = "force-dynamic" export const dynamic = "force-dynamic"
/** Import EPUB nhiều chương + ghi NAS chậm có thể kéo dài nhiều phút (đặc biệt khi Next làm proxy BFF). */
export const maxDuration = 900
const readerApiOrigin = (process.env.READER_API_ORIGIN || "http://localhost:8000").replace(/\/+$/, "") const readerApiOrigin = (process.env.READER_API_ORIGIN || "http://localhost:8000").replace(/\/+$/, "")
@@ -21,13 +24,13 @@ async function proxyToReaderApi(req: NextRequest, path: string[]) {
} }
const isBodyMethod = req.method !== "GET" && req.method !== "HEAD" const isBodyMethod = req.method !== "GET" && req.method !== "HEAD"
const upstream = await fetch(targetUrl, { const upstream = await readerApiLongFetch(targetUrl, {
method: req.method, method: req.method,
headers, headers,
body: isBodyMethod ? req.body : undefined, body: isBodyMethod ? req.body : undefined,
cache: "no-store", cache: "no-store",
duplex: "half", duplex: "half",
} as any) })
return new NextResponse(upstream.body, { return new NextResponse(upstream.body, {
status: upstream.status, status: upstream.status,
app/mod/import/import-batch-client.tsx
+474
View File
@@ -0,0 +1,474 @@
"use client"
import { useState } from "react"
import type { InputHTMLAttributes } from "react"
import { Loader2 } from "lucide-react"
import { Input } from "@/components/ui/input"
import { Progress } from "@/components/ui/progress"
import { toast } from "sonner"
/** Đồng bộ với `MOD_EPUB_MAX_CHAPTERS` trên reader-api. */
const BATCH_IMPORT_MAX_CHAPTERS = 4000
/** Một EPUB: preview + AI + parse + import — vượt quá thì hủy và sang file kế. */
const BATCH_IMPORT_MAX_MS_PER_FILE = 25 * 60 * 1000
type Genre = { id: string; name: string }
type BatchRow = {
fileName: string
ok: boolean
novelId?: string
/** Tiêu đề sau preview (dùng so trùng DB / hiển thị cho mod). */
resolvedTitle?: string
error?: string
skippedDuplicate?: boolean
skippedGuard?: boolean
chapters?: number
}
function isAbortError(e: unknown): boolean {
return e instanceof DOMException && e.name === "AbortError"
}
/** Đường dẫn hiển thị khi chọn từ thư mục (Chrome: folder/book.epub) */
function getFileLabel(file: File): string {
const rp = (file as File & { webkitRelativePath?: string }).webkitRelativePath
return rp && rp.trim().length > 0 ? rp.trim() : file.name || "upload.epub"
}
function epubFilesOnly(files: FileList | File[]): File[] {
return Array.from(files).filter((f) => /\.epub$/i.test(f.name))
}
/** Giống chuẩn hóa tiêu đề khi so `Novel.lower(title)` trên API. */
function normalizeNovelTitle(raw: string): string {
return raw.split(/\s+/).join(" ").trim()
}
export function ImportBatchClient() {
const [splitMode, setSplitMode] = useState<"toc" | "regex">("toc")
const [chapterStartPattern, setChapterStartPattern] = useState(
"^\\s*(?:[#>*\\-\\[]\\s*)*(?:ch(?:u\\.?|ương|uong)?|chapter|hồi|hoi|quyển|quyen|phần|phan|tập|tap)\\s*\\d+(?:[\\.:\\-\\)]\\s*|\\s+).+$",
)
const [replaceExisting, setReplaceExisting] = useState(false)
const [running, setRunning] = useState(false)
const [progress, setProgress] = useState({ current: 0, total: 0 })
const [rows, setRows] = useState<BatchRow[]>([])
const fetchGenres = async (signal?: AbortSignal): Promise<Genre[]> => {
const res = await fetch("/api/mod/the-loai", { credentials: "include", signal })
const data = await res.json()
if (!res.ok) throw new Error(data?.error || data?.detail || "Không lấy được thể loại")
return Array.isArray(data) ? (data as Genre[]) : []
}
const ensureGenreIdsByNames = async (names: string[], signal?: AbortSignal): Promise<string[]> => {
const uniqueNames = [...new Set(names.map((n) => n.trim()).filter(Boolean))].slice(0, 6)
if (uniqueNames.length === 0) return []
let genreList = await fetchGenres(signal)
const ids: string[] = []
for (const name of uniqueNames) {
const existing = genreList.find((g) => g.name.trim().toLowerCase() === name.toLowerCase())
if (existing) {
ids.push(existing.id)
continue
}
const res = await fetch("/api/mod/the-loai", {
method: "POST",
headers: { "Content-Type": "application/json" },
credentials: "include",
signal,
body: JSON.stringify({ name, description: "" }),
})
const data = await res.json().catch(() => ({}))
if (!res.ok) {
throw new Error(data?.error || data?.detail || `Không thể tạo thể loại: ${name}`)
}
if (data?.id) {
ids.push(data.id)
}
genreList = await fetchGenres(signal)
}
return [...new Set(ids)].slice(0, 6)
}
const processOneFile = async (file: File, signal: AbortSignal): Promise<BatchRow> => {
const displayPath = getFileLabel(file)
const baseName = file.name || "upload.epub"
const formPreview = new FormData()
formPreview.append("file", file)
const r1 = await fetch("/api/import/uploads/preview", {
method: "POST",
credentials: "include",
body: formPreview,
signal,
})
const d1 = await r1.json().catch(() => ({}))
if (!r1.ok) {
return {
fileName: displayPath,
ok: false,
error: String(d1?.detail || "Bước 1: preview EPUB thất bại"),
}
}
const title =
String(d1?.suggested?.title || "").trim() || baseName.replace(/\.epub$/i, "") || "Untitled"
const author = String(d1?.suggested?.author || "").trim() || "Unknown"
const normalizedTitle = normalizeNovelTitle(title) || "Untitled"
if (!replaceExisting) {
const checkUrl = `/api/mod/truyen/by-title?title=${encodeURIComponent(normalizedTitle)}`
const cr = await fetch(checkUrl, { credentials: "include", signal })
const cd = await cr.json().catch(() => ({}))
if (cr.ok && cd?.exists === true && cd?.novel?.id) {
const nt = String(cd.novel.title || normalizedTitle)
const nid = String(cd.novel.id)
return {
fileName: displayPath,
ok: false,
skippedDuplicate: true,
resolvedTitle: title,
novelId: nid,
error: `Đã có trong DB — bỏ qua batch · «${nt}» · novelId: ${nid}`,
}
}
}
const formAi = new FormData()
formAi.append("file", file)
formAi.append("splitMode", splitMode)
if (splitMode === "regex") formAi.append("chapterRegex", chapterStartPattern)
formAi.append("title", title)
formAi.append("authorName", author)
const r2 = await fetch("/api/mod/epub/ai-suggest", {
method: "POST",
credentials: "include",
body: formAi,
signal,
})
const d2 = await r2.json().catch(() => ({}))
if (!r2.ok) {
return {
fileName: displayPath,
ok: false,
resolvedTitle: title,
error: String(d2?.detail || "Bước 2: AI gợi ý thất bại"),
}
}
const genreNames: string[] = (d2?.suggestedGenres || []).slice(0, 6)
let genreIds: string[] = []
try {
genreIds = await ensureGenreIdsByNames(genreNames, signal)
} catch (e) {
return {
fileName: displayPath,
ok: false,
resolvedTitle: title,
error: e instanceof Error ? e.message : "Bước 2: tạo/ghép thể loại thất bại",
}
}
const description = String(d2?.shortDescription || "").trim()
const statusRaw = String(d2?.suggestedStatus || "").trim()
const status =
statusRaw === "Hoàn thành" || statusRaw === "Tạm ngưng" || statusRaw === "Đang ra" ? statusRaw : "Đang ra"
const formParse = new FormData()
formParse.append("file", file)
formParse.append("preview", "true")
formParse.append("splitMode", splitMode)
if (splitMode === "regex") formParse.append("chapterRegex", chapterStartPattern)
const r3 = await fetch("/api/mod/epub", { method: "POST", credentials: "include", body: formParse, signal })
const d3 = await r3.json().catch(() => ({}))
if (!r3.ok) {
return {
fileName: displayPath,
ok: false,
resolvedTitle: title,
error: String(d3?.detail || "Bước 3: tách chương (preview) thất bại"),
}
}
const chapterCount = Number(d3?.novel?.totalChapters ?? d3?.chapterCount ?? 0)
if (d3?.importBlocked === true || chapterCount > BATCH_IMPORT_MAX_CHAPTERS) {
return {
fileName: displayPath,
ok: false,
skippedGuard: true,
resolvedTitle: title,
chapters: chapterCount || undefined,
error: String(
d3?.importBlockedReason ||
`Quá ${BATCH_IMPORT_MAX_CHAPTERS.toLocaleString()} chương (${chapterCount.toLocaleString()}) — đã bỏ qua`,
),
}
}
const sampleLen = Array.isArray(d3?.sample) ? d3.sample.length : 0
const chaptersPrevLen = Array.isArray(d3?.chaptersPreview) ? d3.chaptersPreview.length : 0
if (chapterCount <= 0 && sampleLen <= 0 && chaptersPrevLen <= 0) {
return {
fileName: displayPath,
ok: false,
resolvedTitle: title,
error: "Bước 3: không tách được chương với cấu hình TOC/Regex hiện tại",
}
}
const formImport = new FormData()
formImport.append("file", file)
formImport.append("preview", "false")
formImport.append("splitMode", splitMode)
if (splitMode === "regex") formImport.append("chapterRegex", chapterStartPattern)
formImport.append("title", title)
formImport.append("authorName", author)
formImport.append("status", status)
formImport.append("description", description)
formImport.append("genreIds", genreIds.join(","))
formImport.append("replaceExisting", String(replaceExisting))
const r4 = await fetch("/api/mod/epub", { method: "POST", credentials: "include", body: formImport, signal })
const d4 = await r4.json().catch(() => ({}))
if (r4.status === 409 && d4?.code === "DUPLICATE_TITLE") {
const ex = d4?.existingNovel as { id?: string; title?: string } | undefined
const nid = ex?.id ? String(ex.id) : ""
const nt = ex?.title ? String(ex.title) : ""
return {
fileName: displayPath,
ok: false,
skippedDuplicate: true,
resolvedTitle: title,
novelId: nid || undefined,
error: nid
? `Đã có trong DB — bỏ qua · «${nt || title}» · novelId: ${nid}`
: String(d4?.error || "Trùng tiêu đề"),
}
}
if (!r4.ok) {
const detail = String(d4?.detail || "Bước 4: import thất bại")
const limitHit =
r4.status === 400 &&
(detail.includes("giới hạn") || detail.includes("Quá giới hạn") || detail.includes("chương sau khi tách"))
return {
fileName: displayPath,
ok: false,
skippedGuard: limitHit,
resolvedTitle: title,
error: detail,
}
}
return {
fileName: displayPath,
ok: true,
resolvedTitle: title,
novelId: String(d4?.novelId || ""),
chapters: Number(d4?.totalChapters ?? chapterCount),
}
}
const runBatch = async (fileList: FileList | null) => {
if (!fileList?.length) {
toast.error("Chọn ít nhất một file EPUB")
return
}
const list = epubFilesOnly(fileList)
if (!list.length) {
toast.error("Không thấy file .epub (kiểm tra đuôi hoặc chọn đúng thư mục chứa EPUB)")
return
}
setRows([])
setRunning(true)
setProgress({ current: 0, total: list.length })
const out: BatchRow[] = []
for (let i = 0; i < list.length; i++) {
setProgress({ current: i + 1, total: list.length })
const ac = new AbortController()
const limitTimer = window.setTimeout(() => ac.abort(), BATCH_IMPORT_MAX_MS_PER_FILE)
try {
const row = await processOneFile(list[i], ac.signal)
out.push(row)
} catch (e) {
if (isAbortError(e)) {
out.push({
fileName: getFileLabel(list[i]),
ok: false,
skippedGuard: true,
error: `Quá ${Math.round(BATCH_IMPORT_MAX_MS_PER_FILE / 60000)} phút cho một file — đã bỏ qua, chuyển file kế`,
})
} else {
out.push({
fileName: getFileLabel(list[i]),
ok: false,
error: e instanceof Error ? e.message : "Lỗi không xác định",
})
}
} finally {
window.clearTimeout(limitTimer)
}
setRows([...out])
}
setRunning(false)
const okCount = out.filter((r) => r.ok).length
const dupSkip = out.filter((r) => !r.ok && r.skippedDuplicate).length
const guardSkip = out.filter((r) => !r.ok && r.skippedGuard).length
const errCount = out.filter((r) => !r.ok && !r.skippedDuplicate && !r.skippedGuard).length
toast.success(
`Hoàn tất ${list.length} file: ${okCount} import OK · ${dupSkip} đã có (bỏ qua) · ${guardSkip} bỏ qua (giới hạn/thời gian) · ${errCount} lỗi`,
{ duration: 8000 },
)
}
const pct = progress.total ? Math.round((progress.current / progress.total) * 100) : 0
return (
<div className="space-y-6 p-4 md:p-6">
<div>
<h1 className="text-2xl font-bold">Import nhiều EPUB (tự động)</h1>
</div>
<section className="space-y-3 rounded-xl border p-4">
<p className="text-sm font-medium">Cấu hình chung</p>
<div className="flex flex-wrap items-center gap-3 rounded-md border bg-muted/30 p-3">
<label className="text-sm">Tách chương:</label>
<select
className="rounded border px-2 py-1 text-sm"
value={splitMode}
onChange={(e) => setSplitMode(e.target.value as "toc" | "regex")}
disabled={running}
>
<option value="toc">TOC</option>
<option value="regex">Regex tiếng Việt</option>
</select>
{splitMode === "regex" && (
<Input
className="max-w-xl font-mono text-xs"
value={chapterStartPattern}
onChange={(e) => setChapterStartPattern(e.target.value)}
disabled={running}
placeholder="Regex bắt đầu chương"
/>
)}
</div>
<label className="flex items-center gap-2 text-sm">
<input type="checkbox" checked={replaceExisting} onChange={(e) => setReplaceExisting(e.target.checked)} disabled={running} />
Ghi đè nếu trùng tiêu đề (tắt = batch chỉ skip khi đã có truyện cùng tiêu đề)
</label>
<p className="text-xs text-muted-foreground">
An toàn batch: tối đa {BATCH_IMPORT_MAX_CHAPTERS.toLocaleString()} chương sau khi tách mỗi file; quá{" "}
{Math.round(BATCH_IMPORT_MAX_MS_PER_FILE / 60000)} phút/file thì bỏ qua và xử lý file tiếp theo.
</p>
</section>
<section className="space-y-4 rounded-xl border p-4">
<div className="space-y-2">
<p className="text-sm font-medium">Thư mục cha (quét đệ quy)</p>
<Input
type="file"
{...({ webkitdirectory: "", mozdirectory: "" } as InputHTMLAttributes<HTMLInputElement>)}
disabled={running}
className="cursor-pointer"
onChange={(e) => {
const fl = e.target.files
void runBatch(fl)
e.currentTarget.value = ""
}}
/>
</div>
{running && (
<div className="space-y-2">
<div className="flex items-center gap-2 text-sm text-muted-foreground">
<Loader2 className="h-4 w-4 animate-spin" />
Đang xử lý {progress.current}/{progress.total}…
</div>
<Progress value={pct} />
</div>
)}
</section>
{rows.length > 0 && (
<section className="rounded-xl border">
<div className="border-b px-4 py-2 text-sm font-medium">Kết quả</div>
<div className="flex flex-wrap gap-3 border-b bg-muted/20 px-4 py-2 text-xs text-muted-foreground">
<span>
Thành công:{" "}
<strong className="text-emerald-600">{rows.filter((r) => r.ok).length}</strong>
</span>
<span>
Đã có (skip):{" "}
<strong className="text-amber-700">{rows.filter((r) => r.skippedDuplicate).length}</strong>
</span>
<span>
Bỏ qua an toàn:{" "}
<strong className="text-amber-700">{rows.filter((r) => r.skippedGuard).length}</strong>
</span>
<span>
Lỗi:{" "}
<strong className="text-destructive">
{rows.filter((r) => !r.ok && !r.skippedDuplicate && !r.skippedGuard).length}
</strong>
</span>
<span>
Tổng: <strong className="text-foreground">{rows.length}</strong>
</span>
</div>
<div className="max-h-[480px] overflow-auto">
<table className="w-full text-left text-sm">
<thead>
<tr className="border-b bg-muted/40 text-xs text-muted-foreground">
<th className="px-3 py-2">File</th>
<th className="px-3 py-2">Tiêu đề (preview)</th>
<th className="px-3 py-2">Trạng thái</th>
<th className="px-3 py-2">Chi tiết</th>
</tr>
</thead>
<tbody>
{rows.map((r, idx) => (
<tr key={`${r.fileName}-${idx}`} className="border-b last:border-0">
<td className="max-w-[220px] break-all px-3 py-2 align-top">{r.fileName}</td>
<td className="max-w-[200px] break-words px-3 py-2 align-top text-xs">
{r.resolvedTitle || "—"}
</td>
<td className="whitespace-nowrap px-3 py-2 align-top">
{r.ok ? (
<span className="text-emerald-600">OK</span>
) : r.skippedDuplicate ? (
<span className="text-amber-600">Đã có · skip</span>
) : r.skippedGuard ? (
<span className="text-amber-600">Bỏ qua</span>
) : (
<span className="text-destructive">Lỗi</span>
)}
</td>
<td className="min-w-[200px] px-3 py-2 align-top text-xs text-muted-foreground">
{r.ok && r.novelId ? (
<>
novelId: {r.novelId}
{typeof r.chapters === "number" ? ` · ${r.chapters} chương` : ""}
</>
) : (
r.error || "—"
)}
</td>
</tr>
))}
</tbody>
</table>
</div>
</section>
)}
</div>
)
}
app/mod/import/import-client.tsx
+4
View File
@@ -257,6 +257,10 @@ export function ImportClient() {
setSelectedGenreIds(ensuredIds) setSelectedGenreIds(ensuredIds)
} }
setShortDescription(data?.shortDescription || "") setShortDescription(data?.shortDescription || "")
const st = typeof data?.suggestedStatus === "string" ? data.suggestedStatus.trim() : ""
if (st === "Đang ra" || st === "Hoàn thành" || st === "Tạm ngưng") {
setStatus(st)
}
toast.success("Đã áp dụng gợi ý AI") toast.success("Đã áp dụng gợi ý AI")
} catch (error) { } catch (error) {
toast.error(error instanceof Error ? error.message : "AI suggest lỗi") toast.error(error instanceof Error ? error.message : "AI suggest lỗi")
app/mod/import/page.tsx
+16 -1
View File
@@ -1,5 +1,20 @@
import { ImportBatchClient } from "./import-batch-client"
import { ImportClient } from "./import-client" import { ImportClient } from "./import-client"
import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs"
export default function ModImportPage() { export default function ModImportPage() {
return <ImportClient /> return (
<Tabs defaultValue="single" className="w-full">
<TabsList className="mx-4 mt-4 md:mx-6">
<TabsTrigger value="single">Một EPUB</TabsTrigger>
<TabsTrigger value="batch">Nhiều EPUB (tự động)</TabsTrigger>
</TabsList>
<TabsContent value="single" className="mt-0">
<ImportClient />
</TabsContent>
<TabsContent value="batch" className="mt-0">
<ImportBatchClient />
</TabsContent>
</Tabs>
)
} }
docker-compose.yml
-1
View File
@@ -12,7 +12,6 @@ services:
environment: environment:
# KHÔNG SỬ DỤNG DẤU NGOẶC KÉP "" TRONG DOCKER COMPOSE # KHÔNG SỬ DỤNG DẤU NGOẶC KÉP "" TRONG DOCKER COMPOSE
- DATABASE_URL=postgresql://reader:reader%40123@master-02:5432/reader?schema=public - DATABASE_URL=postgresql://reader:reader%40123@master-02:5432/reader?schema=public
- MONGODB_URI=mongodb://root:virtus%40123@master-02:27017/reader?authSource=admin
- NEXTAUTH_SECRET=your-super-secret-key - NEXTAUTH_SECRET=your-super-secret-key
# Sửa thành domain name thực tế bạn đang truy cập # Sửa thành domain name thực tế bạn đang truy cập
- NEXTAUTH_URL=http://master-02:3003 - NEXTAUTH_URL=http://master-02:3003
lib/reader-api-long-fetch.ts
+19
View File
@@ -0,0 +1,19 @@
import { Agent, fetch as undiciFetch } from "undici"
/** Tránh mặc định undici (headers/body ~300s) cắt kết nối khi API + NAS xử lý lâu. */
const longRunningAgent = new Agent({
connectTimeout: 120_000,
headersTimeout: 3_600_000,
bodyTimeout: 3_600_000,
})
export async function readerApiLongFetch(
input: string | URL,
init?: RequestInit & { duplex?: "half" },
): Promise<Response> {
const res = await undiciFetch(input, {
...init,
dispatcher: longRunningAgent,
} as Parameters<typeof undiciFetch>[1])
return res as unknown as Response
}
next.config.mjs
+9
View File
@@ -1,7 +1,16 @@
import path from "node:path"
import { fileURLToPath } from "node:url"
const __dirname = path.dirname(fileURLToPath(import.meta.url))
/** @type {import('next').NextConfig} */ /** @type {import('next').NextConfig} */
const readerApiOrigin = (process.env.READER_API_ORIGIN || "http://localhost:8000").replace(/\/+$/, "") const readerApiOrigin = (process.env.READER_API_ORIGIN || "http://localhost:8000").replace(/\/+$/, "")
const nextConfig = { const nextConfig = {
// Tránh Turbopack suy luận sai root (lỗi: tìm next từ `reader/app` → dev server exit 1).
turbopack: {
root: __dirname,
},
output: "standalone", output: "standalone",
typescript: { typescript: {
ignoreBuildErrors: true, ignoreBuildErrors: true,
package.json
+1
View File
@@ -64,6 +64,7 @@
"recharts": "2.15.0", "recharts": "2.15.0",
"sonner": "^1.7.1", "sonner": "^1.7.1",
"tailwind-merge": "^3.3.1", "tailwind-merge": "^3.3.1",
"undici": "^8.2.0",
"vaul": "^1.1.2", "vaul": "^1.1.2",
"zod": "^3.24.1" "zod": "^3.24.1"
}, },
pnpm-lock.yaml
Generated
+9
View File
@@ -173,6 +173,9 @@ importers:
tailwind-merge: tailwind-merge:
specifier: ^3.3.1 specifier: ^3.3.1
version: 3.4.0 version: 3.4.0
undici:
specifier: ^8.2.0
version: 8.2.0
vaul: vaul:
specifier: ^1.1.2 specifier: ^1.1.2
version: 1.1.2(@types/react-dom@19.2.3(@types/react@19.2.14))(@types/react@19.2.14)(react-dom@19.2.4(react@19.2.4))(react@19.2.4) version: 1.1.2(@types/react-dom@19.2.3(@types/react@19.2.14))(@types/react@19.2.14)(react-dom@19.2.4(react@19.2.4))(react@19.2.4)
@@ -2475,6 +2478,10 @@ packages:
undici-types@6.21.0: undici-types@6.21.0:
resolution: {integrity: sha512-iwDZqg0QAGrg9Rav5H4n0M64c3mkR59cJ6wQp+7C4nI0gsmExaedaYLNO44eT4AtBBwjbTiGPMlt2Md0T9H9JQ==} resolution: {integrity: sha512-iwDZqg0QAGrg9Rav5H4n0M64c3mkR59cJ6wQp+7C4nI0gsmExaedaYLNO44eT4AtBBwjbTiGPMlt2Md0T9H9JQ==}
undici@8.2.0:
resolution: {integrity: sha512-Z+4Hx9GE26Lh9Upwfnc8C7SsrpBPGaM/Gm6kMFtiG7c+5IvQKlXi/t+9x9DrrCh29cww5TSP9YdVaBcnLDs5fQ==}
engines: {node: '>=22.19.0'}
update-browserslist-db@1.2.3: update-browserslist-db@1.2.3:
resolution: {integrity: sha512-Js0m9cx+qOgDxo0eMiFGEueWztz+d4+M3rGlmKPT+T4IS/jP4ylw3Nwpu6cpTTP8R1MAC1kF4VbdLt3ARf209w==} resolution: {integrity: sha512-Js0m9cx+qOgDxo0eMiFGEueWztz+d4+M3rGlmKPT+T4IS/jP4ylw3Nwpu6cpTTP8R1MAC1kF4VbdLt3ARf209w==}
hasBin: true hasBin: true
@@ -5083,6 +5090,8 @@ snapshots:
undici-types@6.21.0: {} undici-types@6.21.0: {}
undici@8.2.0: {}
update-browserslist-db@1.2.3(browserslist@4.28.1): update-browserslist-db@1.2.3(browserslist@4.28.1):
dependencies: dependencies:
browserslist: 4.28.1 browserslist: 4.28.1
prisma/schema.prisma
+1 -1
View File
@@ -161,7 +161,7 @@ model Comment {
content String @db.Text content String @db.Text
userId String userId String
novelId String novelId String
chapterId String? // Có thể bình luận riêng tư cho từng chương (Lưu chapterId từ MongoDB) chapterId String? // Bình luận theo chương (id chương từ backend / ChapterMeta)
user User @relation(fields: [userId], references: [id], onDelete: Cascade) user User @relation(fields: [userId], references: [id], onDelete: Cascade)
novel Novel @relation(fields: [novelId], references: [id], onDelete: Cascade) novel Novel @relation(fields: [novelId], references: [id], onDelete: Cascade)
test-db.js
-17
View File
@@ -1,17 +0,0 @@
const { MongoClient } = require('mongodb');
async function check() {
const uri = process.env.MONGODB_URI || "mongodb://localhost:27017/reader";
const client = new MongoClient(uri);
try {
await client.connect();
const db = client.db();
// Just find the latest 5 chapters inserted
const docs = await db.collection('Chapter').find({}).sort({ _id: -1 }).limit(10).toArray();
console.log("Recent chapters:");
docs.forEach(d => console.log(d.novelId, d.number, d.title));
} finally {
await client.close();
}
}
check();
tsconfig.tsbuildinfo
+1 -1
View File
File diff suppressed because one or more lines are too long