diff --git a/docs/安全审查风险整改任务清单.md b/docs/安全审查风险整改任务清单.md index 97c1480..29b4ee9 100644 --- a/docs/安全审查风险整改任务清单.md +++ b/docs/安全审查风险整改任务清单.md @@ -854,9 +854,19 @@ go vet ./... ## 2026-08-30 配置快照发布失败重试整改记录 -- **本轮状态**?`[x] 已完成?等待复审` -- **实施内容**?统一配置启动加载?配置应用?配置回滚?数据库切换和插件 Secret 迁移的运行时快照发布入口?当磁盘事务已成功提交但运行时快照暂不可用时?保留可重试状态并将配置服务标记为降级? -- **接口提示**?新增稳定错误码 `CONFIG_RUNTIME_SNAPSHOT_UNAVAILABLE`?返回 HTTP 503?并明确提示配置已保存?运行时尚未切换?可稍后重试应用? -- **新增测试**?覆盖应用阶段快照发布失败后的磁盘代次?运行时旧快照?降级状态和再次应用成功?保留回滚票据失败重试及并发保护测试? -- **验证结果**?服务端 `go test ./... -count=1 -timeout 30m` 和 `go vet ./...` 通过?未将真实 MySQL?PostgreSQL?Ubuntu?Windows?浏览器和生产部署验收标记为完成? -- **遗留边界**?事务提交后快照发布失败的跨进程恢复仍依赖服务重载或后续重试?真实多数据库复制?校验?原子切换和部署联调继续按实机清单验收? +- **本轮状态**：`[x] 已完成，等待复审` +- **实施内容**：统一配置启动加载、配置应用、配置回滚、数据库切换和插件 Secret 迁移的运行时快照发布入口；当磁盘事务已成功提交但运行时快照暂不可用时，保留可重试状态并将配置服务标记为降级。 +- **接口提示**：新增稳定错误码 `CONFIG_RUNTIME_SNAPSHOT_UNAVAILABLE`，返回 HTTP 503，并明确提示配置已保存、运行时尚未切换，可稍后重试应用。 +- **新增测试**：覆盖应用阶段快照发布失败后的磁盘代次、运行时旧快照、降级状态和再次应用成功；保留回滚票据失败重试及并发保护测试。 +- **验证结果**：服务端 `go test ./... -count=1 -timeout 30m` 和 `go vet ./...` 通过；未将真实 MySQL、PostgreSQL、Ubuntu、Windows、浏览器和生产部署验收标记为完成。 +- **遗留边界**：事务提交后快照发布失败的跨进程恢复仍依赖服务重载或后续重试；真实多数据库复制、校验、原子切换和部署联调继续按实机清单验收。 + +## 2026-08-31 IMAP 删除语义增量复审 + +- **关联任务**：SEC-020，以及统一复审清单中的用户隔离、静态检查和回归测试项目。 +- **状态**：`[x] 已完成，等待真实客户端和部署环境复审`。 +- **实施内容**：为 `user_email` 增加持久化 `is_deleted` 标志和版本 14 数据库迁移；`SELECT`、UID/序列号 `FETCH` 和 `STORE` 统一返回或更新 `\Seen`、`\Deleted`；普通 `EXPUNGE` 仅清除当前邮箱中已标记删除的邮件，`UID EXPUNGE` 仅清除指定 UID 集合中已标记删除的邮件。 +- **边界加固**：所有查询和更新同时限制当前用户与当前邮箱；修复 UID FETCH 返回错误序列号的问题；按序列号降序发送 EXPUNGE 响应；支持 `*` 和 `n:*` 动态 UID 范围，限制单次最多展开 10000 个 UID，防止超大范围占用资源。 +- **修改范围**：`server/models/user_email.go`、`server/db/migration/imap_flags.go`、`server/services/list/list.go`、`server/services/del_email/del_email.go`、`server/listen/imap_server/session_*.go` 及相关测试。 +- **自动化验证**：`go test ./services/list ./listen/imap_server ./db/migration ./services/del_email -count=1 -timeout 10m` 已通过；本轮最终提交前继续执行服务端全量测试、竞态测试、`go vet`、前端 ESLint 和生产构建。 +- **开放项**：HSTS 与 Nginx 反向代理的实际上线行为、真实 IMAP 客户端兼容性、Ubuntu/Windows 部署和真实 MySQL/PostgreSQL 迁移仍需实机复审，不据此升级为最终发布通过。 diff --git a/docs/超级管理员重置与邮箱验证码实施任务清单.md b/docs/超级管理员重置与邮箱验证码实施任务清单.md index c009aa1..507dd19 100644 --- a/docs/超级管理员重置与邮箱验证码实施任务清单.md +++ b/docs/超级管理员重置与邮箱验证码实施任务清单.md @@ -1395,3 +1395,34 @@ AD-065 + CG-053～CG-056 → AD-066 → CG-057 - **本地验证**：服务端全量 `go test ./... -count=1` 通过；插件控制器和 HTTP 安全头专项测试通过。 - **未完成真实验收**：超级管理员本机重置通道仍需在 Ubuntu 22.04、Windows 部署端和实际权限模型中复审；邮箱验证码、SMTP 投递、WebAuthn、真实浏览器和多实例流程不得标记为最终通过。 - **复审日期**：2026-08-29（本机自动化复审）。 + +## 2026-08-31 全量代码状态复核 + +> 本节是 2026-08-31 对当前代码库的增量状态口径。原任务条目中的“未开始”保留为历史计划文本，不再代表当前实现状态；未完成真实平台或外部系统验收的任务不得标记为 `[✓]`。 + +| 任务范围 | 当前状态 | 复核结论 | +|---|---|---| +| AD-001～AD-008 | `[x] 已完成，等待复审` | 进程模型、权限矩阵、威胁模型、敏感字段、测试矩阵、独立管理程序、严格命令解析和部署管理员检查均已有实现与自动化测试。 | +| AD-009～AD-011 | `[x] 已完成，等待实机复审` | systemd、Windows 服务/ACL、版本绑定和升级约束已有代码或部署文件；真实 Ubuntu 与 Windows 服务环境尚未验收。 | +| AD-012～AD-021 | `[x] 已完成，等待实机复审` | 最小控制协议、UDS、Named Pipe、一次性 nonce、统一密码策略、事务重置、凭据撤销、审计及 Web 超级管理员恢复拒绝均已有实现和负向测试。 | +| AD-022～AD-025 | `[x] 已完成，等待复审` | 外部邮箱模型、加密存储、独立指纹、归属校验、脱敏查询和绑定失效事务已实现。 | +| AD-026～AD-031 | `[x] 已完成，等待复审` | 临时 HMAC 密钥、容量上限、验证码生成、用途绑定、恒定比较、原子消费、统一清理和重启失效已实现。 | +| AD-032～AD-037 | `[x] 已完成，等待外部系统复审` | SMTP Relay 接入、验证码不落盘投递、失败分类、多维限流、运营数据和 14 天分批清理已实现并具备自动化测试。 | +| AD-038 | `[!] 暂缓或存在阻塞` | 三数据库清理代码和方言已具备；真实 MySQL、PostgreSQL 的清理、空间复用和故障恢复尚未执行。 | +| AD-039～AD-059 | `[x] 已完成，等待浏览器与 SMTP 复审` | 统一登录标识、受限会话、二次验证策略、防枚举恢复、分角色恢复、一次性授权、V1 API、前端页面、审计和告警边界均已有实现。 | +| AD-060 | `[x] 已完成，等待复审` | 单元、并发和故障路径自动化测试已建立。 | +| AD-061～AD-064 | `[!] 暂缓或存在阻塞` | 真实三数据库、Ubuntu、Windows、真实浏览器三角色流程需要对应环境和人工交互验收。 | +| AD-065 | `[x] 已完成，等待独立安全复审` | 关键负向、重放、越权、枚举、降级和敏感数据测试已覆盖；生产观测链路仍需复核。 | +| AD-066 | `[!] 暂缓或存在阻塞` | 最终发布评审依赖 AD-038、AD-061～AD-064 以及真实 SMTP、多实例和生产安全审查。 | + +### 当前实现证据 + +- 管理程序与本机控制通道：`server/cmd/smail-admin`、`server/services/adminreset`、`deploy/systemd`。 +- 外部邮箱与验证码：`server/services/externalemail`、`server/services/verification`。 +- 恢复和强认证：`server/services/recovery`、`server/services/webauthn`、`server/services/totp`、`server/services/credentials`。 +- API 与前端：`server/controllers/security.go`、`server/controllers/webauthn.go`、`server/listen/http_server/api_v1_compat.go`、`fe/src/views` 下登录验证、密码恢复、账户安全和强制改密页面。 +- 自动化复核：服务端全量、竞态和静态检查以及前端 ESLint、生产构建在本轮最终提交前统一执行并记录。 + +### 发布结论 + +三份清单要求的可在当前工作区完成的代码实现已进入待复审状态；真实 Ubuntu 22.04/24.04、Windows Service/ACL/Named Pipe、真实 SMTP Relay、MySQL、PostgreSQL、浏览器三角色、多实例和最终发布安全评审保持阻塞。超级管理员自身重置仍只允许受保护的 UDS 或 Named Pipe，本轮未增加 HTTP、localhost TCP、局域网或公网 TCP 重置入口。 diff --git a/docs/配置系统重构实施任务清单.md b/docs/配置系统重构实施任务清单.md index 65012f8..471be9a 100644 --- a/docs/配置系统重构实施任务清单.md +++ b/docs/配置系统重构实施任务清单.md @@ -1265,9 +1265,44 @@ CG-053～CG-056 → AD-066 → CG-057 ## 2026-08-30 配置快照发布失败重试整改记录 -- **本轮状态**?`[x] 已完成?等待复审` -- **实施内容**?统一配置启动加载?配置应用?配置回滚?数据库切换和插件 Secret 迁移的运行时快照发布入口?当磁盘事务已成功提交但运行时快照暂不可用时?保留可重试状态并将配置服务标记为降级? -- **接口提示**?新增稳定错误码 `CONFIG_RUNTIME_SNAPSHOT_UNAVAILABLE`?返回 HTTP 503?并明确提示配置已保存?运行时尚未切换?可稍后重试应用? -- **新增测试**?覆盖应用阶段快照发布失败后的磁盘代次?运行时旧快照?降级状态和再次应用成功?保留回滚票据失败重试及并发保护测试? -- **验证结果**?服务端 `go test ./... -count=1 -timeout 30m` 和 `go vet ./...` 通过?未将真实 MySQL?PostgreSQL?Ubuntu?Windows?浏览器和生产部署验收标记为完成? -- **遗留边界**?事务提交后快照发布失败的跨进程恢复仍依赖服务重载或后续重试?真实多数据库复制?校验?原子切换和部署联调继续按实机清单验收? +- **本轮状态**：`[x] 已完成，等待复审` +- **实施内容**：统一配置启动加载、配置应用、配置回滚、数据库切换和插件 Secret 迁移的运行时快照发布入口；当磁盘事务已成功提交但运行时快照暂不可用时，保留可重试状态并将配置服务标记为降级。 +- **接口提示**：新增稳定错误码 `CONFIG_RUNTIME_SNAPSHOT_UNAVAILABLE`，返回 HTTP 503，并明确提示配置已保存、运行时尚未切换，可稍后重试应用。 +- **新增测试**：覆盖应用阶段快照发布失败后的磁盘代次、运行时旧快照、降级状态和再次应用成功；保留回滚票据失败重试及并发保护测试。 +- **验证结果**：服务端 `go test ./... -count=1 -timeout 30m` 和 `go vet ./...` 通过；未将真实 MySQL、PostgreSQL、Ubuntu、Windows、浏览器和生产部署验收标记为完成。 +- **遗留边界**：事务提交后快照发布失败的跨进程恢复仍依赖服务重载或后续重试；真实多数据库复制、校验、原子切换和部署联调继续按实机清单验收。 + +## 2026-08-31 全量代码状态复核 + +> 本节是 2026-08-31 对当前实现的增量状态口径。原任务条目中的旧状态保留为实施历史；以下状态优先用于判断当前代码完成度，但不能替代真实平台和生产发布验收。 + +| 任务范围 | 当前状态 | 复核结论 | +|---|---|---| +| CG-001 | `[✓] 复审通过` | 需求、边界和依赖关系已经冻结。 | +| CG-002～CG-007 | `[x] 已完成，等待复审` | 敏感字段、显式迁移、测试基线、配置模型、严格解析和验证器均已实现。 | +| CG-008～CG-009 | `[✓] 复审通过` | CredentialVault 和不可变运行时快照已有测试证据。 | +| CG-010～CG-014 | `[x] 已完成，等待复审` | ConfigManager、事务恢复、备份回滚、状态/审计/受控重启、旧配置迁移均已有实现和自动化测试。 | +| CG-015 | `[x] 已完成，等待实机复审` | SSL 分区和受控存储已实现；真实证书、Nginx 和平台权限仍需验收。 | +| CG-016～CG-021 | `[✓] 复审通过` | 插件目录、授权、专属配置下发和三个插件源码整改已有本机自动化复审证据。 | +| CG-022～CG-044 | `[x] 已完成，等待复审` | SMTP/邮箱策略、三级角色、统一认证、密码、部署端重置、恢复仓储、密保、恢复码、WebAuthn、恢复状态机、API 和前端页面均已有实现。 | +| CG-045 | `[!] 暂缓或存在阻塞` | 真实浏览器三角色、可访问性和完整交互流程尚未人工验收。 | +| CG-046～CG-050 | `[x] 已完成，等待部署复审` | setupState、初始化授权、V1 兼容、功能开关和旧写路径收敛已有代码和测试。 | +| CG-051 | `[!] 暂缓或存在阻塞` | Windows、Ubuntu、MySQL、PostgreSQL 和 SQLite 真实部署矩阵尚未全部完成。 | +| CG-052～CG-055 | `[x] 已完成，等待独立复审` | 自动化回归、故障路径以及配置、认证、插件安全审查已形成代码和测试证据。 | +| CG-056～CG-057 | `[!] 暂缓或存在阻塞` | 浏览器数据泄露复审和最终发布验收依赖真实环境、人工交互及生产安全评审。 | + +### 过时记录修正 + +旧记录中“真实数据库复制执行器尚未接入”的结论已经过时。当前已存在 `server/services/dbmigration/sql_executor.go`、`sqlite_executor.go`、运行时执行器工厂、复制校验、配置切换和相关测试。该修正只表示代码已接入，不表示真实 MySQL/PostgreSQL 数据复制、失败回滚和原子切换已经实机通过。 + +### 当前实现证据 + +- 配置核心：`server/config/model.go`、`parser.go`、`validator.go`、`secret.go`、`snapshot.go`、`manager.go`、`manager_v2.go`、`transaction.go`、`storage.go`。 +- 备份、审计和迁移：`server/config/manager_backups.go`、`audit_store.go`、`audit_query.go`、`database_migration.go`。 +- 数据库复制与切换：`server/services/dbmigration/service.go`、`sql_executor.go`、`sqlite_executor.go`。 +- 受控重启和接口：`server/services/restart`、`server/controllers/config.go`、`config_audit.go`、`database_migration.go`、`restart.go`。 +- 自动化复核：服务端全量测试、竞态检测、`go vet`、前端 ESLint 和生产构建在本轮最终提交前统一执行并记录。 + +### 发布结论 + +配置系统重构的工作区代码已进入待复审状态，但 CG-045、CG-051、CG-056 和 CG-057 仍受真实浏览器、跨平台、三数据库、多实例和生产发布评审阻塞。不得将“代码完成”表述为“生产验收完成”。 diff --git a/server/db/migration/imap_flags.go b/server/db/migration/imap_flags.go new file mode 100644 index 0000000..6a78b28 --- /dev/null +++ b/server/db/migration/imap_flags.go @@ -0,0 +1,19 @@ +package migration + +import ( + "context" + "fmt" +) + +func applyIMAPDeletedFlagSchema(ctx context.Context, exec executor, dialect Dialect) error { + definitions := map[Dialect]string{ + DialectSQLite: `INTEGER NOT NULL DEFAULT 0`, + DialectMySQL: `TINYINT NOT NULL DEFAULT 0`, + DialectPostgres: `SMALLINT NOT NULL DEFAULT 0`, + } + definition, ok := definitions[dialect] + if !ok { + return fmt.Errorf("unsupported migration dialect %q", dialect) + } + return ensureColumn(ctx, exec, dialect, "user_email", "is_deleted", definition) +} diff --git a/server/db/migration/registry.go b/server/db/migration/registry.go index fb1141f..0c03807 100644 --- a/server/db/migration/registry.go +++ b/server/db/migration/registry.go @@ -15,5 +15,6 @@ func DefaultMigrations() []Migration { DefineForDialects(11, "create_plugin_configuration_tables", pluginSettingsStatements()), DefineCustom(12, "expand_user_email_login_addresses", "expand_user_email_login_addresses_v1", applyUserEmailAddressSchema), DefineForDialects(13, "create_protocol_credentials", protocolCredentialStatements()), + DefineCustom(14, "expand_imap_deleted_flag", "expand_imap_deleted_flag_v1", applyIMAPDeletedFlagSchema), } } diff --git a/server/db/migration/registry_test.go b/server/db/migration/registry_test.go index 82ed909..2332a9a 100644 --- a/server/db/migration/registry_test.go +++ b/server/db/migration/registry_test.go @@ -40,7 +40,7 @@ func TestDefaultMigrationsCreateCompleteSQLiteSchema(t *testing.T) { if !exists { t.Fatal("column webauthn_credential.discoverable was not created") } - for _, column := range []string{"address", "address_canonical", "is_primary", "verified_at", "created_at", "updated_at"} { + for _, column := range []string{"address", "address_canonical", "is_primary", "verified_at", "created_at", "updated_at", "is_deleted"} { exists, err := columnExists(context.Background(), db, DialectSQLite, "user_email", column) if err != nil { t.Fatal(err) @@ -102,7 +102,7 @@ func TestUserEmailLoginAddressMigrationUpgradesLegacyTableAndIsIdempotent(t *tes if _, err := runner.Run(context.Background()); err != nil { t.Fatal(err) } - for _, column := range []string{"address", "address_canonical", "is_primary", "verified_at", "created_at", "updated_at"} { + for _, column := range []string{"address", "address_canonical", "is_primary", "verified_at", "created_at", "updated_at", "is_deleted"} { exists, err := columnExists(context.Background(), db, DialectSQLite, "user_email", column) if err != nil { t.Fatal(err) @@ -144,6 +144,13 @@ func TestUserEmailLoginAddressMigrationRejectsDuplicateCanonicalAddresses(t *tes } } +func TestIMAPDeletedFlagMigrationRejectsUnexpectedDialect(t *testing.T) { + err := applyIMAPDeletedFlagSchema(context.Background(), nil, Dialect("unsupported")) + if err == nil || !strings.Contains(err.Error(), "unsupported") { + t.Fatalf("unexpected dialect error = %v", err) + } +} + func TestUserEmailLoginAddressMigrationRejectsUnexpectedDialect(t *testing.T) { err := applyUserEmailAddressSchema(context.Background(), nil, Dialect("unsupported")) if err == nil || !strings.Contains(err.Error(), "unsupported") { diff --git a/server/dto/response/email.go b/server/dto/response/email.go index 301ed91..4af07c4 100644 --- a/server/dto/response/email.go +++ b/server/dto/response/email.go @@ -5,6 +5,7 @@ import "git.pcux.cn/xuans/Smail/models" type EmailResponseData struct { models.Email `xorm:"extends"` IsRead int8 `json:"is_read"` + IsDeleted int8 `json:"-"` SerialNumber int `json:"serial_number"` UeId int `json:"ue_id"` } diff --git a/server/listen/imap_server/imap_server_test.go b/server/listen/imap_server/imap_server_test.go index 99a57f1..60a01ae 100644 --- a/server/listen/imap_server/imap_server_test.go +++ b/server/listen/imap_server/imap_server_test.go @@ -12,6 +12,7 @@ import ( "mime" "os" "path/filepath" + "reflect" "strings" "testing" "time" @@ -618,6 +619,32 @@ func TestStore(t *testing.T) { } assertReadState(t, relation.ID, userID, 0) + if _, err := clientLogin.Store(uidSet(uid), &imap.StoreFlags{ + Op: imap.StoreFlagsAdd, + Flags: []imap.Flag{imap.FlagDeleted}, + }, &imap.StoreOptions{}).Collect(); err != nil { + t.Fatalf("add \\Deleted: %v", err) + } + assertDeletedState(t, relation.ID, userID, 1) + + if _, err := clientLogin.Store(uidSet(uid), &imap.StoreFlags{ + Op: imap.StoreFlagsDel, + Silent: true, + Flags: []imap.Flag{imap.FlagDeleted}, + }, &imap.StoreOptions{}).Collect(); err != nil { + t.Fatalf("delete \\Deleted: %v", err) + } + assertDeletedState(t, relation.ID, userID, 0) + + if _, err := clientLogin.Store(uidSet(uid), &imap.StoreFlags{ + Op: imap.StoreFlagsSet, + Flags: []imap.Flag{imap.FlagSeen, imap.FlagDeleted}, + }, &imap.StoreOptions{}).Collect(); err != nil { + t.Fatalf("set supported flags: %v", err) + } + assertReadState(t, relation.ID, userID, 1) + assertDeletedState(t, relation.ID, userID, 1) + if _, err := clientLogin.Store(uidSet(uid), &imap.StoreFlags{ Op: imap.StoreFlagsSet, Flags: nil, @@ -625,6 +652,7 @@ func TestStore(t *testing.T) { t.Fatalf("set empty flags: %v", err) } assertReadState(t, relation.ID, userID, 0) + assertDeletedState(t, relation.ID, userID, 0) } func uidSet(uid imap.UID) imap.UIDSet { @@ -662,39 +690,152 @@ func TestExpunge(t *testing.T) { t.Fatal(err) } + userID := clientLoginUserID(t) var targets []models.UserEmail if err := db.Instance.Table("user_email"). - Where("user_id = ? AND email_id > 0", 1). + Where("user_id = ? AND status = ? AND group_id = 0 AND email_id > 0 AND is_deleted = 0", userID, 0). OrderBy("id ASC"). Limit(2). Find(&targets); err != nil { t.Fatal(err) } - if len(targets) == 0 { - t.Fatal("no email relations available for expunge") + if len(targets) < 2 { + t.Fatal("not enough INBOX relations available for expunge") } - uids := make([]imap.UID, 0, len(targets)) - for _, target := range targets { - uids = append(uids, imap.UID(target.ID)) + if _, err := clientLogin.Store(uidSet(imap.UID(targets[0].ID)), &imap.StoreFlags{ + Op: imap.StoreFlagsAdd, + Flags: []imap.Flag{imap.FlagDeleted}, + }, &imap.StoreOptions{}).Collect(); err != nil { + t.Fatalf("mark message deleted: %v", err) } - res, err := clientLogin.UIDExpunge(imap.UIDSetNum(uids...)).Collect() + assertDeletedState(t, targets[0].ID, userID, 1) + assertDeletedState(t, targets[1].ID, userID, 0) + expungeSequence := assertDeletedFetchFlags(t, imap.UID(targets[0].ID)) + + expunged, err := clientLogin.UIDExpunge(imap.UIDSetNum(imap.UID(targets[0].ID), imap.UID(targets[1].ID))).Collect() if err != nil { - t.Errorf("%+v", err) + t.Fatalf("UID EXPUNGE: %v", err) } - t.Logf("%+v", res) - - for _, target := range targets { - var remaining models.UserEmail - found, err := db.Instance.ID(target.ID).Get(&remaining) - if err != nil { - t.Fatal(err) - } - if found { - t.Errorf("email relation %d was not expunged", target.ID) - } + if len(expunged) != 1 || expunged[0] != expungeSequence { + t.Fatalf("UID EXPUNGE sequences = %v, want [%d]", expunged, expungeSequence) } + var remaining models.UserEmail + found, err := db.Instance.ID(targets[0].ID).Get(&remaining) + if err != nil { + t.Fatal(err) + } + if found { + t.Errorf("marked email relation %d was not expunged", targets[0].ID) + } + remaining = models.UserEmail{} + found, err = db.Instance.ID(targets[1].ID).Get(&remaining) + if err != nil { + t.Fatal(err) + } + if !found { + t.Errorf("unmarked email relation %d was expunged", targets[1].ID) + } + + if _, err := clientLogin.Store(uidSet(imap.UID(targets[1].ID)), &imap.StoreFlags{ + Op: imap.StoreFlagsAdd, + Flags: []imap.Flag{imap.FlagDeleted}, + }, &imap.StoreOptions{}).Collect(); err != nil { + t.Fatalf("mark remaining message deleted: %v", err) + } + remainingSequence := assertDeletedFetchFlags(t, imap.UID(targets[1].ID)) + expunged, err = clientLogin.Expunge().Collect() + if err != nil { + t.Fatalf("EXPUNGE: %v", err) + } + if len(expunged) != 1 || expunged[0] != remainingSequence { + t.Fatalf("EXPUNGE sequences = %v, want [%d]", expunged, remainingSequence) + } + remaining = models.UserEmail{} + found, err = db.Instance.ID(targets[1].ID).Get(&remaining) + if err != nil { + t.Fatal(err) + } + if found { + t.Errorf("marked email relation %d was not expunged", targets[1].ID) + } +} + +func TestExpandIMAPUIDSet(t *testing.T) { + tests := []struct { + name string + set imap.UIDSet + maxUID int + want []int + wantErr bool + }{ + {name: "static", set: imap.UIDSet{{Start: 2, Stop: 4}}, want: []int{2, 3, 4}}, + {name: "last", set: imap.UIDSet{{Start: 0, Stop: 0}}, maxUID: 8, want: []int{8}}, + {name: "dynamic range", set: imap.UIDSet{{Start: 6, Stop: 0}}, maxUID: 8, want: []int{6, 7, 8}}, + {name: "dynamic range beyond maximum", set: imap.UIDSet{{Start: 10, Stop: 0}}, maxUID: 8, want: []int{8}}, + {name: "overlapping ranges", set: imap.UIDSet{{Start: 2, Stop: 4}, {Start: 3, Stop: 5}}, want: []int{2, 3, 4, 5}}, + {name: "maximum UID", set: imap.UIDSet{{Start: imap.UID(^uint32(0)), Stop: imap.UID(^uint32(0))}}, want: []int{int(^uint32(0))}}, + {name: "empty mailbox", set: imap.UIDSet{{Start: 0, Stop: 0}}, maxUID: 0, want: []int{}}, + {name: "too large", set: imap.UIDSet{{Start: 1, Stop: maxIMAPExpungeUIDs + 1}}, wantErr: true}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got, err := expandIMAPUIDSet(&tt.set, tt.maxUID) + if tt.wantErr { + if err == nil { + t.Fatal("expected an error") + } + return + } + if err != nil { + t.Fatalf("expand UID set: %v", err) + } + if !reflect.DeepEqual(got, tt.want) { + t.Fatalf("expanded UIDs = %v, want %v", got, tt.want) + } + }) + } +} + +func assertDeletedState(t *testing.T, relationID, userID int, expected int8) { + t.Helper() + var relation models.UserEmail + found, err := db.Instance.Where("id = ? AND user_id = ?", relationID, userID).Get(&relation) + if err != nil { + t.Fatalf("load relation deleted state: %v", err) + } + if !found || relation.IsDeleted != expected { + t.Fatalf("relation %d deleted state = %d, want %d", relationID, relation.IsDeleted, expected) + } +} + +func assertDeletedFetchFlags(t *testing.T, uid imap.UID) uint32 { + t.Helper() + uidMessages, err := clientLogin.Fetch(imap.UIDSetNum(uid), &imap.FetchOptions{Flags: true, UID: true}).Collect() + if err != nil { + t.Fatalf("UID FETCH deleted flags: %v", err) + } + if len(uidMessages) != 1 || uidMessages[0].UID != uid || !hasIMAPFlag(uidMessages[0].Flags, imap.FlagDeleted) { + t.Fatalf("UID FETCH flags = %+v, want \\Deleted for UID %d", uidMessages, uid) + } + sequenceMessages, err := clientLogin.Fetch(imap.SeqSetNum(uidMessages[0].SeqNum), &imap.FetchOptions{Flags: true, UID: true}).Collect() + if err != nil { + t.Fatalf("sequence FETCH deleted flags: %v", err) + } + if len(sequenceMessages) != 1 || sequenceMessages[0].UID != uid || !hasIMAPFlag(sequenceMessages[0].Flags, imap.FlagDeleted) { + t.Fatalf("sequence FETCH flags = %+v, want \\Deleted for UID %d", sequenceMessages, uid) + } + return uidMessages[0].SeqNum +} + +func hasIMAPFlag(flags []imap.Flag, expected imap.Flag) bool { + for _, flag := range flags { + if flag == expected { + return true + } + } + return false } func TestExamine(t *testing.T) { diff --git a/server/listen/imap_server/session_expunge.go b/server/listen/imap_server/session_expunge.go index 9f717b3..f2b5688 100644 --- a/server/listen/imap_server/session_expunge.go +++ b/server/listen/imap_server/session_expunge.go @@ -1,47 +1,154 @@ package imap_server import ( + "fmt" + "log/slog" + "sort" + "git.pcux.cn/xuans/Smail/services/del_email" + "git.pcux.cn/xuans/Smail/services/list" "github.com/emersion/go-imap/v2" "github.com/emersion/go-imap/v2/imapserver" "github.com/spf13/cast" - "log/slog" ) +const maxIMAPExpungeUIDs = 10000 + +type imapExpungeTarget struct { + uid int + sequence uint32 +} + func (s *serverSession) Expunge(w *imapserver.ExpungeWriter, uids *imap.UIDSet) error { - if uids == nil && len(s.deleteUidList) == 0 { + if s == nil || s.ctx == nil || s.ctx.UserID <= 0 { + return &imap.Error{Type: imap.StatusResponseTypeNo, Text: "Expunge failed"} + } + + maxUID := 0 + var err error + if uids != nil && uids.Dynamic() { + maxUID, err = list.GetMaxUIDByMailbox(s.ctx, s.currentMailbox) + if err != nil { + slog.Error("query IMAP maximum UID failed", "error", err) + return &imap.Error{Type: imap.StatusResponseTypeNo, Text: "Expunge failed"} + } + } + requestedUIDs, err := expandIMAPUIDSet(uids, maxUID) + if err != nil { + return &imap.Error{Type: imap.StatusResponseTypeBad, Text: err.Error()} + } + targets, err := markedDeletedMessages(s, requestedUIDs) + if err != nil { + slog.Error("query IMAP deleted messages failed", "error", err) + return &imap.Error{Type: imap.StatusResponseTypeNo, Text: "Expunge failed"} + } + if len(targets) == 0 { return nil } - uidList := []int{} - if uids != nil { - for _, uidRange := range *uids { - if uidRange.Start > 0 && uidRange.Stop > 0 { - for i := uidRange.Start; i <= uidRange.Stop; i++ { - uidList = append(uidList, cast.ToInt(uint32(i))) - } + uidList := make([]int, 0, len(targets)) + for _, target := range targets { + uidList = append(uidList, target.uid) + } + if err := del_email.DelByUID(s.ctx, uidList); err != nil { + slog.Error("delete IMAP messages failed", "error", err) + return &imap.Error{Type: imap.StatusResponseTypeNo, Text: "Expunge failed"} + } + + sort.Slice(targets, func(i, j int) bool { + return targets[i].sequence > targets[j].sequence + }) + for _, target := range targets { + if w != nil { + if err := w.WriteExpunge(target.sequence); err != nil { + return err } } } - - if len(s.deleteUidList) > 0 { - uidList = append(uidList, s.deleteUidList...) - } - - if len(uidList) == 0 { - return nil - } - - slog.Debug("DeleteUidList:", slog.Any("uidList", uidList)) - - err := del_email.DelByUID(s.ctx, uidList) - s.deleteUidList = []int{} - if err != nil { - return &imap.Error{ - Type: imap.StatusResponseTypeNo, - Text: err.Error(), - } - } - return nil } + +func expandIMAPUIDSet(uids *imap.UIDSet, maxUID int) ([]int, error) { + if uids == nil { + return nil, nil + } + result := make([]int, 0) + seen := make(map[int]struct{}) + appendUID := func(uid imap.UID) error { + value := cast.ToInt(uint32(uid)) + if value <= 0 { + return nil + } + if _, exists := seen[value]; exists { + return nil + } + if len(result) >= maxIMAPExpungeUIDs { + return fmt.Errorf("UID range is too large") + } + seen[value] = struct{}{} + result = append(result, value) + return nil + } + for _, uidRange := range *uids { + start := uidRange.Start + stop := uidRange.Stop + if start == 0 { + if maxUID > 0 { + if err := appendUID(imap.UID(maxUID)); err != nil { + return nil, err + } + } + continue + } + if stop == 0 { + if maxUID <= 0 { + continue + } + if imap.UID(maxUID) < start { + if err := appendUID(imap.UID(maxUID)); err != nil { + return nil, err + } + continue + } + stop = imap.UID(maxUID) + } + if stop < start { + return nil, fmt.Errorf("invalid UID range") + } + span := uint64(stop-start) + 1 + if span > maxIMAPExpungeUIDs { + return nil, fmt.Errorf("UID range is too large") + } + for value := start; ; value++ { + if err := appendUID(value); err != nil { + return nil, err + } + if value == stop { + break + } + } + } + return result, nil +} + +func markedDeletedMessages(s *serverSession, requested []int) ([]imapExpungeTarget, error) { + relations, err := list.GetUEListByUID(s.ctx, s.currentMailbox, 0, 0, requested) + if err != nil { + return nil, err + } + targets := make([]imapExpungeTarget, 0, len(relations)) + for _, relation := range relations { + if relation == nil || relation.ID <= 0 || relation.IsDeleted != 1 { + continue + } + sequence, err := imapUint32(relation.SerialNumber) + if err != nil || sequence == 0 { + return nil, fmt.Errorf("invalid message sequence number") + } + targets = append(targets, imapExpungeTarget{ + uid: relation.ID, + sequence: sequence, + }) + } + return targets, nil +} diff --git a/server/listen/imap_server/session_fetch.go b/server/listen/imap_server/session_fetch.go index 4474791..018496f 100644 --- a/server/listen/imap_server/session_fetch.go +++ b/server/listen/imap_server/session_fetch.go @@ -331,11 +331,14 @@ func write(ctx *context.Context, w *imapserver.FetchWriter, emailList []*respons writer.WriteRFC822Size(cast.ToInt64(len(emailContent))) } if options.Flags { + flags := make([]imap.Flag, 0, 2) if email.IsRead == 1 { - writer.WriteFlags([]imap.Flag{imap.FlagSeen}) - } else { - writer.WriteFlags([]imap.Flag{}) + flags = append(flags, imap.FlagSeen) } + if email.IsDeleted == 1 { + flags = append(flags, imap.FlagDeleted) + } + writer.WriteFlags(flags) } if options.InternalDate { writer.WriteInternalDate(email.CreateTime) diff --git a/server/listen/imap_server/session_select.go b/server/listen/imap_server/session_select.go index 785a016..5be9db9 100644 --- a/server/listen/imap_server/session_select.go +++ b/server/listen/imap_server/session_select.go @@ -37,8 +37,8 @@ func (s *serverSession) Select(mailbox string, options *imap.SelectOptions) (*im } ret := &imap.SelectData{ - Flags: []imap.Flag{imap.FlagSeen}, - PermanentFlags: []imap.Flag{imap.FlagSeen}, + Flags: []imap.Flag{imap.FlagSeen, imap.FlagDeleted}, + PermanentFlags: []imap.Flag{imap.FlagSeen, imap.FlagDeleted}, NumMessages: numMessages, UIDNext: uidNext, UIDValidity: uidValidity, diff --git a/server/listen/imap_server/session_store.go b/server/listen/imap_server/session_store.go index d28d90d..bb5e684 100644 --- a/server/listen/imap_server/session_store.go +++ b/server/listen/imap_server/session_store.go @@ -6,7 +6,6 @@ import ( "git.pcux.cn/xuans/Smail/db" "git.pcux.cn/xuans/Smail/dto/response" "git.pcux.cn/xuans/Smail/models" - "git.pcux.cn/xuans/Smail/services/del_email" "git.pcux.cn/xuans/Smail/services/list" "git.pcux.cn/xuans/Smail/utils/array" "github.com/emersion/go-imap/v2" @@ -56,16 +55,24 @@ func (s *serverSession) Store(w *imapserver.FetchWriter, numSet imap.NumSet, fla deletedRequested := array.InArray(imap.FlagDeleted, flags.Flags) updateReadState := false readState := false + updateDeletedState := false + deletedState := false switch flags.Op { case imap.StoreFlagsSet: updateReadState = true readState = seenRequested + updateDeletedState = true + deletedState = deletedRequested case imap.StoreFlagsAdd: updateReadState = seenRequested readState = true + updateDeletedState = deletedRequested + deletedState = true case imap.StoreFlagsDel: updateReadState = seenRequested readState = false + updateDeletedState = deletedRequested + deletedState = false default: return &imap.Error{Type: imap.StatusResponseTypeBad, Text: "Unsupported STORE operation"} } @@ -83,18 +90,16 @@ func (s *serverSession) Store(w *imapserver.FetchWriter, numSet imap.NumSet, fla } } - if deletedRequested && flags.Op == imap.StoreFlagsAdd { + if updateDeletedState { for _, data := range emailList { - s.deleteUidList = append(s.deleteUidList, data.UeId) - } - } - - if len(s.deleteUidList) > 0 { - err := del_email.DelByUID(s.ctx, s.deleteUidList) - s.deleteUidList = []int{} - if err != nil { - log.WithContext(s.ctx).WithError(err).Error("delete IMAP message failed") - return &imap.Error{Type: imap.StatusResponseTypeNo, Text: "Store failed"} + if err := updateDeletedByUID(s.ctx.UserID, data.UeId, deletedState); err != nil { + log.WithContext(s.ctx).WithError(err).Error("update IMAP message deleted flag failed") + return &imap.Error{Type: imap.StatusResponseTypeNo, Text: "Store failed"} + } + data.IsDeleted = 0 + if deletedState { + data.IsDeleted = 1 + } } } @@ -105,11 +110,14 @@ func (s *serverSession) Store(w *imapserver.FetchWriter, numSet imap.NumSet, fla return &imap.Error{Type: imap.StatusResponseTypeNo, Text: "Store failed"} } message := w.CreateMessage(sequence) + updatedFlags := make([]imap.Flag, 0, 2) if data.IsRead == 1 { - message.WriteFlags([]imap.Flag{imap.FlagSeen}) - } else { - message.WriteFlags(nil) + updatedFlags = append(updatedFlags, imap.FlagSeen) } + if data.IsDeleted == 1 { + updatedFlags = append(updatedFlags, imap.FlagDeleted) + } + message.WriteFlags(updatedFlags) if err := message.Close(); err != nil { return err } @@ -137,3 +145,22 @@ func updateReadByUID(userID, relationID int, read bool) error { } return nil } + +func updateDeletedByUID(userID, relationID int, deleted bool) error { + if userID <= 0 || relationID <= 0 { + return fmt.Errorf("invalid mailbox relation") + } + value := int8(0) + if deleted { + value = 1 + } + affected, err := db.Instance.Where("id = ? and user_id = ?", relationID, userID). + Cols("is_deleted").Update(&models.UserEmail{IsDeleted: value}) + if err != nil { + return fmt.Errorf("update mailbox relation deleted flag: %w", err) + } + if affected != 1 { + return fmt.Errorf("mailbox relation not found") + } + return nil +} diff --git a/server/models/user_email.go b/server/models/user_email.go index a500ad6..a27bab4 100644 --- a/server/models/user_email.go +++ b/server/models/user_email.go @@ -13,6 +13,7 @@ type UserEmail struct { CreatedAt time.Time `xorm:"created_at datetime created" json:"created_at"` UpdatedAt time.Time `xorm:"updated_at datetime updated" json:"updated_at"` IsRead int8 `xorm:"is_read tinyint(1) comment('是否已读')" json:"is_read"` + IsDeleted int8 `xorm:"is_deleted tinyint(1) notnull default(0)" json:"-"` GroupId int `xorm:"group_id int notnull default(0) comment('分组id')'" json:"group_id"` Status int8 `xorm:"status tinyint(4) notnull default(0) comment('0未发送或收件，1已发送，2发送失败，3删除 4草稿 5广告')" json:"status"` // 0未发送或收件，1已发送，2发送失败 3删除 4草稿箱(Drafts) 5骚扰邮件(Junk) Created time.Time `xorm:"create datetime created index('idx_create_time')"` diff --git a/server/services/del_email/del_email.go b/server/services/del_email/del_email.go index dc178aa..55cd719 100644 --- a/server/services/del_email/del_email.go +++ b/server/services/del_email/del_email.go @@ -6,7 +6,6 @@ import ( "git.pcux.cn/xuans/Smail/db" "git.pcux.cn/xuans/Smail/models" "git.pcux.cn/xuans/Smail/utils/context" - log "github.com/sirupsen/logrus" "github.com/spf13/cast" "log/slog" "xorm.io/xorm" @@ -80,14 +79,14 @@ func DelByUID(ctx *context.Context, ids []int) error { } for _, id := range ids { var ue models.UserEmail - found, err := session.Table("user_email").Where(Eq{"id": id, "user_id": ctx.UserID}).Get(&ue) + found, err := session.Table("user_email").Where(Eq{"id": id, "user_id": ctx.UserID, "is_deleted": 1}).Get(&ue) if err != nil { rollbackDeleteSession(session) return err } if !found || ue.ID == 0 { - log.WithContext(ctx).Warn("no user email found") - continue + rollbackDeleteSession(session) + return fmt.Errorf("email relation %d is missing or not marked deleted", id) } emailId := ue.EmailID diff --git a/server/services/del_email/del_email_test.go b/server/services/del_email/del_email_test.go index 2108aad..e1d9c3b 100644 --- a/server/services/del_email/del_email_test.go +++ b/server/services/del_email/del_email_test.go @@ -100,3 +100,54 @@ func TestForcedDeleteRemovesLastOwnedEmail(t *testing.T) { t.Fatal("unreferenced owned email was not deleted") } } + +func TestDelByUIDRequiresDeletedFlag(t *testing.T) { + initDeleteTestDB(t) + email := insertDeleteTestEmail(t) + relation := models.UserEmail{UserID: 1, EmailID: email.Id} + if _, err := db.Instance.Insert(&relation); err != nil { + t.Fatal(err) + } + + if err := DelByUID(&context.Context{UserID: 1}, []int{relation.ID}); err == nil { + t.Fatal("unmarked relation was accepted for expunge") + } + + found, err := db.Instance.ID(relation.ID).Get(&models.UserEmail{}) + if err != nil { + t.Fatal(err) + } + if !found { + t.Fatal("unmarked relation was deleted") + } +} + +func TestDelByUIDRollsBackBatchWhenRelationIsNotMarked(t *testing.T) { + initDeleteTestDB(t) + firstEmail := insertDeleteTestEmail(t) + secondEmail := insertDeleteTestEmail(t) + firstRelation := models.UserEmail{UserID: 1, EmailID: firstEmail.Id, IsDeleted: 1} + if _, err := db.Instance.Insert(&firstRelation); err != nil { + t.Fatal(err) + } + secondRelation := models.UserEmail{UserID: 1, EmailID: secondEmail.Id} + if _, err := db.Instance.Insert(&secondRelation); err != nil { + t.Fatal(err) + } + relations := []models.UserEmail{firstRelation, secondRelation} + + err := DelByUID(&context.Context{UserID: 1}, []int{relations[0].ID, relations[1].ID}) + if err == nil { + t.Fatal("mixed marked and unmarked batch was accepted") + } + + for _, relation := range relations { + found, loadErr := db.Instance.ID(relation.ID).Get(&models.UserEmail{}) + if loadErr != nil { + t.Fatal(loadErr) + } + if !found { + t.Fatalf("relation %d was not rolled back", relation.ID) + } + } +} diff --git a/server/services/list/list.go b/server/services/list/list.go index 5b87f01..2409db8 100644 --- a/server/services/list/list.go +++ b/server/services/list/list.go @@ -2,6 +2,8 @@ package list import ( "fmt" + "strings" + "git.pcux.cn/xuans/Smail/consts" "git.pcux.cn/xuans/Smail/db" "git.pcux.cn/xuans/Smail/dto" @@ -9,8 +11,8 @@ import ( "git.pcux.cn/xuans/Smail/models" "git.pcux.cn/xuans/Smail/utils/context" "git.pcux.cn/xuans/Smail/utils/query" + . "xorm.io/builder" ) -import . "xorm.io/builder" func GetEmailList(ctx *context.Context, tagInfo dto.SearchTag, keyword string, pop3List bool, offset, limit int) (emailList []*response.EmailResponseData, total int64, err error) { return getList(ctx, tagInfo, keyword, pop3List, offset, limit) @@ -127,153 +129,122 @@ type ImapListReq struct { End int } -func GetUEListByUID(ctx *context.Context, groupName string, star, end int, uidList []int) ([]*response.UserEmailUIDData, error) { - var ue []*response.UserEmailUIDData - sql := "SELECT id,email_id, is_read, ROW_NUMBER() OVER (ORDER BY id) AS serial_number FROM `user_email` WHERE user_id = ? " - +func buildIMAPMailboxRelationQuery(ctx *context.Context, groupName string) (string, []any, error) { + if ctx == nil || ctx.UserID <= 0 { + return "", nil, fmt.Errorf("invalid mailbox user") + } + inner := "SELECT id,email_id,is_read,is_deleted,ROW_NUMBER() OVER (ORDER BY id) AS serial_number FROM user_email WHERE user_id = ?" params := []any{ctx.UserID} + switch groupName { + case "INBOX": + inner += " and status = ? and group_id = 0" + params = append(params, 0) + case "Sent Messages": + inner += " and status = ? and group_id = 0" + params = append(params, 1) + case "Drafts": + inner += " and status = ? and group_id = 0" + params = append(params, 4) + case "Deleted Messages": + inner += " and status = ? and group_id = 0" + params = append(params, 3) + case "Junk": + inner += " and status = ? and group_id = 0" + params = append(params, 5) + default: + var mailbox models.Group + found, err := db.Instance.Table("group").Where("user_id=? and full_path=?", ctx.UserID, groupName).Get(&mailbox) + if err != nil { + return "", nil, fmt.Errorf("query mailbox: %w", err) + } + if !found || mailbox.ID == 0 { + return "", nil, fmt.Errorf("mailbox not found") + } + inner += " and group_id = ?" + params = append(params, mailbox.ID) + } + return inner, params, nil +} +func GetUEListByUID(ctx *context.Context, groupName string, star, end int, uidList []int) ([]*response.UserEmailUIDData, error) { + if star < 0 || end < 0 || (end > 0 && star > end) { + return nil, fmt.Errorf("invalid UID range") + } + inner, params, err := buildIMAPMailboxRelationQuery(ctx, groupName) + if err != nil { + return nil, err + } + filters := make([]string, 0, 3) if len(uidList) > 0 { placeholders, uidArgs, ok := query.IntList(uidList, 10000) if !ok { return nil, fmt.Errorf("invalid UID list") } - sql += fmt.Sprintf(" and id in (%s)", placeholders) + filters = append(filters, fmt.Sprintf("id in (%s)", placeholders)) params = append(params, uidArgs...) } if star > 0 { - sql += " and id >=?" + filters = append(filters, "id >= ?") params = append(params, star) } if end > 0 { - sql += " and id <=?" + filters = append(filters, "id <= ?") params = append(params, end) } - - switch groupName { - case "INBOX": - sql += " and status =? and group_id = 0" - params = append(params, 0) - case "Sent Messages": - sql += " and status =? and group_id = 0" - params = append(params, 1) - case "Drafts": - sql += " and status =? and group_id = 0" - params = append(params, 4) - case "Deleted Messages": - sql += " and status =? and group_id = 0" - params = append(params, 3) - case "Junk": - sql += " and status =? and group_id = 0" - params = append(params, 5) - default: - var group models.Group - found, err := db.Instance.Table("group").Where("user_id=? and full_path=?", ctx.UserID, groupName).Get(&group) - if err != nil { - return nil, err - } - if !found || group.ID == 0 { - return nil, nil - } - - sql += " and group_id = ?" - params = append(params, group.ID) + sql := "SELECT * FROM (" + inner + ") mailbox" + if len(filters) > 0 { + sql += " WHERE " + strings.Join(filters, " AND ") } - - if err := db.Instance.SQL(sql, params...).Find(&ue); err != nil { - return nil, err + sql += " ORDER BY id" + var relations []*response.UserEmailUIDData + if err := db.Instance.SQL(sql, params...).Find(&relations); err != nil { + return nil, fmt.Errorf("query mailbox UID relations: %w", err) } - return ue, nil + return relations, nil +} + +func GetMaxUIDByMailbox(ctx *context.Context, groupName string) (int, error) { + inner, params, err := buildIMAPMailboxRelationQuery(ctx, groupName) + if err != nil { + return 0, err + } + var maxUID int + if _, err := db.Instance.SQL("SELECT COALESCE(MAX(id), 0) FROM ("+inner+") mailbox", params...).Get(&maxUID); err != nil { + return 0, fmt.Errorf("query mailbox maximum UID: %w", err) + } + return maxUID, nil } func getEmailListByUidList(ctx *context.Context, groupName string, req ImapListReq) ([]*response.EmailResponseData, error) { - var ret []*response.EmailResponseData - var ue []*response.UserEmailUIDData if req.Star < 0 || req.End < 0 || (req.End > 0 && req.Star > req.End) { return nil, fmt.Errorf("invalid UID range") } - - buildQuery := func(filterColumn string, baseParams []any) (string, []any, bool) { - sql := "SELECT id,email_id,is_read,ROW_NUMBER() OVER (ORDER BY id) AS serial_number FROM `user_email` WHERE user_id = ?" - params := append([]any{}, baseParams...) - if filterColumn != "" { - sql += " and " + filterColumn + " = ?" - if filterColumn == "status" { - sql += " and group_id = 0" - } - } - if req.Star > 0 { - sql += " and id >= ?" - params = append(params, req.Star) - if req.End > 0 { - sql += " and id <= ?" - params = append(params, req.End) - } - return sql, params, true + inner, params, err := buildIMAPMailboxRelationQuery(ctx, groupName) + if err != nil { + return nil, err + } + sql := "SELECT * FROM (" + inner + ") mailbox WHERE " + if req.Star > 0 { + sql += "id >= ?" + params = append(params, req.Star) + if req.End > 0 { + sql += " and id <= ?" + params = append(params, req.End) } + } else { placeholders, uidArgs, ok := query.IntList(req.UidList, 10000) if !ok { - return "", nil, false + return nil, fmt.Errorf("invalid UID list") } - sql += fmt.Sprintf(" and id in (%s)", placeholders) + sql += fmt.Sprintf("id in (%s)", placeholders) params = append(params, uidArgs...) - return sql, params, true } - - var sql string - var params []any - var ok bool - switch groupName { - case "INBOX": - sql, params, ok = buildQuery("status", []any{ctx.UserID, 0}) - case "Sent Messages": - sql, params, ok = buildQuery("status", []any{ctx.UserID, 1}) - case "Drafts": - sql, params, ok = buildQuery("status", []any{ctx.UserID, 4}) - case "Deleted Messages": - sql, params, ok = buildQuery("status", []any{ctx.UserID, 3}) - case "Junk": - sql, params, ok = buildQuery("status", []any{ctx.UserID, 5}) - default: - var group models.Group - found, err := db.Instance.Table("group").Where("user_id=? and full_path=?", ctx.UserID, groupName).Get(&group) - if err != nil { - return nil, fmt.Errorf("query mailbox: %w", err) - } - if !found || group.ID == 0 { - return nil, fmt.Errorf("mailbox not found") - } - sql, params, ok = buildQuery("group_id", []any{ctx.UserID, group.ID}) - } - if !ok { - return nil, fmt.Errorf("invalid UID list") - } - if err := db.Instance.SQL(sql, params...).Find(&ue); err != nil { + sql += " ORDER BY id" + var relations []*response.UserEmailUIDData + if err := db.Instance.SQL(sql, params...).Find(&relations); err != nil { return nil, fmt.Errorf("query mailbox UID list: %w", err) } - - ueMap := map[int]*response.UserEmailUIDData{} - var emailIds []int - for _, email := range ue { - ueMap[email.EmailID] = email - emailIds = append(emailIds, email.EmailID) - } - if len(emailIds) == 0 { - return ret, nil - } - if err := db.Instance.Table("email").Select("*").Where(Eq{"id": emailIds}).Find(&ret); err != nil { - return nil, fmt.Errorf("query mailbox messages: %w", err) - } - for i, data := range ret { - item := ueMap[data.Id] - if item == nil { - continue - } - ret[i].IsRead = item.IsRead - ret[i].SerialNumber = item.SerialNumber - ret[i].UeId = item.ID - } - return ret, nil + return hydrateIMAPEmailResponses(relations) } func GetEmailListByGroup(ctx *context.Context, groupName string, req ImapListReq, uid bool) ([]*response.EmailResponseData, error) { @@ -286,41 +257,11 @@ func GetEmailListByGroup(ctx *context.Context, groupName string, req ImapListReq if req.Star < 0 || req.End < 0 || (req.End > 0 && req.Star > req.End) { return nil, fmt.Errorf("invalid sequence range") } - - var ret []*response.EmailResponseData - var ue []*response.UserEmailUIDData - inner := "SELECT id,email_id,is_read,ROW_NUMBER() OVER (ORDER BY id) AS serial_number FROM `user_email` WHERE user_id = ?" - params := []any{ctx.UserID} - switch groupName { - case "INBOX": - inner += " and status = ? and group_id = 0" - params = append(params, 0) - case "Sent Messages": - inner += " and status = ? and group_id = 0" - params = append(params, 1) - case "Drafts": - inner += " and status = ? and group_id = 0" - params = append(params, 4) - case "Deleted Messages": - inner += " and status = ? and group_id = 0" - params = append(params, 3) - case "Junk": - inner += " and status = ? and group_id = 0" - params = append(params, 5) - default: - var group models.Group - found, err := db.Instance.Table("group").Where("user_id=? and full_path=?", ctx.UserID, groupName).Get(&group) - if err != nil { - return nil, fmt.Errorf("query mailbox: %w", err) - } - if !found || group.ID == 0 { - return nil, fmt.Errorf("mailbox not found") - } - inner += " and group_id = ?" - params = append(params, group.ID) + inner, params, err := buildIMAPMailboxRelationQuery(ctx, groupName) + if err != nil { + return nil, err } - - sql := "SELECT * FROM (" + inner + ") a WHERE " + sql := "SELECT * FROM (" + inner + ") mailbox WHERE " if req.Star > 0 { sql += "serial_number >= ?" params = append(params, req.Star) @@ -336,30 +277,49 @@ func GetEmailListByGroup(ctx *context.Context, groupName string, req ImapListReq sql += fmt.Sprintf("serial_number in (%s)", placeholders) params = append(params, sequenceArgs...) } - if err := db.Instance.SQL(sql, params...).Find(&ue); err != nil { + sql += " ORDER BY serial_number" + var relations []*response.UserEmailUIDData + if err := db.Instance.SQL(sql, params...).Find(&relations); err != nil { return nil, fmt.Errorf("query mailbox sequence list: %w", err) } + return hydrateIMAPEmailResponses(relations) +} - ueMap := map[int]*response.UserEmailUIDData{} - var emailIds []int - for _, email := range ue { - ueMap[email.EmailID] = email - emailIds = append(emailIds, email.EmailID) +func hydrateIMAPEmailResponses(relations []*response.UserEmailUIDData) ([]*response.EmailResponseData, error) { + result := make([]*response.EmailResponseData, 0, len(relations)) + if len(relations) == 0 { + return result, nil } - if len(emailIds) == 0 { - return ret, nil - } - if err := db.Instance.Table("email").Select("*").Where(Eq{"id": emailIds}).Find(&ret); err != nil { - return nil, fmt.Errorf("query mailbox messages: %w", err) - } - for i, data := range ret { - item := ueMap[data.Id] - if item == nil { + emailIDs := make([]int, 0, len(relations)) + for _, relation := range relations { + if relation == nil || relation.EmailID <= 0 { continue } - ret[i].IsRead = item.IsRead - ret[i].SerialNumber = item.SerialNumber - ret[i].UeId = item.ID + emailIDs = append(emailIDs, relation.EmailID) } - return ret, nil + if len(emailIDs) == 0 { + return result, nil + } + var stored []*response.EmailResponseData + if err := db.Instance.Table("email").Select("*").Where(Eq{"id": emailIDs}).Find(&stored); err != nil { + return nil, fmt.Errorf("query mailbox messages: %w", err) + } + messageByID := make(map[int]*response.EmailResponseData, len(stored)) + for _, message := range stored { + if message != nil { + messageByID[message.Id] = message + } + } + for _, relation := range relations { + message := messageByID[relation.EmailID] + if message == nil { + continue + } + message.IsRead = relation.IsRead + message.IsDeleted = relation.IsDeleted + message.SerialNumber = relation.SerialNumber + message.UeId = relation.ID + result = append(result, message) + } + return result, nil }