完善IMAP删除语义并同步三份整改清单
Some checks are pending
Security Checks / security (push) Waiting to run

This commit is contained in:
smail 2026-08-31 10:54:08 +08:00
parent 99988bdebc
commit e9981cff70
16 changed files with 649 additions and 256 deletions

View File

@ -854,9 +854,19 @@ go vet ./...
## 2026-08-30 配置快照发布失败重试整改记录
- **本轮状态**?`[x] 已完成?等待复审`
- **实施内容**?统一配置启动加载?配置应用?配置回滚?数据库切换和插件 Secret 迁移的运行时快照发布入口?当磁盘事务已成功提交但运行时快照暂不可用时?保留可重试状态并将配置服务标记为降级?
- **接口提示**?新增稳定错误码 `CONFIG_RUNTIME_SNAPSHOT_UNAVAILABLE`?返回 HTTP 503?并明确提示配置已保存?运行时尚未切换?可稍后重试应用?
- **新增测试**?覆盖应用阶段快照发布失败后的磁盘代次?运行时旧快照?降级状态和再次应用成功?保留回滚票据失败重试及并发保护测试?
- **验证结果**?服务端 `go test ./... -count=1 -timeout 30m` 和 `go vet ./...` 通过?未将真实 MySQL?PostgreSQL?Ubuntu?Windows?浏览器和生产部署验收标记为完成?
- **遗留边界**?事务提交后快照发布失败的跨进程恢复仍依赖服务重载或后续重试?真实多数据库复制?校验?原子切换和部署联调继续按实机清单验收?
- **本轮状态**:`[x] 已完成,等待复审`
- **实施内容**:统一配置启动加载、配置应用、配置回滚、数据库切换和插件 Secret 迁移的运行时快照发布入口;当磁盘事务已成功提交但运行时快照暂不可用时,保留可重试状态并将配置服务标记为降级。
- **接口提示**:新增稳定错误码 `CONFIG_RUNTIME_SNAPSHOT_UNAVAILABLE`,返回 HTTP 503,并明确提示配置已保存、运行时尚未切换,可稍后重试应用。
- **新增测试**:覆盖应用阶段快照发布失败后的磁盘代次、运行时旧快照、降级状态和再次应用成功;保留回滚票据失败重试及并发保护测试。
- **验证结果**:服务端 `go test ./... -count=1 -timeout 30m` 和 `go vet ./...` 通过;未将真实 MySQL、PostgreSQL、Ubuntu、Windows、浏览器和生产部署验收标记为完成。
- **遗留边界**:事务提交后快照发布失败的跨进程恢复仍依赖服务重载或后续重试;真实多数据库复制、校验、原子切换和部署联调继续按实机清单验收。
## 2026-08-31 IMAP 删除语义增量复审
- **关联任务**:SEC-020,以及统一复审清单中的用户隔离、静态检查和回归测试项目。
- **状态**:`[x] 已完成,等待真实客户端和部署环境复审`。
- **实施内容**:为 `user_email` 增加持久化 `is_deleted` 标志和版本 14 数据库迁移;`SELECT`、UID/序列号 `FETCH` 和 `STORE` 统一返回或更新 `\Seen`、`\Deleted`;普通 `EXPUNGE` 仅清除当前邮箱中已标记删除的邮件,`UID EXPUNGE` 仅清除指定 UID 集合中已标记删除的邮件。
- **边界加固**:所有查询和更新同时限制当前用户与当前邮箱;修复 UID FETCH 返回错误序列号的问题;按序列号降序发送 EXPUNGE 响应;支持 `*` 和 `n:*` 动态 UID 范围,限制单次最多展开 10000 个 UID,防止超大范围占用资源。
- **修改范围**:`server/models/user_email.go`、`server/db/migration/imap_flags.go`、`server/services/list/list.go`、`server/services/del_email/del_email.go`、`server/listen/imap_server/session_*.go` 及相关测试。
- **自动化验证**:`go test ./services/list ./listen/imap_server ./db/migration ./services/del_email -count=1 -timeout 10m` 已通过;本轮最终提交前继续执行服务端全量测试、竞态测试、`go vet`、前端 ESLint 和生产构建。
- **开放项**:HSTS 与 Nginx 反向代理的实际上线行为、真实 IMAP 客户端兼容性、Ubuntu/Windows 部署和真实 MySQL/PostgreSQL 迁移仍需实机复审,不据此升级为最终发布通过。

View File

@ -1395,3 +1395,34 @@ AD-065 + CG-053~CG-056 → AD-066 → CG-057
- **本地验证**:服务端全量 `go test ./... -count=1` 通过;插件控制器和 HTTP 安全头专项测试通过。
- **未完成真实验收**:超级管理员本机重置通道仍需在 Ubuntu 22.04、Windows 部署端和实际权限模型中复审;邮箱验证码、SMTP 投递、WebAuthn、真实浏览器和多实例流程不得标记为最终通过。
- **复审日期**:2026-08-29(本机自动化复审)。
## 2026-08-31 全量代码状态复核
> 本节是 2026-08-31 对当前代码库的增量状态口径。原任务条目中的“未开始”保留为历史计划文本,不再代表当前实现状态;未完成真实平台或外部系统验收的任务不得标记为 `[✓]`。
| 任务范围 | 当前状态 | 复核结论 |
|---|---|---|
| AD-001~AD-008 | `[x] 已完成,等待复审` | 进程模型、权限矩阵、威胁模型、敏感字段、测试矩阵、独立管理程序、严格命令解析和部署管理员检查均已有实现与自动化测试。 |
| AD-009~AD-011 | `[x] 已完成,等待实机复审` | systemd、Windows 服务/ACL、版本绑定和升级约束已有代码或部署文件;真实 Ubuntu 与 Windows 服务环境尚未验收。 |
| AD-012~AD-021 | `[x] 已完成,等待实机复审` | 最小控制协议、UDS、Named Pipe、一次性 nonce、统一密码策略、事务重置、凭据撤销、审计及 Web 超级管理员恢复拒绝均已有实现和负向测试。 |
| AD-022~AD-025 | `[x] 已完成,等待复审` | 外部邮箱模型、加密存储、独立指纹、归属校验、脱敏查询和绑定失效事务已实现。 |
| AD-026~AD-031 | `[x] 已完成,等待复审` | 临时 HMAC 密钥、容量上限、验证码生成、用途绑定、恒定比较、原子消费、统一清理和重启失效已实现。 |
| AD-032~AD-037 | `[x] 已完成,等待外部系统复审` | SMTP Relay 接入、验证码不落盘投递、失败分类、多维限流、运营数据和 14 天分批清理已实现并具备自动化测试。 |
| AD-038 | `[!] 暂缓或存在阻塞` | 三数据库清理代码和方言已具备;真实 MySQL、PostgreSQL 的清理、空间复用和故障恢复尚未执行。 |
| AD-039~AD-059 | `[x] 已完成,等待浏览器与 SMTP 复审` | 统一登录标识、受限会话、二次验证策略、防枚举恢复、分角色恢复、一次性授权、V1 API、前端页面、审计和告警边界均已有实现。 |
| AD-060 | `[x] 已完成,等待复审` | 单元、并发和故障路径自动化测试已建立。 |
| AD-061~AD-064 | `[!] 暂缓或存在阻塞` | 真实三数据库、Ubuntu、Windows、真实浏览器三角色流程需要对应环境和人工交互验收。 |
| AD-065 | `[x] 已完成,等待独立安全复审` | 关键负向、重放、越权、枚举、降级和敏感数据测试已覆盖;生产观测链路仍需复核。 |
| AD-066 | `[!] 暂缓或存在阻塞` | 最终发布评审依赖 AD-038、AD-061~AD-064 以及真实 SMTP、多实例和生产安全审查。 |
### 当前实现证据
- 管理程序与本机控制通道:`server/cmd/smail-admin`、`server/services/adminreset`、`deploy/systemd`。
- 外部邮箱与验证码:`server/services/externalemail`、`server/services/verification`。
- 恢复和强认证:`server/services/recovery`、`server/services/webauthn`、`server/services/totp`、`server/services/credentials`。
- API 与前端:`server/controllers/security.go`、`server/controllers/webauthn.go`、`server/listen/http_server/api_v1_compat.go`、`fe/src/views` 下登录验证、密码恢复、账户安全和强制改密页面。
- 自动化复核:服务端全量、竞态和静态检查以及前端 ESLint、生产构建在本轮最终提交前统一执行并记录。
### 发布结论
三份清单要求的可在当前工作区完成的代码实现已进入待复审状态;真实 Ubuntu 22.04/24.04、Windows Service/ACL/Named Pipe、真实 SMTP Relay、MySQL、PostgreSQL、浏览器三角色、多实例和最终发布安全评审保持阻塞。超级管理员自身重置仍只允许受保护的 UDS 或 Named Pipe,本轮未增加 HTTP、localhost TCP、局域网或公网 TCP 重置入口。

View File

@ -1265,9 +1265,44 @@ CG-053~CG-056 → AD-066 → CG-057
## 2026-08-30 配置快照发布失败重试整改记录
- **本轮状态**?`[x] 已完成?等待复审`
- **实施内容**?统一配置启动加载?配置应用?配置回滚?数据库切换和插件 Secret 迁移的运行时快照发布入口?当磁盘事务已成功提交但运行时快照暂不可用时?保留可重试状态并将配置服务标记为降级?
- **接口提示**?新增稳定错误码 `CONFIG_RUNTIME_SNAPSHOT_UNAVAILABLE`?返回 HTTP 503?并明确提示配置已保存?运行时尚未切换?可稍后重试应用?
- **新增测试**?覆盖应用阶段快照发布失败后的磁盘代次?运行时旧快照?降级状态和再次应用成功?保留回滚票据失败重试及并发保护测试?
- **验证结果**?服务端 `go test ./... -count=1 -timeout 30m` 和 `go vet ./...` 通过?未将真实 MySQL?PostgreSQL?Ubuntu?Windows?浏览器和生产部署验收标记为完成?
- **遗留边界**?事务提交后快照发布失败的跨进程恢复仍依赖服务重载或后续重试?真实多数据库复制?校验?原子切换和部署联调继续按实机清单验收?
- **本轮状态**:`[x] 已完成,等待复审`
- **实施内容**:统一配置启动加载、配置应用、配置回滚、数据库切换和插件 Secret 迁移的运行时快照发布入口;当磁盘事务已成功提交但运行时快照暂不可用时,保留可重试状态并将配置服务标记为降级。
- **接口提示**:新增稳定错误码 `CONFIG_RUNTIME_SNAPSHOT_UNAVAILABLE`,返回 HTTP 503,并明确提示配置已保存、运行时尚未切换,可稍后重试应用。
- **新增测试**:覆盖应用阶段快照发布失败后的磁盘代次、运行时旧快照、降级状态和再次应用成功;保留回滚票据失败重试及并发保护测试。
- **验证结果**:服务端 `go test ./... -count=1 -timeout 30m` 和 `go vet ./...` 通过;未将真实 MySQL、PostgreSQL、Ubuntu、Windows、浏览器和生产部署验收标记为完成。
- **遗留边界**:事务提交后快照发布失败的跨进程恢复仍依赖服务重载或后续重试;真实多数据库复制、校验、原子切换和部署联调继续按实机清单验收。
## 2026-08-31 全量代码状态复核
> 本节是 2026-08-31 对当前实现的增量状态口径。原任务条目中的旧状态保留为实施历史;以下状态优先用于判断当前代码完成度,但不能替代真实平台和生产发布验收。
| 任务范围 | 当前状态 | 复核结论 |
|---|---|---|
| CG-001 | `[✓] 复审通过` | 需求、边界和依赖关系已经冻结。 |
| CG-002~CG-007 | `[x] 已完成,等待复审` | 敏感字段、显式迁移、测试基线、配置模型、严格解析和验证器均已实现。 |
| CG-008~CG-009 | `[✓] 复审通过` | CredentialVault 和不可变运行时快照已有测试证据。 |
| CG-010~CG-014 | `[x] 已完成,等待复审` | ConfigManager、事务恢复、备份回滚、状态/审计/受控重启、旧配置迁移均已有实现和自动化测试。 |
| CG-015 | `[x] 已完成,等待实机复审` | SSL 分区和受控存储已实现;真实证书、Nginx 和平台权限仍需验收。 |
| CG-016~CG-021 | `[✓] 复审通过` | 插件目录、授权、专属配置下发和三个插件源码整改已有本机自动化复审证据。 |
| CG-022~CG-044 | `[x] 已完成,等待复审` | SMTP/邮箱策略、三级角色、统一认证、密码、部署端重置、恢复仓储、密保、恢复码、WebAuthn、恢复状态机、API 和前端页面均已有实现。 |
| CG-045 | `[!] 暂缓或存在阻塞` | 真实浏览器三角色、可访问性和完整交互流程尚未人工验收。 |
| CG-046~CG-050 | `[x] 已完成,等待部署复审` | setupState、初始化授权、V1 兼容、功能开关和旧写路径收敛已有代码和测试。 |
| CG-051 | `[!] 暂缓或存在阻塞` | Windows、Ubuntu、MySQL、PostgreSQL 和 SQLite 真实部署矩阵尚未全部完成。 |
| CG-052~CG-055 | `[x] 已完成,等待独立复审` | 自动化回归、故障路径以及配置、认证、插件安全审查已形成代码和测试证据。 |
| CG-056~CG-057 | `[!] 暂缓或存在阻塞` | 浏览器数据泄露复审和最终发布验收依赖真实环境、人工交互及生产安全评审。 |
### 过时记录修正
旧记录中“真实数据库复制执行器尚未接入”的结论已经过时。当前已存在 `server/services/dbmigration/sql_executor.go`、`sqlite_executor.go`、运行时执行器工厂、复制校验、配置切换和相关测试。该修正只表示代码已接入,不表示真实 MySQL/PostgreSQL 数据复制、失败回滚和原子切换已经实机通过。
### 当前实现证据
- 配置核心:`server/config/model.go`、`parser.go`、`validator.go`、`secret.go`、`snapshot.go`、`manager.go`、`manager_v2.go`、`transaction.go`、`storage.go`。
- 备份、审计和迁移:`server/config/manager_backups.go`、`audit_store.go`、`audit_query.go`、`database_migration.go`。
- 数据库复制与切换:`server/services/dbmigration/service.go`、`sql_executor.go`、`sqlite_executor.go`。
- 受控重启和接口:`server/services/restart`、`server/controllers/config.go`、`config_audit.go`、`database_migration.go`、`restart.go`。
- 自动化复核:服务端全量测试、竞态检测、`go vet`、前端 ESLint 和生产构建在本轮最终提交前统一执行并记录。
### 发布结论
配置系统重构的工作区代码已进入待复审状态,但 CG-045、CG-051、CG-056 和 CG-057 仍受真实浏览器、跨平台、三数据库、多实例和生产发布评审阻塞。不得将“代码完成”表述为“生产验收完成”。

View File

@ -0,0 +1,19 @@
package migration
import (
"context"
"fmt"
)
func applyIMAPDeletedFlagSchema(ctx context.Context, exec executor, dialect Dialect) error {
definitions := map[Dialect]string{
DialectSQLite: `INTEGER NOT NULL DEFAULT 0`,
DialectMySQL: `TINYINT NOT NULL DEFAULT 0`,
DialectPostgres: `SMALLINT NOT NULL DEFAULT 0`,
}
definition, ok := definitions[dialect]
if !ok {
return fmt.Errorf("unsupported migration dialect %q", dialect)
}
return ensureColumn(ctx, exec, dialect, "user_email", "is_deleted", definition)
}

View File

@ -15,5 +15,6 @@ func DefaultMigrations() []Migration {
DefineForDialects(11, "create_plugin_configuration_tables", pluginSettingsStatements()),
DefineCustom(12, "expand_user_email_login_addresses", "expand_user_email_login_addresses_v1", applyUserEmailAddressSchema),
DefineForDialects(13, "create_protocol_credentials", protocolCredentialStatements()),
DefineCustom(14, "expand_imap_deleted_flag", "expand_imap_deleted_flag_v1", applyIMAPDeletedFlagSchema),
}
}

View File

@ -40,7 +40,7 @@ func TestDefaultMigrationsCreateCompleteSQLiteSchema(t *testing.T) {
if !exists {
t.Fatal("column webauthn_credential.discoverable was not created")
}
for _, column := range []string{"address", "address_canonical", "is_primary", "verified_at", "created_at", "updated_at"} {
for _, column := range []string{"address", "address_canonical", "is_primary", "verified_at", "created_at", "updated_at", "is_deleted"} {
exists, err := columnExists(context.Background(), db, DialectSQLite, "user_email", column)
if err != nil {
t.Fatal(err)
@ -102,7 +102,7 @@ func TestUserEmailLoginAddressMigrationUpgradesLegacyTableAndIsIdempotent(t *tes
if _, err := runner.Run(context.Background()); err != nil {
t.Fatal(err)
}
for _, column := range []string{"address", "address_canonical", "is_primary", "verified_at", "created_at", "updated_at"} {
for _, column := range []string{"address", "address_canonical", "is_primary", "verified_at", "created_at", "updated_at", "is_deleted"} {
exists, err := columnExists(context.Background(), db, DialectSQLite, "user_email", column)
if err != nil {
t.Fatal(err)
@ -144,6 +144,13 @@ func TestUserEmailLoginAddressMigrationRejectsDuplicateCanonicalAddresses(t *tes
}
}
func TestIMAPDeletedFlagMigrationRejectsUnexpectedDialect(t *testing.T) {
err := applyIMAPDeletedFlagSchema(context.Background(), nil, Dialect("unsupported"))
if err == nil || !strings.Contains(err.Error(), "unsupported") {
t.Fatalf("unexpected dialect error = %v", err)
}
}
func TestUserEmailLoginAddressMigrationRejectsUnexpectedDialect(t *testing.T) {
err := applyUserEmailAddressSchema(context.Background(), nil, Dialect("unsupported"))
if err == nil || !strings.Contains(err.Error(), "unsupported") {

View File

@ -5,6 +5,7 @@ import "git.pcux.cn/xuans/Smail/models"
type EmailResponseData struct {
models.Email `xorm:"extends"`
IsRead int8 `json:"is_read"`
IsDeleted int8 `json:"-"`
SerialNumber int `json:"serial_number"`
UeId int `json:"ue_id"`
}

View File

@ -12,6 +12,7 @@ import (
"mime"
"os"
"path/filepath"
"reflect"
"strings"
"testing"
"time"
@ -618,6 +619,32 @@ func TestStore(t *testing.T) {
}
assertReadState(t, relation.ID, userID, 0)
if _, err := clientLogin.Store(uidSet(uid), &imap.StoreFlags{
Op: imap.StoreFlagsAdd,
Flags: []imap.Flag{imap.FlagDeleted},
}, &imap.StoreOptions{}).Collect(); err != nil {
t.Fatalf("add \\Deleted: %v", err)
}
assertDeletedState(t, relation.ID, userID, 1)
if _, err := clientLogin.Store(uidSet(uid), &imap.StoreFlags{
Op: imap.StoreFlagsDel,
Silent: true,
Flags: []imap.Flag{imap.FlagDeleted},
}, &imap.StoreOptions{}).Collect(); err != nil {
t.Fatalf("delete \\Deleted: %v", err)
}
assertDeletedState(t, relation.ID, userID, 0)
if _, err := clientLogin.Store(uidSet(uid), &imap.StoreFlags{
Op: imap.StoreFlagsSet,
Flags: []imap.Flag{imap.FlagSeen, imap.FlagDeleted},
}, &imap.StoreOptions{}).Collect(); err != nil {
t.Fatalf("set supported flags: %v", err)
}
assertReadState(t, relation.ID, userID, 1)
assertDeletedState(t, relation.ID, userID, 1)
if _, err := clientLogin.Store(uidSet(uid), &imap.StoreFlags{
Op: imap.StoreFlagsSet,
Flags: nil,
@ -625,6 +652,7 @@ func TestStore(t *testing.T) {
t.Fatalf("set empty flags: %v", err)
}
assertReadState(t, relation.ID, userID, 0)
assertDeletedState(t, relation.ID, userID, 0)
}
func uidSet(uid imap.UID) imap.UIDSet {
@ -662,39 +690,152 @@ func TestExpunge(t *testing.T) {
t.Fatal(err)
}
userID := clientLoginUserID(t)
var targets []models.UserEmail
if err := db.Instance.Table("user_email").
Where("user_id = ? AND email_id > 0", 1).
Where("user_id = ? AND status = ? AND group_id = 0 AND email_id > 0 AND is_deleted = 0", userID, 0).
OrderBy("id ASC").
Limit(2).
Find(&targets); err != nil {
t.Fatal(err)
}
if len(targets) == 0 {
t.Fatal("no email relations available for expunge")
if len(targets) < 2 {
t.Fatal("not enough INBOX relations available for expunge")
}
uids := make([]imap.UID, 0, len(targets))
for _, target := range targets {
uids = append(uids, imap.UID(target.ID))
if _, err := clientLogin.Store(uidSet(imap.UID(targets[0].ID)), &imap.StoreFlags{
Op: imap.StoreFlagsAdd,
Flags: []imap.Flag{imap.FlagDeleted},
}, &imap.StoreOptions{}).Collect(); err != nil {
t.Fatalf("mark message deleted: %v", err)
}
res, err := clientLogin.UIDExpunge(imap.UIDSetNum(uids...)).Collect()
assertDeletedState(t, targets[0].ID, userID, 1)
assertDeletedState(t, targets[1].ID, userID, 0)
expungeSequence := assertDeletedFetchFlags(t, imap.UID(targets[0].ID))
expunged, err := clientLogin.UIDExpunge(imap.UIDSetNum(imap.UID(targets[0].ID), imap.UID(targets[1].ID))).Collect()
if err != nil {
t.Errorf("%+v", err)
t.Fatalf("UID EXPUNGE: %v", err)
}
t.Logf("%+v", res)
for _, target := range targets {
var remaining models.UserEmail
found, err := db.Instance.ID(target.ID).Get(&remaining)
if err != nil {
t.Fatal(err)
}
if found {
t.Errorf("email relation %d was not expunged", target.ID)
}
if len(expunged) != 1 || expunged[0] != expungeSequence {
t.Fatalf("UID EXPUNGE sequences = %v, want [%d]", expunged, expungeSequence)
}
var remaining models.UserEmail
found, err := db.Instance.ID(targets[0].ID).Get(&remaining)
if err != nil {
t.Fatal(err)
}
if found {
t.Errorf("marked email relation %d was not expunged", targets[0].ID)
}
remaining = models.UserEmail{}
found, err = db.Instance.ID(targets[1].ID).Get(&remaining)
if err != nil {
t.Fatal(err)
}
if !found {
t.Errorf("unmarked email relation %d was expunged", targets[1].ID)
}
if _, err := clientLogin.Store(uidSet(imap.UID(targets[1].ID)), &imap.StoreFlags{
Op: imap.StoreFlagsAdd,
Flags: []imap.Flag{imap.FlagDeleted},
}, &imap.StoreOptions{}).Collect(); err != nil {
t.Fatalf("mark remaining message deleted: %v", err)
}
remainingSequence := assertDeletedFetchFlags(t, imap.UID(targets[1].ID))
expunged, err = clientLogin.Expunge().Collect()
if err != nil {
t.Fatalf("EXPUNGE: %v", err)
}
if len(expunged) != 1 || expunged[0] != remainingSequence {
t.Fatalf("EXPUNGE sequences = %v, want [%d]", expunged, remainingSequence)
}
remaining = models.UserEmail{}
found, err = db.Instance.ID(targets[1].ID).Get(&remaining)
if err != nil {
t.Fatal(err)
}
if found {
t.Errorf("marked email relation %d was not expunged", targets[1].ID)
}
}
func TestExpandIMAPUIDSet(t *testing.T) {
tests := []struct {
name string
set imap.UIDSet
maxUID int
want []int
wantErr bool
}{
{name: "static", set: imap.UIDSet{{Start: 2, Stop: 4}}, want: []int{2, 3, 4}},
{name: "last", set: imap.UIDSet{{Start: 0, Stop: 0}}, maxUID: 8, want: []int{8}},
{name: "dynamic range", set: imap.UIDSet{{Start: 6, Stop: 0}}, maxUID: 8, want: []int{6, 7, 8}},
{name: "dynamic range beyond maximum", set: imap.UIDSet{{Start: 10, Stop: 0}}, maxUID: 8, want: []int{8}},
{name: "overlapping ranges", set: imap.UIDSet{{Start: 2, Stop: 4}, {Start: 3, Stop: 5}}, want: []int{2, 3, 4, 5}},
{name: "maximum UID", set: imap.UIDSet{{Start: imap.UID(^uint32(0)), Stop: imap.UID(^uint32(0))}}, want: []int{int(^uint32(0))}},
{name: "empty mailbox", set: imap.UIDSet{{Start: 0, Stop: 0}}, maxUID: 0, want: []int{}},
{name: "too large", set: imap.UIDSet{{Start: 1, Stop: maxIMAPExpungeUIDs + 1}}, wantErr: true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := expandIMAPUIDSet(&tt.set, tt.maxUID)
if tt.wantErr {
if err == nil {
t.Fatal("expected an error")
}
return
}
if err != nil {
t.Fatalf("expand UID set: %v", err)
}
if !reflect.DeepEqual(got, tt.want) {
t.Fatalf("expanded UIDs = %v, want %v", got, tt.want)
}
})
}
}
func assertDeletedState(t *testing.T, relationID, userID int, expected int8) {
t.Helper()
var relation models.UserEmail
found, err := db.Instance.Where("id = ? AND user_id = ?", relationID, userID).Get(&relation)
if err != nil {
t.Fatalf("load relation deleted state: %v", err)
}
if !found || relation.IsDeleted != expected {
t.Fatalf("relation %d deleted state = %d, want %d", relationID, relation.IsDeleted, expected)
}
}
func assertDeletedFetchFlags(t *testing.T, uid imap.UID) uint32 {
t.Helper()
uidMessages, err := clientLogin.Fetch(imap.UIDSetNum(uid), &imap.FetchOptions{Flags: true, UID: true}).Collect()
if err != nil {
t.Fatalf("UID FETCH deleted flags: %v", err)
}
if len(uidMessages) != 1 || uidMessages[0].UID != uid || !hasIMAPFlag(uidMessages[0].Flags, imap.FlagDeleted) {
t.Fatalf("UID FETCH flags = %+v, want \\Deleted for UID %d", uidMessages, uid)
}
sequenceMessages, err := clientLogin.Fetch(imap.SeqSetNum(uidMessages[0].SeqNum), &imap.FetchOptions{Flags: true, UID: true}).Collect()
if err != nil {
t.Fatalf("sequence FETCH deleted flags: %v", err)
}
if len(sequenceMessages) != 1 || sequenceMessages[0].UID != uid || !hasIMAPFlag(sequenceMessages[0].Flags, imap.FlagDeleted) {
t.Fatalf("sequence FETCH flags = %+v, want \\Deleted for UID %d", sequenceMessages, uid)
}
return uidMessages[0].SeqNum
}
func hasIMAPFlag(flags []imap.Flag, expected imap.Flag) bool {
for _, flag := range flags {
if flag == expected {
return true
}
}
return false
}
func TestExamine(t *testing.T) {

View File

@ -1,47 +1,154 @@
package imap_server
import (
"fmt"
"log/slog"
"sort"
"git.pcux.cn/xuans/Smail/services/del_email"
"git.pcux.cn/xuans/Smail/services/list"
"github.com/emersion/go-imap/v2"
"github.com/emersion/go-imap/v2/imapserver"
"github.com/spf13/cast"
"log/slog"
)
const maxIMAPExpungeUIDs = 10000
type imapExpungeTarget struct {
uid int
sequence uint32
}
func (s *serverSession) Expunge(w *imapserver.ExpungeWriter, uids *imap.UIDSet) error {
if uids == nil && len(s.deleteUidList) == 0 {
if s == nil || s.ctx == nil || s.ctx.UserID <= 0 {
return &imap.Error{Type: imap.StatusResponseTypeNo, Text: "Expunge failed"}
}
maxUID := 0
var err error
if uids != nil && uids.Dynamic() {
maxUID, err = list.GetMaxUIDByMailbox(s.ctx, s.currentMailbox)
if err != nil {
slog.Error("query IMAP maximum UID failed", "error", err)
return &imap.Error{Type: imap.StatusResponseTypeNo, Text: "Expunge failed"}
}
}
requestedUIDs, err := expandIMAPUIDSet(uids, maxUID)
if err != nil {
return &imap.Error{Type: imap.StatusResponseTypeBad, Text: err.Error()}
}
targets, err := markedDeletedMessages(s, requestedUIDs)
if err != nil {
slog.Error("query IMAP deleted messages failed", "error", err)
return &imap.Error{Type: imap.StatusResponseTypeNo, Text: "Expunge failed"}
}
if len(targets) == 0 {
return nil
}
uidList := []int{}
if uids != nil {
for _, uidRange := range *uids {
if uidRange.Start > 0 && uidRange.Stop > 0 {
for i := uidRange.Start; i <= uidRange.Stop; i++ {
uidList = append(uidList, cast.ToInt(uint32(i)))
}
uidList := make([]int, 0, len(targets))
for _, target := range targets {
uidList = append(uidList, target.uid)
}
if err := del_email.DelByUID(s.ctx, uidList); err != nil {
slog.Error("delete IMAP messages failed", "error", err)
return &imap.Error{Type: imap.StatusResponseTypeNo, Text: "Expunge failed"}
}
sort.Slice(targets, func(i, j int) bool {
return targets[i].sequence > targets[j].sequence
})
for _, target := range targets {
if w != nil {
if err := w.WriteExpunge(target.sequence); err != nil {
return err
}
}
}
if len(s.deleteUidList) > 0 {
uidList = append(uidList, s.deleteUidList...)
}
if len(uidList) == 0 {
return nil
}
slog.Debug("DeleteUidList:", slog.Any("uidList", uidList))
err := del_email.DelByUID(s.ctx, uidList)
s.deleteUidList = []int{}
if err != nil {
return &imap.Error{
Type: imap.StatusResponseTypeNo,
Text: err.Error(),
}
}
return nil
}
func expandIMAPUIDSet(uids *imap.UIDSet, maxUID int) ([]int, error) {
if uids == nil {
return nil, nil
}
result := make([]int, 0)
seen := make(map[int]struct{})
appendUID := func(uid imap.UID) error {
value := cast.ToInt(uint32(uid))
if value <= 0 {
return nil
}
if _, exists := seen[value]; exists {
return nil
}
if len(result) >= maxIMAPExpungeUIDs {
return fmt.Errorf("UID range is too large")
}
seen[value] = struct{}{}
result = append(result, value)
return nil
}
for _, uidRange := range *uids {
start := uidRange.Start
stop := uidRange.Stop
if start == 0 {
if maxUID > 0 {
if err := appendUID(imap.UID(maxUID)); err != nil {
return nil, err
}
}
continue
}
if stop == 0 {
if maxUID <= 0 {
continue
}
if imap.UID(maxUID) < start {
if err := appendUID(imap.UID(maxUID)); err != nil {
return nil, err
}
continue
}
stop = imap.UID(maxUID)
}
if stop < start {
return nil, fmt.Errorf("invalid UID range")
}
span := uint64(stop-start) + 1
if span > maxIMAPExpungeUIDs {
return nil, fmt.Errorf("UID range is too large")
}
for value := start; ; value++ {
if err := appendUID(value); err != nil {
return nil, err
}
if value == stop {
break
}
}
}
return result, nil
}
func markedDeletedMessages(s *serverSession, requested []int) ([]imapExpungeTarget, error) {
relations, err := list.GetUEListByUID(s.ctx, s.currentMailbox, 0, 0, requested)
if err != nil {
return nil, err
}
targets := make([]imapExpungeTarget, 0, len(relations))
for _, relation := range relations {
if relation == nil || relation.ID <= 0 || relation.IsDeleted != 1 {
continue
}
sequence, err := imapUint32(relation.SerialNumber)
if err != nil || sequence == 0 {
return nil, fmt.Errorf("invalid message sequence number")
}
targets = append(targets, imapExpungeTarget{
uid: relation.ID,
sequence: sequence,
})
}
return targets, nil
}

View File

@ -331,11 +331,14 @@ func write(ctx *context.Context, w *imapserver.FetchWriter, emailList []*respons
writer.WriteRFC822Size(cast.ToInt64(len(emailContent)))
}
if options.Flags {
flags := make([]imap.Flag, 0, 2)
if email.IsRead == 1 {
writer.WriteFlags([]imap.Flag{imap.FlagSeen})
} else {
writer.WriteFlags([]imap.Flag{})
flags = append(flags, imap.FlagSeen)
}
if email.IsDeleted == 1 {
flags = append(flags, imap.FlagDeleted)
}
writer.WriteFlags(flags)
}
if options.InternalDate {
writer.WriteInternalDate(email.CreateTime)

View File

@ -37,8 +37,8 @@ func (s *serverSession) Select(mailbox string, options *imap.SelectOptions) (*im
}
ret := &imap.SelectData{
Flags: []imap.Flag{imap.FlagSeen},
PermanentFlags: []imap.Flag{imap.FlagSeen},
Flags: []imap.Flag{imap.FlagSeen, imap.FlagDeleted},
PermanentFlags: []imap.Flag{imap.FlagSeen, imap.FlagDeleted},
NumMessages: numMessages,
UIDNext: uidNext,
UIDValidity: uidValidity,

View File

@ -6,7 +6,6 @@ import (
"git.pcux.cn/xuans/Smail/db"
"git.pcux.cn/xuans/Smail/dto/response"
"git.pcux.cn/xuans/Smail/models"
"git.pcux.cn/xuans/Smail/services/del_email"
"git.pcux.cn/xuans/Smail/services/list"
"git.pcux.cn/xuans/Smail/utils/array"
"github.com/emersion/go-imap/v2"
@ -56,16 +55,24 @@ func (s *serverSession) Store(w *imapserver.FetchWriter, numSet imap.NumSet, fla
deletedRequested := array.InArray(imap.FlagDeleted, flags.Flags)
updateReadState := false
readState := false
updateDeletedState := false
deletedState := false
switch flags.Op {
case imap.StoreFlagsSet:
updateReadState = true
readState = seenRequested
updateDeletedState = true
deletedState = deletedRequested
case imap.StoreFlagsAdd:
updateReadState = seenRequested
readState = true
updateDeletedState = deletedRequested
deletedState = true
case imap.StoreFlagsDel:
updateReadState = seenRequested
readState = false
updateDeletedState = deletedRequested
deletedState = false
default:
return &imap.Error{Type: imap.StatusResponseTypeBad, Text: "Unsupported STORE operation"}
}
@ -83,18 +90,16 @@ func (s *serverSession) Store(w *imapserver.FetchWriter, numSet imap.NumSet, fla
}
}
if deletedRequested && flags.Op == imap.StoreFlagsAdd {
if updateDeletedState {
for _, data := range emailList {
s.deleteUidList = append(s.deleteUidList, data.UeId)
}
}
if len(s.deleteUidList) > 0 {
err := del_email.DelByUID(s.ctx, s.deleteUidList)
s.deleteUidList = []int{}
if err != nil {
log.WithContext(s.ctx).WithError(err).Error("delete IMAP message failed")
return &imap.Error{Type: imap.StatusResponseTypeNo, Text: "Store failed"}
if err := updateDeletedByUID(s.ctx.UserID, data.UeId, deletedState); err != nil {
log.WithContext(s.ctx).WithError(err).Error("update IMAP message deleted flag failed")
return &imap.Error{Type: imap.StatusResponseTypeNo, Text: "Store failed"}
}
data.IsDeleted = 0
if deletedState {
data.IsDeleted = 1
}
}
}
@ -105,11 +110,14 @@ func (s *serverSession) Store(w *imapserver.FetchWriter, numSet imap.NumSet, fla
return &imap.Error{Type: imap.StatusResponseTypeNo, Text: "Store failed"}
}
message := w.CreateMessage(sequence)
updatedFlags := make([]imap.Flag, 0, 2)
if data.IsRead == 1 {
message.WriteFlags([]imap.Flag{imap.FlagSeen})
} else {
message.WriteFlags(nil)
updatedFlags = append(updatedFlags, imap.FlagSeen)
}
if data.IsDeleted == 1 {
updatedFlags = append(updatedFlags, imap.FlagDeleted)
}
message.WriteFlags(updatedFlags)
if err := message.Close(); err != nil {
return err
}
@ -137,3 +145,22 @@ func updateReadByUID(userID, relationID int, read bool) error {
}
return nil
}
func updateDeletedByUID(userID, relationID int, deleted bool) error {
if userID <= 0 || relationID <= 0 {
return fmt.Errorf("invalid mailbox relation")
}
value := int8(0)
if deleted {
value = 1
}
affected, err := db.Instance.Where("id = ? and user_id = ?", relationID, userID).
Cols("is_deleted").Update(&models.UserEmail{IsDeleted: value})
if err != nil {
return fmt.Errorf("update mailbox relation deleted flag: %w", err)
}
if affected != 1 {
return fmt.Errorf("mailbox relation not found")
}
return nil
}

View File

@ -13,6 +13,7 @@ type UserEmail struct {
CreatedAt time.Time `xorm:"created_at datetime created" json:"created_at"`
UpdatedAt time.Time `xorm:"updated_at datetime updated" json:"updated_at"`
IsRead int8 `xorm:"is_read tinyint(1) comment('是否已读')" json:"is_read"`
IsDeleted int8 `xorm:"is_deleted tinyint(1) notnull default(0)" json:"-"`
GroupId int `xorm:"group_id int notnull default(0) comment('分组id')'" json:"group_id"`
Status int8 `xorm:"status tinyint(4) notnull default(0) comment('0未发送或收件,1已发送,2发送失败,3删除 4草稿 5广告')" json:"status"` // 0未发送或收件,1已发送,2发送失败 3删除 4草稿箱(Drafts) 5骚扰邮件(Junk)
Created time.Time `xorm:"create datetime created index('idx_create_time')"`

View File

@ -6,7 +6,6 @@ import (
"git.pcux.cn/xuans/Smail/db"
"git.pcux.cn/xuans/Smail/models"
"git.pcux.cn/xuans/Smail/utils/context"
log "github.com/sirupsen/logrus"
"github.com/spf13/cast"
"log/slog"
"xorm.io/xorm"
@ -80,14 +79,14 @@ func DelByUID(ctx *context.Context, ids []int) error {
}
for _, id := range ids {
var ue models.UserEmail
found, err := session.Table("user_email").Where(Eq{"id": id, "user_id": ctx.UserID}).Get(&ue)
found, err := session.Table("user_email").Where(Eq{"id": id, "user_id": ctx.UserID, "is_deleted": 1}).Get(&ue)
if err != nil {
rollbackDeleteSession(session)
return err
}
if !found || ue.ID == 0 {
log.WithContext(ctx).Warn("no user email found")
continue
rollbackDeleteSession(session)
return fmt.Errorf("email relation %d is missing or not marked deleted", id)
}
emailId := ue.EmailID

View File

@ -100,3 +100,54 @@ func TestForcedDeleteRemovesLastOwnedEmail(t *testing.T) {
t.Fatal("unreferenced owned email was not deleted")
}
}
func TestDelByUIDRequiresDeletedFlag(t *testing.T) {
initDeleteTestDB(t)
email := insertDeleteTestEmail(t)
relation := models.UserEmail{UserID: 1, EmailID: email.Id}
if _, err := db.Instance.Insert(&relation); err != nil {
t.Fatal(err)
}
if err := DelByUID(&context.Context{UserID: 1}, []int{relation.ID}); err == nil {
t.Fatal("unmarked relation was accepted for expunge")
}
found, err := db.Instance.ID(relation.ID).Get(&models.UserEmail{})
if err != nil {
t.Fatal(err)
}
if !found {
t.Fatal("unmarked relation was deleted")
}
}
func TestDelByUIDRollsBackBatchWhenRelationIsNotMarked(t *testing.T) {
initDeleteTestDB(t)
firstEmail := insertDeleteTestEmail(t)
secondEmail := insertDeleteTestEmail(t)
firstRelation := models.UserEmail{UserID: 1, EmailID: firstEmail.Id, IsDeleted: 1}
if _, err := db.Instance.Insert(&firstRelation); err != nil {
t.Fatal(err)
}
secondRelation := models.UserEmail{UserID: 1, EmailID: secondEmail.Id}
if _, err := db.Instance.Insert(&secondRelation); err != nil {
t.Fatal(err)
}
relations := []models.UserEmail{firstRelation, secondRelation}
err := DelByUID(&context.Context{UserID: 1}, []int{relations[0].ID, relations[1].ID})
if err == nil {
t.Fatal("mixed marked and unmarked batch was accepted")
}
for _, relation := range relations {
found, loadErr := db.Instance.ID(relation.ID).Get(&models.UserEmail{})
if loadErr != nil {
t.Fatal(loadErr)
}
if !found {
t.Fatalf("relation %d was not rolled back", relation.ID)
}
}
}

View File

@ -2,6 +2,8 @@ package list
import (
"fmt"
"strings"
"git.pcux.cn/xuans/Smail/consts"
"git.pcux.cn/xuans/Smail/db"
"git.pcux.cn/xuans/Smail/dto"
@ -9,8 +11,8 @@ import (
"git.pcux.cn/xuans/Smail/models"
"git.pcux.cn/xuans/Smail/utils/context"
"git.pcux.cn/xuans/Smail/utils/query"
. "xorm.io/builder"
)
import . "xorm.io/builder"
func GetEmailList(ctx *context.Context, tagInfo dto.SearchTag, keyword string, pop3List bool, offset, limit int) (emailList []*response.EmailResponseData, total int64, err error) {
return getList(ctx, tagInfo, keyword, pop3List, offset, limit)
@ -127,153 +129,122 @@ type ImapListReq struct {
End int
}
func GetUEListByUID(ctx *context.Context, groupName string, star, end int, uidList []int) ([]*response.UserEmailUIDData, error) {
var ue []*response.UserEmailUIDData
sql := "SELECT id,email_id, is_read, ROW_NUMBER() OVER (ORDER BY id) AS serial_number FROM `user_email` WHERE user_id = ? "
func buildIMAPMailboxRelationQuery(ctx *context.Context, groupName string) (string, []any, error) {
if ctx == nil || ctx.UserID <= 0 {
return "", nil, fmt.Errorf("invalid mailbox user")
}
inner := "SELECT id,email_id,is_read,is_deleted,ROW_NUMBER() OVER (ORDER BY id) AS serial_number FROM user_email WHERE user_id = ?"
params := []any{ctx.UserID}
switch groupName {
case "INBOX":
inner += " and status = ? and group_id = 0"
params = append(params, 0)
case "Sent Messages":
inner += " and status = ? and group_id = 0"
params = append(params, 1)
case "Drafts":
inner += " and status = ? and group_id = 0"
params = append(params, 4)
case "Deleted Messages":
inner += " and status = ? and group_id = 0"
params = append(params, 3)
case "Junk":
inner += " and status = ? and group_id = 0"
params = append(params, 5)
default:
var mailbox models.Group
found, err := db.Instance.Table("group").Where("user_id=? and full_path=?", ctx.UserID, groupName).Get(&mailbox)
if err != nil {
return "", nil, fmt.Errorf("query mailbox: %w", err)
}
if !found || mailbox.ID == 0 {
return "", nil, fmt.Errorf("mailbox not found")
}
inner += " and group_id = ?"
params = append(params, mailbox.ID)
}
return inner, params, nil
}
func GetUEListByUID(ctx *context.Context, groupName string, star, end int, uidList []int) ([]*response.UserEmailUIDData, error) {
if star < 0 || end < 0 || (end > 0 && star > end) {
return nil, fmt.Errorf("invalid UID range")
}
inner, params, err := buildIMAPMailboxRelationQuery(ctx, groupName)
if err != nil {
return nil, err
}
filters := make([]string, 0, 3)
if len(uidList) > 0 {
placeholders, uidArgs, ok := query.IntList(uidList, 10000)
if !ok {
return nil, fmt.Errorf("invalid UID list")
}
sql += fmt.Sprintf(" and id in (%s)", placeholders)
filters = append(filters, fmt.Sprintf("id in (%s)", placeholders))
params = append(params, uidArgs...)
}
if star > 0 {
sql += " and id >=?"
filters = append(filters, "id >= ?")
params = append(params, star)
}
if end > 0 {
sql += " and id <=?"
filters = append(filters, "id <= ?")
params = append(params, end)
}
switch groupName {
case "INBOX":
sql += " and status =? and group_id = 0"
params = append(params, 0)
case "Sent Messages":
sql += " and status =? and group_id = 0"
params = append(params, 1)
case "Drafts":
sql += " and status =? and group_id = 0"
params = append(params, 4)
case "Deleted Messages":
sql += " and status =? and group_id = 0"
params = append(params, 3)
case "Junk":
sql += " and status =? and group_id = 0"
params = append(params, 5)
default:
var group models.Group
found, err := db.Instance.Table("group").Where("user_id=? and full_path=?", ctx.UserID, groupName).Get(&group)
if err != nil {
return nil, err
}
if !found || group.ID == 0 {
return nil, nil
}
sql += " and group_id = ?"
params = append(params, group.ID)
sql := "SELECT * FROM (" + inner + ") mailbox"
if len(filters) > 0 {
sql += " WHERE " + strings.Join(filters, " AND ")
}
if err := db.Instance.SQL(sql, params...).Find(&ue); err != nil {
return nil, err
sql += " ORDER BY id"
var relations []*response.UserEmailUIDData
if err := db.Instance.SQL(sql, params...).Find(&relations); err != nil {
return nil, fmt.Errorf("query mailbox UID relations: %w", err)
}
return ue, nil
return relations, nil
}
func GetMaxUIDByMailbox(ctx *context.Context, groupName string) (int, error) {
inner, params, err := buildIMAPMailboxRelationQuery(ctx, groupName)
if err != nil {
return 0, err
}
var maxUID int
if _, err := db.Instance.SQL("SELECT COALESCE(MAX(id), 0) FROM ("+inner+") mailbox", params...).Get(&maxUID); err != nil {
return 0, fmt.Errorf("query mailbox maximum UID: %w", err)
}
return maxUID, nil
}
func getEmailListByUidList(ctx *context.Context, groupName string, req ImapListReq) ([]*response.EmailResponseData, error) {
var ret []*response.EmailResponseData
var ue []*response.UserEmailUIDData
if req.Star < 0 || req.End < 0 || (req.End > 0 && req.Star > req.End) {
return nil, fmt.Errorf("invalid UID range")
}
buildQuery := func(filterColumn string, baseParams []any) (string, []any, bool) {
sql := "SELECT id,email_id,is_read,ROW_NUMBER() OVER (ORDER BY id) AS serial_number FROM `user_email` WHERE user_id = ?"
params := append([]any{}, baseParams...)
if filterColumn != "" {
sql += " and " + filterColumn + " = ?"
if filterColumn == "status" {
sql += " and group_id = 0"
}
}
if req.Star > 0 {
sql += " and id >= ?"
params = append(params, req.Star)
if req.End > 0 {
sql += " and id <= ?"
params = append(params, req.End)
}
return sql, params, true
inner, params, err := buildIMAPMailboxRelationQuery(ctx, groupName)
if err != nil {
return nil, err
}
sql := "SELECT * FROM (" + inner + ") mailbox WHERE "
if req.Star > 0 {
sql += "id >= ?"
params = append(params, req.Star)
if req.End > 0 {
sql += " and id <= ?"
params = append(params, req.End)
}
} else {
placeholders, uidArgs, ok := query.IntList(req.UidList, 10000)
if !ok {
return "", nil, false
return nil, fmt.Errorf("invalid UID list")
}
sql += fmt.Sprintf(" and id in (%s)", placeholders)
sql += fmt.Sprintf("id in (%s)", placeholders)
params = append(params, uidArgs...)
return sql, params, true
}
var sql string
var params []any
var ok bool
switch groupName {
case "INBOX":
sql, params, ok = buildQuery("status", []any{ctx.UserID, 0})
case "Sent Messages":
sql, params, ok = buildQuery("status", []any{ctx.UserID, 1})
case "Drafts":
sql, params, ok = buildQuery("status", []any{ctx.UserID, 4})
case "Deleted Messages":
sql, params, ok = buildQuery("status", []any{ctx.UserID, 3})
case "Junk":
sql, params, ok = buildQuery("status", []any{ctx.UserID, 5})
default:
var group models.Group
found, err := db.Instance.Table("group").Where("user_id=? and full_path=?", ctx.UserID, groupName).Get(&group)
if err != nil {
return nil, fmt.Errorf("query mailbox: %w", err)
}
if !found || group.ID == 0 {
return nil, fmt.Errorf("mailbox not found")
}
sql, params, ok = buildQuery("group_id", []any{ctx.UserID, group.ID})
}
if !ok {
return nil, fmt.Errorf("invalid UID list")
}
if err := db.Instance.SQL(sql, params...).Find(&ue); err != nil {
sql += " ORDER BY id"
var relations []*response.UserEmailUIDData
if err := db.Instance.SQL(sql, params...).Find(&relations); err != nil {
return nil, fmt.Errorf("query mailbox UID list: %w", err)
}
ueMap := map[int]*response.UserEmailUIDData{}
var emailIds []int
for _, email := range ue {
ueMap[email.EmailID] = email
emailIds = append(emailIds, email.EmailID)
}
if len(emailIds) == 0 {
return ret, nil
}
if err := db.Instance.Table("email").Select("*").Where(Eq{"id": emailIds}).Find(&ret); err != nil {
return nil, fmt.Errorf("query mailbox messages: %w", err)
}
for i, data := range ret {
item := ueMap[data.Id]
if item == nil {
continue
}
ret[i].IsRead = item.IsRead
ret[i].SerialNumber = item.SerialNumber
ret[i].UeId = item.ID
}
return ret, nil
return hydrateIMAPEmailResponses(relations)
}
func GetEmailListByGroup(ctx *context.Context, groupName string, req ImapListReq, uid bool) ([]*response.EmailResponseData, error) {
@ -286,41 +257,11 @@ func GetEmailListByGroup(ctx *context.Context, groupName string, req ImapListReq
if req.Star < 0 || req.End < 0 || (req.End > 0 && req.Star > req.End) {
return nil, fmt.Errorf("invalid sequence range")
}
var ret []*response.EmailResponseData
var ue []*response.UserEmailUIDData
inner := "SELECT id,email_id,is_read,ROW_NUMBER() OVER (ORDER BY id) AS serial_number FROM `user_email` WHERE user_id = ?"
params := []any{ctx.UserID}
switch groupName {
case "INBOX":
inner += " and status = ? and group_id = 0"
params = append(params, 0)
case "Sent Messages":
inner += " and status = ? and group_id = 0"
params = append(params, 1)
case "Drafts":
inner += " and status = ? and group_id = 0"
params = append(params, 4)
case "Deleted Messages":
inner += " and status = ? and group_id = 0"
params = append(params, 3)
case "Junk":
inner += " and status = ? and group_id = 0"
params = append(params, 5)
default:
var group models.Group
found, err := db.Instance.Table("group").Where("user_id=? and full_path=?", ctx.UserID, groupName).Get(&group)
if err != nil {
return nil, fmt.Errorf("query mailbox: %w", err)
}
if !found || group.ID == 0 {
return nil, fmt.Errorf("mailbox not found")
}
inner += " and group_id = ?"
params = append(params, group.ID)
inner, params, err := buildIMAPMailboxRelationQuery(ctx, groupName)
if err != nil {
return nil, err
}
sql := "SELECT * FROM (" + inner + ") a WHERE "
sql := "SELECT * FROM (" + inner + ") mailbox WHERE "
if req.Star > 0 {
sql += "serial_number >= ?"
params = append(params, req.Star)
@ -336,30 +277,49 @@ func GetEmailListByGroup(ctx *context.Context, groupName string, req ImapListReq
sql += fmt.Sprintf("serial_number in (%s)", placeholders)
params = append(params, sequenceArgs...)
}
if err := db.Instance.SQL(sql, params...).Find(&ue); err != nil {
sql += " ORDER BY serial_number"
var relations []*response.UserEmailUIDData
if err := db.Instance.SQL(sql, params...).Find(&relations); err != nil {
return nil, fmt.Errorf("query mailbox sequence list: %w", err)
}
return hydrateIMAPEmailResponses(relations)
}
ueMap := map[int]*response.UserEmailUIDData{}
var emailIds []int
for _, email := range ue {
ueMap[email.EmailID] = email
emailIds = append(emailIds, email.EmailID)
func hydrateIMAPEmailResponses(relations []*response.UserEmailUIDData) ([]*response.EmailResponseData, error) {
result := make([]*response.EmailResponseData, 0, len(relations))
if len(relations) == 0 {
return result, nil
}
if len(emailIds) == 0 {
return ret, nil
}
if err := db.Instance.Table("email").Select("*").Where(Eq{"id": emailIds}).Find(&ret); err != nil {
return nil, fmt.Errorf("query mailbox messages: %w", err)
}
for i, data := range ret {
item := ueMap[data.Id]
if item == nil {
emailIDs := make([]int, 0, len(relations))
for _, relation := range relations {
if relation == nil || relation.EmailID <= 0 {
continue
}
ret[i].IsRead = item.IsRead
ret[i].SerialNumber = item.SerialNumber
ret[i].UeId = item.ID
emailIDs = append(emailIDs, relation.EmailID)
}
return ret, nil
if len(emailIDs) == 0 {
return result, nil
}
var stored []*response.EmailResponseData
if err := db.Instance.Table("email").Select("*").Where(Eq{"id": emailIDs}).Find(&stored); err != nil {
return nil, fmt.Errorf("query mailbox messages: %w", err)
}
messageByID := make(map[int]*response.EmailResponseData, len(stored))
for _, message := range stored {
if message != nil {
messageByID[message.Id] = message
}
}
for _, relation := range relations {
message := messageByID[relation.EmailID]
if message == nil {
continue
}
message.IsRead = relation.IsRead
message.IsDeleted = relation.IsDeleted
message.SerialNumber = relation.SerialNumber
message.UeId = relation.ID
result = append(result, message)
}
return result, nil
}