完成完整候选反代隔离验收并补齐收尾回归证据
Some checks are pending
Security Checks / security (push) Waiting to run

This commit is contained in:
smail 2026-09-18 02:17:40 +08:00
parent 1d4b9c4135
commit cfa41e1efe
6 changed files with 399 additions and 17 deletions

View File

@ -0,0 +1,72 @@
# 完整候选反代隔离验收记录
## 范围与结论
远程主机为 `47.108.78.12`,主机名 `ikun2`。本轮从远程时间 2026 年 9 月 18 日 01:51 开始,仅在远程执行应用和自动测试;本机用于编辑、Git 和文件传输。所有业务流程使用新建隔离实例及空数据库,不清理或复用线上业务数据,不删除历史证据,不暂停线上服务。
- [✓] 完整候选六个 Linux 可执行文件上传后逐一校验长度、SHA-256、ELF64 和 x86_64 标识。
- [✓] 真实独立 Nginx 反代到 HTTPS 8002,HTTP 8001 初始化和运行端口通过;通过正式配置 API 设置公网 HTTPS 443 和可信代理范围。
- [✓] 公网 80 及应用 8001 跳转均保留路径和查询参数,不携带内部 8002;匿名恢复入口、页面登录地址和实际生产入口脚本可用。
- [✓] 两个真实连接来源分别验证登录限流;被限制来源伪造转发头仍被拒绝,另一来源能够正常登录。
- [✓] 空库首次安装、登录会话、SMTP/IMAP/POP3 认证、内部 25 STARTTLS 中文正文及附件收信、混合认证过载后的恢复通过;双库完整性正常,线上不存在测试邮件。
- [✓] 测试进程与线上独立网络、挂载隔离;最终反代测试四个线上服务 PID、InvocationID 和重启计数前后一致,没有新增内核 OOM。
- [✓] 6 项测试收尾模拟回归通过,覆盖资源查询超时、停止及强制终止失败后继续处理其余测试单元、拒绝业务服务名。
- [!] 最新应用尚未替换线上版本。这是隔离候选验收,不是正式部署、真实浏览器闭环、可信 CA 正式换证或整体生产放行。
## 候选组成
应用源码为 `c1dd533c805cbfda7a366a1170d2e710f04f86a1`,版本 `review-c1dd533`。本轮验收工具修改前提交并推送中文基线 `1d4b9c4`,该基线不改变应用源码。
主前端采用上一轮远程生产构建,详见《远程前端构建验收记录》。Telegram 界面也由同一版本归档源码远程构建,其 16 个输入文件已在前序构建阶段核对。依赖安装日志位于 `/var/tmp/smail-review-telegram-build-20260918/deps-0140.txt`,最终构建日志为 `build-0143.txt`;实际 Node 路径修正后 `vue-tsc --build`、Vite 均通过,构建峰值约 395 MiB、交换空间为零。首次私有临时目录造成的启动失败日志保留。
Linux 候选由干净源码归档、上述新界面产物及重新编译的三个插件组成,主程序使用 `bundled` 标签;前序交叉编译使用 Go 1.26.6、`CGO_ENABLED=0`、`GOOS=linux`、`GOARCH=amd64`、`-mod=readonly -trimpath`。本轮未再次编译或更改应用源码。
本机产物位于 `out/release-review-c1dd533-linux-amd64`,远程位于 root 所有的 `0700` 目录 `/var/tmp/smail-review-candidate-c1dd533`。`receipt.json` 记录六个文件的长度和摘要。
| 项目 | SHA-256 |
| --- | --- |
| 主程序 | `b2ceda68f26e3753be7124d11ce5bc06e309a119dd13ef7254bf403c3e0a4792` |
| 管理工具 | `2a99f6bef10092b72d550e195141a572ec02fae541fb3615d3b529bd15a193eb` |
| 配置迁移工具 | `b2016a293798c82190f7334b623237b3f460816e67ae9bdb2d592ea6a4ef8c55` |
| 主前端归档 | `91f775c666e01a24e47b7ac35132ac171a087aac9641c5bffb3456113d311f01` |
| Telegram 界面归档 | `07f05f902dbe58c7e49d757b53c3f47c4a6c1442ffd4cc7c83db56af03d061de` |
首次管理工具版本断言将额外的根目录输出误判为版本不匹配,在运行业务测试前退出;随后按实际版本输出行核对为 `review-c1dd533`。主程序没有被当作支持 `--version` 的命令直接在宿主环境启动。
## 反代验收方式
`test/tools/runtime_review.py` 新增可选 `--nginx-template`,保持原有直连默认路径。`runtime_proxy.py` 仅替换部署模板中的测试域名和证书、临时文件路径;保留 `proxy_pass https://127.0.0.1:8002`、上游证书校验、SNI 和覆盖客户端转发头的配置。模板摘要为 `bf6816c38f9c639ab17fd7344fff4b06afc5a2388f74318e18c60b8b3311ebd8`。
证书由隔离测试创建,仅供当前客户端及测试 Nginx 信任,不写入宿主信任库,不复制正式私钥。公网端口指隔离网络中的 80/443,不对实际互联网开放。应用、探针、Nginx 的内存硬上限分别为 128/48/32 MiB,交换空间均禁止;反代预检至少要求 320 MiB 可用内存,不通过时拒绝执行。
先在 8001 完成正常初始化,再通过真实反代登录、读取配置代次、验证及保存候选、显式应用配置。`publicHttpsPort` 设置为 443,`trustedProxyCidrs` 仅为 `127.0.0.1/32`,没有直接改写测试数据库或配置文件来绕过正式流程。
来源测试从 `127.0.0.2` 连续发送十次错误密码,使账号与 IP 组合限流生效;该来源改用正确密码并伪造 `X-Forwarded-For`、`X-Real-IP` 仍被拒绝。随后从 `127.0.0.3` 使用正确密码登录成功。该验证覆盖当前单代理拓扑,不等同于任意多层代理、防火墙或公网攻击容量验收。
恢复页以匿名连接访问,验证 `/public-password-recovery` 跳转至 `/rest/`、页面 `no-store`、注入登录地址为 `https://mail.runtime.invalid/#/login`,并解析 HTML 后实际读取生产脚本入口。没有执行浏览器 JavaScript、恢复邮件送达或密码重置,不能据此宣称恢复业务闭环完成。
## 运行证据与失败记录
- `/var/tmp/smail-review-runtime-20260918-015323`:修改工具前的完整候选直连基线通过,探针 24.91 秒,应用峰值 77,230,080 字节。
- `/var/tmp/smail-review-runtime-20260918-020045`:首次反代失败。生成配置为 root 组且权限 0640,受限 Nginx 无法读取;记录保留,线上不变,测试进程已停止。
- `/var/tmp/smail-review-runtime-20260918-020252`:修复为 root 所有、smail 组的 0640 后反代通过,探针 39.28 秒。
- `/var/tmp/smail-review-runtime-20260918-020719`:补齐恢复页和收尾逻辑后的反代验收通过,探针 40.32 秒,应用峰值 77,225,984 字节,Nginx 峰值 3,612,672 字节,两者交换空间为零。
- `/var/tmp/smail-review-runtime-20260918-020823`:最新工具的默认直连回归通过,探针 25.59 秒,应用峰值 77,254,656 字节,交换空间为零;证明可选反代模式未破坏原直连流程。
每个运行目录保留 `result.json`、`probe-result.json`、`probe.log`、`service.log`,失败目录也保留。最终结果中的 `resource_errors`、`cleanup_errors` 均为空,`live_message_absent`、`live_unchanged` 为真,两库 `integrity_check` 均为 `ok`。停止操作只接受工具生成的单元名,查询异常仍执行收尾;这些措施不代替未来宿主掉电等完整故障演练。
收尾模拟单元 `smail-runtime-cleanup-tests-20260918-0206b` 六项通过,峰值 14.9 MiB、交换空间为零。首次 `0206` 因 `PrivateTmp` 隐藏工具目录而没有执行测试;第二次仅增加指定工具目录的只读挂载,继续保留生产目录不可访问和独立网络限制。模拟用例不调用真实停止命令。
02:16 在相同限制下补跑六项模拟回归,全部通过,单元 `smail-runtime-cleanup-tests-20260918-021606` 运行 406 毫秒、峰值 15.1 MiB、交换空间为零。持久日志为 `/var/tmp/smail-review-proxy-c1dd533/cleanup-tests-20260918-021606.txt`。模拟单元以 root 执行,不宣称其为非特权用户回归。
02:15 远程收尾复核:四个线上服务仍 active,主进程及 InvocationID 与测试前一致;Nginx 配置检查、MySQL 就绪探测通过,Smail 与 Scert 的系统信任 HTTPS 页面均返回 200。本轮测试无运行中单元,历史失败单元记录继续保留。四个远程验收工具与本机 SHA-256 逐一一致。此时专用 DNS 配置仍不存在,旧续期 timer 为 active,未把候选脚本当作已经上线。
## 后续门槛
- [ ] 对线上部署建立可回退备份,部署候选并复核真实公网配置和传输入口。
- [ ] 真实桌面浏览器首次安装、登录、网页提交邮件、最终队列状态和实际收件结果。
- [ ] 受保护本地 IPC 管理员恢复及旧会话、协议凭据撤销,凭据轮换与访问审计。
- [ ] 队列与数据库故障、目标数据库兼容性、竞态、容量和异机恢复。
- [!] 01:51 只读复核时专用 DNS 配置仍缺失、旧续期 timer 仍启用;全新 DNS 挑战、正式新证书发布及外部告警仍未验收。不得将本轮隔离证书当作可信证书自动续期完成。
维持整体结论:不满足生产放行。以上通过项是完整目标中的增量证据,不缩减原发布验收范围。

View File

@ -2,6 +2,17 @@
## 范围与结论 ## 范围与结论
### 当前进展:完整候选反代隔离运行通过
- [✓] 2026 年 9 月 18 日,`review-c1dd533` 完整 Linux 候选六文件上传至 47 服务器,长度、摘要、ELF 格式核对通过;主前端和 Telegram 界面均使用新生产构建。
- [✓] 02:07:19 开始的新空实例通过真实独立 Nginx 反代验收,应用 8001/8002、公网 443、可信代理配置均通过正式 API 生效;HTTP 跳转和恢复页登录地址不再继承内部 8002。
- [✓] 两个真实来源的登录限流相互隔离,伪造转发头不能绕过已触发的限制;首次安装、三协议认证、内部 25 中文附件收信、过载恢复、双库完整性通过。
- [✓] 线上四个服务进程及 InvocationID 不变,线上无测试邮件,未停止业务服务,未发生新增 OOM;6 项测试收尾异常模拟回归通过。
- [✓] 最新工具默认直连回归通过;02:15 两站点可信 HTTPS、MySQL 就绪和服务状态收尾通过,02:16 六项收尾回归补齐持久日志,四个工具的本机及远程摘要一致。
- [!] 候选尚未替换线上版本,浏览器业务闭环、真实恢复邮件、管理员恢复撤销、DNS 自动续期、容量与灾难恢复等门槛继续未完成。隔离通过不等于线上配置已经整改生效。
详见《完整候选反代隔离验收记录》。下方较早章节保留历史阶段状态,当前增量结论以上方为准。
### 当前进展:远程生产前端构建完成 ### 当前进展:远程生产前端构建完成
- [✓] 2026 年 9 月 18 日 01:30,完整锁定依赖下远程前端回归 24 个文件、185 项通过,零失败、零跳过。 - [✓] 2026 年 9 月 18 日 01:30,完整锁定依赖下远程前端回归 24 个文件、185 项通过,零失败、零跳过。

View File

@ -0,0 +1,70 @@
'''隔离验收收尾回归,模拟异常,不停止真实服务。'''
from pathlib import Path
import subprocess
import sys
import unittest
from unittest.mock import patch
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'tools'))
import runtime_review as review
class RuntimeCleanupTest(unittest.TestCase):
app = 'smail-runtime-review-20260918-020000'
probe = 'smail-runtime-probe-20260918-020000'
proxy = 'smail-runtime-nginx-20260918-020000'
def test_resources_then_stop_all(self):
result = {}
with patch.object(review.subprocess, 'check_output', return_value='Result=success') as stats, \
patch.object(review, 'stop_test_units', return_value=[]) as stop:
review.collect_and_stop(result, self.app, self.probe, self.proxy)
self.assertEqual(stats.call_count, 2)
stop.assert_called_once_with([self.proxy, self.probe, self.app])
self.assertEqual(result['cleanup_errors'], [])
self.assertEqual(result['resource_errors'], [])
def test_resource_timeout_still_stops_all(self):
result = {}
with patch.object(review.subprocess, 'check_output', side_effect=subprocess.TimeoutExpired('show', 15)), \
patch.object(review, 'stop_test_units', return_value=[]) as stop:
review.collect_and_stop(result, self.app, self.probe, self.proxy)
self.assertEqual(len(result['resource_errors']), 2)
stop.assert_called_once_with([self.proxy, self.probe, self.app])
def test_stop_timeout_and_kill_error_do_not_skip_remaining(self):
success = subprocess.CompletedProcess([], 0, '', '')
with patch.object(review.subprocess, 'run', side_effect=[
subprocess.TimeoutExpired('stop', 25), OSError(), success, success,
]) as run:
errors = review.stop_test_units([self.proxy, self.probe, self.app])
self.assertEqual([item['action'] for item in errors], ['stop', 'kill'])
self.assertEqual(run.call_args_list[-1].args[0], ['systemctl', 'stop', self.app])
def test_missing_unit_is_not_failure(self):
missing = subprocess.CompletedProcess([], 5, '', 'Unit not loaded.')
with patch.object(review.subprocess, 'run', return_value=missing):
self.assertEqual(review.stop_test_units([self.probe]), [])
def test_stop_failure_attempts_kill_and_reports_failure(self):
failed = subprocess.CompletedProcess([], 1, '', 'Failed to stop unit')
success = subprocess.CompletedProcess([], 0, '', '')
with patch.object(review.subprocess, 'run', side_effect=[failed, success]) as run:
errors = review.stop_test_units([self.app])
self.assertEqual(errors[0]['error'], 'CalledProcessError')
self.assertEqual(run.call_args_list[1].args[0], [
'systemctl', 'kill', '--kill-whom=all', '--signal=SIGKILL', self.app,
])
def test_reject_business_units_before_any_stop(self):
for name in ('nginx', 'smail.service', self.app + ' other', '../mysql.service'):
with self.subTest(name=name), patch.object(review.subprocess, 'run') as run:
with self.assertRaises(AssertionError):
review.stop_test_units([self.app, name])
run.assert_not_called()
if __name__ == '__main__':
unittest.main()

View File

@ -26,20 +26,26 @@ ADDRESS = ACCOUNT + '@runtime.invalid'
class LocalHTTPS(http.client.HTTPSConnection): class LocalHTTPS(http.client.HTTPSConnection):
def connect(self): def connect(self):
raw = socket.create_connection(('127.0.0.1', self.port), self.timeout) raw = socket.create_connection(('127.0.0.1', self.port), self.timeout, self.source_address)
self.sock = self._context.wrap_socket(raw, server_hostname=self.host) self.sock = self._context.wrap_socket(raw, server_hostname=self.host)
class Client: class Client:
def __init__(self, context=None): def __init__(self, context=None, port=None, source=None):
self.context = context self.context = context
self.port = port or (443 if context else 80)
self.source = (source, 0) if source else None
self.cookies = SimpleCookie() self.cookies = SimpleCookie()
def request(self, path, payload=None, method='POST'): def request(self, path, payload=None, method='POST', extra_headers=None):
connection = (LocalHTTPS(HOST, 443, timeout=30, context=self.context) connection = (LocalHTTPS(HOST, self.port, timeout=30, context=self.context,
if self.context else http.client.HTTPConnection('127.0.0.1', 80, timeout=30)) source_address=self.source)
if self.context else http.client.HTTPConnection('127.0.0.1', self.port, timeout=30))
origin = 'https://' + HOST if self.context else 'http://127.0.0.1' origin = 'https://' + HOST if self.context else 'http://127.0.0.1'
if self.port != (443 if self.context else 80):
origin += ':' + str(self.port)
headers = {'Origin': origin, 'Content-Type': 'application/json'} headers = {'Origin': origin, 'Content-Type': 'application/json'}
headers.update(extra_headers or {})
if self.cookies: if self.cookies:
headers['Cookie'] = '; '.join(item.key + '=' + item.coded_value for item in self.cookies.values()) headers['Cookie'] = '; '.join(item.key + '=' + item.coded_value for item in self.cookies.values())
if 'smail_csrf' in self.cookies: if 'smail_csrf' in self.cookies:
@ -138,6 +144,7 @@ def main():
parser = argparse.ArgumentParser() parser = argparse.ArgumentParser()
parser.add_argument('root', type=Path) parser.add_argument('root', type=Path)
parser.add_argument('--live-net', required=True) parser.add_argument('--live-net', required=True)
parser.add_argument('--proxy', action='store_true')
args = parser.parse_args() args = parser.parse_args()
assert str(Path('/proc/self/ns/net').readlink()) != args.live_net, '禁止在宿主网络执行测试' assert str(Path('/proc/self/ns/net').readlink()) != args.live_net, '禁止在宿主网络执行测试'
root = args.root.resolve(strict=True) root = args.root.resolve(strict=True)
@ -145,7 +152,8 @@ def main():
result = {'stage': 'setup', 'passed': False} result = {'stage': 'setup', 'passed': False}
started = time.monotonic() started = time.monotonic()
try: try:
client = Client() setup_port = 8001 if args.proxy else 80
client = Client(port=setup_port)
for _ in range(60): for _ in range(60):
try: try:
assert client.request('/', method='GET')[0] == 200 assert client.request('/', method='GET')[0] == 200
@ -156,7 +164,7 @@ def main():
raise AssertionError('初始化服务未就绪') raise AssertionError('初始化服务未就绪')
assert 'smail_setup_token' in client.cookies, '初始化令牌未下发' assert 'smail_setup_token' in client.cookies, '初始化令牌未下发'
assert client.cookies['smail_setup_token']['httponly'] assert client.cookies['smail_setup_token']['httponly']
assert Client().request('/api/v1/setup', {'step': 'database', 'action': 'get'})[0] == 403 assert Client(port=setup_port).request('/api/v1/setup', {'step': 'database', 'action': 'get'})[0] == 403
assert not (root / 'data/smail.db').exists(), '业务库必须从空数据开始' assert not (root / 'data/smail.db').exists(), '业务库必须从空数据开始'
setup = '/api/v1/setup' setup = '/api/v1/setup'
client.success(setup, {'step': 'database', 'action': 'set', 'db_type': 'sqlite', 'db_dsn': ''}) client.success(setup, {'step': 'database', 'action': 'set', 'db_type': 'sqlite', 'db_dsn': ''})
@ -185,6 +193,9 @@ def main():
assert 'smail_csrf' in client.cookies assert 'smail_csrf' in client.cookies
assert client.success('/api/v1/user/info').get('account') == ACCOUNT assert client.success('/api/v1/user/info').get('account') == ACCOUNT
result['setup'] = {'empty_database': True, 'token_required': True, 'login': True, 'session': True} result['setup'] = {'empty_database': True, 'token_required': True, 'login': True, 'session': True}
if args.proxy:
from runtime_proxy import configure_proxy
result['proxy'] = configure_proxy(client, context, login, Client, LocalHTTPS)
result['stage'] = 'protocols' result['stage'] = 'protocols'
codes = {} codes = {}
for protocol in ('smtp', 'imap', 'pop3'): for protocol in ('smtp', 'imap', 'pop3'):
@ -256,6 +267,9 @@ def main():
assert client.success('/api/v1/login', login).get('account') == ACCOUNT assert client.success('/api/v1/login', login).get('account') == ACCOUNT
round_result['recovered'] = True round_result['recovered'] = True
(root.parent / 'probe-result.json').write_text(json.dumps(result, ensure_ascii=False, indent=2), encoding='utf-8') (root.parent / 'probe-result.json').write_text(json.dumps(result, ensure_ascii=False, indent=2), encoding='utf-8')
if args.proxy:
from runtime_proxy import verify_proxy_sources
result['proxy']['sources'] = verify_proxy_sources(context, login, Client)
result['stage'] = 'complete' result['stage'] = 'complete'
result['passed'] = True result['passed'] = True
finally: finally:

146
test/tools/runtime_proxy.py Normal file
View File

@ -0,0 +1,146 @@
'''复用部署模板,在独立网络中验收 Nginx 与 8001/8002,不访问线上实例。'''
import http.client
import shutil
import time
HOST = 'mail.runtime.invalid'
def prepare_nginx(target, root, template):
directory = target / 'nginx'
directory.mkdir(mode=0o750)
shutil.chown(directory, user='smail', group='smail')
source = template.read_text(encoding='utf-8')
assert 'proxy_pass https://127.0.0.1:8002;' in source
assert 'proxy_ssl_verify on;' in source
assert 'proxy_set_header X-Forwarded-For $remote_addr;' in source
site = source.replace('solit.cn', 'runtime.invalid')
site = site.replace('/opt/smail/ssl/custom.crt', str(root / 'ssl/runtime.crt'))
site = site.replace('/opt/smail/ssl/custom.key', str(root / 'ssl/runtime.key'))
site = site.replace('/etc/ssl/certs/ca-certificates.crt', str(root / 'ssl/runtime.crt'))
site = site.replace('/var/lib/smail-acme', str(directory / 'acme'))
(directory / 'site.conf').write_text(site, encoding='utf-8')
config = '\n'.join([
'worker_processes 1;',
'pid ' + str(directory / 'nginx.pid') + ';',
'error_log ' + str(directory / 'error.log') + ' warn;',
'events { worker_connections 128; }',
'http {',
' access_log off;',
*(' ' + name + '_temp_path ' + str(directory / name) + ';'
for name in ('client_body', 'proxy', 'fastcgi', 'uwsgi', 'scgi')),
' include ' + str(directory / 'site.conf') + ';',
'}',
'',
])
(directory / 'nginx.conf').write_text(config, encoding='utf-8')
for name in ('site.conf', 'nginx.conf'):
path = directory / name
path.chmod(0o640)
shutil.chown(path, user='root', group='smail')
def configure_proxy(client, context, login, client_type, connection_type):
status = client.success('/api/v1/config/status', method='GET')
assert status['available']
payload = client.success('/api/v1/config/base', method='GET')
assert payload['http']['port'] == 8001 and payload['http']['httpsPort'] == 8002
payload.pop('databasePasswordStatus', None)
payload['http']['publicHttpsPort'] = 443
payload['http']['trustedProxyCidrs'] = ['127.0.0.1/32']
request = {'expectedGeneration': status['activeGeneration'], 'payload': payload,
'confirmation': {'confirmed': True, 'method': 'recent_auth'}}
client.success('/api/v1/config/base/validate', request)
candidate = client.success('/api/v1/config/base', request, 'PUT')
client.success('/api/v1/config/base/apply', {
'generation': candidate['generation'], 'candidateId': candidate['candidateId'],
})
for _ in range(30):
try:
active = client.success('/api/v1/config/base', method='GET')
assert active['http']['publicHttpsPort'] == 443
assert active['http']['trustedProxyCidrs'] == ['127.0.0.1/32']
break
except (OSError, http.client.HTTPException, AssertionError):
time.sleep(1)
else:
raise AssertionError('反代配置未通过正式接口生效')
assert active['http']['port'] == 8001 and active['http']['httpsPort'] == 8002
assert client_type(context, port=8002).success('/api/v1/login', login)['account'] == login['account']
redirects = {}
for port in (80, 8001):
connection = http.client.HTTPConnection('127.0.0.1', port, timeout=10)
try:
connection.request('GET', '/login?review=port', headers={'Host': HOST})
response = connection.getresponse()
response.read()
assert response.status in (301, 302, 307, 308), (port, response.status)
location = response.getheader('Location')
assert location == 'https://' + HOST + '/login?review=port', (port, location)
redirects[str(port)] = location
finally:
connection.close()
recovery = verify_recovery_page(context, connection_type)
return {'configuration_api': True, 'internal_http': 8001, 'internal_https': 8002,
'public_https': 443, 'upstream_tls_verified': True, 'redirects': redirects,
'recovery': recovery}
def verify_recovery_page(context, connection_type):
from html.parser import HTMLParser
class PageParser(HTMLParser):
def __init__(self):
super().__init__()
self.login_url = None
self.scripts = []
def handle_starttag(self, tag, attrs):
attributes = dict(attrs)
if tag == 'div' and attributes.get('id') == 'app':
self.login_url = attributes.get('data-login-url')
if tag == 'script' and attributes.get('src'):
self.scripts.append(attributes['src'])
def fetch(path):
connection = connection_type(HOST, 443, timeout=10, context=context)
try:
connection.request('GET', path)
response = connection.getresponse()
content = response.read()
return response.status, dict(response.getheaders()), content
finally:
connection.close()
status, headers, _ = fetch('/public-password-recovery')
assert status == 302 and headers.get('Location') == 'https://' + HOST + '/rest/'
status, headers, content = fetch('/rest/')
assert status == 200 and headers.get('Cache-Control') == 'no-store'
page = PageParser()
page.feed(content.decode('utf-8'))
assert page.login_url == 'https://' + HOST + '/#/login'
assert page.scripts, '恢复页面缺少生产构建入口'
for script in page.scripts:
assert script.startswith('/assets/') and '..' not in script
status, headers, content = fetch(script)
assert status == 200 and content and 'javascript' in headers.get('Content-Type', '')
return {'entry_redirect': True, 'login_url': page.login_url,
'no_store': True, 'scripts_loaded': len(page.scripts)}
def verify_proxy_sources(context, login, client_type):
first = client_type(context, source='127.0.0.2')
second = client_type(context, source='127.0.0.3')
wrong = {**login, 'password': 'invalid-proxy-review-password'}
for _ in range(10):
status, data, _ = first.request('/api/v1/login', wrong)
assert status == 200 and data is not None and data['errorNo'] != 0
status, data, _ = first.request('/api/v1/login', login, extra_headers={
'X-Forwarded-For': '127.0.0.3', 'X-Real-IP': '127.0.0.3',
})
assert status == 200 and data is not None and data['errorNo'] != 0, '伪造来源绕过登录限流'
assert second.success('/api/v1/login', login)['account'] == login['account'], '不同来源被错误合并限流'
return {'blocked_source': '127.0.0.2', 'unaffected_source': '127.0.0.3',
'forged_headers_rejected': True, 'independent_login': True}

View File

@ -5,6 +5,7 @@ import hashlib
import json import json
import os import os
from pathlib import Path from pathlib import Path
import re
import shutil import shutil
import sqlite3 import sqlite3
import subprocess import subprocess
@ -16,14 +17,51 @@ def run(*args):
def service_state(): def service_state():
return {name: run('systemctl', 'show', name, '--property=MainPID,NRestarts,ActiveState') return {name: run('systemctl', 'show', name, '--property=MainPID,NRestarts,ActiveState,InvocationID')
for name in ('smail.service', 'mysql.service', 'Scert.service')} for name in ('smail.service', 'nginx.service', 'mysql.service', 'Scert.service')}
def stop_test_units(names):
assert all(re.fullmatch(r'smail-runtime-(review|probe|nginx)-[0-9]{8}-[0-9]{6}', name)
for name in names), '只能停止本轮隔离测试单元'
errors = []
for name in names:
try:
stopped = subprocess.run(['systemctl', 'stop', name], capture_output=True,
text=True, timeout=25)
if stopped.returncode and 'not loaded' not in stopped.stderr:
raise subprocess.CalledProcessError(stopped.returncode, 'systemctl')
except (OSError, subprocess.SubprocessError) as error:
errors.append({'unit': name, 'action': 'stop', 'error': type(error).__name__})
try:
subprocess.run(['systemctl', 'kill', '--kill-whom=all', '--signal=SIGKILL', name],
check=True, capture_output=True, timeout=15)
except (OSError, subprocess.SubprocessError) as kill_error:
errors.append({'unit': name, 'action': 'kill', 'error': type(kill_error).__name__})
return errors
def collect_and_stop(result, unit, probe_unit, proxy_unit=None):
result['resource_errors'] = []
try:
for key, name in [('resources', unit)] + ([('nginx_resources', proxy_unit)] if proxy_unit else []):
try:
result[key] = subprocess.check_output([
'systemctl', 'show', name,
'--property=MemoryPeak,MemoryCurrent,MemorySwapPeak,Result,ExecMainStatus,NRestarts',
], text=True, timeout=15).strip()
except (OSError, subprocess.SubprocessError) as error:
result['resource_errors'].append({'unit': name, 'error': type(error).__name__})
finally:
names = ([proxy_unit] if proxy_unit else []) + [probe_unit, unit]
result['cleanup_errors'] = stop_test_units(names)
def main(): def main():
parser = argparse.ArgumentParser() parser = argparse.ArgumentParser()
parser.add_argument('--main-sha', required=True) parser.add_argument('--main-sha', required=True)
parser.add_argument('--binary', type=Path, default=Path('/opt/smail/smail')) parser.add_argument('--binary', type=Path, default=Path('/opt/smail/smail'))
parser.add_argument('--nginx-template', type=Path)
args = parser.parse_args() args = parser.parse_args()
assert os.geteuid() == 0 and run('hostname') == 'ikun2' assert os.geteuid() == 0 and run('hostname') == 'ikun2'
os.umask(0o027) os.umask(0o027)
@ -42,7 +80,8 @@ def main():
with (live / 'smail').open('rb') as stream: with (live / 'smail').open('rb') as stream:
live_sha = hashlib.file_digest(stream, 'sha256').hexdigest() live_sha = hashlib.file_digest(stream, 'sha256').hexdigest()
memory = dict(line.split(':', 1) for line in Path('/proc/meminfo').read_text().splitlines()) memory = dict(line.split(':', 1) for line in Path('/proc/meminfo').read_text().splitlines())
assert int(memory['MemAvailable'].split()[0]) >= 272 * 1024, '可用内存不足,暂停隔离验收' required_memory = 320 if args.nginx_template else 272
assert int(memory['MemAvailable'].split()[0]) >= required_memory * 1024, '可用内存不足,暂停隔离验收'
stamp = time.strftime('%Y%m%d-%H%M%S') stamp = time.strftime('%Y%m%d-%H%M%S')
target = Path('/var/tmp') / ('smail-review-runtime-' + stamp) target = Path('/var/tmp') / ('smail-review-runtime-' + stamp)
target.mkdir(mode=0o750) target.mkdir(mode=0o750)
@ -55,6 +94,8 @@ def main():
shutil.copy2(binary, root / 'smail') shutil.copy2(binary, root / 'smail')
(root / 'smail').chmod(0o755) (root / 'smail').chmod(0o755)
shutil.copy2(Path(__file__).with_name('runtime_probe.py'), target / 'runtime_probe.py') shutil.copy2(Path(__file__).with_name('runtime_probe.py'), target / 'runtime_probe.py')
if args.nginx_template:
shutil.copy2(Path(__file__).with_name('runtime_proxy.py'), target / 'runtime_proxy.py')
subprocess.run(['openssl', 'req', '-x509', '-newkey', 'rsa:2048', '-nodes', '-days', '2', subprocess.run(['openssl', 'req', '-x509', '-newkey', 'rsa:2048', '-nodes', '-days', '2',
'-keyout', str(root / 'ssl/runtime.key'), '-out', str(root / 'ssl/runtime.crt'), '-keyout', str(root / 'ssl/runtime.key'), '-out', str(root / 'ssl/runtime.crt'),
'-subj', '/CN=mail.runtime.invalid', '-addext', '-subj', '/CN=mail.runtime.invalid', '-addext',
@ -68,8 +109,19 @@ def main():
before = service_state() before = service_state()
unit = 'smail-runtime-review-' + stamp unit = 'smail-runtime-review-' + stamp
probe_unit = 'smail-runtime-probe-' + stamp probe_unit = 'smail-runtime-probe-' + stamp
proxy_unit = 'smail-runtime-nginx-' + stamp
result = {'target': str(target), 'unit': unit, 'main_sha256': args.main_sha, result = {'target': str(target), 'unit': unit, 'main_sha256': args.main_sha,
'source': str(binary), 'live_sha256': live_sha, 'before': before} 'source': str(binary), 'live_sha256': live_sha, 'before': before,
'topology': 'nginx-8001-8002' if args.nginx_template else 'direct-80-443'}
service_env = []
probe_args = []
if args.nginx_template:
from runtime_proxy import prepare_nginx
prepare_nginx(target, root, args.nginx_template)
result['nginx_template_sha256'] = hashlib.sha256(args.nginx_template.read_bytes()).hexdigest()
service_env = ['--setenv=setup_port=8001', '--setenv=SMAIL_HTTP_PORT=8001',
'--setenv=SMAIL_HTTPS_PORT=8002']
probe_args = ['--proxy']
started = time.strftime('%Y-%m-%dT%H:%M:%S') started = time.strftime('%Y-%m-%dT%H:%M:%S')
try: try:
subprocess.run([ subprocess.run([
@ -85,7 +137,7 @@ def main():
'--property=BindPaths=' + str(root) + ':/opt/smail ' + str(control) + ':/run/smail', '--property=BindPaths=' + str(root) + ':/opt/smail ' + str(control) + ':/run/smail',
'--property=ReadWritePaths=/opt/smail /run/smail', '--property=ReadWritePaths=/opt/smail /run/smail',
'--property=WorkingDirectory=/opt/smail', '--setenv=SMAIL_ROOT=/opt/smail', '--property=WorkingDirectory=/opt/smail', '--setenv=SMAIL_ROOT=/opt/smail',
'--setenv=GOMEMLIMIT=64MiB', '--setenv=GOMAXPROCS=1', '/opt/smail/smail', '--setenv=GOMEMLIMIT=64MiB', '--setenv=GOMAXPROCS=1', *service_env, '/opt/smail/smail',
], check=True) ], check=True)
pid = run('systemctl', 'show', unit, '--property=MainPID', '--value') pid = run('systemctl', 'show', unit, '--property=MainPID', '--value')
assert pid != '0' assert pid != '0'
@ -93,6 +145,24 @@ def main():
assert str(Path('/proc/' + pid + '/ns/net').readlink()) != live_net assert str(Path('/proc/' + pid + '/ns/net').readlink()) != live_net
assert Path('/proc/' + pid + '/root/opt/smail/smail').stat().st_ino == (root / 'smail').stat().st_ino assert Path('/proc/' + pid + '/root/opt/smail/smail').stat().st_ino == (root / 'smail').stat().st_ino
assert (root / 'smail').stat().st_ino != (live / 'smail').stat().st_ino assert (root / 'smail').stat().st_ino != (live / 'smail').stat().st_ino
if args.nginx_template:
subprocess.run([
'systemd-run', '--unit=' + proxy_unit, '--property=Type=exec',
'--property=User=smail', '--property=Group=smail',
'--property=NetworkNamespacePath=/proc/' + pid + '/ns/net',
'--property=PrivateTmp=no', '--property=ProtectSystem=strict',
'--property=ProtectHome=yes', '--property=NoNewPrivileges=yes',
'--property=InaccessiblePaths=/opt/smail /run/smail /etc/nginx',
'--property=ReadWritePaths=' + str(target / 'nginx'),
'--property=CapabilityBoundingSet=CAP_NET_BIND_SERVICE',
'--property=AmbientCapabilities=CAP_NET_BIND_SERVICE',
'--property=MemoryMax=32M', '--property=MemorySwapMax=0',
'--property=CPUQuota=25%', '--property=TasksMax=16',
'--property=RuntimeMaxSec=300s', '--property=OOMPolicy=stop',
'--property=KillMode=control-group', '--property=TimeoutStopSec=10s',
'/usr/sbin/nginx', '-p', str(target / 'nginx'),
'-c', str(target / 'nginx/nginx.conf'), '-g', 'daemon off;',
], check=True)
probe = subprocess.run([ probe = subprocess.run([
'systemd-run', '--wait', '--pipe', '--unit=' + probe_unit, 'systemd-run', '--wait', '--pipe', '--unit=' + probe_unit,
'--property=User=smail', '--property=Group=smail', '--property=User=smail', '--property=Group=smail',
@ -103,7 +173,7 @@ def main():
'--property=MemorySwapMax=0', '--property=CPUQuota=25%', '--property=TasksMax=32', '--property=MemorySwapMax=0', '--property=CPUQuota=25%', '--property=TasksMax=32',
'--property=RuntimeMaxSec=240s', '--property=OOMPolicy=stop', '--property=RuntimeMaxSec=240s', '--property=OOMPolicy=stop',
'--property=KillMode=control-group', '--property=TimeoutStopSec=10s', '--property=KillMode=control-group', '--property=TimeoutStopSec=10s',
'python3', str(target / 'runtime_probe.py'), str(root), '--live-net', live_net, 'python3', str(target / 'runtime_probe.py'), str(root), '--live-net', live_net, *probe_args,
], text=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, timeout=270) ], text=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, timeout=270)
(target / 'probe.log').write_text(probe.stdout, encoding='utf-8') (target / 'probe.log').write_text(probe.stdout, encoding='utf-8')
result['probe_code'] = probe.returncode result['probe_code'] = probe.returncode
@ -116,10 +186,7 @@ def main():
assert database.execute('SELECT COUNT(*) FROM email WHERE msg_id=?', (message_id,)).fetchone()[0] == 0 assert database.execute('SELECT COUNT(*) FROM email WHERE msg_id=?', (message_id,)).fetchone()[0] == 0
result['live_message_absent'] = True result['live_message_absent'] = True
finally: finally:
result['resources'] = run('systemctl', 'show', unit, collect_and_stop(result, unit, probe_unit, proxy_unit if args.nginx_template else None)
'--property=MemoryPeak,MemoryCurrent,Result,ExecMainStatus,NRestarts')
subprocess.run(['systemctl', 'stop', unit], check=True)
subprocess.run(['systemctl', 'stop', probe_unit], check=False, stderr=subprocess.DEVNULL)
result['after'] = service_state() result['after'] = service_state()
with (live / 'smail').open('rb') as stream: with (live / 'smail').open('rb') as stream:
result['live_unchanged'] = (result['after'] == before result['live_unchanged'] = (result['after'] == before
@ -136,6 +203,8 @@ def main():
(target / 'result.json').write_text(json.dumps(result, ensure_ascii=False, indent=2), encoding='utf-8') (target / 'result.json').write_text(json.dumps(result, ensure_ascii=False, indent=2), encoding='utf-8')
print(json.dumps(result, ensure_ascii=False, indent=2)) print(json.dumps(result, ensure_ascii=False, indent=2))
assert result['live_unchanged'] and result['oom_events'] == 0 assert result['live_unchanged'] and result['oom_events'] == 0
assert not result['cleanup_errors'], '测试单元未全部停止,需要核查保留的结果'
assert not result['resource_errors'], '资源证据不完整,需要核查保留的结果'
assert result['integrity'] == {'smail.db': [('ok',)], 'smail-state.db': [('ok',)]} assert result['integrity'] == {'smail.db': [('ok',)], 'smail-state.db': [('ok',)]}