fix: Add dedicated document LLM, improve Markdown detection and conte…

[bittergreen]

Description

fix: Add dedicated document LLM, improve Markdown detection and context-aware structured extraction

Type of change

• Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

• Unit Test: tests/mem_reader/test_document_parser_llm.py

Checklist

• I have performed a self-review of my own code | 我已自行检查了自己的代码
• I have commented my code in hard-to-understand areas | 我已在难以理解的地方对代码进行了注释
• I have added tests that prove my fix is effective or that my feature works | 我已添加测试以证明我的修复有效或功能正常
• I have created related documentation issue/PR in MemOS-Docs (if applicable) | 我已在 MemOS-Docs 中创建了相关的文档 issue/PR（如果适用）
• I have linked the issue to this PR (if applicable) | 我已将 issue 链接到此 PR（如果适用）
• I have mentioned the person who will review this PR | 我已提及将审查此 PR 的人

Reviewer Checklist

• closes #xxxx (Replace xxxx with the GitHub issue number)
• Made sure Checks passed
• Tests have been provided

[Memtensor-AI]

🤖 Open Code Review

Target: PR #2155
Task: f476d14aa5ba5a60
Base: dev-v2.0.25
Head: wq-dev-v2.0.25

🔍 OpenCodeReview found 3 issue(s) in this PR.

---

1. src/memos/configs/mem_reader.py (L40-L43)

The description omits the fallback behaviour when this field is None. All sibling fields (image_parser_llm, preference_extractor_llm) explicitly document what they fall back to, which helps operators configure the system correctly. From the implementation in multi_modal_struct.py, when document_parser_llm is None the FileContentParser receives None (no LLM at all, unlike image_parser_llm which falls back to general_llm). This difference in fallback policy is significant and worth surfacing in the description.

💡 Suggested Change

Before:

    document_parser_llm: LLMConfigFactory | None = Field(
        default=None,
        description="Dedicated LLM for document content extraction",
    )

After:

    document_parser_llm: LLMConfigFactory | None = Field(
        default=None,
        description="Dedicated LLM for document content extraction. "
        "If not set, no LLM is used for document parsing (unlike image_parser_llm, there is no automatic fallback).",
    )

---

2. src/memos/mem_reader/read_multi_modal/file_content_parser.py (L1083-L1091)

The guard if not filename does not account for whitespace-only filenames (e.g., " "). Such a value passes the guard and produces a misleading Filename: line in the document context sent to the LLM. Strip filename before the guard and before emitting it to avoid this.

Suggested fix:

filename = filename.strip() if filename else ""

if not filename and not h1_titles:
    return None
...
if filename:
    lines.append(f"Filename: {filename}")

💡 Suggested Change

Before:

        if not filename and not h1_titles:
            return None

        lines = [
            "Document-level context for disambiguation only. "
            "Do not create memories from this context unless the current chunk supports them."
        ]
        if filename:
            lines.append(f"Filename: {filename}")

After:

        filename = filename.strip() if filename else ""

        if not filename and not h1_titles:
            return None

        lines = [
            "Document-level context for disambiguation only. "
            "Do not create memories from this context unless the current chunk supports them."
        ]
        if filename:
            lines.append(f"Filename: {filename}")

---

3. src/memos/mem_reader/read_multi_modal/multi_modal_parser.py (L63)

Behavioral regression: document_parser_llm does not fall back to llm, so fine-mode document extraction silently degrades when only llm is configured.

Before this change, FileContentParser always received llm and used it for LLM-based chunk extraction in fine mode. Now, when document_parser_llm is None (the default), FileContentParser receives None, causing parse_fine() to hit the if not self.llm: guard on line 871 and return raw "no_llm" fallback items — even when a perfectly capable llm is provided.

This silently breaks all existing deployments that do not explicitly set document_parser_llm in their config.

The fix is to mirror the same fallback pattern used for image_parser_llm:

self.document_parser_llm = document_parser_llm if document_parser_llm is not None else llm

💡 Suggested Change

Before:

        self.document_parser_llm = document_parser_llm

After:

        # Document parser LLM for file extraction, falls back to main llm
        self.document_parser_llm = document_parser_llm if document_parser_llm is not None else llm

Generated by cloud-assistant via Open Code Review.

[Memtensor-AI]

✅ Automated Test Results: PASSED

All tests passed (15/15 executed). memos_python_core/changed-repo-python: 15/15. Duration: 5s [advisory, non-gating] AI-generated tests on branch test/auto-gen-f476d14aa5ba5a60-20260724104934: 119/119 passed — these do NOT affect the PR verdict; review the branch manually.

Branch: wq-dev-v2.0.25