添加知识库与子代理功能。添加 debug 模式。
This commit is contained in:
@@ -1,8 +1,7 @@
|
|||||||
# 安智伴 · AI 数字人陪护终端(Android)
|
# 安智伴 · AI 数字人陪护终端(Android)
|
||||||
|
|
||||||
English | [中文](./README_zh.md)
|
|
||||||
|
|
||||||
> 本项目与 [digital-human-monitor-v2](https://github.com/user/digital-human-monitor-v2)(家属监护 Web 端)配合使用,共同构成完整的"AI 数字人 + 家庭远程关怀"解决方案。
|
> 本项目与 家属监护 Web 端配合使用,共同构成完整的"AI 数字人 + 家庭远程关怀"解决方案。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -61,13 +60,14 @@ DUIX SDK 仅负责"给定 PCM 音频 → 渲染对应口型/表情动画",不
|
|||||||
|
|
||||||
这种解耦使得 AI 模型可独立升级,不影响渲染管线。
|
这种解耦使得 AI 模型可独立升级,不影响渲染管线。
|
||||||
|
|
||||||
### 3. 工具调用实现双向留言
|
### 3. 工具调用 + 实现双向留言
|
||||||
|
|
||||||
通过 Qwen 的 Function Calling 能力,在 AI 对话中注册两个工具函数:
|
通过 Qwen 的 Function Calling 能力,在 AI 对话中注册三个工具函数:
|
||||||
|
- `consult_psychology_knowledge_base`:Omni 提炼老人问题 → Android 端调用 qwen3.6-plus,并把 WHO ICOPE 2024 全文作为上下文 → 返回权威说明
|
||||||
- `leave_message_for_family`:老人口述 → AI 提取内容 → 调用工具 → 消息存入云端
|
- `leave_message_for_family`:老人口述 → AI 提取内容 → 调用工具 → 消息存入云端
|
||||||
- `fetch_family_messages`:老人想听留言 → AI 调用工具 → 拉取消息 → 口播转达
|
- `fetch_family_messages`:老人想听留言 → AI 调用工具 → 拉取消息 → 口播转达
|
||||||
|
|
||||||
AI 自主判断何时调用工具,无需老人学习任何操作指令。
|
AI 自主判断何时调用工具,无需老人学习任何操作指令。涉及权威照护依据时,实时语音模型不会直接吞入整份知识库上下文,而是把问题转交给长上下文子代理处理。
|
||||||
|
|
||||||
### 4. 面向老人的交互设计
|
### 4. 面向老人的交互设计
|
||||||
|
|
||||||
@@ -132,9 +132,9 @@ AI 自主判断何时调用工具,无需老人学习任何操作指令。
|
|||||||
| 层级 | 技术 |
|
| 层级 | 技术 |
|
||||||
|------|------|
|
|------|------|
|
||||||
| **数字人渲染** | NCNN (端侧推理) + OpenCV 4.6 + OpenGL ES 2.0 + ONNX Runtime |
|
| **数字人渲染** | NCNN (端侧推理) + OpenCV 4.6 + OpenGL ES 2.0 + ONNX Runtime |
|
||||||
| **AI 对话** | 千问 Qwen 3.5 Omni Plus Realtime (WebSocket) |
|
| **AI 对话** | 千问 Qwen 3.5 Omni Plus Realtime (WebSocket) + Qwen 3.6 Plus 知识子代理 (HTTP) |
|
||||||
| **音频处理** | AudioRecord (16kHz) → PCM → Base64 → WebSocket;AudioTrack 播放 |
|
| **音频处理** | AudioRecord (16kHz) → PCM → Base64 → WebSocket;AudioTrack 播放 |
|
||||||
| **网络通信** | OkHttp 4 (HTTP API) + OkHttp WebSocket (Qwen Realtime) |
|
| **网络通信** | OkHttp 4 (HTTP API / qwen3.6-plus) + OkHttp WebSocket (Qwen Realtime) |
|
||||||
| **本地存储** | SharedPreferences (设备 UUID、配置) |
|
| **本地存储** | SharedPreferences (设备 UUID、配置) |
|
||||||
| **构建系统** | Gradle 8.13.2 + AGP 8.9.1 + JDK 17 (Temurin) |
|
| **构建系统** | Gradle 8.13.2 + AGP 8.9.1 + JDK 17 (Temurin) |
|
||||||
| **语言** | Kotlin (应用层) + Java (SDK 层) + C++ (Native 推理层) |
|
| **语言** | Kotlin (应用层) + Java (SDK 层) + C++ (Native 推理层) |
|
||||||
@@ -159,23 +159,28 @@ dh_aigc_android/
|
|||||||
│ └── third/ # 预编译依赖 (NCNN, OpenCV, ONNX, libjpeg)
|
│ └── third/ # 预编译依赖 (NCNN, OpenCV, ONNX, libjpeg)
|
||||||
│
|
│
|
||||||
├── test/ # 演示应用(应用模块)
|
├── test/ # 演示应用(应用模块)
|
||||||
│ └── src/main/java/ai/guiji/duix/test/
|
│ └── src/main/
|
||||||
│ ├── ui/activity/
|
│ ├── java/ai/guiji/duix/test/
|
||||||
│ │ ├── MainActivity.kt # 主页:模型选择与下载管理
|
│ │ ├── ui/activity/
|
||||||
│ │ ├── CallActivity.kt # 核心:数字人渲染 + AI 对话 + 留言桥梁
|
│ │ │ ├── MainActivity.kt # 主页:模型选择与下载管理
|
||||||
│ │ ├── SettingsActivity.kt # 设备设置
|
│ │ │ ├── CallActivity.kt # 核心:数字人渲染 + AI 对话 + 留言桥梁 + 知识子代理
|
||||||
│ │ └── VoiceCloneSetupActivity.kt # 声音克隆配置
|
│ │ │ ├── SettingsActivity.kt # 设备设置
|
||||||
│ ├── service/
|
│ │ │ └── VoiceCloneSetupActivity.kt # 声音克隆配置
|
||||||
│ │ ├── FamilyBridgeService.kt # 家庭监护 API 客户端
|
│ │ ├── realtime/
|
||||||
│ │ ├── OmniRealtimeClient.kt # Qwen Omni WebSocket 客户端
|
│ │ │ ├── OmniRealtimeClient.java # Qwen Omni WebSocket 客户端
|
||||||
│ │ ├── QwenTtsRealtimeClient.kt # Qwen TTS 客户端
|
│ │ │ └── QwenTtsRealtimeClient.kt # Qwen TTS 客户端
|
||||||
│ │ ├── VoiceCloneService.kt # 声音克隆服务
|
│ │ ├── service/
|
||||||
│ │ └── DashScopeConfig.java # API 配置常量
|
│ │ │ ├── FamilyBridgeService.kt # 家庭监护 API 客户端
|
||||||
│ ├── audio/
|
│ │ │ ├── PsychologyKnowledgeAgentService.kt # qwen3.6-plus 长上下文知识子代理
|
||||||
│ │ ├── AudioResampler.java # PCM 重采样
|
│ │ │ ├── VoiceCloneService.kt # 声音克隆服务
|
||||||
│ │ └── StreamingAudioRecorder.java # 实时麦克风录音
|
│ │ │ └── DashScopeConfig.java # API 配置常量
|
||||||
│ └── camera/
|
│ │ ├── audio/
|
||||||
│ └── CameraFrameCapture.java # 摄像头帧采集
|
│ │ │ ├── AudioResampler.java # PCM 重采样
|
||||||
|
│ │ │ └── StreamingAudioRecorder.java # 实时麦克风录音
|
||||||
|
│ │ └── camera/
|
||||||
|
│ │ └── CameraFrameCapture.java # 摄像头帧采集
|
||||||
|
│ └── assets/knowledge/
|
||||||
|
│ └── who_icope_2024.txt # WHO ICOPE 2024 权威知识库全文
|
||||||
│
|
│
|
||||||
└── qwen-docs/ # 千问 Omni-Realtime API 参考文档
|
└── qwen-docs/ # 千问 Omni-Realtime API 参考文档
|
||||||
├── main.md # 协议总览
|
├── main.md # 协议总览
|
||||||
@@ -277,6 +282,7 @@ pushPcm(音频) → MFCC特征提取 → NCNN口型网络推理
|
|||||||
|
|
||||||
| 工具名 | 参数 | 用途 |
|
| 工具名 | 参数 | 用途 |
|
||||||
|--------|------|------|
|
|--------|------|------|
|
||||||
|
| `consult_psychology_knowledge_base` | `question` (核心问题), `focus` (聚焦点,可选) | 调用 qwen3.6-plus + WHO ICOPE 全文,返回权威照护说明 |
|
||||||
| `leave_message_for_family` | `recipientRelation` (称谓), `message` (内容), `importance` (优先级) | 老人留言给家人 |
|
| `leave_message_for_family` | `recipientRelation` (称谓), `message` (内容), `importance` (优先级) | 老人留言给家人 |
|
||||||
| `fetch_family_messages` | `count` (数量 1-5) | 拉取家人留言 |
|
| `fetch_family_messages` | `count` (数量 1-5) | 拉取家人留言 |
|
||||||
|
|
||||||
@@ -390,15 +396,9 @@ API 密钥和服务地址在 `DashScopeConfig.java` 中配置:
|
|||||||
```java
|
```java
|
||||||
MONITOR_BASE_URL // 监护 Web 服务地址
|
MONITOR_BASE_URL // 监护 Web 服务地址
|
||||||
REALTIME_BASE_URL // Qwen Omni WebSocket 地址
|
REALTIME_BASE_URL // Qwen Omni WebSocket 地址
|
||||||
|
MULTIMODAL_GENERATION_URL // qwen3.6-plus HTTP 地址
|
||||||
DEFAULT_API_KEY // DashScope API Key
|
DEFAULT_API_KEY // DashScope API Key
|
||||||
|
PSYCHOLOGY_SUBAGENT_MODEL // 心理学知识子代理模型
|
||||||
|
PSYCHOLOGY_KNOWLEDGE_ASSET_PATH // 内置 WHO ICOPE 知识库 asset 路径
|
||||||
AUDIO_SAMPLE_RATE // 音频采样率 (24000)
|
AUDIO_SAMPLE_RATE // 音频采样率 (24000)
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 关联项目
|
|
||||||
|
|
||||||
| 项目 | 角色 | 仓库 |
|
|
||||||
|------|------|------|
|
|
||||||
| **dh_aigc_android**(本项目) | 老人端:AI 数字人陪护 + 设备行为上报 | 当前仓库 |
|
|
||||||
| **digital-human-monitor-v2** | 家属端:远程监护 Web 面板 + 消息管理 + 推送通知 | [digital-human-monitor-v2](../digital-human-monitor-v2/) |
|
|
||||||
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large
Load Diff
@@ -36,30 +36,18 @@ public class OmniRealtimeClient {
|
|||||||
private static final String INPUT_AUDIO_TRANSCRIPTION_MODEL = "gummy-realtime-v1";
|
private static final String INPUT_AUDIO_TRANSCRIPTION_MODEL = "gummy-realtime-v1";
|
||||||
private static final int MAX_IMAGE_BYTES = 500 * 1024;
|
private static final int MAX_IMAGE_BYTES = 500 * 1024;
|
||||||
private static final long MIN_IMAGE_APPEND_INTERVAL_MS = 800L;
|
private static final long MIN_IMAGE_APPEND_INTERVAL_MS = 800L;
|
||||||
private static final String DEFAULT_INSTRUCTIONS = "你是一名面向中国老年群体的心理陪伴 AI。你的职责是为老人提供温和、耐心、尊重、稳定的情感陪伴、情绪支持、回忆唤起、生活关怀和风险识别服务。你不是医生,也不是心理治疗师,不能替代专业诊疗,但应在必要时建议联系家属、社区、医生或急救资源。\n"
|
private static final String DEFAULT_INSTRUCTIONS = "你是一名面向中国老年群体的心理陪伴 AI。你的职责是为老人提供温和、耐心、尊重、稳定的情感陪伴、情绪支持、回忆唤起、生活关怀和风险识别服务。\n"
|
||||||
+ "\n"
|
+ "\n"
|
||||||
+ "请始终遵循以下原则:\n"
|
+ "请始终遵循以下原则:\n"
|
||||||
+ "\n"
|
+ "\n"
|
||||||
+ "先共情,再回应问题;\n"
|
|
||||||
+ "\n"
|
|
||||||
+ "先接住情绪,再给建议;\n"
|
+ "先接住情绪,再给建议;\n"
|
||||||
+ "\n"
|
+ "\n"
|
||||||
+ "使用简短、朴素、生活化的中文,始终用“您”称呼;\n"
|
+ "使用简短、朴素、生活化的中文,始终用“您”称呼;\n"
|
||||||
+ "\n"
|
+ "\n"
|
||||||
+ "不说教、不训诫、不否定、不敷衍;\n"
|
|
||||||
+ "\n"
|
|
||||||
+ "不把老人当小孩,不制造虚假亲密;\n"
|
|
||||||
+ "\n"
|
|
||||||
+ "一次不提供超过两个建议;\n"
|
+ "一次不提供超过两个建议;\n"
|
||||||
+ "\n"
|
+ "\n"
|
||||||
+ "鼓励老人表达感受,可适度引导回忆人生经历、兴趣、家庭和过往成就;\n"
|
+ "鼓励老人表达感受,可适度引导回忆人生经历、兴趣、家庭和过往成就;\n"
|
||||||
+ "\n"
|
+ "\n"
|
||||||
+ "对孤独、丧偶、被忽视、无价值感、失眠、焦虑、怕拖累家人等议题保持高度敏感;\n"
|
|
||||||
+ "\n"
|
|
||||||
+ "若出现自杀、自伤、重度绝望、幻觉妄想、意识混乱、虐待、严重医疗异常等风险,必须立即建议联系真人帮助和专业机构;\n"
|
|
||||||
+ "\n"
|
|
||||||
+ "不编造医疗知识、政策信息、机构电话,不确定时明确说明。\n"
|
|
||||||
+ "\n"
|
|
||||||
+ "你的回复风格应温和、平稳、尊重、有耐心。每次回答尽量包含:一句共情、一句理解或复述、一句温和追问或一个微小建议。核心目标是让老人感到:我被听见了,我被尊重了,有人愿意陪我。";
|
+ "你的回复风格应温和、平稳、尊重、有耐心。每次回答尽量包含:一句共情、一句理解或复述、一句温和追问或一个微小建议。核心目标是让老人感到:我被听见了,我被尊重了,有人愿意陪我。";
|
||||||
|
|
||||||
private final OkHttpClient okHttpClient;
|
private final OkHttpClient okHttpClient;
|
||||||
|
|||||||
@@ -4,6 +4,9 @@ public final class DashScopeConfig {
|
|||||||
|
|
||||||
public static final String DEFAULT_API_KEY = "sk-7c32083ace99472e97209e611b9ecbb1";
|
public static final String DEFAULT_API_KEY = "sk-7c32083ace99472e97209e611b9ecbb1";
|
||||||
public static final String REALTIME_BASE_URL = "wss://dashscope.aliyuncs.com/api-ws/v1/realtime";
|
public static final String REALTIME_BASE_URL = "wss://dashscope.aliyuncs.com/api-ws/v1/realtime";
|
||||||
|
public static final String MULTIMODAL_GENERATION_URL = "https://dashscope.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation";
|
||||||
|
public static final String PSYCHOLOGY_SUBAGENT_MODEL = "qwen3.6-plus";
|
||||||
|
public static final String PSYCHOLOGY_KNOWLEDGE_ASSET_PATH = "knowledge/who_icope_2024.txt";
|
||||||
public static final String VOICE_CLONE_URL = "https://dashscope.aliyuncs.com/api/v1/services/audio/tts/customization";
|
public static final String VOICE_CLONE_URL = "https://dashscope.aliyuncs.com/api/v1/services/audio/tts/customization";
|
||||||
public static final String VOICE_CLONE_MODEL = "qwen-voice-enrollment";
|
public static final String VOICE_CLONE_MODEL = "qwen-voice-enrollment";
|
||||||
public static final String VOICE_CLONE_TTS_MODEL = "qwen3-tts-vc-realtime-2026-01-15";
|
public static final String VOICE_CLONE_TTS_MODEL = "qwen3-tts-vc-realtime-2026-01-15";
|
||||||
|
|||||||
@@ -0,0 +1,21 @@
|
|||||||
|
package ai.guiji.duix.test.service
|
||||||
|
|
||||||
|
import android.content.Context
|
||||||
|
|
||||||
|
class DebugSettingsStore(context: Context) {
|
||||||
|
|
||||||
|
private val prefs = context.getSharedPreferences(PREFS_NAME, Context.MODE_PRIVATE)
|
||||||
|
|
||||||
|
fun isChatDebugEnabled(): Boolean {
|
||||||
|
return prefs.getBoolean(KEY_CHAT_DEBUG_ENABLED, false)
|
||||||
|
}
|
||||||
|
|
||||||
|
fun setChatDebugEnabled(enabled: Boolean) {
|
||||||
|
prefs.edit().putBoolean(KEY_CHAT_DEBUG_ENABLED, enabled).apply()
|
||||||
|
}
|
||||||
|
|
||||||
|
companion object {
|
||||||
|
private const val PREFS_NAME = "app_debug_settings"
|
||||||
|
private const val KEY_CHAT_DEBUG_ENABLED = "chat_debug_enabled"
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,194 @@
|
|||||||
|
package ai.guiji.duix.test.service
|
||||||
|
|
||||||
|
import ai.guiji.duix.test.App
|
||||||
|
import android.content.Context
|
||||||
|
import okhttp3.MediaType.Companion.toMediaType
|
||||||
|
import okhttp3.Request
|
||||||
|
import okhttp3.RequestBody.Companion.toRequestBody
|
||||||
|
import org.json.JSONArray
|
||||||
|
import org.json.JSONObject
|
||||||
|
import java.util.concurrent.TimeUnit
|
||||||
|
|
||||||
|
class PsychologyKnowledgeAgentService(
|
||||||
|
context: Context,
|
||||||
|
) {
|
||||||
|
|
||||||
|
private val appContext = context.applicationContext
|
||||||
|
private val client = App.getOkHttpClient().newBuilder()
|
||||||
|
.connectTimeout(30, TimeUnit.SECONDS)
|
||||||
|
.writeTimeout(120, TimeUnit.SECONDS)
|
||||||
|
.readTimeout(180, TimeUnit.SECONDS)
|
||||||
|
.callTimeout(180, TimeUnit.SECONDS)
|
||||||
|
.build()
|
||||||
|
|
||||||
|
private val knowledgeText: String by lazy {
|
||||||
|
appContext.assets.open(DashScopeConfig.PSYCHOLOGY_KNOWLEDGE_ASSET_PATH)
|
||||||
|
.bufferedReader(Charsets.UTF_8)
|
||||||
|
.use { reader ->
|
||||||
|
reader.readText()
|
||||||
|
.replace('\u000c', '\n')
|
||||||
|
.replace("\u0000", "")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fun consultKnowledge(
|
||||||
|
question: String,
|
||||||
|
focus: String? = null,
|
||||||
|
): KnowledgeAnswer {
|
||||||
|
val normalizedQuestion = question.trim()
|
||||||
|
require(normalizedQuestion.isNotEmpty()) { "问题不能为空" }
|
||||||
|
|
||||||
|
val request = Request.Builder()
|
||||||
|
.url(DashScopeConfig.MULTIMODAL_GENERATION_URL)
|
||||||
|
.addHeader("Authorization", "Bearer " + DashScopeConfig.DEFAULT_API_KEY)
|
||||||
|
.addHeader("Content-Type", "application/json")
|
||||||
|
.post(
|
||||||
|
buildRequestPayload(
|
||||||
|
normalizedQuestion,
|
||||||
|
focus?.trim().takeIf { !it.isNullOrEmpty() },
|
||||||
|
).toString().toRequestBody(JSON_MEDIA_TYPE)
|
||||||
|
)
|
||||||
|
.build()
|
||||||
|
|
||||||
|
client.newCall(request).execute().use { response ->
|
||||||
|
val body = response.body?.string().orEmpty()
|
||||||
|
if (!response.isSuccessful) {
|
||||||
|
throw IllegalStateException(parseErrorMessage(body).ifEmpty { "心理学知识子代理调用失败" })
|
||||||
|
}
|
||||||
|
|
||||||
|
val json = JSONObject(body)
|
||||||
|
val message = json.optJSONObject("output")
|
||||||
|
?.optJSONArray("choices")
|
||||||
|
?.optJSONObject(0)
|
||||||
|
?.optJSONObject("message")
|
||||||
|
val content = extractMessageContent(message)
|
||||||
|
if (content.isBlank()) {
|
||||||
|
throw IllegalStateException("心理学知识子代理没有返回内容")
|
||||||
|
}
|
||||||
|
|
||||||
|
val usage = json.optJSONObject("usage")
|
||||||
|
return KnowledgeAnswer(
|
||||||
|
outputText = content.trim(),
|
||||||
|
model = DashScopeConfig.PSYCHOLOGY_SUBAGENT_MODEL,
|
||||||
|
requestId = json.optString("request_id").ifBlank { null },
|
||||||
|
inputTokens = usage?.optInt("input_tokens") ?: 0,
|
||||||
|
outputTokens = usage?.optInt("output_tokens") ?: 0,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun buildRequestPayload(
|
||||||
|
question: String,
|
||||||
|
focus: String?,
|
||||||
|
): JSONObject {
|
||||||
|
return JSONObject().apply {
|
||||||
|
put("model", DashScopeConfig.PSYCHOLOGY_SUBAGENT_MODEL)
|
||||||
|
put(
|
||||||
|
"input",
|
||||||
|
JSONObject().apply {
|
||||||
|
put(
|
||||||
|
"messages",
|
||||||
|
JSONArray()
|
||||||
|
.put(
|
||||||
|
JSONObject().apply {
|
||||||
|
put("role", "system")
|
||||||
|
put("content", buildSystemPrompt())
|
||||||
|
}
|
||||||
|
)
|
||||||
|
.put(
|
||||||
|
JSONObject().apply {
|
||||||
|
put("role", "user")
|
||||||
|
put("content", buildUserPrompt(question, focus))
|
||||||
|
}
|
||||||
|
)
|
||||||
|
)
|
||||||
|
}
|
||||||
|
)
|
||||||
|
put(
|
||||||
|
"parameters",
|
||||||
|
JSONObject().apply {
|
||||||
|
put("result_format", "message")
|
||||||
|
put("enable_thinking", false)
|
||||||
|
put("temperature", 0.2)
|
||||||
|
put("max_tokens", 900)
|
||||||
|
}
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun buildSystemPrompt(): String {
|
||||||
|
return buildString {
|
||||||
|
append(knowledgeText)
|
||||||
|
append("以上是完整知识库全文\n")
|
||||||
|
append("你是一个面向老年陪伴场景的权威照护知识子代理。")
|
||||||
|
append("你只能依据系统里提供的 WHO ICOPE 2024 全文回答,不要编造,不要输出与原文冲突的结论。")
|
||||||
|
append("如果知识库未提供足够直接依据,要明确说明。")
|
||||||
|
append("请使用简体中文输出,提供权威心理学指示和说明。不需要包括生活化语言,你只是一个查询专家,不负责指导。直接输出无格式纯文本,答案简练不超过100字。")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun buildUserPrompt(
|
||||||
|
question: String,
|
||||||
|
focus: String?,
|
||||||
|
): String {
|
||||||
|
return buildString {
|
||||||
|
append("请基于上面的 WHO ICOPE 2024 全文,回答下面这个需要查询的问题。\n")
|
||||||
|
append("核心问题:")
|
||||||
|
append(question)
|
||||||
|
if (!focus.isNullOrBlank()) {
|
||||||
|
append("\n特别关注:")
|
||||||
|
append(focus)
|
||||||
|
}
|
||||||
|
append("\n请避免空泛安慰,优先给出可落地的权威说明。")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun extractMessageContent(message: JSONObject?): String {
|
||||||
|
if (message == null) {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
val content = message.opt("content")
|
||||||
|
return when (content) {
|
||||||
|
is String -> content
|
||||||
|
is JSONArray -> buildString {
|
||||||
|
for (index in 0 until content.length()) {
|
||||||
|
val part = content.opt(index)
|
||||||
|
val text = when (part) {
|
||||||
|
is JSONObject -> part.optString("text")
|
||||||
|
is String -> part
|
||||||
|
else -> ""
|
||||||
|
}.trim()
|
||||||
|
if (text.isNotEmpty()) {
|
||||||
|
if (isNotEmpty()) {
|
||||||
|
append('\n')
|
||||||
|
}
|
||||||
|
append(text)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else -> ""
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun parseErrorMessage(body: String): String {
|
||||||
|
return try {
|
||||||
|
val json = JSONObject(body)
|
||||||
|
json.optJSONObject("error")?.optString("message").orEmpty()
|
||||||
|
.ifBlank { json.optString("message") }
|
||||||
|
} catch (_: Exception) {
|
||||||
|
""
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
data class KnowledgeAnswer(
|
||||||
|
val outputText: String,
|
||||||
|
val model: String,
|
||||||
|
val requestId: String?,
|
||||||
|
val inputTokens: Int,
|
||||||
|
val outputTokens: Int,
|
||||||
|
)
|
||||||
|
|
||||||
|
companion object {
|
||||||
|
private val JSON_MEDIA_TYPE = "application/json; charset=utf-8".toMediaType()
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -13,7 +13,9 @@ import ai.guiji.duix.test.databinding.ActivityCallBinding
|
|||||||
import ai.guiji.duix.test.realtime.OmniRealtimeClient
|
import ai.guiji.duix.test.realtime.OmniRealtimeClient
|
||||||
import ai.guiji.duix.test.realtime.QwenTtsRealtimeClient
|
import ai.guiji.duix.test.realtime.QwenTtsRealtimeClient
|
||||||
import ai.guiji.duix.test.service.DashScopeConfig
|
import ai.guiji.duix.test.service.DashScopeConfig
|
||||||
|
import ai.guiji.duix.test.service.DebugSettingsStore
|
||||||
import ai.guiji.duix.test.service.FamilyBridgeService
|
import ai.guiji.duix.test.service.FamilyBridgeService
|
||||||
|
import ai.guiji.duix.test.service.PsychologyKnowledgeAgentService
|
||||||
import ai.guiji.duix.test.ui.adapter.MotionAdapter
|
import ai.guiji.duix.test.ui.adapter.MotionAdapter
|
||||||
import ai.guiji.duix.test.ui.dialog.AudioRecordDialog
|
import ai.guiji.duix.test.ui.dialog.AudioRecordDialog
|
||||||
import ai.guiji.duix.test.ui.widget.AudioSpectrumView
|
import ai.guiji.duix.test.ui.widget.AudioSpectrumView
|
||||||
@@ -32,9 +34,7 @@ import android.widget.Toast
|
|||||||
import com.bumptech.glide.Glide
|
import com.bumptech.glide.Glide
|
||||||
import org.json.JSONArray
|
import org.json.JSONArray
|
||||||
import org.json.JSONObject
|
import org.json.JSONObject
|
||||||
import java.io.File
|
|
||||||
import java.io.FileInputStream
|
import java.io.FileInputStream
|
||||||
import java.io.FileOutputStream
|
|
||||||
import java.util.UUID
|
import java.util.UUID
|
||||||
import java.util.concurrent.ExecutorService
|
import java.util.concurrent.ExecutorService
|
||||||
import java.util.concurrent.Executors
|
import java.util.concurrent.Executors
|
||||||
@@ -55,18 +55,22 @@ class CallActivity : BaseActivity() {
|
|||||||
private fun applyMessage(msg: String){
|
private fun applyMessage(msg: String){
|
||||||
if (debug){
|
if (debug){
|
||||||
runOnUiThread {
|
runOnUiThread {
|
||||||
binding.tvDebug.visibility = View.VISIBLE
|
binding.scrollDebug.visibility = View.VISIBLE
|
||||||
if (mMessage.length > 10000){
|
if (mMessage.length > 60000){
|
||||||
mMessage = ""
|
mMessage = mMessage.takeLast(40000)
|
||||||
}
|
}
|
||||||
mMessage = "${StringUtils.dateToStringMS4()} $msg\n$mMessage"
|
mMessage += "${StringUtils.dateToStringMS4()} $msg\n"
|
||||||
binding.tvDebug.text = mMessage
|
binding.tvDebug.text = mMessage
|
||||||
|
binding.scrollDebug.post {
|
||||||
|
binding.scrollDebug.fullScroll(View.FOCUS_DOWN)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
private lateinit var binding: ActivityCallBinding
|
private lateinit var binding: ActivityCallBinding
|
||||||
|
private val debugSettingsStore by lazy { DebugSettingsStore(applicationContext) }
|
||||||
private var duix: DUIX? = null
|
private var duix: DUIX? = null
|
||||||
private var mDUIXRender: DUIXRenderer? = null
|
private var mDUIXRender: DUIXRenderer? = null
|
||||||
private var mModelInfo: ModelInfo?=null // 加载的模型信息
|
private var mModelInfo: ModelInfo?=null // 加载的模型信息
|
||||||
@@ -80,13 +84,14 @@ class CallActivity : BaseActivity() {
|
|||||||
private val mainHandler = Handler(Looper.getMainLooper())
|
private val mainHandler = Handler(Looper.getMainLooper())
|
||||||
private var assistantTextBuffer = StringBuilder()
|
private var assistantTextBuffer = StringBuilder()
|
||||||
private var hasStartedPushForResponse = false
|
private var hasStartedPushForResponse = false
|
||||||
private val enableWebSearch = false
|
private val enableWebSearch = true
|
||||||
private val enableToolCalling = true
|
private val enableToolCalling = true
|
||||||
private var isCloneMode = false
|
private var isCloneMode = false
|
||||||
private var clonedVoice = ""
|
private var clonedVoice = ""
|
||||||
private var cloneTtsModel = DashScopeConfig.VOICE_CLONE_TTS_MODEL
|
private var cloneTtsModel = DashScopeConfig.VOICE_CLONE_TTS_MODEL
|
||||||
private var ttsRealtimeClient: QwenTtsRealtimeClient? = null
|
private var ttsRealtimeClient: QwenTtsRealtimeClient? = null
|
||||||
private val familyBridgeService by lazy { FamilyBridgeService(applicationContext) }
|
private val familyBridgeService by lazy { FamilyBridgeService(applicationContext) }
|
||||||
|
private val psychologyKnowledgeAgentService by lazy { PsychologyKnowledgeAgentService(applicationContext) }
|
||||||
private val bridgeExecutor: ExecutorService = Executors.newSingleThreadExecutor()
|
private val bridgeExecutor: ExecutorService = Executors.newSingleThreadExecutor()
|
||||||
private var currentSessionId: String? = null
|
private var currentSessionId: String? = null
|
||||||
private var pendingConversationPrompt: FamilyBridgeService.ConversationPromptInjection? = null
|
private var pendingConversationPrompt: FamilyBridgeService.ConversationPromptInjection? = null
|
||||||
@@ -103,7 +108,7 @@ class CallActivity : BaseActivity() {
|
|||||||
|
|
||||||
modelUrl = intent.getStringExtra("modelUrl") ?: ""
|
modelUrl = intent.getStringExtra("modelUrl") ?: ""
|
||||||
voice = intent.getStringExtra("voice") ?: "Ethan"
|
voice = intent.getStringExtra("voice") ?: "Ethan"
|
||||||
debug = intent.getBooleanExtra("debug", false)
|
debug = intent.getBooleanExtra("debug", debugSettingsStore.isChatDebugEnabled())
|
||||||
isCloneMode = intent.getBooleanExtra("cloneMode", false)
|
isCloneMode = intent.getBooleanExtra("cloneMode", false)
|
||||||
clonedVoice = intent.getStringExtra("clonedVoice") ?: ""
|
clonedVoice = intent.getStringExtra("clonedVoice") ?: ""
|
||||||
cloneTtsModel = intent.getStringExtra("cloneTtsModel") ?: DashScopeConfig.VOICE_CLONE_TTS_MODEL
|
cloneTtsModel = intent.getStringExtra("cloneTtsModel") ?: DashScopeConfig.VOICE_CLONE_TTS_MODEL
|
||||||
@@ -136,16 +141,6 @@ class CallActivity : BaseActivity() {
|
|||||||
requestPermission(arrayOf(Manifest.permission.RECORD_AUDIO), 1)
|
requestPermission(arrayOf(Manifest.permission.RECORD_AUDIO), 1)
|
||||||
}
|
}
|
||||||
|
|
||||||
binding.btnPlayPCM.setOnClickListener {
|
|
||||||
applyMessage("start play pcm")
|
|
||||||
playPCMStream()
|
|
||||||
}
|
|
||||||
|
|
||||||
binding.btnPlayWAV.setOnClickListener {
|
|
||||||
applyMessage("start play wav")
|
|
||||||
playWAVFile()
|
|
||||||
}
|
|
||||||
|
|
||||||
binding.btnRandomMotion.setOnClickListener {
|
binding.btnRandomMotion.setOnClickListener {
|
||||||
applyMessage("start random motion")
|
applyMessage("start random motion")
|
||||||
duix?.startRandomMotion(true)
|
duix?.startRandomMotion(true)
|
||||||
@@ -238,16 +233,9 @@ class CallActivity : BaseActivity() {
|
|||||||
applyMessage("init ok")
|
applyMessage("init ok")
|
||||||
runOnUiThread {
|
runOnUiThread {
|
||||||
binding.btnRecord.isEnabled = true
|
binding.btnRecord.isEnabled = true
|
||||||
binding.btnPlayPCM.isEnabled = true
|
|
||||||
binding.btnPlayWAV.isEnabled = true
|
|
||||||
binding.switchMute.isEnabled = true
|
binding.switchMute.isEnabled = true
|
||||||
binding.btnAIConversation.isEnabled = true
|
binding.btnAIConversation.isEnabled = true
|
||||||
|
|
||||||
if (debug) {
|
|
||||||
binding.btnPlayPCM.visibility = View.VISIBLE
|
|
||||||
binding.btnPlayWAV.visibility = View.VISIBLE
|
|
||||||
}
|
|
||||||
|
|
||||||
mModelInfo?.let { modelInfo ->
|
mModelInfo?.let { modelInfo ->
|
||||||
if (modelInfo.motionRegions.isNotEmpty()) {
|
if (modelInfo.motionRegions.isNotEmpty()) {
|
||||||
val names = ArrayList<String>()
|
val names = ArrayList<String>()
|
||||||
@@ -305,6 +293,9 @@ class CallActivity : BaseActivity() {
|
|||||||
return buildString {
|
return buildString {
|
||||||
append(OmniRealtimeClient.getDefaultInstructions())
|
append(OmniRealtimeClient.getDefaultInstructions())
|
||||||
append("\n\n")
|
append("\n\n")
|
||||||
|
append("当用户需要关于记忆下降、情绪低落、焦虑失眠、孤独感、认知照护、老年照护路径或家属陪伴方式的权威依据时,优先调用 consult_psychology_knowledge_base 工具,不要凭经验给出确定性的专业判断。\n")
|
||||||
|
append("调用 consult_psychology_knowledge_base 时,把老人当前最核心的问题写入 question;如果你想限定查询角度,例如陪伴建议、风险判断、是否需要就医,就写入 focus。\n")
|
||||||
|
append("如果只是日常闲聊或普通安慰,不必调用 consult_psychology_knowledge_base 工具。\n")
|
||||||
append("当用户明确要给儿子、女儿、老伴、家里人或亲友带话、留言、报平安时,必须调用 leave_message_for_family 工具,不要假装已经转达。\n")
|
append("当用户明确要给儿子、女儿、老伴、家里人或亲友带话、留言、报平安时,必须调用 leave_message_for_family 工具,不要假装已经转达。\n")
|
||||||
append("当用户问家里人有没有留言、想听家人说了什么、想看看新消息时,必须调用 fetch_family_messages 工具,不要自己编造留言。\n")
|
append("当用户问家里人有没有留言、想听家人说了什么、想看看新消息时,必须调用 fetch_family_messages 工具,不要自己编造留言。\n")
|
||||||
append("工具执行结果返回后,再用温和、简短、生活化的中文向老人说明情况。")
|
append("工具执行结果返回后,再用温和、简短、生活化的中文向老人说明情况。")
|
||||||
@@ -350,7 +341,27 @@ class CallActivity : BaseActivity() {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
val psychologyKnowledgeParameters = JSONObject().apply {
|
||||||
|
put("type", "object")
|
||||||
|
put("properties", JSONObject().apply {
|
||||||
|
put("question", JSONObject().apply {
|
||||||
|
put("type", "string")
|
||||||
|
put("description", "需要从权威知识库中查询的核心问题,应尽量包含老人当前的真实困扰或具体疑问。")
|
||||||
|
})
|
||||||
|
put("focus", JSONObject().apply {
|
||||||
|
put("type", "string")
|
||||||
|
put("description", "希望子代理特别聚焦的角度,例如陪伴建议、风险判断、何时就医、家属沟通方式。")
|
||||||
|
})
|
||||||
|
})
|
||||||
|
put("required", JSONArray().put("question"))
|
||||||
|
}
|
||||||
|
|
||||||
return listOf(
|
return listOf(
|
||||||
|
OmniRealtimeClient.ToolDefinition(
|
||||||
|
"consult_psychology_knowledge_base",
|
||||||
|
"当需要基于 WHO ICOPE 2024 等权威照护知识回答认知、情绪、睡眠、孤独、照护与家属支持问题时使用。当老人提出具体问题比如食欲不振时,必须使用此工具进行查询,而不是凭经验直接回答。",
|
||||||
|
psychologyKnowledgeParameters,
|
||||||
|
),
|
||||||
OmniRealtimeClient.ToolDefinition(
|
OmniRealtimeClient.ToolDefinition(
|
||||||
"leave_message_for_family",
|
"leave_message_for_family",
|
||||||
"当老人明确要求给家里人带话、留言、报平安或请转告时使用。",
|
"当老人明确要求给家里人带话、留言、报平安或请转告时使用。",
|
||||||
@@ -385,46 +396,6 @@ class CallActivity : BaseActivity() {
|
|||||||
bridgeExecutor.shutdownNow()
|
bridgeExecutor.shutdownNow()
|
||||||
}
|
}
|
||||||
|
|
||||||
private fun playPCMStream(){
|
|
||||||
val thread = Thread {
|
|
||||||
duix?.startPush()
|
|
||||||
val inputStream = assets.open("pcm/2.pcm")
|
|
||||||
val buffer = ByteArray(320)
|
|
||||||
var length: Int
|
|
||||||
while (inputStream.read(buffer).also { length = it } > 0){
|
|
||||||
val data = buffer.copyOfRange(0, length)
|
|
||||||
duix?.pushPcm(data)
|
|
||||||
}
|
|
||||||
duix?.stopPush()
|
|
||||||
inputStream.close()
|
|
||||||
}
|
|
||||||
thread.start()
|
|
||||||
}
|
|
||||||
|
|
||||||
private fun playWAVFile(){
|
|
||||||
val thread = Thread {
|
|
||||||
val wavName = "1.wav"
|
|
||||||
val wavFile = File(mContext.externalCacheDir, wavName)
|
|
||||||
if (!wavFile.exists()){
|
|
||||||
// copy assets -> sd card
|
|
||||||
val inputStream = assets.open("wav/$wavName")
|
|
||||||
if (!mContext.externalCacheDir!!.exists()){
|
|
||||||
mContext.externalCacheDir!!.mkdirs()
|
|
||||||
}
|
|
||||||
val out = FileOutputStream(wavFile)
|
|
||||||
val buffer = ByteArray(1024)
|
|
||||||
var length: Int
|
|
||||||
while ((inputStream.read(buffer).also { length = it }) > 0) {
|
|
||||||
out.write(buffer, 0, length)
|
|
||||||
}
|
|
||||||
out.close()
|
|
||||||
inputStream.close()
|
|
||||||
}
|
|
||||||
duix?.playAudio(wavFile.absolutePath)
|
|
||||||
}
|
|
||||||
thread.start()
|
|
||||||
}
|
|
||||||
|
|
||||||
override fun permissionsGet(get: Boolean, code: Int) {
|
override fun permissionsGet(get: Boolean, code: Int) {
|
||||||
super.permissionsGet(get, code)
|
super.permissionsGet(get, code)
|
||||||
when (code) {
|
when (code) {
|
||||||
@@ -496,6 +467,9 @@ class CallActivity : BaseActivity() {
|
|||||||
binding.btnCameraToggle.contentDescription = getString(R.string.camera_off)
|
binding.btnCameraToggle.contentDescription = getString(R.string.camera_off)
|
||||||
isCameraEnabled = false
|
isCameraEnabled = false
|
||||||
applyMessage("AI对话: 已连接, model=${OmniRealtimeClient.DEFAULT_MODEL}, search=${if (enableWebSearch) "on" else "off"}, tools=${if (enableToolCalling) "on" else "off"}, cloneMode=$isCloneMode")
|
applyMessage("AI对话: 已连接, model=${OmniRealtimeClient.DEFAULT_MODEL}, search=${if (enableWebSearch) "on" else "off"}, tools=${if (enableToolCalling) "on" else "off"}, cloneMode=$isCloneMode")
|
||||||
|
if (enableWebSearch && enableToolCalling) {
|
||||||
|
// applyMessage("AI对话: 当前配置请求 search=on 且 tools=on,但 Qwen Omni Realtime 协议规定两者不兼容,实际下发时将由服务端连接配置决定仅生效其中一项")
|
||||||
|
}
|
||||||
if (initialConversationPrompt != null) {
|
if (initialConversationPrompt != null) {
|
||||||
applyMessage("AI对话: 检测到家属留言,准备主动播报")
|
applyMessage("AI对话: 检测到家属留言,准备主动播报")
|
||||||
val requested = omniClient?.requestResponse() == true
|
val requested = omniClient?.requestResponse() == true
|
||||||
@@ -554,6 +528,7 @@ class CallActivity : BaseActivity() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
override fun onUserTranscript(transcript: String) {
|
override fun onUserTranscript(transcript: String) {
|
||||||
|
applyMessage("用户语音转写: $transcript")
|
||||||
mainHandler.post {
|
mainHandler.post {
|
||||||
binding.tvSubtitle.text = getString(R.string.ai_call_user_said, transcript)
|
binding.tvSubtitle.text = getString(R.string.ai_call_user_said, transcript)
|
||||||
}
|
}
|
||||||
@@ -568,6 +543,7 @@ class CallActivity : BaseActivity() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
override fun onAssistantTranscriptDone(transcript: String) {
|
override fun onAssistantTranscriptDone(transcript: String) {
|
||||||
|
applyMessage("助手回复: $transcript")
|
||||||
mainHandler.post {
|
mainHandler.post {
|
||||||
assistantTextBuffer.clear()
|
assistantTextBuffer.clear()
|
||||||
assistantTextBuffer.append(transcript)
|
assistantTextBuffer.append(transcript)
|
||||||
@@ -579,6 +555,7 @@ class CallActivity : BaseActivity() {
|
|||||||
override fun onToolCallRequested(toolName: String, callId: String, argumentsJson: String) {
|
override fun onToolCallRequested(toolName: String, callId: String, argumentsJson: String) {
|
||||||
mainHandler.post {
|
mainHandler.post {
|
||||||
binding.tvSubtitle.text = when (toolName) {
|
binding.tvSubtitle.text = when (toolName) {
|
||||||
|
"consult_psychology_knowledge_base" -> "我来帮您查一下更稳妥的照护资料。"
|
||||||
"leave_message_for_family" -> "我来帮您把这句话记给家里人。"
|
"leave_message_for_family" -> "我来帮您把这句话记给家里人。"
|
||||||
"fetch_family_messages" -> "我来帮您看看家里人的新留言。"
|
"fetch_family_messages" -> "我来帮您看看家里人的新留言。"
|
||||||
else -> getString(R.string.ai_call_thinking)
|
else -> getString(R.string.ai_call_thinking)
|
||||||
@@ -888,6 +865,7 @@ class CallActivity : BaseActivity() {
|
|||||||
enqueueBridgeTask("工具调用处理失败") {
|
enqueueBridgeTask("工具调用处理失败") {
|
||||||
val arguments = runCatching { JSONObject(argumentsJson) }.getOrElse { JSONObject() }
|
val arguments = runCatching { JSONObject(argumentsJson) }.getOrElse { JSONObject() }
|
||||||
val result = executeRealtimeTool(toolName, callId, arguments)
|
val result = executeRealtimeTool(toolName, callId, arguments)
|
||||||
|
applyMessage("AI工具输出: $toolName, callId=$callId, status=${result.status}, output=${result.outputText}")
|
||||||
uploadConversationTurnInternal(
|
uploadConversationTurnInternal(
|
||||||
FamilyBridgeService.ROLE_TOOL,
|
FamilyBridgeService.ROLE_TOOL,
|
||||||
result.outputText,
|
result.outputText,
|
||||||
@@ -911,6 +889,7 @@ class CallActivity : BaseActivity() {
|
|||||||
arguments: JSONObject,
|
arguments: JSONObject,
|
||||||
): ToolExecutionResult {
|
): ToolExecutionResult {
|
||||||
return when (toolName) {
|
return when (toolName) {
|
||||||
|
"consult_psychology_knowledge_base" -> executePsychologyKnowledgeTool(callId, arguments)
|
||||||
"leave_message_for_family" -> executeLeaveMessageTool(callId, arguments)
|
"leave_message_for_family" -> executeLeaveMessageTool(callId, arguments)
|
||||||
"fetch_family_messages" -> executeFetchFamilyMessagesTool(callId, arguments)
|
"fetch_family_messages" -> executeFetchFamilyMessagesTool(callId, arguments)
|
||||||
else -> {
|
else -> {
|
||||||
@@ -930,6 +909,56 @@ class CallActivity : BaseActivity() {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private fun executePsychologyKnowledgeTool(callId: String, arguments: JSONObject): ToolExecutionResult {
|
||||||
|
val question = arguments.optString("question").trim()
|
||||||
|
val focus = arguments.optString("focus").trim().ifBlank { null }
|
||||||
|
|
||||||
|
if (question.isEmpty()) {
|
||||||
|
val outputText = "还没有拿到要查询的具体问题,请先用一句话说明想查什么。"
|
||||||
|
runCatching {
|
||||||
|
familyBridgeService.recordToolCall(
|
||||||
|
toolName = "consult_psychology_knowledge_base",
|
||||||
|
arguments = arguments,
|
||||||
|
outputText = outputText,
|
||||||
|
status = FamilyBridgeService.STATUS_FAILURE,
|
||||||
|
callId = callId,
|
||||||
|
sessionId = currentSessionId,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
return ToolExecutionResult(outputText, FamilyBridgeService.STATUS_FAILURE)
|
||||||
|
}
|
||||||
|
|
||||||
|
return runCatching {
|
||||||
|
val result = psychologyKnowledgeAgentService.consultKnowledge(question, focus)
|
||||||
|
familyBridgeService.recordToolCall(
|
||||||
|
toolName = "consult_psychology_knowledge_base",
|
||||||
|
arguments = arguments,
|
||||||
|
outputText = result.outputText,
|
||||||
|
status = FamilyBridgeService.STATUS_SUCCESS,
|
||||||
|
callId = callId,
|
||||||
|
sessionId = currentSessionId,
|
||||||
|
)
|
||||||
|
applyMessage(
|
||||||
|
"心理学子代理完成: model=${result.model}, requestId=${result.requestId ?: "-"}, inputTokens=${result.inputTokens}, outputTokens=${result.outputTokens}"
|
||||||
|
)
|
||||||
|
ToolExecutionResult(result.outputText, FamilyBridgeService.STATUS_SUCCESS)
|
||||||
|
}.getOrElse { error ->
|
||||||
|
val outputText = "权威照护资料查询暂时失败。请先基于安全原则回应:保持温和陪伴,避免给出确定性的医学结论;如果涉及持续恶化、明显痛苦、自伤他伤或急性危险,请建议尽快联系家属、医生或急救。"
|
||||||
|
runCatching {
|
||||||
|
familyBridgeService.recordToolCall(
|
||||||
|
toolName = "consult_psychology_knowledge_base",
|
||||||
|
arguments = arguments,
|
||||||
|
outputText = outputText,
|
||||||
|
status = FamilyBridgeService.STATUS_FAILURE,
|
||||||
|
callId = callId,
|
||||||
|
sessionId = currentSessionId,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
applyMessage("心理学子代理失败: ${error.message}")
|
||||||
|
ToolExecutionResult(outputText, FamilyBridgeService.STATUS_FAILURE)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
private fun executeLeaveMessageTool(callId: String, arguments: JSONObject): ToolExecutionResult {
|
private fun executeLeaveMessageTool(callId: String, arguments: JSONObject): ToolExecutionResult {
|
||||||
val message = arguments.optString("message").trim()
|
val message = arguments.optString("message").trim()
|
||||||
val relation = arguments.optString("recipientRelation").trim().ifBlank { "家里人" }
|
val relation = arguments.optString("recipientRelation").trim().ifBlank { "家里人" }
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ import ai.guiji.duix.test.R
|
|||||||
import ai.guiji.duix.test.databinding.ActivityMainBinding
|
import ai.guiji.duix.test.databinding.ActivityMainBinding
|
||||||
import ai.guiji.duix.test.model.AvatarCatalog
|
import ai.guiji.duix.test.model.AvatarCatalog
|
||||||
import ai.guiji.duix.test.model.AvatarModel
|
import ai.guiji.duix.test.model.AvatarModel
|
||||||
|
import ai.guiji.duix.test.service.DebugSettingsStore
|
||||||
import ai.guiji.duix.test.service.FamilyBridgeService
|
import ai.guiji.duix.test.service.FamilyBridgeService
|
||||||
import ai.guiji.duix.test.ui.adapter.AvatarAdapter
|
import ai.guiji.duix.test.ui.adapter.AvatarAdapter
|
||||||
import ai.guiji.duix.test.ui.dialog.LoadingDialog
|
import ai.guiji.duix.test.ui.dialog.LoadingDialog
|
||||||
@@ -19,6 +20,7 @@ import java.io.File
|
|||||||
class MainActivity : BaseActivity() {
|
class MainActivity : BaseActivity() {
|
||||||
|
|
||||||
private lateinit var binding: ActivityMainBinding
|
private lateinit var binding: ActivityMainBinding
|
||||||
|
private val debugSettingsStore by lazy { DebugSettingsStore(applicationContext) }
|
||||||
private val familyBridgeService by lazy { FamilyBridgeService(applicationContext) }
|
private val familyBridgeService by lazy { FamilyBridgeService(applicationContext) }
|
||||||
private var mLoadingDialog: LoadingDialog? = null
|
private var mLoadingDialog: LoadingDialog? = null
|
||||||
private var mLastProgress = 0
|
private var mLastProgress = 0
|
||||||
@@ -89,7 +91,7 @@ class MainActivity : BaseActivity() {
|
|||||||
val intent = Intent(mContext, CallActivity::class.java)
|
val intent = Intent(mContext, CallActivity::class.java)
|
||||||
intent.putExtra("modelUrl", model.modelUrl)
|
intent.putExtra("modelUrl", model.modelUrl)
|
||||||
intent.putExtra("voice", model.voice)
|
intent.putExtra("voice", model.voice)
|
||||||
intent.putExtra("debug", false)
|
intent.putExtra("debug", debugSettingsStore.isChatDebugEnabled())
|
||||||
startActivity(intent)
|
startActivity(intent)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ package ai.guiji.duix.test.ui.activity
|
|||||||
import ai.guiji.duix.test.BuildConfig
|
import ai.guiji.duix.test.BuildConfig
|
||||||
import ai.guiji.duix.test.databinding.ActivitySettingsBinding
|
import ai.guiji.duix.test.databinding.ActivitySettingsBinding
|
||||||
import ai.guiji.duix.test.service.DashScopeConfig
|
import ai.guiji.duix.test.service.DashScopeConfig
|
||||||
|
import ai.guiji.duix.test.service.DebugSettingsStore
|
||||||
import ai.guiji.duix.test.service.FamilyBridgeService
|
import ai.guiji.duix.test.service.FamilyBridgeService
|
||||||
import android.content.Intent
|
import android.content.Intent
|
||||||
import android.net.Uri
|
import android.net.Uri
|
||||||
@@ -12,6 +13,7 @@ import org.json.JSONObject
|
|||||||
class SettingsActivity : BaseActivity() {
|
class SettingsActivity : BaseActivity() {
|
||||||
|
|
||||||
private lateinit var binding: ActivitySettingsBinding
|
private lateinit var binding: ActivitySettingsBinding
|
||||||
|
private val debugSettingsStore by lazy { DebugSettingsStore(applicationContext) }
|
||||||
private val familyBridgeService by lazy { FamilyBridgeService(applicationContext) }
|
private val familyBridgeService by lazy { FamilyBridgeService(applicationContext) }
|
||||||
|
|
||||||
override fun onCreate(savedInstanceState: Bundle?) {
|
override fun onCreate(savedInstanceState: Bundle?) {
|
||||||
@@ -23,11 +25,18 @@ class SettingsActivity : BaseActivity() {
|
|||||||
binding.rowDeviceCode.setOnClickListener {
|
binding.rowDeviceCode.setOnClickListener {
|
||||||
startActivity(Intent(mContext, DeviceSettingsActivity::class.java))
|
startActivity(Intent(mContext, DeviceSettingsActivity::class.java))
|
||||||
}
|
}
|
||||||
|
binding.rowDebugMode.setOnClickListener {
|
||||||
|
binding.switchDebugMode.performClick()
|
||||||
|
}
|
||||||
|
|
||||||
binding.tvVersionValue.text = "${BuildConfig.VERSION_NAME} (${BuildConfig.VERSION_CODE})"
|
binding.tvVersionValue.text = "${BuildConfig.VERSION_NAME} (${BuildConfig.VERSION_CODE})"
|
||||||
binding.tvServerValue.text = Uri.parse(DashScopeConfig.MONITOR_BASE_URL).host
|
binding.tvServerValue.text = Uri.parse(DashScopeConfig.MONITOR_BASE_URL).host
|
||||||
?: DashScopeConfig.MONITOR_BASE_URL
|
?: DashScopeConfig.MONITOR_BASE_URL
|
||||||
binding.tvDeviceValue.text = familyBridgeService.getDeviceUuid()
|
binding.tvDeviceValue.text = familyBridgeService.getDeviceUuid()
|
||||||
|
binding.switchDebugMode.isChecked = debugSettingsStore.isChatDebugEnabled()
|
||||||
|
binding.switchDebugMode.setOnCheckedChangeListener { _, isChecked ->
|
||||||
|
debugSettingsStore.setChatDebugEnabled(isChecked)
|
||||||
|
}
|
||||||
|
|
||||||
Thread {
|
Thread {
|
||||||
runCatching {
|
runCatching {
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ import ai.guiji.duix.test.model.AvatarCatalog
|
|||||||
import ai.guiji.duix.test.model.AvatarModel
|
import ai.guiji.duix.test.model.AvatarModel
|
||||||
import ai.guiji.duix.test.model.VoiceCloneProfile
|
import ai.guiji.duix.test.model.VoiceCloneProfile
|
||||||
import ai.guiji.duix.test.service.DashScopeConfig
|
import ai.guiji.duix.test.service.DashScopeConfig
|
||||||
|
import ai.guiji.duix.test.service.DebugSettingsStore
|
||||||
import ai.guiji.duix.test.service.VoiceCloneProfileStore
|
import ai.guiji.duix.test.service.VoiceCloneProfileStore
|
||||||
import ai.guiji.duix.test.service.VoiceCloneService
|
import ai.guiji.duix.test.service.VoiceCloneService
|
||||||
import ai.guiji.duix.test.ui.adapter.AvatarAdapter
|
import ai.guiji.duix.test.ui.adapter.AvatarAdapter
|
||||||
@@ -27,6 +28,7 @@ class VoiceCloneSetupActivity : BaseActivity() {
|
|||||||
private lateinit var binding: ActivityVoiceCloneSetupBinding
|
private lateinit var binding: ActivityVoiceCloneSetupBinding
|
||||||
private lateinit var avatarAdapter: AvatarAdapter
|
private lateinit var avatarAdapter: AvatarAdapter
|
||||||
private val mainHandler = Handler(Looper.getMainLooper())
|
private val mainHandler = Handler(Looper.getMainLooper())
|
||||||
|
private val debugSettingsStore by lazy { DebugSettingsStore(applicationContext) }
|
||||||
private val profileStore by lazy { VoiceCloneProfileStore(this) }
|
private val profileStore by lazy { VoiceCloneProfileStore(this) }
|
||||||
private val voiceCloneService = VoiceCloneService()
|
private val voiceCloneService = VoiceCloneService()
|
||||||
private val recorder = VoiceCloneRecorder()
|
private val recorder = VoiceCloneRecorder()
|
||||||
@@ -222,7 +224,7 @@ class VoiceCloneSetupActivity : BaseActivity() {
|
|||||||
val intent = Intent(mContext, CallActivity::class.java)
|
val intent = Intent(mContext, CallActivity::class.java)
|
||||||
intent.putExtra("modelUrl", selectedAvatar.modelUrl)
|
intent.putExtra("modelUrl", selectedAvatar.modelUrl)
|
||||||
intent.putExtra("voice", selectedAvatar.voice)
|
intent.putExtra("voice", selectedAvatar.voice)
|
||||||
intent.putExtra("debug", false)
|
intent.putExtra("debug", debugSettingsStore.isChatDebugEnabled())
|
||||||
intent.putExtra("cloneMode", true)
|
intent.putExtra("cloneMode", true)
|
||||||
intent.putExtra("clonedVoice", profile.voice)
|
intent.putExtra("clonedVoice", profile.voice)
|
||||||
intent.putExtra("cloneTtsModel", profile.targetModel)
|
intent.putExtra("cloneTtsModel", profile.targetModel)
|
||||||
|
|||||||
@@ -15,16 +15,29 @@
|
|||||||
android:layout_width="match_parent"
|
android:layout_width="match_parent"
|
||||||
android:layout_height="match_parent" />
|
android:layout_height="match_parent" />
|
||||||
|
|
||||||
<TextView
|
<ScrollView
|
||||||
android:id="@+id/tvDebug"
|
android:id="@+id/scrollDebug"
|
||||||
android:textSize="12sp"
|
android:layout_width="0dp"
|
||||||
|
android:layout_height="220dp"
|
||||||
android:visibility="gone"
|
android:visibility="gone"
|
||||||
android:textColor="@color/white"
|
|
||||||
android:background="#66000000"
|
android:background="#66000000"
|
||||||
android:padding="6dp"
|
android:fadeScrollbars="false"
|
||||||
|
android:fillViewport="true"
|
||||||
|
android:scrollbars="vertical"
|
||||||
app:layout_constraintTop_toTopOf="parent"
|
app:layout_constraintTop_toTopOf="parent"
|
||||||
android:layout_width="match_parent"
|
app:layout_constraintStart_toStartOf="parent"
|
||||||
android:layout_height="200dp"/>
|
app:layout_constraintEnd_toEndOf="parent">
|
||||||
|
|
||||||
|
<TextView
|
||||||
|
android:id="@+id/tvDebug"
|
||||||
|
android:layout_width="match_parent"
|
||||||
|
android:layout_height="wrap_content"
|
||||||
|
android:fontFamily="monospace"
|
||||||
|
android:padding="8dp"
|
||||||
|
android:textColor="@color/white"
|
||||||
|
android:textIsSelectable="true"
|
||||||
|
android:textSize="12sp" />
|
||||||
|
</ScrollView>
|
||||||
|
|
||||||
|
|
||||||
<androidx.appcompat.widget.SwitchCompat
|
<androidx.appcompat.widget.SwitchCompat
|
||||||
@@ -160,30 +173,4 @@
|
|||||||
android:layout_width="match_parent"
|
android:layout_width="match_parent"
|
||||||
android:layout_height="wrap_content"/>
|
android:layout_height="wrap_content"/>
|
||||||
|
|
||||||
<Button
|
|
||||||
android:id="@+id/btnPlayPCM"
|
|
||||||
android:text="@string/play_pcm"
|
|
||||||
android:visibility="gone"
|
|
||||||
app:layout_constraintBottom_toTopOf="@+id/btnAIConversation"
|
|
||||||
android:background="@drawable/shape_common_btn"
|
|
||||||
android:layout_margin="6dp"
|
|
||||||
android:textSize="13sp"
|
|
||||||
android:textColor="@color/white"
|
|
||||||
android:enabled="false"
|
|
||||||
android:layout_width="match_parent"
|
|
||||||
android:layout_height="wrap_content"/>
|
|
||||||
|
|
||||||
<Button
|
|
||||||
android:id="@+id/btnPlayWAV"
|
|
||||||
android:text="@string/play_wav"
|
|
||||||
android:visibility="gone"
|
|
||||||
app:layout_constraintBottom_toTopOf="@+id/btnPlayPCM"
|
|
||||||
android:background="@drawable/shape_common_btn"
|
|
||||||
android:layout_margin="6dp"
|
|
||||||
android:textSize="13sp"
|
|
||||||
android:textColor="@color/white"
|
|
||||||
android:enabled="false"
|
|
||||||
android:layout_width="match_parent"
|
|
||||||
android:layout_height="wrap_content"/>
|
|
||||||
|
|
||||||
</androidx.constraintlayout.widget.ConstraintLayout>
|
</androidx.constraintlayout.widget.ConstraintLayout>
|
||||||
@@ -87,6 +87,53 @@
|
|||||||
android:contentDescription="@null" />
|
android:contentDescription="@null" />
|
||||||
</LinearLayout>
|
</LinearLayout>
|
||||||
|
|
||||||
|
<LinearLayout
|
||||||
|
android:id="@+id/rowDebugMode"
|
||||||
|
android:layout_width="match_parent"
|
||||||
|
android:layout_height="wrap_content"
|
||||||
|
android:layout_marginTop="16dp"
|
||||||
|
android:background="@drawable/shape_warm_feature_card"
|
||||||
|
android:clickable="true"
|
||||||
|
android:focusable="true"
|
||||||
|
android:foreground="?attr/selectableItemBackground"
|
||||||
|
android:gravity="center_vertical"
|
||||||
|
android:orientation="horizontal"
|
||||||
|
android:padding="18dp">
|
||||||
|
|
||||||
|
<LinearLayout
|
||||||
|
android:layout_width="0dp"
|
||||||
|
android:layout_height="wrap_content"
|
||||||
|
android:layout_weight="1"
|
||||||
|
android:orientation="vertical">
|
||||||
|
|
||||||
|
<TextView
|
||||||
|
android:layout_width="wrap_content"
|
||||||
|
android:layout_height="wrap_content"
|
||||||
|
android:text="@string/settings_debug_title"
|
||||||
|
android:textColor="@color/warm_text_primary"
|
||||||
|
android:textSize="18sp"
|
||||||
|
android:textStyle="bold" />
|
||||||
|
|
||||||
|
<TextView
|
||||||
|
android:layout_width="match_parent"
|
||||||
|
android:layout_height="wrap_content"
|
||||||
|
android:layout_marginTop="6dp"
|
||||||
|
android:lineSpacingExtra="4dp"
|
||||||
|
android:text="@string/settings_debug_desc"
|
||||||
|
android:textColor="@color/warm_text_secondary"
|
||||||
|
android:textSize="14sp" />
|
||||||
|
</LinearLayout>
|
||||||
|
|
||||||
|
<androidx.appcompat.widget.SwitchCompat
|
||||||
|
android:id="@+id/switchDebugMode"
|
||||||
|
android:layout_width="wrap_content"
|
||||||
|
android:layout_height="wrap_content"
|
||||||
|
android:layout_marginStart="12dp"
|
||||||
|
android:contentDescription="@string/settings_debug_title"
|
||||||
|
android:importantForAutofill="no"
|
||||||
|
android:text="" />
|
||||||
|
</LinearLayout>
|
||||||
|
|
||||||
<LinearLayout
|
<LinearLayout
|
||||||
android:layout_width="match_parent"
|
android:layout_width="match_parent"
|
||||||
android:layout_height="wrap_content"
|
android:layout_height="wrap_content"
|
||||||
|
|||||||
@@ -80,6 +80,8 @@
|
|||||||
<string name="settings_subtitle">把设备信息、家人连接入口和当前版本都放在这里,找起来更方便。</string>
|
<string name="settings_subtitle">把设备信息、家人连接入口和当前版本都放在这里,找起来更方便。</string>
|
||||||
<string name="settings_device_code_title">家人连接设备码</string>
|
<string name="settings_device_code_title">家人连接设备码</string>
|
||||||
<string name="settings_device_code_desc">打开设备二维码页面后,家里人扫一扫就能把这台设备绑定到自己的账号。</string>
|
<string name="settings_device_code_desc">打开设备二维码页面后,家里人扫一扫就能把这台设备绑定到自己的账号。</string>
|
||||||
|
<string name="settings_debug_title">显示调试详情</string>
|
||||||
|
<string name="settings_debug_desc">开启后,聊天页面会显示完整的交互日志,包括工具调用的输入和输出。</string>
|
||||||
<string name="settings_about_title">关于这台设备</string>
|
<string name="settings_about_title">关于这台设备</string>
|
||||||
<string name="settings_version_label">应用版本</string>
|
<string name="settings_version_label">应用版本</string>
|
||||||
<string name="settings_server_label">家属面板域名</string>
|
<string name="settings_server_label">家属面板域名</string>
|
||||||
|
|||||||
@@ -82,6 +82,8 @@
|
|||||||
<string name="settings_subtitle">Keep device information, family connection entry and current version together in one place.</string>
|
<string name="settings_subtitle">Keep device information, family connection entry and current version together in one place.</string>
|
||||||
<string name="settings_device_code_title">Family device code</string>
|
<string name="settings_device_code_title">Family device code</string>
|
||||||
<string name="settings_device_code_desc">Open the QR code page for your family to scan and bind this elder device.</string>
|
<string name="settings_device_code_desc">Open the QR code page for your family to scan and bind this elder device.</string>
|
||||||
|
<string name="settings_debug_title">Show debug details</string>
|
||||||
|
<string name="settings_debug_desc">When enabled, the chat page will show full interaction logs, including tool input and output.</string>
|
||||||
<string name="settings_about_title">About this device</string>
|
<string name="settings_about_title">About this device</string>
|
||||||
<string name="settings_version_label">App version</string>
|
<string name="settings_version_label">App version</string>
|
||||||
<string name="settings_server_label">Family panel domain</string>
|
<string name="settings_server_label">Family panel domain</string>
|
||||||
|
|||||||
Reference in New Issue
Block a user