长沙营销型网站建设制作网站后台制作这么做

长沙营销型网站建设制作,网站后台制作这么做,淘宝客导购网站建设,贵阳经开区建设管理局网站DeepResearch代码浅析概述代码#xff1a;DeepResearch主要看一下inference下面的ReAct推理流程。inference├── eval_data│ ├── example_with_file.jsonl│ ├── example.jsonl│ └── file_corpus│ └── hello.txt├── file_tools│ ├── __pycache__│ │…DeepResearch代码浅析概述代码DeepResearch主要看一下inference下面的ReAct推理流程。inference├── eval_data│ ├── example_with_file.jsonl│ ├── example.jsonl│ └── file_corpus│ └── hello.txt├── file_tools│ ├── __pycache__│ │ └── file_parser.cpython-313.pyc│ ├── file_parser.py│ ├── idp.py│ ├── utils.py│ ├── video_agent.py│ └── video_analysis.py├── prompt.py├── react_agent.py├── run_multi_react.py├── run_react_infer.sh├── tool_file.py├── tool_python.py├── tool_scholar.py├── tool_search.py└── tool_visit.py代码的入口是run_react_infer.sh中的run_multi_react.py文件run_multi_react.py负责初始化节点环境加载数据集加载模型配置进行多次rollout采样。react_agent是ReAct 架构的Agent负责迭代输出调用工具。from react_agent import MultiTurnReactAgenttest_agent MultiTurnReactAgent(llmllm_cfg,function_list[search, visit, google_scholar, PythonInterpreter])react_agent主体的ReAct agent统一调度处理模型的输出进行tool extract and execute和tool response的拼接执行ReAct的全部流程给出最后的执行状态处理运行中的异常现象定义工具from tool_file import *from tool_scholar import *from tool_python import *from tool_search import *from tool_visit import *OBS_START tool_responseOBS_END \n/tool_response# 定义工具放在TOOL_MAP中TOOL_CLASS [FileParser(),Scholar(),Visit(),Search(),PythonInterpreter(),]TOOL_MAP {tool.name: tool for tool in TOOL_CLASS}在MultiTurnReactAgent类中使用def call_server() 调用llm apidef call_server(self, msgs, planning_port, max_tries10):openai_api_key EMPTYopenai_api_base fhttp://127.0.0.1:{planning_port}/v1client OpenAI(api_keyopenai_api_key,base_urlopenai_api_base,timeout600.0,)执行ReAct流程可能出现的情况返回answer 出现answer /answer未达到轮次限制达到/未达到上下文token数量限制未返回answer超出轮次限制后达到上下问token数量限制后返回答案没有answer超时工具调用错误tool_call 的json格式错误def _run(self, data: str, model: str, **kwargs) - List[List[Message]]:############################################################## 初始化question和最多调用轮次num_llm_calls_available# 记录start_time拼接最开始的message#############################################################self.modelmodeltry:question data[item][question]except:raw_msg data[item][messages][1][content]question raw_msg.split(User:)[1].strip() if User: in raw_msg else raw_msgstart_time time.time()planning_port data[planning_port]answer data[item][answer]self.user_prompt questionsystem_prompt SYSTEM_PROMPTcur_date today_date()system_prompt system_prompt str(cur_date)messages [{role: system, content: system_prompt}, {role: user, content: question}]num_llm_calls_available MAX_LLM_CALL_PER_RUNround 0############################################################## 开始迭代每一个iter生成tool_call 或是answer#############################################################while num_llm_calls_available 0:# Check whether time is reached############################################################## 检查是否超时2.5小时#############################################################if time.time() - start_time 150 * 60: # 150 minutes in secondsprediction No answer found after 2h30minstermination No answer found after 2h30minsresult {question: question,answer: answer,messages: messages,prediction: prediction,termination: termination}return result############################################################## 更新调用llm次数 num_llm_calls_available# 获取llm的返回值 content#############################################################round 1num_llm_calls_available - 1content self.call_server(messages, planning_port)print(fRound {round}: {content})############################################################## 进行content中关键tool的提取############################################################## 舍弃content中tool_response的部分应为obs应该是user输入的而不是llm生成的if tool_response in content:pos content.find(tool_response)content content[:pos]messages.append({role: assistant, content: content.strip()})# 查看content中是否有工具调用 tool_callif tool_call in content and /tool_call in content:tool_call content.split(tool_call)[1].split(/tool_call)[0]try:# 使用python解释器运行code_rawif python in tool_call.lower():try:code_rawcontent.split(tool_call)[1].split(/tool_call)[0].split(code)[1].split(/code)[0].strip()result TOOL_MAP[PythonInterpreter].call(code_raw)except:result [Python Interpreter Error]: Formatting error.# 调用其他的工具else:tool_call json5.loads(tool_call)tool_name tool_call.get(name, )tool_args tool_call.get(arguments, {})result self.custom_call_tool(tool_name, tool_args)# 如果llm生成的tool formart错误则将错误信息写入messages中可以使用约束采样避免格式错误except:result Error: Tool call is not a valid JSON. Tool call must contain a valid name and arguments field.result tool_response\n result \n/tool_response# print(result)# 把tool response写入到user中messages.append({role: user, content: result})# 如果模型生成的content中有answer /answer则已经输出答案if answer in content and /answer in content:termination answerbreak# 如果没有可用轮次记录失败信息if num_llm_calls_available 0 and answer not in content:messages[-1][content] Sorry, the number of llm calls exceeds the limit.max_tokens 110 * 1024token_count self.count_tokens(messages)print(fround: {round}, token count: {token_count})############################################################## ReAct的累积上下文token长度达到阈值强制给出回答#############################################################if token_count max_tokens:print(fToken quantity exceeds the limit: {token_count} {max_tokens})messages[-1][content] You have now reached the maximum context length you can handle. You should stop making tool calls and, based on all the information above, think again and provide what you consider the most likely answer in the following format:thinkyour final thinking/think\nansweryour answer/answercontent self.call_server(messages, planning_port)messages.append({role: assistant, content: content.strip()})# token数达到阈值后成功返回结果if answer in content and /answer in content:prediction messages[-1][content].split(answer)[1].split(/answer)[0]termination generate an answer as token limit reached# 未返回结果else:prediction messages[-1][content]termination format error: generate an answer as token limit reachedresult {question: question,answer: answer,messages: messages,prediction: prediction,termination: termination}return result# 这里termination忽略了token超限制后是否给出answer的情况if answer in messages[-1][content]:prediction messages[-1][content].split(answer)[1].split(/answer)[0]termination answerelse:prediction No answer found.termination answer not foundif num_llm_calls_available 0:termination exceed available llm callsresult {question: question,answer: answer,messages: messages,prediction: prediction,termination: termination}return result工具调用tool_python执行python代码。(code;Interpreter)→(stdout,stderr)def call(self, params, files None, timeout 50, **kwargs) - str:try:# params 即为要执行的code代码codeparamslast_error None# 尝试多次for attempt in range(8):try:# Randomly sample an endpoint for each attemptendpoint random.choice(SANDBOX_FUSION_ENDPOINTS)print(fAttempt {attempt 1}/5 using endpoint: {endpoint})# 执行codecode_result run_code(RunCodeRequest(codecode, languagepython, run_timeouttimeout), max_attempts1, client_timeouttimeout, endpointendpoint)print([Python] Code Result, code_result)result []# 记录code 的标准输出和错误if code_result.run_result.stdout:result.append(fstdout:\n{code_result.run_result.stdout})if code_result.run_result.stderr:result.append(fstderr:\n{code_result.run_result.stderr})if code_result.run_result.execution_time timeout-1:result.append(f[PythonInterpreter Error] TimeoutError: Execution timed out.)result \n.join(result)print(SUCCESS RUNNING TOOL)return result if result.strip() else Finished execution.# code执行超时except Timeout as e:last_error f[Python Interpreter Error] TimeoutError: Execution timed out on endpoint {endpoint}.print(fTimeout on attempt {attempt 1}: {last_error})if attempt 4: # Last attemptreturn last_errorcontinue# code执行错误except Exception as e:last_error f[Python Interpreter Error]: {str(e)} on endpoint {endpoint}print(fError on attempt {attempt 1}: {last_error})if attempt 4: # Last attemptreturn last_errorcontinuereturn last_error if last_error else [Python Interpreter Error]: All attempts failed.except Exception as e:return f[Python Interpreter Error]: {str(e)}tool_visit搜索具体的url并根据goal总结返回。(url,goal;π)→summaryJINA_API_KEYS os.getenv(JINA_API_KEYS, )def readpage_jina(self, url: str, goal: str) - str:Attempt to read webpage content by alternating between jina and aidata services.Args:url: The URL to readgoal: The goal/purpose of reading the pageReturns:str: The webpage content or error message# def call_server用于根据goal总结网页的内容summary_page_func self.call_servermax_retries int(os.getenv(VISIT_SERVER_MAX_RETRIES, 1))# 使用jina将url的网页信息转化为 markdown格式content self.html_readpage_jina(url)############################################################## 处理markdown的网页信息 content############################################################## 如果网页信息可以被jina提取if content and not content.startswith([visit] Failed to read page.) and content ! [visit] Empty content. and not content.startswith([document_parser]):# pre-process 先处理content的token长度避免llm的上下文超长content truncate_to_tokens(content, max_tokens95000)# 总结promoptmessages [{role:user,content: EXTRACTOR_PROMPT.format(webpage_contentcontent, goalgoal)}]parse_retry_times 0# 得到网页总结后的信息 rawraw summary_page_func(messages, max_retriesmax_retries)summary_retries 3# 如果raw少于10个字符那么认为总结失败因为raw是json格式json {rational:..., evidence:..., summary:...}while len(raw) 10 and summary_retries 0:# 尝试截断30%的长度truncate_length int(0.7 * len(content)) if summary_retries 0 else 25000status_msg (f[visit] Summary url[{url}] fattempt {3 - summary_retries 1}/3, fcontent length: {len(content)}, ftruncating to {truncate_length} chars) if summary_retries 0 else (f[visit] Summary url[{url}] failed after 3 attempts, ffinal truncation to 25000 chars) # 截断30%不行尝试只留下25000字符print(status_msg)content content[:truncate_length]extraction_prompt EXTRACTOR_PROMPT.format(webpage_contentcontent,goalgoal)messages [{role: user, content: extraction_prompt}]raw summary_page_func(messages, max_retriesmax_retries)summary_retries - 1# 解析总结的格式parse_retry_times 2if isinstance(raw, str):raw raw.replace(json, ).replace(, ).strip()while parse_retry_times 3:try:raw json.loads(raw)breakexcept:# 解析失败的话就重新生成总结raw summary_page_func(messages, max_retriesmax_retries)parse_retry_times 1# 解析失败if parse_retry_times 3:useful_information The useful information in {url} for user goal {goal} as follows: \n\n.format(urlurl, goalgoal)useful_information Evidence in page: \n The provided webpage content could not be accessed. Please check the URL or file format. \n\nuseful_information Summary: \n The webpage content could not be processed, and therefore, no information is available. \n\n# 解析成功把evidence和summary一并返回else:useful_information The useful information in {url} for user goal {goal} as follows: \n\n.format(urlurl, goalgoal)useful_information Evidence in page: \n str(raw[evidence]) \n\nuseful_information Summary: \n str(raw[summary]) \n\nif len(useful_information) 10 and summary_retries 0:print([visit] Could not generate valid summary after maximum retries)useful_information [visit] Failed to read pagereturn useful_information# If no valid content was obtained after all retries# 如果网页的原始信息就不合理jina无法提取返回失败信息else:useful_information The useful information in {url} for user goal {goal} as follows: \n\n.format(urlurl, goalgoal)useful_information Evidence in page: \n The provided webpage content could not be accessed. Please check the URL or file format. \n\nuseful_information Summary: \n The webpage content could not be processed, and therefore, no information is available. \n\nreturn useful_informationjina举例输入https://r.jina.ai/url 即 jina提取出的markdown原始网页image-20251017120415289jina由三部分组成titleurlmarkdown content图片的url信息超链接等Title: Jina Reader API完全指南4种实用集成方案详解 | AI开发教程URL Source: https://www.axtonliu.ai/newsletters/ai-2/posts/jina-reader-api-four-usage-methods-guideMarkdown Content:构建知识库或者分析各种文章数据是大家使用 AI 很重要的一个应用场景tool_file根据url的文件和goal返回总结信息类似于tool_visit。但是要借助于file_tools进行指定url文件的读取visit是借用jina进行指定url网页信息的读取。input:- query/goal: str- Docs: List[file]/List[url]- file type: pdf, docx, pptx, txt, html, csv, tsv, xlsx, xls, doc, zip, .mp4, .mov, .avi, .mkv, .webm, .mp3, .wav, .aac, .ogg, .flacoutput:- answer: str- useful_information: strtool_search调用google 进行search。(q;Enginer)→docstool_scholar类似于tool_search区别在于 tool_scholar在goole scholar上进行文章的搜索Prompt分为react的system prompt以及visit 总结的extract promptSYSTEM_PROMPT You are a deep research assistant. Your core function is to conduct thorough, multi-source investigations into any topic. You must handle both broad, open-domain inquiries and queries within specialized academic fields. For every request, synthesize information from credible, diverse sources to deliver a comprehensive, accurate, and objective response. When you have gathered sufficient information and are ready to provide the definitive response, you must enclose the entire final answer within answer/answer tags.# ToolsYou may call one or more functions to assist with the user query.You are provided with function signatures within tools/tools XML tags:tools{type: function, function: {name: search, description: Perform Google web searches then returns a string of the top search results. Accepts multiple queries., parameters: {type: object, properties: {query: {type: array, items: {type: string, description: The search query.}, minItems: 1, description: The list of search queries.}}, required: [query]}}}{type: function, function: {name: visit, description: Visit webpage(s) and return the summary of the content., parameters: {type: object, properties: {url: {type: array, items: {type: string}, description: The URL(s) of the webpage(s) to visit. Can be a single URL or an array of URLs.}, goal: {type: string, description: The specific information goal for visiting webpage(s).}}, required: [url, goal]}}}{type: function, function: {name: PythonInterpreter, description: Executes Python code in a sandboxed environment. To use this tool, you must follow this format:1. The arguments JSON object must be empty: {}.2. The Python code to be executed must be placed immediately after the JSON block, enclosed within code and /code tags.IMPORTANT: Any output you want to see MUST be printed to standard output using the print() function.Example of a correct call:tool_call{name: PythonInterpreter, arguments: {}}codeimport numpy as np# Your code hereprint(fThe result is: {np.mean([1,2,3])})/code/tool_call, parameters: {type: object, properties: {}, required: []}}}{type: function, function: {name: google_scholar, description: Leverage Google Scholar to retrieve relevant information from academic publications. Accepts multiple queries. This tool will also return results from google search, parameters: {type: object, properties: {query: {type: array, items: {type: string, description: The search query.}, minItems: 1, description: The list of search queries for Google Scholar.}}, required: [query]}}}{type: function, function: {name: parse_file, description: This is a tool that can be used to parse multiple user uploaded local files such as PDF, DOCX, PPTX, TXT, CSV, XLSX, DOC, ZIP, MP4, MP3., parameters: {type: object, properties: {files: {type: array, items: {type: string}, description: The file name of the user uploaded local files to be parsed.}}, required: [files]}}}/toolsFor each function call, return a json object with function name and arguments within tool_call/tool_call XML tags:tool_call{name: function-name, arguments: args-json-object}/tool_callCurrent date: EXTRACTOR_PROMPT Please process the following webpage content and user goal to extract relevant information:## **Webpage Content**{webpage_content}## **User Goal**{goal}## **Task Guidelines**1. **Content Scanning for Rational**: Locate the **specific sections/data** directly related to the users goal within the webpage content2. **Key Extraction for Evidence**: Identify and extract the **most relevant information** from the content, you never miss any important information, output the **full original context** of the content as far as possible, it can be more than three paragraphs.3. **Summary Output for Summary**: Organize into a concise paragraph with logical flow, prioritizing clarity and judge the contribution of the information to the goal.**Final Output Format using JSON format has rational, evidence, summary feilds**