vip网站怎么做seo网站排名优化公司

vip网站怎么做,seo网站排名优化公司,网站建设支付安全,注册一个网站要多少钱文章目录EAGLE 系列模型的演进与核心机制关键参数与训练逻辑思考参考来源#xff1a;https://docs.sglang.com.cn/backend/speculative_decoding.html https://github.com/SafeAILab/EAGLE EAGLE3 https://arxiv.org/pdf/2503.01840 EAGLE 系列模型的演进与核心机制 EAGLE 基…文章目录EAGLE 系列模型的演进与核心机制关键参数与训练逻辑思考参考来源https://docs.sglang.com.cn/backend/speculative_decoding.htmlhttps://github.com/SafeAILab/EAGLEEAGLE3 https://arxiv.org/pdf/2503.01840EAGLE 系列模型的演进与核心机制EAGLE 基础架构草稿模型通过特征序列和 token 序列预测下一个特征向量基于原始 LLM 的最后一个隐藏状态生成候选。采样后的 token 与原始序列以树状结构扩展分支因子由speculative_eagle_topk控制确保上下文连贯性。扩展后的树结构重新作为输入迭代生成。EAGLE-2 的优化引入动态分支评估机制草稿模型主动评估扩展分支的可能性提前终止低概率分支的扩展。扩展阶段结束后通过重排序筛选前speculative_num_draft_tokens个节点作为最终草稿 token减少冗余计算。将--speculative-token-map参数设置为true以启用高频 token 优化功能。该参数通常在模型推理或训练配置文件中进行设置。EAGLE-3 的改进移除特征预测目标整合低层与中间层特征提升表示能力。采用 on-policy 训练方式使模型在推理阶段的行为与训练目标更一致进一步优化生成质量与效率。关键参数与训练逻辑speculative_eagle_topk控制每步扩展的分支数量影响生成多样性与计算开销。speculative_num_draft_tokens决定保留的候选 token 数量平衡生成速度与准确性。On-policy 训练通过对齐训练与推理阶段的策略减少分布偏移问题。https://github.com/SafeAILab/EAGLE/blob/main/eagle/traineagle3/cnets.py核心代码部分def_prepare_decoder_attention_mask(self,attention_mask,input_shape,inputs_embeds,past_key_values_length):# create causal mask# [bsz, seq_len] - [bsz, 1, tgt_seq_len, src_seq_len]combined_attention_maskNoneifinput_shape[-1]1:combined_attention_mask_make_causal_mask(input_shape,inputs_embeds.dtype,deviceinputs_embeds.device,past_key_values_lengthpast_key_values_length,)ifattention_maskisnotNone:# [bsz, seq_len] - [bsz, 1, tgt_seq_len, src_seq_len]expanded_attn_mask_expand_mask(attention_mask,inputs_embeds.dtype,tgt_leninput_shape[-1]).to(inputs_embeds.device)combined_attention_mask(expanded_attn_maskifcombined_attention_maskisNoneelseexpanded_attn_maskcombined_attention_mask)returncombined_attention_masktorch.no_grad()defdataprepare(self,input_ids,attention_mask,loss_mask):deviceinput_ids.device outsself.target_model(input_idsinput_ids,attention_maskattention_mask)hidden_states0outs.hidden_states[0]hidden_states1outs.hidden_states[1]hidden_states2outs.hidden_states[2]hidden_statestorch.cat((hidden_states0,hidden_states1,hidden_states2),dim-1)# hidden_statestorch.cat((hidden_states0,hidden_states1),dim-1)targetouts.logits targetpadding(target,leftFalse)input_idspadding(input_ids,leftFalse)iftargetisnotNone:targettarget.to(device)loss_maskloss_mask[...,None]loss_maskloss_mask.to(device)returnhidden_states,target,loss_mask,input_idsdefforward(self,# hidden_states,input_ids,attention_mask:Optional[torch.Tensor]None,position_ids:Optional[torch.LongTensor]None,past_key_values:Optional[List[torch.FloatTensor]]None,use_cache:Optional[bool]None,output_attentions:Optional[bool]None,output_hidden_states:Optional[bool]None,loss_mask:Optional[torch.Tensor]None,):hidden_states,target,loss_mask,input_idsself.dataprepare(input_ids,attention_mask,loss_mask)batch_size,seq_length,_hidden_states.shape seq_length_with_pastseq_length past_key_values_length0# with torch.no_grad():# inputs_embeds self.embed_tokens(input_ids)# inputs_embeds inputs_embeds.detach()ifself.trainingandself.gradient_checkpointingandnothidden_states.requires_grad:hidden_states.requires_gradTruehidden_statesself.fc(hidden_states)ifpast_key_valuesisnotNone:past_key_values_lengthpast_key_values[0][0].shape[2]seq_length_with_pastseq_length_with_pastpast_key_values_lengthifposition_idsisNone:devicehidden_states.device position_idstorch.arange(past_key_values_length,seq_lengthpast_key_values_length,dtypetorch.long,devicedevice)position_idsposition_ids.unsqueeze(0).view(-1,seq_length)else:position_idsposition_ids.view(-1,seq_length).long()ifattention_maskisNone:attention_masktorch.ones((batch_size,seq_length_with_past),dtypetorch.bool,devicehidden_states.device)attention_maskself._prepare_decoder_attention_mask(attention_mask,(batch_size,seq_length),hidden_states,past_key_values_length)ifself.gradient_checkpointingandself.training:ifuse_cache:use_cacheFalseplosses[]vlosses[]acces[]cache_hidden[[],[]]foridxinrange(self.length):lastidxself.length-1inputs_embedsself.embed_tokens(input_ids)ifself.trainingandself.gradient_checkpointingandnotinputs_embeds.requires_grad:inputs_embeds.requires_gradTrueinputs_embedsinputs_embeds.to(hidden_states.dtype)ifself.gradient_checkpointingandself.training:defcreate_custom_forward(module):defcustom_forward(*inputs):# None for past_key_valuereturnmodule(*inputs,None,output_attentions)returncustom_forward layer_outputs,cache_hiddentorch.utils.checkpoint.checkpoint(create_custom_forward(self.midlayer),inputs_embeds,hidden_states,cache_hidden,attention_mask,position_ids,)else:layer_outputs,cache_hiddenself.midlayer(input_embinputs_embeds,hidden_stateshidden_states,cache_hiddencache_hidden,attention_maskattention_mask,position_idsposition_ids,past_key_valueNone,output_attentionsoutput_attentions,use_cacheTrue,)hidden_states_outlayer_outputs[0]# cache_hidden.append(layer_outputs[1])# kv_cahce layer_outputs[-1]withtorch.no_grad():# hidden_states_target padding(hidden_states, leftFalse)target_headtarget target_max_tokentarget_head.argmax(-1)# Move d2t to the same device as target_max_tokenself.t2dself.t2d.to(target_max_token.device)target_maskself.t2d[target_max_token]target_masktarget_mask[...,None].int()position_masktarget_mask*loss_mask target_headtarget_head[...,self.t2d]target_headtarget_head.float()target_pnn.Softmax(dim2)(target_head)target_ptarget_p.detach()hidden_stateshidden_states_out hidden_states_outself.norm(hidden_states_out)logitsself.lm_head(hidden_states_out)logitslogits.float()out_logpnn.LogSoftmax(dim2)(logits)plogptarget_p*out_logp loss-torch.sum(position_mask*plogp,2).mean()plosses.append(loss)withtorch.no_grad():acces.append(((logits.argmax(-1)target_p.argmax(-1))*position_mask.squeeze(-1)).sum().item()/(loss_mask.sum().item()1e-6))ifnotlast:input_idspadding(input_ids,leftFalse)targetpadding(target,leftFalse)loss_maskpadding(loss_mask,leftFalse)returnplosses,vlosses,acces思考》 FASTMTP与EAGLE3相比谁更快一些