Int8OPTDecoderLayer Class Reference

Public Member Functions
	Int8OPTDecoderLayer (std::string param_path, const struct model_config config, int layer_idx, LayerNormQ self_attn_layer_norm, LayerNormQ final_layer_norm, W8A8B8O8LinearReLU fc1, W8A8BFP32OFP32Linear fc2, BMM_S8T_S8N_F32T qk_bmm, BMM_S8T_S8N_S8T pv_bmm, W8A8B8O8Linear k_proj, W8A8B8O8Linear v_proj, W8A8B8O8Linear q_proj, W8A8BFP32OFP32Linear out_proj)
struct Int8OPTDecoderLayer_output	forward (const struct Int8OPTDecoderLayer_input &input)

Public Attributes
int	embed_dim
int	num_attention_heads
int	hidden_dim
int	layer_idx
LayerNormQ	self_attn_layer_norm
LayerNormQ	final_layer_norm
W8A8B8O8LinearReLU	fc1
W8A8BFP32OFP32Linear	fc2
Int8OPTAttention	attn
std::string	profile_name = "Int8OPTDecoderLayer"

---

The documentation for this class was generated from the following files:

• llm/include/nn_modules/Int8OPTDecoderLayer.h
• llm/src/nn_modules/Int8OPTDecoderLayer.cc