Int4llamaAttention Class Reference

Public Member Functions
	Int4llamaAttention (std::string param_path, const struct model_config config, int layer_idx)
struct Int4llamaAttention_output	forward (std::string param_path, const struct Int4llamaAttention_input &input)

Static Public Member Functions
static void	initialized_memory (const struct model_config config)

Public Attributes
int *	q_weight = nullptr
int *	k_weight = nullptr
int *	v_weight = nullptr
int *	o_weight = nullptr
int *	qkv_weight = nullptr
float *	cos_buf = nullptr
float *	sin_buf = nullptr

---

The documentation for this class was generated from the following files:

• llm/include/nn_modules/Int4llamaAttention.h
• llm/src/nn_modules/non_cuda/Int4llamaAttention.cc