Vitis HLS for Llama2 Acceleration (Part 0)

Plan

• ✅ Global configuration header file
• ✅ Divide the llama2 decoder layer into 2 regions
• ❌ Analyze the llama2 decoder layer dataflow
• ❌ Convert original software implementation to vitis hls hardware implementation
• ❌ Compare allo generated hls code
• ❌ Measure the speedup
• ❌ Integrate actual llama2 large language model

Global Configuration Header File

#define HIDDEN_SIZE         4096
#define INTERMEDIATE_SIZE   11008
#define NUM_ATTENTION_HEADS 32
#define NUM_KEY_VALUE_HEADS 32

#define NUM_HIDDEN_LAYERS   32
#define VOCAB_SIZE          32000

References

• karpathy/llama2.c
• cornell-zhang/allo-pldi24-artifact