Large language models have transformed natural language processing through their ability to generate and interpret humanlike text. Most modern language models rely on transformer architectures, whose self-attention mechanisms provide powerful capabilities for learning relationships within long sequences. However, training and running these models can require substantial computational resources and energy, creating challenges for their wider use in embedded systems, edge devices and other environments where computing resources are limited.