Diary

16th December, 2024

#####

• LLM context window is the amount of text an LLM can take as input and formulate a response for. Prompt engineering is tweaking the text in this context window.
• The longer the inpute in the context window, the higher the compute cost.
• Neural attention memory models (NAMMs) can be used to eliminate redundant tokens and improve performance.