- Google Colab is an alternative to Jupyter notebooks that run on their GPUs for free.
RAG - Retrieval Augmented Generation
- It is simply a technique to improve prompts, like a Knowledege Base powered skill, which adds relevant details to the prompt.
- Simplest implementation: when sending a system prefix, add the context dynamically (e.g. collect all data from documents inside of KB for all documents that contain a word from the prompt).
- When testing RAG, remember that the conversation history is always sent to the LLM, so the results for a prompt may be based on a previous answer, and not the RAG itself. Always use a fresh chat ensure correctness.
- Auto-regressive LLM: predicts a future token from the past.
- Auto-encoding LLM: produces output based on the full input. Example: classication; sentiment analysis; vector embeddings calculation.
- Vectors mathematically represent the “meaning” of an input. It converts unstructured text chunks and queries into numerical arrays that capture semantic meaning rather than exact words. The vectorization result can then be fed as text into a regressive LLM to enhance its context.