Image source: YouTube
Table of Contents
- Introduction - AI, ML, DL, and Gen AI
- Getting started with InstructLab
- Glossary
- Some facts
- References
Introduction - AI, ML, DL, and Gen AI
- Artificial Intelligence (AI): The broader field of creating intelligent agents, aiming to mimic human intelligence.
- Machine Learning (ML): A subset of AI focused on teaching computers to learn from data without explicit programming.
- Deep Learning (DL): A subset of ML using artificial neural networks to analyze complex patterns in data.
- Generative AI (Gen AI): A subfield focused on creating new content, such as text, images, or code, based on learned patterns.
InstructLab is a game-changer for enhancing large language models (LLMs). This open-source project, co-created by IBM and Red Hat, makes it easier to align LLMs with user intent, opening up possibilities for innovative AI applications.
Podman AI Lab is Red Hat’s answer to simplifying the AI development process. This extension provides a local environment with essential open-source tools and curated recipes to guide you through building AI solutions.
Red Hat OpenShift AI is a powerful platform for deploying and scaling AI applications across hybrid cloud environments. Built on open-source technologies, it offers a trusted foundation for teams to experiment, serve models, and deliver innovative AI-driven apps.
AI Knowledge vs Skill
Knowledge in AI:
- Knowledge represents the information and data that an AI system has access to, including facts, rules, and concepts about the world or a specific domain. It’s the foundational material that the AI uses to make decisions, predictions, and perform tasks.
- AI systems accumulate knowledge through data training (e.g., large datasets of text, images, or structured data) and can retrieve and apply this knowledge when required.
- Knowledge can be encoded as rules (in rule-based systems) or derived from patterns in data (in machine learning systems).
Examples:
- A language model like GPT is trained on vast amounts of text data, providing it with knowledge about grammar, facts, and even context.
- A medical diagnostic AI might have knowledge about symptoms, diseases, and treatments stored in its database or learned from medical literature.
Skill in AI:
- Skill refers to the ability of an AI to apply knowledge effectively to perform tasks or solve problems. It involves leveraging knowledge in a way that demonstrates proficiency in a specific task or set of tasks.
- AI develops skills through training, experience, and fine-tuning. Skill in AI can be thought of as the execution part, where the system demonstrates its capability to solve real-world problems using the knowledge it possesses.
Examples:
- A self-driving car’s ability to navigate roads by understanding traffic rules and reacting to real-time conditions.
- A machine learning model’s skill to recognize objects in images, using knowledge about how certain objects appear.
Relationship Between Knowledge and Skill in AI:
- Knowledge is the foundation, while skill is the application of that knowledge.
- An AI system may have vast knowledge but limited skill if it cannot effectively use the information to solve tasks.
- Conversely, a highly skilled AI must be backed by a strong knowledge base to perform its tasks accurately and consistently.
Getting started with InstructLab
Prerequisites
System Requirements:
- Operating System: Currently supported on Apple M1/M2/M3 Macs and Linux systems.
- Software:
- C++ compiler
- Python 3.9 or later
- Storage: Approximately 60GB of free disk space. Note that this requirement can vary based on the size of the LLM you’re working with. Please be aware that the actual disk space needed will depend on several factors, including the size of the LLM model itself.
NOTE: The following commands are executed from a Fedora 40 machine (32GB/12vCPU).
Installing InstructLab
- Create a Python virtual environment.
This is an optional step to isolate the testing environment. You can deploy the ilab and components directly into your Python environment if required.
$ python3 -m venv aienv
$ cd aienv
$ source bin/activate
(aienv) $ aienv
- Install
ilabCLI
This task will take some time and also consume some disk space (~5GB) as there will be a lot of dependencies to install.
$ pip3 install git+https://github.com/instructlab/[email protected]
# Check version
$ ilab --version
ilab, version 0.16.1
# Sub-commands
Commands:
chat Run a chat using the modified model
check (Deprecated) Check that taxonomy is valid
convert Converts model to GGUF
diff Lists taxonomy files that have changed since...
download Download the model(s) to train
generate Generates synthetic data to enhance your example data
init Initializes environment for InstructLab
list (Deprecated) Lists taxonomy files that have changed since...
serve Start a local server
sysinfo Print system information
test Runs basic test to ensure model correctness
train Takes synthetic data generated locally with `ilab...
Initialize ilab
Prepare the workspace (any directory) and init ilab.
$ ilab init
Welcome to InstructLab CLI. This guide will help you to setup your environment.
Please provide the following values to initiate the environment [press Enter for defaults]:
Path to taxonomy repo [taxonomy]:
`taxonomy` seems to not exist or is empty. Should I clone https://github.com/instructlab/taxonomy.git for you? [y/N]: y
Cloning https://github.com/instructlab/taxonomy.git...
Generating `config.yaml` in the current directory...
Initialization completed successfully, you're ready to start using `ilab`. Enjoy!
Download a Model
$ ilab download --repository instructlab/granite-7b-lab-GGUF --filename=granite-7b-lab-Q4_K_M.gguf
$ ls models/
granite-7b-lab-Q4_K_M.gguf
Serving model
With the model downloaded, we’re ready to serve it. Serving means setting up a system that allows other programs to interact with the model. It’s like creating an API where other applications can send requests and receive responses.
$ ilab serve --model-path models/granite-7b-lab-Q4_K_M.gguf
INFO 2024-07-26 13:12:46,897 lab.py:340 Using model 'models/granite-7b-lab-Q4_K_M.gguf' with -1 gpu-layers and 4096 max context size.
INFO 2024-07-26 13:12:48,483 server.py:206 Starting server process, press CTRL+C to shutdown server...
INFO 2024-07-26 13:12:48,483 server.py:207 After application startup complete see http://127.0.0.1:8000/docs for API.
Now the model is serving and waiting for any request.
You can test the model by starting ilab model chat (or ilab chat) in another terminal as follows.
Activate Python venv
$ cd ~/aienv/
$ source bin/activate
(aienv)
Goto the InstructLab directory (or use absoluate path)
$ cd ~/workarea/ilab/
$ ilab chat -m models/granite-7b-lab-Q4_K_M.gguf
╭──────────────────────────────────────────────── system ────────────────────────────────────────────────╮
│ Welcome to InstructLab Chat w/ MODELS/GRANITE-7B-LAB-Q4_K_M.GGUF (type /h for help) │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────╯
>>> [S][default]
Now we can ask questions here.
>>> what is a container in 20 words? [S][default]
╭────────────────────────────────── models/granite-7b-lab-Q4_K_M.gguf ───────────────────────────────────╮
│ A container is a lightweight, portable, and executable software package containing code, libraries, │
│ dependencies, and system tools. It offers consistent, efficient, scalable, and isolated application │
│ deployment across various environments. │
╰─────────────────────────────────────────────────────────────────────────────── elapsed 13.398 seconds ─╯
>>> [S][default]
Exit from the chat prompt.
>>> exit
Glossary
- LAB - Large-scale Alignment for chatBots
- LLM - Large Language Models
- Gen AI - generative artificial intelligence
Some facts
- Llama2–70B model — a large language model released by Meta.ai.
References
- InstructLab Project
- InstructLab Taxonomy Repo
- InstructLab Repo
- JJ and Paul continue introducing and getting started with InstructLab (2024) (Video)
- InstructLab – “Ever imagined the ease of tuning pre-trained LLMs? InstructLab makes it a reality. Let’s delve into how it sets itself apart from other model tuning methods.” (Blog)
- Generative AI Development with Podman AI Lab, InstructLab, & OpenShift AI (Video)