This workshop will provide researchers interested in LLMs with the skills to launch and run inference-based workflows with open-source models on YCRC systems. Specifically, attendees will learn:
What GPU resources are available for each YCRC system
How to identify what GPU is needed for different LLMs
How to launch an LLM on YCRC systems
How to conduct inference via direct prompting with an LLM using ollama
How to modify an LLM's parameters for reproducibility/consistency in response
How to implement an LLM within python using jupyter and ollama
Understand additional considerations for RAG and fine-tuning.
This is an advanced instruction. Working introductory knowledge of HPC on Yale's clusters is required. If you are a new user, we recommend watching the Introduction to HPC video and its accompanying slides, and completing the intro exercises before attending this workshop.
