Privacy issues with cloud-based LLMs like ChatGPT are becoming more prevalent as AI gets more pervasive. A strong substitute is provided by Apple's on-device AI approach, which is fueled by Core ML and Swift. In order to avoid cloud fees, latency, and data privacy concerns, this talk delves deeply into running large language models (LLMs) only on iOS devices.

What we'll cover:

Why on-device LLMs are important: offline functionality, cost savings, and privacy advantages.
Core ML Optimization: how to compress and convert Hugging Face models like Mistral and Phi-3 for iOS, including Metal GPU acceleration and 4-bit/8-bit quantization.
Swift integration: developing a responsive SwiftUI chatbot that manages memory, generates text in real time, and uses local LLM inference.
Challenges and solutions: using Apple's MLX framework, fallbacks for older devices, and tradeoffs between model size and speed.

The talk includes a live demo of a fully offline ChatGPT-like app running on an iPhone, plus benchmarks across devices. You will leave with actionable steps to implement private, efficient AI in your apps—no server required. Key takeaways:

Practical steps to deploy LLMs on iOS using Core ML + Swift.
Techniques to balance model size, speed, and accuracy.
Future-proofing apps for Apple’s on-device AI roadmap.

Target audience: intermediate/advanced iOS developers interested in AI, privacy, or cutting-edge Swift capabilities.

Использование AI в процессах разработки

Спикеры

Александр Юдин

Другие доклады по теме «Trends»