In this Civo Navigate 2024 session, Engin Diri shares his unexpected journey into managing Open-Source Large Language Models (LLMs) in a cloud infrastructure setting. Discover the three key strategies he proposed to clients, weighing the pros and cons of each approach. Engin also addresses the challenges faced along the way and the solutions implemented to overcome them.

Learn more about Civo Navigate here -► https://www.civo.com/navigate

====================
Get free credit to try the world’s first K3s-powered, managed Kubernetes service.
Sign up to Civo -► https://www.civo.com
Get started with Kubernetes with Civo Academy: https://www.civo.com/academy

Subscribe to our YouTube Channel -► http://civo.io/subscribe

Follow Civo:

• Twitter -► https://twitter.com/civocloud
• Github -► https://github.com/civo
• LinkedIn -► https://linkedin.com/company/civocloud
• Facebook -► https://facebook.com/civocloud

0:00 - Introduction

1:30 - Overview of Serverless LLM Inferences

3:00 - The Importance of Open Source LLMs

6:00 - Addressing Cost and Privacy Concerns

8:15 - Understanding Proprietary vs. Open Source LLMs

11:45 - Challenges of Open Source LLMs

14:00 - Key Considerations for Serverless vs. Dedicated LLM Deployments

17:30 - The Benefits of Serverless Approaches

20:00 - Cost Analysis for Serverless vs. Dedicated Hosting

22:15 - Tools and Technologies: Carpenter and Pulumi

25:00 - Conclusion: Choosing the Right Approach

26:30 - Q&A and Closing Remarks