Rami's Readings #62 - 📢 01.ai & DeepSeek Release New Models

The latest on AI, LLMs, new Yi & DeepSeek models, KAN, xLSTM, Underwater Mortgages, NYC & SF Millionaires, and more.

Welcome to Rami’s Readings #62 - a weekly digest of interesting articles, papers, videos, and X threads from my various sources across the Internet. Expect a list of reads covering AI, technology, business, culture, fashion, travel, and more. Learn about what I do at ramisayar.com/about.

---

---

Thank you to all my wonderful subscribers for your warm messages of congratulations, thoughtful ideas, and collaboration offers. I am genuinely excited to explore the fantastic projects you've proposed!

Rami's Readings #61 - 🏖️ Sabbatical Announcement

---

🤖 AI Reads

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

Notes: DeepSeek launched a MoE model. I haven’t had a time to try it, but I was using DeepSeek’s code generative model as my default for a while on my M1 due to excellent inference speed. #China #AI

Updated Yi-1.5 LLM Released as an Apache 2.0-Licensed Model

Notes: A good update to the Yi LLMs from 01.ai (Kai-Fu Lee’s startup). Again, another good Chinese model. Yi-1.5 34B almost matches Meta’s Llama 3 70B, while being half the size.

xLSTM: Extended Long Short-Term Memory and Kolmogorov-Arnold Networks

Notes: Very interesting architectures for LLMs if they can scale on hardware (other than Transformers and State Space Models). Kolmogorov-Arnold Networks was heavily discussed on the Twittersphere last week.

Every LLM Company Is a Search Company, and Search Is Hard: The Future of LLM Retrieval Systems

Notes: I agree. Search is hard. 😂

Musings on Building a Generative AI Product

Notes: From LinkedIn, I highly recommend reading this short article on GenAI products in practice.

Automatic evaluation is the holy grail, but still a work in progress.

Introducing ChatQA-1.5: Surpassing GPT-4 on Conversational QA and RAG

Notes: From Nvidia. Always interesting to explore.

LoRA Land: 310 Fine-tuned LLMs that Rival GPT-4, A Technical Report

Notes: Good survey for those interested in fine-tuned LLMs.

How AI apps make money

💼 Business Reads

How a Ballooning Public Sector Is Reshaping Canada’s Economy

Notes: This article is paywalled. Public sector employment as a share of total employment is above 21.5% in Canada and growing (a significantly larger share than the US or Italy). The growth rate between public and private sector employment is also widening quickly, further evidence of a weakening private sector.

‘Seriously Underwater’ Home Mortgages Tick Up Across the US

Notes: Not good. Seriously, it's not good. Underwater home mortgages limit owners' mobility to better-paying jobs and locations.

No, Low-Skilled Immigrants Don’t Cost Taxpayers Money

Notes: From Tyler Cowen.

Stock Price Prediction Using Time Series, Econometric, Machine Learning, and Deep Learning Models

Notes: Several years old, but still a fun read.

One Out of Every 24 New York City Residents Is Now a Millionaire

Notes: NYC, Bay Area (SF + Silicon Valley), Singapore, LA, Beijing up >40%. Unrelated: I miss NYC. Are you going to NYC Tech Week?

Photo by Jermaine Ee on Unsplash

That is all for this week. Signing off from Redmond.