Rami's Readings #46
The latest on AI, LLMs, Meta, ReplaceAnything, R1, Prompt Engineering, Ellis Island, Bureaucracy, Davos, Oceans, Business Culture, and more.
Welcome to Rami’s Readings #46 - a weekly digest of interesting articles, papers, videos, and X (formerly known as Twitter) threads from my various sources across the Internet. Expect a list of reads covering AI, technology, business, culture, fashion, travel, and more. Learn about what I do at ramisayar.com/about.
Happy New Year! 🎉 As we shake off the confetti and step into a fresh calendar year, today's newsletter is going to be a trove of interesting reads from the past few weeks. My advice to you, dear reader, is to pick and choose what resonates with you the most!
I am excited for this year! Apéro & Intellect was a delight to host and expect a few more this year. Dates and locations will be announced in this newsletter.
I am writing two special newsletters to recap 2023 and discuss what to expect in 2024. Expect those out soon.
🤖 AI Reads
Meta is Developing Open Source AGI, Says Zuckerberg
Notes: Llama 2 is almost single-handedly pushing forward open source AI. Watch Zuck’s video on Instagram. The number of H100s bought represents billions of dollars.
We're currently training our next-gen model Llama 3, and we're building massive compute infrastructure to support our future roadmap, including 350k H100s by the end of this year -- and overall almost 600k H100s equivalents of compute if you include other GPUs.
Self-Rewarding Language Models
Notes: A collaboration between Meta-NYU, published around the same time as Zuck’s Open Source AGI announcement. Claims they are beating GPT-4 and Claude 2 with this technique. Wild! 🔥
MAGNeT from Meta
Notes: Another fantastic audio generation model from Meta. HuggingFace collection.
Transformers are Multi-State RNNs
Notes: Examining the inner-workings of LLMs. Also from Meta!
ReplaceAnything from Alibaba
Notes: Cool demo, no paper or code so far.
R1 Your Pocket Companion
Notes: Quite the hype around this AI device after the announcement at CES! The price point is right for mass consumption. I put in an order. How about you?
Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training
Notes: Paper from Anthropic with implications for AI Safety.
Mixtral of Experts
Notes: Mistral’s paper on Mixtral 8x7b released in 2023.
Principled Instructions Are All You Need for Questioning LLaMA-1/2, GPT-3.5/4
Notes: From Abu Dhabi.
Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding
Notes: Prompt engineering techniques for tabular data.
Applications for Converge 2 Now Open
Participants will receive a $1 million investment from the OpenAI Startup Fund and join our community of builders.
💼 Business Reads
No One’s Name Was Changed at Ellis Island
Notes: Huh! I also believed names were changed at Ellis Island.
Ethan Mollick on R&D Spending of US Companies Compared to Entire Countries
Death by a Thousand Paper Cuts
Notes: David Brooks on unnecessary bureaucracy.
Why the Davos Smart Set Sounds Dumb
Notes: Ouch.
'Predictably Irrational': Keeping (Too Many) Options Open
Notes: Oldie but goodie.
How Crowded Are the Oceans? New Maps Show What Flew under the Radar until Now
Notes: Wow!
Culture Eats Strategy For Breakfast
Notes: Oldie but goodie. Written by my MIT Prof Bill Aulet.
That is all for this week. Signing off from Redmond, WA.