Rami's Readings #57 - 🇨🇦 Canadian Fintech Summit
The latest on CFS, AI, LLMs, Apple, Mixture-of-Depths, Cohere 4 AI, MIT Professor on AI & Jobs, 900 Open Source AI Tools, NeurIPS 2024, Bank of Canada, and more.
Welcome to Rami’s Readings #57 - a weekly digest of interesting articles, papers, videos, and X threads from my various sources across the Internet. Expect a list of reads covering AI, technology, business, culture, fashion, travel, and more. Learn about what I do at ramisayar.com/about.
I am excited to be back in Canada 🇨🇦 this week to speak at the Canadian Fintech Summit about Bing Deep search, Chat, Stories, GPT-4, LLMs at scale, and more. Let me know if you will be in Toronto and want to connect.
🤖 AI Reads
Mixture-of-Depths: Dynamically Allocating Compute in Transformer-based Language Models
Notes: Impressive paper from Google DeepMind & McGill showing a promising technique to improve compute in transformer-based models.
ReALM: Reference Resolution As Language Modeling
Notes: Apple in the house!
We demonstrate large improvements over an existing system with similar functionality across different types of references, with our smallest model obtaining absolute gains of over 5% for on-screen references. We also benchmark against GPT-3.5 and GPT-4, with our smallest model achieving performance comparable to that of GPT-4, and our larger models substantially outperforming it.
Cohere 4 AI Release Command R+ 104B Model with Open Weights
Notes: The open weights research release of the Command R+ from Cohere.
Models All The Way Down
Notes: Beautifully visualized article about dataset ethics and the creation of the LAION-5B dataset. Shared by Janelle Tam. LAION-5B data underpins the training set of Midjourney and Stability AI’s models.
AssemblyAI Launched Universal-1, Our Most Powerful and Accurate Multilingual Speech-to-Text Model to Date
Notes: Beats Whisper (my favorite) and is trained on 12.5M hours of multilingual audio data.
How One Tech Skeptic Decided AI Might Benefit the Middle Class
Notes: More evidence for Generative AI models as copilots from MIT Prof. Autor.
nilsherzig/LLocalSearch: LLocalSearch is a completely locally running search aggregator using LLM Agents
Notes: Neat!
What I Learned from Looking at 900 Most Popular Open Source AI Tools
Notes: Prompt engineering continues to be the superior developer experience, a point I have reiterated multiple times in this newsletter. Chip Huyen's data further supports this observation showing significant growth in repositories aiding prompt engineering. Chips’ data shows the dramatic growth of AI OSS from China - more evidence to not underestimate Chinese AI Talent, a point I have repeatedly made in this newsletter.
💼 Business Reads
Bank of Canada Warns of Low Productivity ‘Emergency,’ Making It Harder to Control Inflation
Notes: Is the BoC finally noticing an economy dominated by real estate investing is not productive? 🙄
Canada also fell behind our G7 peers, with only Italy seeing a larger decline in productivity relative to the United States.
To the best of my knowledge, despite significant advancements in AI research in Montreal and Toronto, and the overrepresentation of Canadians in Big Tech and Silicon Valley AI startups, only a limited number of Canadian companies have successfully implemented AI at scale. Reply to this email if you’d like to chat more.
“The real emergency here is that the Bank of Canada is stuck in the past,” Balsillie said in an e-mail. “This speech could have been written 35 years ago. Vague gesturing at ‘IP’ and ‘AI’ while offering the same old traditional policy prescriptions from the same outdated toolkit will worsen Canada’s productivity problems rather than address them.”
Did One Guy Just Stop a Huge Cyberattack?
Notes: NYTimes wrote about the XZ hack I shared last week. Answer is yes…
That is all for this week. Signing off from Montréal.