Rami's Readings #81 - Anthropic's Contextual Retrieval
Rami's Readings #81 - Anthropic's Contextual Retrieval
The latest on AI, LLMs, Anthropic's Contextual Retrieval, Mistral, Qwen, New York City, Intel, UK Taxation, and more.
Welcome to Rami’s Readings #81 - a weekly digest of interesting articles, papers, videos, and X threads from my various sources across the Internet. Expect a list of reads covering AI, technology, business, culture, fashion, travel, and more. Learn about what I do at ramisayar.com/about.
🤖 AI Reads
Anthropic’s Contextual Retrieval
Notes: Anthropic shared a technique that improves how to add context in document chunks for RAG pipelines, but with a cost tradeoff. Is everything just a tradeoff between speed, cost, and quality? Yes. 😉 Fantastic blog post!
Mistral Released Mistral-Small-Instruct-2409
Notes: Not exactly small at 22B parameters. This model follows Mistral’s launch of Pixtral last week.
Qwen2.5-Coder Technical Report
Notes: Alibaba released the Qwen2.5 series of models on the 19th. In my opinion, the most interesting model is the Qwen2.5-Coder model which has state-of-the-art generation at the size of 1.5B and 7B.
Bespoke-MiniCheck Model for Fact Checking
Notes: Not sure when this model was released. The company was founded in 2024 according to PitchBook.
Moshi: A Speech-Text Foundation Model for Real-Time Dialogue
Notes: From the non-profit lab Kyutai in Paris. Effectively, an open source version of OpenAI’s Advanced Voice Mode. This is the first time I see an inference codebase written in Rust (and not C++).
Our resulting model is the first real-time full-duplex spoken large language model, with a theoretical latency of 160ms, 200ms in practice.
Nvidia Released Nemotron-Mini 4B Instruct
Notes: This model was announced in August but is now available for testing.
On the Diagram of Thought
Notes: Short paper from Tsinghua, Shanghai Artificial Intelligence Laboratory and Qizhi Institute, sparse on the details and data.
GRIN: GRadient-INformed MoE
Notes: Microsoft Research released a MoE model that achieves good performance while only activating a small percentage of the total parameters. GitHub. (Not Microsoft AI, see disclosure in #7).
💼 Business Reads
Why Is New York City So Safe?
Notes: The statistics are wild! NYC is one of the safest places in the country, because you’re simply less likely to die from a car accident.
Chipmaker Qualcomm to Explore Takeover of Intel & Apollo to Offer Multibillion-Dollar Investment in Intel
Notes: Last week, I asked when will Intel be nationalized? It doesn’t seem likely after Qualcomm and Apollo sending offers.
The Economic Consequences of the French Wealth Tax
Notes: This 2008 (last updated 2011) paper is making the rounds again, especially given the proposal to tax unrealized capital gains and the formation of a new French government.
The ISF causes an annual fiscal shortfall of €7 billion, or about twice what it yields; The ISF wealth tax has probably reduced GDP growth by 0.2% per annum, or around 3.5 billion (roughly the same as it yields); In an open world, the ISF wealth tax impoverishes France, shifting the tax burden from wealthy taxpayers leaving the country onto other taxpayers.
London’s Ultra-Rich Flee the Threat of Rising Taxes
Notes: A reminder that capital can leave, if it wasn’t already obvious… Switzerland and the UAE are often favored destinations. However, I didn’t expect Italy to be a new option - I can’t imagine their interesting tax rule on overseas earnings will persist.
In Mayfair’s financial enclave and the sleek offices of advisers to the ultra-rich, the talk is getting louder: everyone knows someone who's thinking about their exit strategy — or already gone.
Signing off from Redmond.