Researchers built a new RAG approach that: - does not need a vector DB. - does not embed data. - involves no chunking. - performs no similarity search. And it hit 98.7% accuracy on a financial benchmark (SOTA). Here's the core problem with RAG that this new approach solves: Traditional RAG chunks documents, embeds them into vectors, and retrieves based on semantic similarity. But similarity ≠ relevance. When you ask "What were the debt trends in 2023?", a vector search returns chunks that look similar. But the actual answer might be buried in some Appendix, referenced on some page, in a section that shares zero semantic overlap with your query. Traditional RAG would likely never find it. PageIndex (open-source) solves this. Instead of chunking and embedding, PageIndex builds a hierarchical tree structure from your documents, like an intelligent table of contents. Then it uses reasoning to traverse that tree. For instance, the model doesn't ask: "What text looks similar to this query?" Instead, it asks: "Based on this document's structure, where would a human expert look for this answer?" That's a fundamentally different approach with: - No arbitrary chunking that breaks context. - No vector DB infrastructure to maintain. - Traceable retrieval to see exactly why it chose a specific section. - The ability to see in-document references ("see Table 5.3") the way a human would. But here's the deeper issue that it solves. Vector search treats every query as independent. But documents have structure and logic, like sections that reference other sections and context that builds across pages. PageIndex respects that structure instead of flattening it into embeddings. Do note that this approach may not make sense in every use case since traditional vector search is still fast, simple, and works well for many applications. But for professional documents that require domain expertise and multi-step reasoning, this tree-based, reasoning-first approach shines. For instance, PageIndex achieved 98.7% accuracy on FinanceBench, significantly outperforming traditional vector-based RAG systems on complex financial document analysis. Everything is fully open-source, so you can see the full implementation in GitHub and try it yourself. I have shared the GitHub repo in the replies!
Avi Chawla@_avichawla
open on x ↗related
Techniques I'd master if building RAG systems that actually work:
Bookmark this.
1. Sliding Window Chunking
2. Semantic Chunking
3. Document Hierarchies
4. Metadata Enrichment
5. Query Expansion
6. Hybrid Search
7. Reranking Models
8. Context Window Packing
9. Lost in the Middle Problem
10. Hypothetical Document Embeddings (HyDE)
11. Multi-Query Retrieval
12. Contextual Compression
13. Sentence Window Retrieval
14. Auto-Merging Retrieval
15. Cross-Encoder Rescoring
16. Temporal Context Decay
17. Negative Sampling
18. MMR (Maximal Marginal Relevance)
19. Graph-Based Retrieval
20. Recursive Retrieval
21. Citation Trackingchunks
22. Context Ablation Testing
23. Adaptive Retrieval@athleticKoder
@OpenRouter≡ 10+2
@tom_doerr
@Zai_org
@every
@chrstnerode≡ 11+11
▶@aidenybai
@realmcore_Introducing Vibe SDK@rauchg
▶@argofowl≡ 2
@OpenRouter≡ 10+2
@tom_doerr
@Zai_org
@every
@chrstnerode≡ 11+11
▶@aidenybai
@realmcore_Introducing Vibe SDK@rauchg
@rohanpaul_ai
@Dr_Singularity
@Sumanth_077
@techNmak
▶@coderabbitaiBest Model Per Use-Case
Presentations - Gemini 2.5
Full-stack apps - GPT-5 Codex, Sonnet 4.5
Docs - Gemini 2.5, GPT-5 thinking
Videos - Sora 2
Images - Nano Banana
Coding - Sonnet 4.5, Grok Code Fast
Browser use - Sonnet 4.5
Doc Processing - Gemini Flash
Enterprise Search - Sonnet 4.5
Data analysis (complex) - Opus 4.1
Agentic workflows - Sonnet 4.5, Haiku 4.5@bindureddy
@GithubProjects
@techwith_ram
▶@_chenglou
@xmayeth+1I took the @karpathy autoresearch loop and pointed it at markets.
25 AI agents debate macro, rates, commodities, sectors, and single stocks daily. Every recommendation scored against real outcomes. Worst agent by rolling Sharpe gets its prompt rewritten by the system. Keep or revert. Same loop, prompts are the weights, Sharpe is the loss function.
Trained the agents on 18 months of market data. 378 iterations. 54 prompt modifications, 16 survived.
The system learned which agents to trust using Darwinian weights — geopolitical, commodities, and the @BillAckman quality compounder rose to the top.
The agents even figured out their own portfolio manager was the weakest link before we did!
Deployed the trained agents. +22% in 173 days. Best pick: AVGO at $152, held for +128%.
The final prompts are evolutionary products — shaped by market feedback, not human intuition. Now running live with my own capital.
https://github.com/chrisworsey55/atlas-gic
Part hedge fund, part research experiment :)@Chris_Worsey
▶@askalphaxivTo keep analyzing pdfs, some tools are qpdf, pdfalyzer, strings.
If you just want to get text out of PDFs, check out marker - https://github.com/datalab-to/marker .
I'm also interested in any weird edge cases anyone else has found - let me know!@VikParuchuriAs a developer,
Please Slap yourself if you are unable to explain even 10 from below terms :
Indexing
Clustering
Denormalization
Normalization
Read replicas
Leader–Follower replication
Multi-leader replication
Quorum
Consensus
CAP theorem
BASE
ACID
Eventual consistency
Strong consistency
Snapshot isolation
MVCC (Multi-Version Concurrency Control)
Two-phase commit (2PC)
Three-phase commit (3PC)
Write-ahead logging (WAL)
Checkpointing
Compaction
Rebalancing
Resharding
Data locality
Hot partition
Split-brain
Failover
High availability (HA)
Horizontal scaling
Vertical scaling
Load balancing
Connection pooling
Caching
Materialized views
Secondary indexes
Composite index
Covering index
Bloom filter
LSM tree
B-tree
Query planner
Cost-based optimizer
Deadlock
Lock escalation
Optimistic locking
Pessimistic locking
Dirty read
Phantom read
Read skew
Write skew
Data skew
Backpressure
Circuit breaker
Throttling
Rate limiting
CDC (Change Data Capture)
Logical replication
Physical replication
Geo-replication
Federation
Data lake
Data warehouse
Columnar storage
Row-based storage
Time-series partitioning
Hash partitioning
Range partitioning
Consistent hashing
Data migration
Schema evolution
Schema registry
Idempotency
Exactly-once semantics@SumitM_X
▶@prathamgrv
@dhruvtwt_Crypto cards are one of the best innovations for off-ramping crypto, I have used these 3 crypto cards in India and in other countries and also online on platforms like swiggy, makemytrip, nykaa etc.
How do they work?
> Load card with crypto from your wallet
> Spend it anywhere just like your normal cards
I’ve reviewed:
@Cypher_HQ_
@AviciMoney
@RedotPay
Based on:
- Exchange rates
- Daily transaction limit
- Fees
- Pros & cons
A thread 🧵@IshitaaPandeyLow-key websites I quietly rely on
1) http://roadmap.sh
Gives you a brutally clear learning path for roles like frontend, backend, DevOps, etc
No fluff, just “learn this → then this → then this”.
2) http://playcode.io
An online playground to quickly test HTML, CSS, JS without setting up anything locally
Perfect for quick experiments and debugging ideas
3) http://usehooks.com
A collection of reusable React hooks with real use cases
Saves time and helps you avoid rewriting the same logic again and again
4) http://devhints.io
Concise cheat sheets for languages, frameworks, and tools. Ideal when you forget syntax and don’t want to read a 20-minute blog
5) http://jsoncrack.com
Turns messy JSON into a clean visual tree
Makes understanding large APIs and configs way easier than staring at raw text
6) http://realtimecolors.com
Lets you generate and preview color palettes instantly
Useful when you want decent UI colors without guessing or copying blindly
7) http://regex101.com
Build, test, and debug regex step by step with explanations Honestly, the fastest way to stop hating regex
8) http://bundlephobia.com
Shows how big an npm package really is before you install it
Helps you avoid bloating your app with “tiny” libraries
9) http://caniuse.com
Tells you which CSS/JS features actually work across browsers Essential before using shiny new features in production
10) http://toolbox.googleapps.com
Google’s own diagnostics tools for DNS, email, headers, and network issues
Surprisingly useful for debugging real-world problems
👉 Which one of these do you already use and which one did you not know existed?@shekhu04
@PrajwalTomar_
▶@coreyhainesco
▶@askalphaxivTo keep analyzing pdfs, some tools are qpdf, pdfalyzer, strings.
If you just want to get text out of PDFs, check out marker - https://github.com/datalab-to/marker .
I'm also interested in any weird edge cases anyone else has found - let me know!@VikParuchuriAs a developer,
Please Slap yourself if you are unable to explain even 10 from below terms :
Indexing
Clustering
Denormalization
Normalization
Read replicas
Leader–Follower replication
Multi-leader replication
Quorum
Consensus
CAP theorem
BASE
ACID
Eventual consistency
Strong consistency
Snapshot isolation
MVCC (Multi-Version Concurrency Control)
Two-phase commit (2PC)
Three-phase commit (3PC)
Write-ahead logging (WAL)
Checkpointing
Compaction
Rebalancing
Resharding
Data locality
Hot partition
Split-brain
Failover
High availability (HA)
Horizontal scaling
Vertical scaling
Load balancing
Connection pooling
Caching
Materialized views
Secondary indexes
Composite index
Covering index
Bloom filter
LSM tree
B-tree
Query planner
Cost-based optimizer
Deadlock
Lock escalation
Optimistic locking
Pessimistic locking
Dirty read
Phantom read
Read skew
Write skew
Data skew
Backpressure
Circuit breaker
Throttling
Rate limiting
CDC (Change Data Capture)
Logical replication
Physical replication
Geo-replication
Federation
Data lake
Data warehouse
Columnar storage
Row-based storage
Time-series partitioning
Hash partitioning
Range partitioning
Consistent hashing
Data migration
Schema evolution
Schema registry
Idempotency
Exactly-once semantics@SumitM_X
▶@prathamgrv
@dhruvtwt_Crypto cards are one of the best innovations for off-ramping crypto, I have used these 3 crypto cards in India and in other countries and also online on platforms like swiggy, makemytrip, nykaa etc.
How do they work?
> Load card with crypto from your wallet
> Spend it anywhere just like your normal cards
I’ve reviewed:
@Cypher_HQ_
@AviciMoney
@RedotPay
Based on:
- Exchange rates
- Daily transaction limit
- Fees
- Pros & cons
A thread 🧵@IshitaaPandeyLow-key websites I quietly rely on
1) http://roadmap.sh
Gives you a brutally clear learning path for roles like frontend, backend, DevOps, etc
No fluff, just “learn this → then this → then this”.
2) http://playcode.io
An online playground to quickly test HTML, CSS, JS without setting up anything locally
Perfect for quick experiments and debugging ideas
3) http://usehooks.com
A collection of reusable React hooks with real use cases
Saves time and helps you avoid rewriting the same logic again and again
4) http://devhints.io
Concise cheat sheets for languages, frameworks, and tools. Ideal when you forget syntax and don’t want to read a 20-minute blog
5) http://jsoncrack.com
Turns messy JSON into a clean visual tree
Makes understanding large APIs and configs way easier than staring at raw text
6) http://realtimecolors.com
Lets you generate and preview color palettes instantly
Useful when you want decent UI colors without guessing or copying blindly
7) http://regex101.com
Build, test, and debug regex step by step with explanations Honestly, the fastest way to stop hating regex
8) http://bundlephobia.com
Shows how big an npm package really is before you install it
Helps you avoid bloating your app with “tiny” libraries
9) http://caniuse.com
Tells you which CSS/JS features actually work across browsers Essential before using shiny new features in production
10) http://toolbox.googleapps.com
Google’s own diagnostics tools for DNS, email, headers, and network issues
Surprisingly useful for debugging real-world problems
👉 Which one of these do you already use and which one did you not know existed?@shekhu04
@PrajwalTomar_
▶@coreyhainesco