FinTrace and PITfall
github.com/fredxing1/fintrace- Category
- AI Agents
- Rank
- No. 3993Tools index
Previous survey · No. 3799 ·
- Pricing
- Open Source
- Type
- AGENT
- Use case
- Research & Education · Model & Agent Evaluation
- GitHub
- 1 stars
- Date
About
FinTrace builds auditable financial research agents that answer using only evidence available at a chosen point in time, with each claim traceable to as-filed data. PITfall is a leakage-controlled benchmark for this: the repository reports that a naive live-data agent answers 82% of impossible questions with the true future value, while the gated agent leaks none. It is research software, not investment advice.
At a glance
FinTrace answers questions about U.S. company filings using only facts filed by a chosen cutoff. Its companion benchmark, PITfall, is worth using when you need to check whether an agent’s apparent forecasting skill comes from access to later data.
Good fit
Researchers testing whether a financial QA or backtesting workflow can reproduce what was knowable on a particular date.
Consider something else when
Skip it if you need live market data or coverage beyond the filings its data layer supports.
Start here
Pick a company and metric in the public demo. Set the cutoff just before the relevant filing, then move it past the filing date. Compare which facts become available and follow the cited filing to check the answer.
Cost and requirements
The Apache-2.0 project uses as-filed SEC EDGAR data. The public demo is documented as requiring no account or API key. Budget separately for storage and any model calls in your own runs.
Limits to know
The cutoff constrains facts served by the data layer. Models may still recall later information from training, and the benchmark results apply to the tested models, data and task set.
Sources checked 2026-11-05. Source-based assessment.
Why it made the leaderboard
Financial LLM agents can leak future data into historical questions. This gates retrieval to as-filed evidence and ships a leakage benchmark (PITfall) to measure it.
Tags
Tech Stack
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.