Financial Datasets
https://github.com/virattt/financial-datasets- Category
- Finance
- Rank
- No. 1599Tools index
Previous survey · No. 1607 ·
- Pricing
- Open Source
- Type
- TOOL
- Builder
- virattt
- GitHub
- 434 stars
- Latest release
- 0.1.16
- Date
About
Curated financial datasets ready for LLM training and evaluation: filings, transcripts, and market data.
What it does
Financial Datasets is a Python library that converts source material into structured question, answer, and supporting-context records. It accepts supplied passages, downloads remote PDFs, or retrieves SEC annual and quarterly reports, splits long material into token-sized chunks, and asks a GPT model for structured output.
Why it's ranked here
This is a focused generator with useful input routes and a clear output contract. Its prompt explicitly pushes for self-contained, source-grounded questions. However, implementation gaps weaken production readiness: model support is restricted, credential handling is misleading, and the declared runtime dependencies do not match the imports.
What's good
The output shape is enforced twice, through the model’s tool schema and Pydantic validation. SEC inputs can target selected report sections. PDF and SEC content is token-chunked with configurable size and overlap. The default prompt rejects unsupported questions, external-document references, fabrication, and answers lacking sufficient context.
Tradeoffs
Only model names beginning with the GPT prefix are accepted. The constructor requires an API key but does not pass it to the OpenAI client, so configuration depends on the client’s environment. Several imported runtime packages are absent from the dependency manifest. PDF downloads also proceed without checking HTTP status or content type.
How to use it well
Use it for prototyping financial question-answer corpora from passages, public PDFs, and SEC reports, especially when you want validated records with supporting context. Review generated facts before training or evaluation. Bring separate tooling for market-data retrieval, dataset storage, quality scoring, and non-OpenAI model providers.
Technical notes+
financial_datasets/generator.py validates api_key but stores only the model, while financial_datasets/llm/openai.py constructs a module-level OpenAI() client without that argument. The retry-decorated chat_completion_request catches exceptions and returns them, preventing Tenacity from seeing a raised failure. financial_datasets/generator.py forwards options as kwargs=kwargs, nesting them and preventing system_prompt and related settings from reaching generate_from_texts. It imports requests and tqdm, and financial_datasets/llm/openai.py imports tenacity, but pyproject.toml does not declare those packages directly. tests/test_generator.py assigns a mock client to the generator instance even though generation uses the module-level client.
Observed
- License
- MIT License
- Primary language
- Python
- Python support
- Python 3.10-compatible releases are declared
- Installation
- Published library installable with pip or Poetry, with Poetry used for repository installation
- Interface
- Python library API
- Supported inputs
- Lists of passages, remote PDF URLs, SEC 10-K reports, and SEC 10-Q reports
- Model backend
- OpenAI client with model names restricted to the GPT prefix
- Test structure
- Pytest suites cover report parsing and dataset generation
Read from README.md, pyproject.toml, financial_datasets/tools.py, financial_datasets/parser.py, financial_datasets/dataset.py, financial_datasets/filings.py, financial_datasets/prompts.py, financial_datasets/generator.py, financial_datasets/llm/openai.py, tests/test_parser.py, tests/test_generator.py, LICENSE.
What it can do
Provide SEC filing datasets
Company identifiers or filing types → Structured SEC filing data
Extract earnings call transcripts
Company names or ticker symbols → Formatted transcript text data
Generate market data feeds
Stock symbols and date ranges → Historical price and volume data
Format datasets for LLM training
Raw financial data → Training-ready dataset files
Create evaluation benchmarks
Financial data requirements → Standardized test datasets
Curate company financial reports
Company selection criteria → Clean, structured financial report data
Tags
Tech Stack
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.