Results on Financial Statement Analysis Tasks

Angle I: LoRA Methods’ Performance on Financial Datasets

On financial statement analysis tasks, fine-tuning the Llama 3.1 8B model [Llama3] provides a substantial performance uplift, improving its capabilities on complex formula construction and calculation tasks where base models fail.

Different LoRA methods excel at different tasks. While standard LoRA [LoRA] is effective, DoRA [DoRA] and rsLoRA [RSLoRA] achieve the highest scores on the formula-based tasks. When compared to the fine-tuned Gemini baseline [Gemini2], the top-performing Llama 3.1 8B variants (using DoRA and rsLoRA) demonstrate superior performance on specific formula construction and calculation. However, the Gemini model shows stronger results on the Financial Math [Financial_Math] dataset.

Angle II: LoRA Suitability for Financial Tasks

LoRA fine-tuning is highly effective for teaching models structured, multi-step financial analysis on XBRL format [XBRL_Analysis]. The most significant performance gains are seen in Formula Construction and Formula Calculation [Financial_Math], where fine-tuning improves the model’s capabilities to high proficiency.

The benefit is less pronounced for the broader FinanceBench [FinanceBench] and Financial Math [Financial_Math] datasets. While scores improve, the gains are more modest. This suggests that while LoRA methods are excellent for instilling specific, XBRL-based structured format, they are less effective at enhancing the model’s general, open-ended mathematical and financial abilities.