Real-Life Key Information Extraction (Part 2): Refining Open-Source LLM Performance
Continuing from Part 1, this guide explores using open-source Large Language Models (LLMs) – Qwen2 2B and LLaMA 3.1 – for invoice information extraction, focusing on overcoming data privacy concerns and hardware limitations (RTX 3060 with 12GB VRAM).
Why Qwen2 2B and LLaMA 3.1?
The choice of these models was driven by resource constraints. Qwen2-VL-2B-Instruct, due to its efficient size, was preferred over larger 7B models. LLaMA 3.1 (8B), accessed via Ollama, was selected for its optimized long-context understanding. Other models, such as Qwen2 in Ollama (lacking image support) and LLaVA (insufficient multilingual capabilities), were deemed unsuitable.
This two-model strategy leverages Qwen2's strength in general key information extraction and LLaMA 3.1's superior long-context handling and JSON output consistency, particularly for multilingual documents. Qwen2 initially extracts raw information, which LLaMA 3.1 then refines and structures into a standardized JSON format. PaddleOCR, as in Part 1, enhances vision capabilities for Qwen2.
A Japanese Invoice Example
A Japanese invoice was used as a test case. The initial OCR process (incorporating language detection and PaddleOCR) yielded the following recognized text:
<code>Recognized Text: {'ori_text': '根岸 東急ストア TEL 045-752-6131 領収証 [TOP2C!UbO J3カード」 クレヅッ 卜でのお支払なら 200円で3ボイン卜 お得なカード! 是非こ入会下さい。 2013年09月02日(月) レジNO. 0102 NOO07さ と う 001131 スダフエウ卜チーネ 23 単198 1396 003271 オインイ年 ユウ10 4238 000805 ソマ一ク スモー一クサーモン 1298 003276 タカナン ナマクリーム35 1298 001093 ヌテラ スフレクト 1398 000335 バナサ 138 000112 アボト 2つ 単158 1316 A000191 タマネキ 429 合計 2,111 (内消費税等 100 現金 10001 お預り合計 110 001 お釣り 7 890', 'ori_language': 'ja', 'text': 'Negishi Tokyu Store TEL 045-752-6131 Receipt [TOP2C!UbO J3 Card] If you pay with a credit card, you can get 3 points for 200 yen.A great value card!Please join us. Monday, September 2, 2013 Cashier No. 0102 NOO07 Satou 001131 Sudafue Bucine 23 Single 198 1396 003271 Oinyen Yu 10 4238 000805 Soma Iku Smo Iku Salmon 1298 003276 Takanan Nama Cream 35 1 298 001093 Nutella Sprect 1398 000335 Banasa 138 000112 Aboto 2 AA 158 1316 A000191 Eggplant 429 Total 2,111 (including consumption tax, etc. 100 Cash 10001 Total deposited 110 001 Change 7 890', 'language': 'en',}</code>
This was compared against a ChatGPT baseline, demonstrating ChatGPT's superior performance in this initial test.
Qwen2 2B Results (Standalone)
Using Qwen2 independently revealed its limitations. The JSON output was incomplete and inaccurate in several fields, highlighting its struggles with consistent JSON formatting and long-context processing.
Combined Qwen2 and LLaMA 3.1 Approach
Employing LLaMA 3.1 as a post-processor to refine Qwen2's output yielded improved, but still imperfect, results. While some key fields were accurately extracted, detailed item information remained problematic.
Future Improvements: Fine-tuning Qwen2VL
The next part will detail fine-tuning the Qwen2VL model using a dataset of receipt data, aiming to significantly improve both accuracy and processing speed for this specialized task.
Conclusion
This experiment showcases the potential and limitations of open-source LLMs for real-world key information extraction. While a combined approach offers improvements over using a single model, further refinement, particularly through model fine-tuning, is necessary to achieve optimal performance. The focus on data privacy and efficient resource utilization remains a key advantage of this open-source approach.
The above is the detailed content of Practical Approaches to Key Information Extraction (Part 2). For more information, please follow other related articles on the PHP Chinese website!

ArraysinPython,especiallyviaNumPy,arecrucialinscientificcomputingfortheirefficiencyandversatility.1)Theyareusedfornumericaloperations,dataanalysis,andmachinelearning.2)NumPy'simplementationinCensuresfasteroperationsthanPythonlists.3)Arraysenablequick

You can manage different Python versions by using pyenv, venv and Anaconda. 1) Use pyenv to manage multiple Python versions: install pyenv, set global and local versions. 2) Use venv to create a virtual environment to isolate project dependencies. 3) Use Anaconda to manage Python versions in your data science project. 4) Keep the system Python for system-level tasks. Through these tools and strategies, you can effectively manage different versions of Python to ensure the smooth running of the project.

NumPyarrayshaveseveraladvantagesoverstandardPythonarrays:1)TheyaremuchfasterduetoC-basedimplementation,2)Theyaremorememory-efficient,especiallywithlargedatasets,and3)Theyofferoptimized,vectorizedfunctionsformathematicalandstatisticaloperations,making

The impact of homogeneity of arrays on performance is dual: 1) Homogeneity allows the compiler to optimize memory access and improve performance; 2) but limits type diversity, which may lead to inefficiency. In short, choosing the right data structure is crucial.

TocraftexecutablePythonscripts,followthesebestpractices:1)Addashebangline(#!/usr/bin/envpython3)tomakethescriptexecutable.2)Setpermissionswithchmod xyour_script.py.3)Organizewithacleardocstringanduseifname=="__main__":formainfunctionality.4

NumPyarraysarebetterfornumericaloperationsandmulti-dimensionaldata,whilethearraymoduleissuitableforbasic,memory-efficientarrays.1)NumPyexcelsinperformanceandfunctionalityforlargedatasetsandcomplexoperations.2)Thearraymoduleismorememory-efficientandfa

NumPyarraysarebetterforheavynumericalcomputing,whilethearraymoduleismoresuitableformemory-constrainedprojectswithsimpledatatypes.1)NumPyarraysofferversatilityandperformanceforlargedatasetsandcomplexoperations.2)Thearraymoduleislightweightandmemory-ef

ctypesallowscreatingandmanipulatingC-stylearraysinPython.1)UsectypestointerfacewithClibrariesforperformance.2)CreateC-stylearraysfornumericalcomputations.3)PassarraystoCfunctionsforefficientoperations.However,becautiousofmemorymanagement,performanceo


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

PhpStorm Mac version
The latest (2018.2.1) professional PHP integrated development tool

Notepad++7.3.1
Easy-to-use and free code editor

SublimeText3 Linux new version
SublimeText3 Linux latest version

mPDF
mPDF is a PHP library that can generate PDF files from UTF-8 encoded HTML. The original author, Ian Back, wrote mPDF to output PDF files "on the fly" from his website and handle different languages. It is slower than original scripts like HTML2FPDF and produces larger files when using Unicode fonts, but supports CSS styles etc. and has a lot of enhancements. Supports almost all languages, including RTL (Arabic and Hebrew) and CJK (Chinese, Japanese and Korean). Supports nested block-level elements (such as P, DIV),

SAP NetWeaver Server Adapter for Eclipse
Integrate Eclipse with SAP NetWeaver application server.
