search
HomeBackend DevelopmentPython TutorialPractical Approaches to Key Information Extraction (Part 2)

Real-Life Key Information Extraction (Part 2): Refining Open-Source LLM Performance

Continuing from Part 1, this guide explores using open-source Large Language Models (LLMs) – Qwen2 2B and LLaMA 3.1 – for invoice information extraction, focusing on overcoming data privacy concerns and hardware limitations (RTX 3060 with 12GB VRAM).

Why Qwen2 2B and LLaMA 3.1?

The choice of these models was driven by resource constraints. Qwen2-VL-2B-Instruct, due to its efficient size, was preferred over larger 7B models. LLaMA 3.1 (8B), accessed via Ollama, was selected for its optimized long-context understanding. Other models, such as Qwen2 in Ollama (lacking image support) and LLaVA (insufficient multilingual capabilities), were deemed unsuitable.

This two-model strategy leverages Qwen2's strength in general key information extraction and LLaMA 3.1's superior long-context handling and JSON output consistency, particularly for multilingual documents. Qwen2 initially extracts raw information, which LLaMA 3.1 then refines and structures into a standardized JSON format. PaddleOCR, as in Part 1, enhances vision capabilities for Qwen2.

A Japanese Invoice Example

A Japanese invoice was used as a test case. The initial OCR process (incorporating language detection and PaddleOCR) yielded the following recognized text:

<code>Recognized Text: 
{'ori_text': '根岸 東急ストア TEL 045-752-6131 領収証 [TOP2C!UbO J3カード」 クレヅッ 卜でのお支払なら 200円で3ボイン卜 お得なカード! 是非こ入会下さい。 2013年09月02日(月) レジNO. 0102 NOO07さ と う 001131 スダフエウ卜チーネ 23 単198 1396 003271 オインイ年 ユウ10 4238 000805 ソマ一ク スモー一クサーモン 1298 003276 タカナン ナマクリーム35 1298 001093 ヌテラ スフレクト 1398 000335 バナサ 138 000112 アボト 2つ 単158 1316 A000191 タマネキ 429 合計 2,111 (内消費税等 100 現金 10001 お預り合計 110 001 お釣り 7 890', 
'ori_language': 'ja', 
'text': 'Negishi Tokyu Store TEL 045-752-6131 Receipt [TOP2C!UbO J3 Card] If you pay with a credit card, you can get 3 points for 200 yen.A great value card!Please join us. Monday, September 2, 2013 Cashier No. 0102 NOO07 Satou 001131 Sudafue Bucine 23 Single 198 1396 003271 Oinyen Yu 10 4238 000805 Soma Iku Smo Iku Salmon 1298 003276 Takanan Nama Cream 35 1 298 001093 Nutella Sprect 1398 000335 Banasa 138 000112 Aboto 2 AA 158 1316 A000191 Eggplant 429 Total 2,111 (including consumption tax, etc. 100 Cash 10001 Total deposited 110 001 Change 7 890', 
'language': 'en',}</code>

Practical Approaches to Key Information Extraction (Part 2)

This was compared against a ChatGPT baseline, demonstrating ChatGPT's superior performance in this initial test.

Qwen2 2B Results (Standalone)

Using Qwen2 independently revealed its limitations. The JSON output was incomplete and inaccurate in several fields, highlighting its struggles with consistent JSON formatting and long-context processing.

Combined Qwen2 and LLaMA 3.1 Approach

Employing LLaMA 3.1 as a post-processor to refine Qwen2's output yielded improved, but still imperfect, results. While some key fields were accurately extracted, detailed item information remained problematic.

Future Improvements: Fine-tuning Qwen2VL

The next part will detail fine-tuning the Qwen2VL model using a dataset of receipt data, aiming to significantly improve both accuracy and processing speed for this specialized task.

Conclusion

This experiment showcases the potential and limitations of open-source LLMs for real-world key information extraction. While a combined approach offers improvements over using a single model, further refinement, particularly through model fine-tuning, is necessary to achieve optimal performance. The focus on data privacy and efficient resource utilization remains a key advantage of this open-source approach.

The above is the detailed content of Practical Approaches to Key Information Extraction (Part 2). For more information, please follow other related articles on the PHP Chinese website!

Statement
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
How are arrays used in scientific computing with Python?How are arrays used in scientific computing with Python?Apr 25, 2025 am 12:28 AM

ArraysinPython,especiallyviaNumPy,arecrucialinscientificcomputingfortheirefficiencyandversatility.1)Theyareusedfornumericaloperations,dataanalysis,andmachinelearning.2)NumPy'simplementationinCensuresfasteroperationsthanPythonlists.3)Arraysenablequick

How do you handle different Python versions on the same system?How do you handle different Python versions on the same system?Apr 25, 2025 am 12:24 AM

You can manage different Python versions by using pyenv, venv and Anaconda. 1) Use pyenv to manage multiple Python versions: install pyenv, set global and local versions. 2) Use venv to create a virtual environment to isolate project dependencies. 3) Use Anaconda to manage Python versions in your data science project. 4) Keep the system Python for system-level tasks. Through these tools and strategies, you can effectively manage different versions of Python to ensure the smooth running of the project.

What are some advantages of using NumPy arrays over standard Python arrays?What are some advantages of using NumPy arrays over standard Python arrays?Apr 25, 2025 am 12:21 AM

NumPyarrayshaveseveraladvantagesoverstandardPythonarrays:1)TheyaremuchfasterduetoC-basedimplementation,2)Theyaremorememory-efficient,especiallywithlargedatasets,and3)Theyofferoptimized,vectorizedfunctionsformathematicalandstatisticaloperations,making

How does the homogenous nature of arrays affect performance?How does the homogenous nature of arrays affect performance?Apr 25, 2025 am 12:13 AM

The impact of homogeneity of arrays on performance is dual: 1) Homogeneity allows the compiler to optimize memory access and improve performance; 2) but limits type diversity, which may lead to inefficiency. In short, choosing the right data structure is crucial.

What are some best practices for writing executable Python scripts?What are some best practices for writing executable Python scripts?Apr 25, 2025 am 12:11 AM

TocraftexecutablePythonscripts,followthesebestpractices:1)Addashebangline(#!/usr/bin/envpython3)tomakethescriptexecutable.2)Setpermissionswithchmod xyour_script.py.3)Organizewithacleardocstringanduseifname=="__main__":formainfunctionality.4

How do NumPy arrays differ from the arrays created using the array module?How do NumPy arrays differ from the arrays created using the array module?Apr 24, 2025 pm 03:53 PM

NumPyarraysarebetterfornumericaloperationsandmulti-dimensionaldata,whilethearraymoduleissuitableforbasic,memory-efficientarrays.1)NumPyexcelsinperformanceandfunctionalityforlargedatasetsandcomplexoperations.2)Thearraymoduleismorememory-efficientandfa

How does the use of NumPy arrays compare to using the array module arrays in Python?How does the use of NumPy arrays compare to using the array module arrays in Python?Apr 24, 2025 pm 03:49 PM

NumPyarraysarebetterforheavynumericalcomputing,whilethearraymoduleismoresuitableformemory-constrainedprojectswithsimpledatatypes.1)NumPyarraysofferversatilityandperformanceforlargedatasetsandcomplexoperations.2)Thearraymoduleislightweightandmemory-ef

How does the ctypes module relate to arrays in Python?How does the ctypes module relate to arrays in Python?Apr 24, 2025 pm 03:45 PM

ctypesallowscreatingandmanipulatingC-stylearraysinPython.1)UsectypestointerfacewithClibrariesforperformance.2)CreateC-stylearraysfornumericalcomputations.3)PassarraystoCfunctionsforefficientoperations.However,becautiousofmemorymanagement,performanceo

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

PhpStorm Mac version

PhpStorm Mac version

The latest (2018.2.1) professional PHP integrated development tool

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Linux new version

SublimeText3 Linux new version

SublimeText3 Linux latest version

mPDF

mPDF

mPDF is a PHP library that can generate PDF files from UTF-8 encoded HTML. The original author, Ian Back, wrote mPDF to output PDF files "on the fly" from his website and handle different languages. It is slower than original scripts like HTML2FPDF and produces larger files when using Unicode fonts, but supports CSS styles etc. and has a lot of enhancements. Supports almost all languages, including RTL (Arabic and Hebrew) and CJK (Chinese, Japanese and Korean). Supports nested block-level elements (such as P, DIV),

SAP NetWeaver Server Adapter for Eclipse

SAP NetWeaver Server Adapter for Eclipse

Integrate Eclipse with SAP NetWeaver application server.