The computer I spent 300 yuan to assemble successfully ran through the local large model

If 2023 is recognized as the first year of AI, then 2024 is likely to be a key year for the popularization of large AI models.

In the past year, a large number of large AI models and AI applications have emerged. Manufacturers such as Meta and Google have also begun to launch their own online/local large models to the public, similar to "AI artificial intelligence". An out-of-reach concept like "intelligence" suddenly comes to people.

Nowadays, people are increasingly exposed to artificial intelligence in their lives. If you look carefully, you will find that almost all of the various AI applications you have access to are deployed on the "cloud". If you want to build a device that can run large models locally, then the hardware is a brand-new AIPC priced at more than 5,000 yuan, which is not friendly to ordinary consumers.

(Photo source: Google)

But compressing the budget to the extreme and then matching a set of "usable" hosts is exactly what everyone needs The ultimate fun for DIYers. The question now is, if you really want to get a computer that can run a large speech model locally, how much will it cost?

My budget is - 300 yuan.

Although the prices of serious memory/storage products have been rising recently, the price of foreign garbage has not changed much. Even a batch of early foreign garbage CPUs and mining card GPUs have experienced price drops. trend. In the context of the current AI trend, I am not even going to stick to office audio and video needs. I decided to challenge myself and use a budget of 300 yuan to try to build a cost-effective entry-level local large-model host.

As for how to operate it specifically and whether the final result is satisfactory, you will naturally know after following me.

Classic old CPU, miner-level GPU

In terms of CPU selection, I directly picked up an Intel® Core™i3-4170T from Xianyu.

This processor has two cores and four threads, a main frequency of 3.2GHz, no turbo frequency capability, a 3MB smart cache, and a Haswell architecture using 22nm process technology. The biggest feature is that it has an HD4400 core display. That's also the key to why I chose it.

(Photo source: Lei Technology)

As the saying goes , low-cost U with low-cost board, the motherboard I chose is an ASUS work package H81M-V3 motherboard, not to mention only two DDR3 memory slots, not even an HDMI output interface, no M2 hard drive, USB 3.0, There are only two SATA III interfaces. The biggest advantage of this thing is the super low second-hand price.

This board U set, which adds up to less than 80 yuan, has outstanding value for money today.

(Photo source: Taobao)

This kind of CPU dissipates heat Don't worry too much.

Xianyu 20 packs of push-down air-cooled heat dissipators were mailed to me. Although the appearance is a bit ugly and crude, it is perfect for me. The low-price configuration is certainly more than enough.

(Photo source: Lei Technology)

As for the graphics card, Recently, a lot of P106-090/P106-100 mining cards have been released on Xianyu. The price of the former is generally around 75 yuan, and the price of the latter is generally around 120 yuan. Considering that my original intention this time is not Coming for games, I finally chose a Zotac P106-090 mining card. This card is not suitable for games due to bandwidth issues, but the price is right here.

Finally, simply equip it with two no-name DDR3 memories to form a dual-channel memory, use a 350W Great Wall power supply to power it, and a 120GB SATA SSD as the system disk. Use We made do with the remaining buffalo silicone grease from the first installation, and finally paired it with a small Computer City case that cost 20 yuan, and we had all the basic hardware for the 300 yuan installation.

(Photo source: Lei Technology)

Run for points if you don’t agree. ! Reasonable performance

#The installation is completed and the chassis is lit!

Let’s do a simple performance test first. As a veteran who has served for many years, the performance of Intel® Core™i3-4170T is just like that, even when using TrottlesStop to unlock power consumption In this case, it is almost equivalent to the level of Core 6th and 7th generation mobile processors.

(Source: Lei Technology)

In the actual test session, the CPU-Z test scored 319.9 points in the single-core test and 881.0 points in the multi-core test. Under the CINEBENCH test standard, the CINEBENCH R20 multi-core score was 732cb and the single-core score was 301cb. Of course, this performance is not outstanding, but it is completely sufficient for daily office work and light entertainment.

(Source: Lei Technology)

Let’s look at the GPU again Partly, the Zotac P106-090 we have is built using a 16nm process. The graphics card core is GP106, the core frequency is 1354MHz, which can be increased to 1531MHz. It has 768 shading units, supports DirectX12, and the video memory specification is 3072MB/192Bit Samsung GDDR5 memory. The memory frequency can reach 2002MHz.

(Photo source: Lei Technology)

It is worth noting that , the interface of P106-090 is PCI-E x4 1.1, while the interface of P106-100 is PCI-E x16 1.1, which leads to obvious performance differences between the two even if they have the same core. I choose The P106-090 graphics card is just because it can provide CUDA computing power at an extremely low price.

As shown in the figure, in the Fire Strike test to evaluate DX11 performance, P106-90 achieved a graphics score of 3287 points in the Extreme test; in the Fire Strike test to evaluate DX12 performance In the Time Spy test, the P106-90 achieved a graphics score of 2251 points in the basic test. This performance is similar to that of the GTX1050, but there is a certain gap between the core and display performance of the R5-7640H.

(Photo source: Lei Technology)

In terms of storage, we This off-brand 128GB Sata SSD hard drive purchased for 40 yuan has sequential read and write speeds of 505.28MB/s and 400.09MB/s, and random 4K read and write speeds of 132.35MB/s and 244.29MB/s, although it is not the same as the M2 SSD hard drive. It's a bit different, but it's definitely more than enough as a system boot disk.

(Photo source: Lei Technology)

As for this pair Channel DDR3 memory, using AIDA64 for memory cache testing, measured read speeds of 20894 MB/s, write speeds of 23629 MB/s, copy speeds of 20901 MB/s, and latency of 67.5ns, for this computer It can be said to be just right.

(Source: Lei Technology)

Large model game, It can actually run

# Since the goal is to create a usable local large-model game console with a budget of several hundred dollars, then the actual experience must be our best A piece of attention.

Try Koblodcpp first. This is an llamacpp startup program that integrates the Koblod.AI interface. It can run local large models in GGUF format that are currently popular on the Internet, and can even integrate The speech model and the drawing model enable the generation of speech and corresponding scenes during dialogue.

(Photo source: Lei Technology)

Unfortunately, The capacity of the GGUF format large models currently shared on Huggingface generally exceeds 3GB. Even if the causallm_7b model with a capacity of only about 5GB is selected, it will still cause the graphics card memory to overflow and trigger the CPU's calculation intervention, ultimately affecting the operating efficiency of the entire large model. reduce.

(Photo source: Lei Technology)

Let’s test it. In this mode, the longest reply of Kobold.AI reaches about 599s (i.e. 10 minutes), and the shortest reply takes about 50s to generate, which is far different from the average cloud model that can provide a reply in 10 seconds. Experience Very average indeed.

In this case, we might as well settle for the next best thing and use Intel AIGC Assistant to experience the local large model. This is a demo that integrates multiple local large language models. Committed to realizing intelligent human-machine dialogue without being connected to the Internet.

When the computer network is turned off, except for the first deployment of a large model, which takes a certain amount of time, subsequent questions can basically be answered within 5 seconds, regardless of whether Answering questions, translating or writing are all very fast. Although logical errors will still occur when dealing with some brain teasers or arithmetic problems, the overall experience in this case is much better than before.

(Photo source: Lei Technology)

Of course, more advanced functions such as character cards and prompt words are definitely not available.

Let’s look at AI painting again. What I use here is the bilibili UP master@ integrated Huiyo launcher, which integrates Stablediffsion WebUI, various SD plug-in downloads and remote use of models. In one, it is a very practical all-in-one starter.

(Photo source: Lei Technology)

Considering P106- 90 only has a 3GB storage limit, so I chose the mistoonAnime MIX_v20 model with a trimmed capacity of only 2GB. It was actually measured that it took about three minutes for P106-90 to generate four 128*160 pictures and four 240*320 pictures. It takes seven to eight minutes, and it takes about six minutes to generate a 512*512 picture.

(Source: Lei Technology)

From actual experience Look, I personally recommend that you start with 128*160 pictures while reducing the number of prompt words and plug-ins. When you encounter a suitable picture, you can then perform AI upgrading and redrawing of a single picture. In this way, the overall experience is quite good. Otherwise, use When generating pictures with the specifications of 512*512, the probability of discarded pictures is not generally high.

(Source: Lei Technology, picture generated with SD)

Now that the local large model can be used, what about the game experience?

Considering the actual hardware configuration, we tested "Far Cry 6", "Shadow of the Tomb Raider", "F1 2020" and "Horizon: Zero Dawn" Let’s take a look at the experience of the four games at FHD resolution.

In the performance test of "Far Cry 6", at the lowest image quality of 1920*1080, the average frame number of this computer was 32fps, even if the FSR performance profile was turned on , the final game frame rate can only fluctuate around 40fps.

(Photo source: Lei Technology)

Look at the racing car again For the game, in "F1 2020", we turned on the 1920*1080 medium image quality and conducted a sunny day test on the Australian track. The final average frame rate could reach 59fps. Even this mining card that has been in service many times can run smoothly.

(Source: Lei Technology)

As a standard test game , "Shadow of the Tomb Raider" naturally did not miss this test. Without turning on FSR and defaulting to the lowest image quality of 1920*1080, the average frame rate of this computer running the game benchmark test is 50fps, which is the closest to a stable 60fps gaming experience among this batch of 3A games.

(Photo source: Lei Technology)

Finally, let’s do it hard For "Horizon: Zero Dawn", because the lowest image quality will also burst the video memory, this computer can barely reach the passing line of 30fps at the lowest image quality of 1280*720, which can only be said to be unsuitable for playing.

(Photo source: Lei Technology)

Overall , affected by strict bandwidth restrictions, the actual performance of P106-090 is not as good as the GTX1050, which has similar running scores. At most, it is slightly better than the Iris Xe 96EU core display. It has been somewhat stretched to cope with 3A games in recent years. .

Summary: It can indeed run large models, but the stability is not good

Advantages:

1. The cost budget is low;

2. It can indeed run local large models.

Disadvantages:

1. Second-hand parts are not guaranteed;

2. The mining card driver is very easy to lose, and the white screen problem occurs has happened.

Can this computer host with a budget of 300 yuan be used?

Although the CPU specifications are old and the mining card performance is unstable, this computer host with a budget of 300 yuan can indeed complete the deployment of local large models, in text generation and AI There is no problem with the drawing experience, and you can even use it as an AI PC.

Even if you run a 3A masterpiece like "Far Cry 6", this machine can still run stably at 60 frames under FHD low image quality, so it can handle "League of Legends" This type of online game should be more than enough, and it can also be used as an entry-level online game host.

(Photo source: Lei Technology)

So what do I think? Are you satisfied with this computer host?

The answer is no. During actual use, the P106-90 mining card has experienced driver failure many times, resulting in a white screen on the computer. You need to use DDDU to uninstall the driver and then reinstall it. Condition. If this is just a little troublesome in testing, then it is definitely difficult to accept in daily use.

It is worth noting that there are currently many foreign junk machines on Xianyu with similar configurations to the machine I assembled, and the prices are generally 350- The price is about 400 yuan. The target audience is obviously young people who have just entered college or have stepped out of the society. I personally recommend that everyone not buy these products. It is no joke that you get what you pay for.

#The bottom line is that picking up trash is risky, so be careful before getting in the car. For players who are tired of playing with high-end computers, it is still very interesting to play with the garbage collector occasionally, but this is just for fun, and they are not really intended to be used as the main machine for a long time. After all, the accessories of the garbage collector are in disrepair. And there is no warranty. If you lose important data due to its failure, it will be worth the loss.

However, by using a budget of 300 yuan to complete the attempt to run a local large-model PC, it shows to a certain extent that hardware performance may not be the main factor restricting the development of AI PC. At least in the usage scenarios of ordinary consumers, instead of improving computing power, what PC manufacturers should do is to help users build the framework and environment of large AI models, and bring users a better experience in terms of systems and software. In other words, AI PC should be something that users can use at their fingertips, rather than assembling a DIY PC ourselves.

The above is the detailed content of The computer I spent 300 yuan to assemble successfully ran through the local large model. For more information, please follow other related articles on the PHP Chinese website!