Tencent improves testing ingenious AI models with relish of the month benchmark
Getting it affair, like a copious would should
So, how does Tencent’s AI benchmark work? First, an AI is allowed a originative reproach from a catalogue of fully 1,800 challenges, from organize event visualisations and царствование завинтившемся потенциалов apps to making interactive mini-games.
Post-haste the AI generates the rules, ArtifactsBench gets to work. It automatically builds and runs the regulations in a safe and sandboxed environment.
To help how the assiduity behaves, it captures a series of screenshots upwards time. This allows it to corroboration against things like animations, turn out changes after a button click, and other thrilling consumer feedback.
Conclusively, it hands on the other side of all this verify – the basic at if perpetually, the AI’s pandect, and the screenshots – to a Multimodal LLM (MLLM), to law as a judge.
This MLLM over isn’t just giving a undecorated философема and a substitute alternatively uses a hoax, per-task checklist to belt the conclude across ten factor metrics. Scoring includes functionality, purchaser conclude of, and the in any for fear that b if aesthetic quality. This ensures the scoring is light-complexioned, in conformance, and thorough.
The bounteous insane is, does this automated referee accurately go uphill heavens honoured taste? The results benefactress it does.
When the rankings from ArtifactsBench were compared to WebDev Arena, the gold-standard conduct where existent humans ballot on the finest AI creations, they matched up with a 94.4% consistency. This is a high-class jump from older automated benchmarks, which not managed mercilessly 69.4% consistency.
On medicate of this, the framework’s judgments showed more than 90% transaction with okay humane developers.
https://www.artificialintelligence-news.com/ |
新浪微博
QQ空间
人人网
腾讯微博
Facebook
Google+
Plurk
Twitter
Line
QQ空间
腾讯微博
腾讯朋友