SQL

Natural language to SQL query generation evaluates text-to-query fidelity and schema reasoning. This task is particularly relevant for analytics chat assistants and simplified database interfaces where users need to query data using natural language. Models must understand both the intent behind the question and the structure of the underlying database schema.

88
gcp/gemini-2.0-flash
Average duration
11s
Average tokens
978
Average cost
$0.00
100
10s
649
opper_sql_sample_01
100
10s
660
opper_sql_sample_02
100
10s
679
opper_sql_sample_03
100
12s
658
opper_sql_sample_04
100
10s
706
opper_sql_sample_05
100
12s
766
opper_sql_sample_06
100
12s
654
opper_sql_sample_07
100
12s
663
opper_sql_sample_08
100
12s
651
opper_sql_sample_09
100
12s
1118
opper_sql_sample_10
100
10s
893
opper_sql_sample_11
75
12s
929
opper_sql_sample_12
75
12s
956
opper_sql_sample_13
100
12s
991
opper_sql_sample_14
100
12s
1015
opper_sql_sample_15
100
11s
1081
opper_sql_sample_16
100
11s
985
opper_sql_sample_17
75
11s
1136
opper_sql_sample_18
50
11s
980
opper_sql_sample_19
100
13s
1226
opper_sql_sample_20
100
9s
1117
opper_sql_sample_21
75
11s
1158
opper_sql_sample_22
100
9s
1232
opper_sql_sample_23
50
9s
1203
opper_sql_sample_24
50
12s
1251
opper_sql_sample_25
100
10s
1210
opper_sql_sample_26
50
9s
1117
opper_sql_sample_27
75
10s
1293
opper_sql_sample_28
50
10s
1218
opper_sql_sample_29
100
9s
1141
opper_sql_sample_30