SQL

Natural language to SQL query generation evaluates text-to-query fidelity and schema reasoning. This task is particularly relevant for analytics chat assistants and simplified database interfaces where users need to query data using natural language. Models must understand both the intent behind the question and the structure of the underlying database schema.

89
openai/o1-mini
Average duration
24s
Average tokens
1835
Average cost
$0.00
100
22s
1268
opper_sql_sample_01
100
23s
1285
opper_sql_sample_02
100
22s
1354
opper_sql_sample_03
100
22s
1538
opper_sql_sample_04
100
19s
1137
opper_sql_sample_05
100
22s
1685
opper_sql_sample_06
100
19s
1016
opper_sql_sample_07
100
19s
1157
opper_sql_sample_08
100
19s
1030
opper_sql_sample_09
100
23s
1711
opper_sql_sample_10
100
19s
1614
opper_sql_sample_11
100
22s
1380
opper_sql_sample_12
100
22s
1687
opper_sql_sample_13
100
23s
1947
opper_sql_sample_14
75
1m 14s
1673
opper_sql_sample_15
100
23s
1995
opper_sql_sample_16
100
23s
1401
opper_sql_sample_17
100
23s
1831
opper_sql_sample_18
50
22s
2497
opper_sql_sample_19
100
22s
2668
opper_sql_sample_20
100
59s
1948
opper_sql_sample_21
75
22s
2312
opper_sql_sample_22
100
23s
1914
opper_sql_sample_23
75
22s
3369
opper_sql_sample_24
100
19s
2423
opper_sql_sample_25
75
23s
2105
opper_sql_sample_26
50
19s
2731
opper_sql_sample_27
75
23s
2259
opper_sql_sample_28
0
19s
2421
opper_sql_sample_29
100
23s
1694
opper_sql_sample_30