SQL

Natural language to SQL query generation evaluates text-to-query fidelity and schema reasoning. This task is particularly relevant for analytics chat assistants and simplified database interfaces where users need to query data using natural language. Models must understand both the intent behind the question and the structure of the underlying database schema.

92
anthropic/claude-3.5-sonnet
Average duration
8s
Average tokens
1070
Average cost
$0.00
100
8s
730
opper_sql_sample_01
100
7s
750
opper_sql_sample_02
100
6s
752
opper_sql_sample_03
100
8s
759
opper_sql_sample_04
100
6s
788
opper_sql_sample_05
100
8s
819
opper_sql_sample_06
100
6s
731
opper_sql_sample_07
100
6s
742
opper_sql_sample_08
100
11s
763
opper_sql_sample_09
100
6s
802
opper_sql_sample_10
100
7s
1032
opper_sql_sample_11
100
8s
1043
opper_sql_sample_12
100
8s
1080
opper_sql_sample_13
100
7s
1083
opper_sql_sample_14
100
6s
1087
opper_sql_sample_15
100
8s
1134
opper_sql_sample_16
100
8s
1095
opper_sql_sample_17
75
9s
1219
opper_sql_sample_18
50
8s
1111
opper_sql_sample_19
100
9s
1188
opper_sql_sample_20
100
6s
1254
opper_sql_sample_21
100
8s
1332
opper_sql_sample_22
100
9s
1403
opper_sql_sample_23
50
7s
1348
opper_sql_sample_24
100
7s
1359
opper_sql_sample_25
100
8s
1385
opper_sql_sample_26
100
10s
1326
opper_sql_sample_27
75
6s
1401
opper_sql_sample_28
0
7s
1327
opper_sql_sample_29
100
6s
1269
opper_sql_sample_30