SQL

Natural language to SQL query generation evaluates text-to-query fidelity and schema reasoning. This task is particularly relevant for analytics chat assistants and simplified database interfaces where users need to query data using natural language. Models must understand both the intent behind the question and the structure of the underlying database schema.

85
fireworks/deepseek-v3
Average duration
7s
Average tokens
897
Average cost
$0.00
100
6s
614
opper_sql_sample_01
100
5s
630
opper_sql_sample_02
100
5s
653
opper_sql_sample_03
100
8s
648
opper_sql_sample_04
100
6s
668
opper_sql_sample_05
75
11s
731
opper_sql_sample_06
100
9s
631
opper_sql_sample_07
100
8s
626
opper_sql_sample_08
100
5s
638
opper_sql_sample_09
100
5s
694
opper_sql_sample_10
100
7s
837
opper_sql_sample_11
100
6s
870
opper_sql_sample_12
100
6s
911
opper_sql_sample_13
100
6s
905
opper_sql_sample_14
75
8s
925
opper_sql_sample_15
0
7s
950
opper_sql_sample_16
100
6s
926
opper_sql_sample_17
75
9s
987
opper_sql_sample_18
50
9s
926
opper_sql_sample_19
100
11s
981
opper_sql_sample_20
100
7s
1052
opper_sql_sample_21
75
7s
1120
opper_sql_sample_22
100
7s
1170
opper_sql_sample_23
50
6s
1127
opper_sql_sample_24
100
7s
1165
opper_sql_sample_25
100
6s
1118
opper_sql_sample_26
50
5s
1068
opper_sql_sample_27
75
8s
1199
opper_sql_sample_28
25
5s
1080
opper_sql_sample_29
100
8s
1048
opper_sql_sample_30