Bad Case Report
This page summarizes the output difference between
analysis_model and baseline_model
after calling auto_find_bad_case.
Statistics for all 20 case and Top3 bad case statistics are now grouped
by output.
output_0 summary
| Metric |
All Average |
All Worst |
Top3 Average |
Top3 Worst |
| L1 |
0.058722 |
0.178026 |
0.151333 |
0.178026 |
| COSINE |
0.932461 |
0.791521 |
0.832904 |
0.791521 |
| ATOL |
1.227576 |
1.972829 |
1.952205 |
1.972829 |
output_0 details
All cases error distribution
L1
| 99th percentile |
99.9th percentile |
99.99th percentile |
| 0.177755 |
0.177999 |
0.178023 |
COSINE
| 99th percentile |
99.9th percentile |
99.99th percentile |
| 0.999677 |
0.999857 |
0.999875 |
ATOL
| 99th percentile |
99.9th percentile |
99.99th percentile |
| 1.967733 |
1.97232 |
1.972778 |
output_1 summary
| Metric |
All Average |
All Worst |
Top3 Average |
Top3 Worst |
| L1 |
0.003957 |
0.004128 |
0.004103 |
0.004128 |
| COSINE |
0.999993 |
0.999993 |
0.999993 |
0.999993 |
| ATOL |
0.013574 |
0.014755 |
0.014676 |
0.014755 |
output_1 details
All cases error distribution
L1
| 99th percentile |
99.9th percentile |
99.99th percentile |
| 0.004125 |
0.004128 |
0.004128 |
COSINE
| 99th percentile |
99.9th percentile |
99.99th percentile |
| 0.999994 |
0.999994 |
0.999994 |
ATOL
| 99th percentile |
99.9th percentile |
99.99th percentile |
| 0.014736 |
0.014753 |
0.014754 |
Bad case tables
The bad case input index of each output
| Name/Index |
L1 |
COSINE |
ATOL |
| 0 |
[11, 8, 13] |
[8, 11, 1] |
[1, 15, 4] |
| 1 |
[14, 18, 1] |
[18, 14, 5] |
[11, 12, 3] |
The metric results of each badcase
| Name/Index |
L1 |
COSINE |
ATOL |
| 0 |
[
0.178026,
0.176601,
0.099371
]
|
[
0.791521,
0.810388,
0.896802
]
|
[
1.972829,
1.946006,
1.937779
]
|
| 1 |
[
0.004128,
0.004112,
0.004067
]
|
[
0.999993,
0.999993,
0.999993
]
|
[
0.014755,
0.014656,
0.014618
]
|
The bad case input index of the worst output
| Name/Index |
L1 |
COSINE |
ATOL |
| 0 |
11 |
8 |
1 |
| 1 |
14 |
18 |
11 |