Preliminary in silico assessment of the specificity of published molecular assays and design of new assays using the available whole genome sequences of 2019-nCoV

SARS-CoV-2 coronavirus nCoV-2019 Diagnostics and Vaccines
45

Updated to 669,911 COVID-19 WGS from GISAID.

Table 1. Results from PSET analysis. The five Noblis assays were compared alongside the four assays from Corman and three assays from the CDC. Along with assays from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore. Each assay was tested using SARS-CoV-2 as the intended target. All off-target hits (TN, PF, FP) are to entries in NCBI BLAST databases (nt and env_nt). *edited 2/2 to include 5 additional assays.

Row Labels PT TP FN Percent True PF FP TN
Noblis.12 652737 14167 3007 99.55% 0 0 680
Noblis.40 659076 9152 1683 99.75% 0 1 1420
Noblis.42 660034 5746 4131 99.38% 0 2 1136
Noblis.44 657855 6362 5694 99.15% 0 1 1786
Noblis.57 648798 8136 12977 98.06% 0 1 853
ncov_e_gene 662714 2536 4661 99.30% 361 15 53
ncov_n_gene 644728 18601 6582 99.02% 0 340 72
ncov_rdrp_1 1 660834 9076 98.65% 433 105 5967
ncov_rdrp_2 17 661222 8672 98.71% 2 73 5916
cdc_n1 641063 23804 5044 99.25% 0 2 381
cdc_n2 641469 21187 7255 98.92% 0 1 372
cdc_n3 641024 23232 5655 99.16% 1 353 36
China_N 337390 53714 278807 58.38% 0 2 691
China_ORF1ab 647227 5924 16760 97.50% 1 0 367
France_nCoV_IP2 655931 6583 7397 98.90% 1 0 367
France_nCoV_IP4 655759 11939 2213 99.67% 0 0 1190
HKU-N 644999 17401 7511 98.88% 327 33 34
HKU-ORF1b-nsp14 659881 5560 4470 99.33% 324 29 452
Japan_NIID_2019-nCOV_N 0 662510 7401 98.90% 0 0 384
Japan_NIID_WH-1_F24381 650617 8728 10566 98.42% 0 1 601
Japan_NIID_WH-1_F501 658512 6520 4879 99.27% 0 1 370
Japan_NIID_WH-1_F509 651610 13233 5068 99.24% 1 0 369
Japan_NIID_WH-1_Seq_F24383 650396 8886 10629 98.41% 0 0 606
Japan_NIID_WH-1_Seq_F519 648904 8640 12367 98.15% 0 1 369
Japan_WuhanCoV-spk1 653563 5734 10614 98.42% 1 1 594
Thailand_WH-NIC_N 651646 13129 5136 99.23% 0 2 375
Chan-N 643184 18978 7749 98.84% 4 3 353
Chan-ORF1ab 24 661441 8446 98.74% 0 5 2
Chan-S 650869 8827 10215 98.48% 5 0 2
Niu-E 657851 7272 4788 99.29% 26 359 0
Won-E 7 665621 4283 99.36% 0 384 1
Won-N 649249 14520 6142 99.08% 8 7 358
Won-ORF1ab 656774 4244 8893 98.67% 5 2 313
Won-S 643506 4541 21864 96.74% 5 1 0
Yip-ORF1ab 656804 5396 7711 98.85% 5 1 1
Young-N 1 664199 5711 99.15% 0 5 1
Young-ORF1ab 656255 10854 2802 99.58% 5 1 0
Young-S 495902 24739 149270 77.72% 4 1 0
C1_COV_ORF1a 640819 10816 18276 97.27% 0 0 0
C2_COV_ORF1b 647087 17247 5577 99.17% 0 0 0
C3_COV_ORF3a 578321 73005 18585 97.23% 0 0 0
C4_COV_ORF8 504638 150508 14765 97.80% 0 0 0
C5_COV_S_gene 650533 9356 10022 98.50% 0 0 0
46

Updated to 752,038 COVID-19 WGS from GISAID.

Table 1. Results from PSET analysis. The five Noblis assays were compared alongside the four assays from Corman and three assays from the CDC. Along with assays from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore. Each assay was tested using SARS-CoV-2 as the intended target. All off-target hits (TN, PF, FP) are to entries in NCBI BLAST databases (nt and env_nt). *edited 2/2 to include 5 additional assays.

assay PT TP FN Percent True PF FP TN
Noblis.12 732527 15800 3711 99.51% 0 0 680
Noblis.40 739757 10398 1883 99.75% 0 1 1420
Noblis.42 740307 6668 5063 99.33% 0 2 1136
Noblis.44 737939 7338 6761 99.10% 0 1 1786
Noblis.57 726361 9491 16186 97.85% 0 1 853
ncov_e_gene 743808 2907 5323 99.29% 361 15 53
ncov_n_gene 723754 20850 7434 99.01% 0 340 72
ncov_rdrp_1 1 739666 12371 98.36% 433 105 5967
ncov_rdrp_2 18 740163 11857 98.42% 2 73 5916
cdc_n1 718432 27037 6569 99.13% 0 2 381
cdc_n2 719964 24044 8030 98.93% 0 1 372
cdc_n3 718854 25880 7304 99.03% 1 353 36
China_N 369207 63326 319505 57.51% 0 2 691
China_ORF1ab 725167 7166 19705 97.38% 1 0 367
France_nCoV_IP2 734414 8080 9544 98.73% 1 0 367
France_nCoV_IP4 734686 14779 2573 99.66% 0 0 1190
HKU-N 723266 20386 8386 98.88% 327 33 34
HKU-ORF1b-nsp14 739700 6517 5821 99.23% 324 29 452
Japan_NIID_2019-nCOV_N 743740 8298 98.90% 0 0 384
Japan_NIID_WH-1_F24381 728025 10519 13494 98.21% 0 1 601
Japan_NIID_WH-1_F501 739150 7536 5352 99.29% 0 1 370
Japan_NIID_WH-1_F509 731464 15006 5568 99.26% 1 0 369
Japan_NIID_WH-1_Seq_F24383 727819 10645 13574 98.20% 0 0 606
Japan_NIID_WH-1_Seq_F519 728918 9847 13273 98.24% 0 1 369
Japan_WuhanCoV-spk1 731615 6892 13531 98.20% 1 1 594
Thailand_WH-NIC_N 730306 15058 6674 99.11% 0 2 375
Chan-N 721969 21393 8676 98.85% 4 3 353
Chan-ORF1ab 29 742643 9366 98.75% 0 5 2
Chan-S 729213 10863 11962 98.41% 5 0 2
Niu-E 737961 8640 5437 99.28% 26 359 0
Won-E 7 747123 4908 99.35% 0 384 1
Won-N 728953 16147 6938 99.08% 8 7 358
Won-ORF1ab 734716 4747 12575 98.33% 5 2 313
Won-S 722231 5920 23887 96.82% 5 1 0
Yip-ORF1ab 736795 6217 9026 98.80% 5 1 1
Young-N 1 744671 7366 99.02% 0 5 1
Young-ORF1ab 737278 11578 3182 99.58% 5 1 0
Young-S 538001 29767 184270 75.50% 4 1 0
C1_COV_ORF1a 719324 12714 20000 97.34% 0 0 0
C2_COV_ORF1b 725254 19702 7082 99.06% 0 0 0
C3_COV_ORF3a 644109 87439 20490 97.28% 0 0 0
C4_COV_ORF8 549217 184807 18014 97.60% 0 0 0
C5_COV_S_gene 728246 10728 13064 98.26% 0 0 0
47

Updated to 771,942 COVID-19 WGS from GISAID.

Table 1. Results from PSET analysis. The five Noblis assays were compared alongside the four assays from Corman and three assays from the CDC. Along with assays from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore. Each assay was tested using SARS-CoV-2 as the intended target. All off-target hits (TN, PF, FP) are to entries in NCBI BLAST databases (nt and env_nt). *edited 2/2 to include 5 additional assays.

Row Labels PT TP FN Percent True PF FP TN
Noblis.12 751902 16270 3770 99.51% 0 0 680
Noblis.40 759303 10714 1925 99.75% 0 1 1420
Noblis.42 759957 6846 5139 99.33% 0 2 1136
Noblis.44 757518 7538 6886 99.11% 0 1 1786
Noblis.57 745115 9858 16969 97.80% 0 1 853
ncov_e_gene 763494 2982 5466 99.29% 361 15 53
ncov_n_gene 742875 21376 7691 99.00% 0 340 72
ncov_rdrp_1 1 758766 13175 98.29% 433 105 5967
ncov_rdrp_2 18 759274 12650 98.36% 2 73 5916
cdc_n1 737065 27906 6971 99.10% 0 2 381
cdc_n2 738615 25089 8238 98.93% 0 1 372
cdc_n3 737539 26645 7758 99.00% 1 353 36
China_N 377301 67268 327373 57.59% 0 2 691
China_ORF1ab 743986 7491 20465 97.35% 1 0 367
France_nCoV_IP2 753238 8580 10124 98.69% 1 0 367
France_nCoV_IP4 753143 16135 2664 99.65% 0 0 1190
HKU-N 741887 21156 8899 98.85% 327 33 34
HKU-ORF1b-nsp14 759029 6634 6279 99.19% 324 29 452
Japan_NIID_2019-nCOV_N 0 763112 8830 98.86% 0 0 384
Japan_NIID_WH-1_F24381 746657 11058 14227 98.16% 0 1 601
Japan_NIID_WH-1_F501 758595 7816 5531 99.28% 0 1 370
Japan_NIID_WH-1_F509 750562 15625 5755 99.25% 1 0 369
Japan_NIID_WH-1_Seq_F24383 746457 11174 14311 98.15% 0 0 606
Japan_NIID_WH-1_Seq_F519 747842 10190 13910 98.20% 0 1 369
Japan_WuhanCoV-spk1 750347 7330 14265 98.15% 1 1 594
Thailand_WH-NIC_N 749358 15508 7076 99.08% 0 2 375
Chan-N 740611 22101 9230 98.80% 4 3 353
Chan-ORF1ab 29 762175 9738 98.74% 0 5 2
Chan-S 748319 11443 12180 98.42% 5 0 2
Niu-E 757456 8908 5578 99.28% 26 359 0
Won-E 7 766909 5026 99.35% 0 384 1
Won-N 747979 16776 7187 99.07% 8 7 358
Won-ORF1ab 753654 4908 13380 98.27% 5 2 313
Won-S 741106 6212 24624 96.81% 5 1 0
Yip-ORF1ab 756396 6385 9161 98.81% 5 1 1
Young-N 1 764112 7829 98.99% 0 5 1
Young-ORF1ab 756776 11889 3277 99.58% 5 1 0
Young-S 549744 31094 191104 75.24% 4 1 0
C1_COV_ORF1a 738462 13273 20207 97.38% 0 0 0
C2_COV_ORF1b 744087 20333 7522 99.03% 0 0 0
C3_COV_ORF3a 657954 92924 21064 97.27% 0 0 0
C4_COV_ORF8 561543 191450 18949 97.55% 0 0 0
C5_COV_S_gene 747066 11032 13844 98.21% 0 0 0
48

Updated to 864,129 COVID-19 WGS from GISAID.

Table 1. Results from PSET analysis. The five Noblis assays were compared alongside the four assays from Corman and three assays from the CDC. Along with assays from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore. Each assay was tested using SARS-CoV-2 as the intended target. All off-target hits (TN, PF, FP) are to entries in NCBI BLAST databases (nt and env_nt). *edited 2/2 to include 5 additional assays.

assay PT TP FN Percent True PF FP TN
Noblis.12 840401 17638 6090 99.30% 0 0 680
Noblis.40 847976 12014 4139 99.52% 0 1 1420
Noblis.42 848605 7971 7553 99.13% 0 2 1136
Noblis.44 846143 8571 9415 98.91% 0 1 1786
Noblis.57 832185 11279 20665 97.61% 0 1 853
ncov_e_gene 852789 3380 7960 99.08% 361 15 53
ncov_n_gene 830135 23466 10528 98.78% 0 340 72
ncov_rdrp_1 1 847038 17090 98.02% 433 105 5967
ncov_rdrp_2 22 847636 16471 98.09% 2 73 5916
cdc_n1 822947 31266 9916 98.85% 0 2 381
cdc_n2 823872 29373 10884 98.74% 0 1 372
cdc_n3 823441 30000 10688 98.76% 1 353 36
China_N 399624 78797 385708 55.36% 0 2 691
China_ORF1ab 830968 8556 24605 97.15% 1 0 367
France_nCoV_IP2 839447 11228 13454 98.44% 1 0 367
France_nCoV_IP4 838177 20933 5019 99.42% 0 0 1190
HKU-N 828840 23701 11588 98.66% 327 33 34
HKU-ORF1b-nsp14 847559 7380 9190 98.94% 324 29 452
Japan_NIID_2019-nCOV_N 0 852557 11572 98.66% 0 0 384
Japan_NIID_WH-1_F24381 833745 12688 17696 97.95% 0 1 601
Japan_NIID_WH-1_F501 847012 8975 8142 99.06% 0 1 370
Japan_NIID_WH-1_F509 837530 18211 8388 99.03% 1 0 369
Japan_NIID_WH-1_Seq_F24383 833552 12776 17801 97.94% 0 0 606
Japan_NIID_WH-1_Seq_F519 835643 11636 16850 98.05% 0 1 369
Japan_WuhanCoV-spk1 837969 8421 17739 97.95% 1 1 594
Thailand_WH-NIC_N 837128 16988 10013 98.84% 0 2 375
Chan-N 827434 24657 12038 98.61% 4 3 353
Chan-ORF1ab 31 851089 13009 98.49% 0 5 2
Chan-S 835700 14176 14253 98.35% 5 0 2
Niu-E 846012 10048 8069 99.07% 26 359 0
Won-E 7 856631 7491 99.13% 0 384 1
Won-N 835339 18800 9990 98.84% 8 7 358
Won-ORF1ab 841261 5592 17276 98.00% 5 2 313
Won-S 828696 7383 28050 96.75% 5 1 0
Yip-ORF1ab 844899 7263 11967 98.62% 5 1 1
Young-N 1 853370 10758 98.76% 0 5 1
Young-ORF1ab 845510 12966 5653 99.35% 5 1 0
Young-S 584432 35469 244228 71.74% 4 1 0
C1_COV_ORF1a 823809 15490 24830 97.13% 0 0 0
C2_COV_ORF1b 830198 23372 10559 98.78% 0 0 0
C3_COV_ORF3a 731586 107574 24969 97.11% 0 0 0
C4_COV_ORF8 597823 243087 23219 97.31% 0 0 0
C5_COV_S_gene 834391 12540 17198 98.01% 0 0 0
49

Updated to 944,000 COVID-19 WGS from GISAID.

Table 1. Results from PSET analysis. The five Noblis assays were compared alongside the four assays from Corman and three assays from the CDC. Along with assays from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore. Each assay was tested using SARS-CoV-2 as the intended target. All off-target hits (TN, PF, FP) are to entries in NCBI BLAST databases (nt and env_nt). *edited 2/2 to include 5 additional assays.

assay PT TP FN Percent True PF FP TN
Noblis.12 917690 18808 7502 99.21% 0 0 680
Noblis.40 925263 13327 5410 99.43% 0 1 1420
Noblis.42 925947 9089 8964 99.05% 0 2 1136
Noblis.44 923309 9683 11008 98.83% 0 1 1786
Noblis.57 908109 12467 23424 97.52% 0 1 853
ncov_e_gene 930642 3835 9523 98.99% 361 15 53
ncov_n_gene 906131 25387 12482 98.68% 0 340 72
ncov_rdrp_1 1 924703 19296 97.96% 433 105 5967
ncov_rdrp_2 23 925343 18634 98.03% 2 73 5916
cdc_n1 897614 34467 11919 98.74% 0 2 381
cdc_n2 899056 32529 12415 98.68% 0 1 372
cdc_n3 898435 33138 12427 98.68% 1 353 36
China_N 416515 89120 438365 53.56% 0 2 691
China_ORF1ab 906737 9591 27672 97.07% 1 0 367
France_nCoV_IP2 915221 13471 15308 98.38% 1 0 367
France_nCoV_IP4 911708 25944 6348 99.33% 0 0 1190
HKU-N 905366 25336 13298 98.59% 327 33 34
HKU-ORF1b-nsp14 925154 7911 10935 98.84% 324 29 452
Japan_NIID_2019-nCOV_N 0 930708 13292 98.59% 0 0 384
Japan_NIID_WH-1_F24381 910062 13848 20090 97.87% 0 1 601
Japan_NIID_WH-1_F501 924278 9988 9734 98.97% 0 1 370
Japan_NIID_WH-1_F509 913880 20134 9986 98.94% 1 0 369
Japan_NIID_WH-1_Seq_F24383 909878 13928 20194 97.86% 0 0 606
Japan_NIID_WH-1_Seq_F519 912289 12988 18723 98.02% 0 1 369
Japan_WuhanCoV-spk1 914607 9243 20150 97.87% 1 1 594
Thailand_WH-NIC_N 913158 18815 12027 98.73% 0 2 375
Chan-N 903351 26880 13769 98.54% 4 3 353
Chan-ORF1ab 33 928668 15299 98.38% 0 5 2
Chan-S 910227 17335 16438 98.26% 5 0 2
Niu-E 923286 11086 9628 98.98% 26 359 0
Won-E 7 934973 9020 99.04% 0 384 1
Won-N 911451 20655 11894 98.74% 8 7 358
Won-ORF1ab 918334 6222 19444 97.94% 5 2 313
Won-S 905490 8338 30172 96.80% 5 1 0
Yip-ORF1ab 922076 8036 13888 98.53% 5 1 1
Young-N 1 931457 12542 98.67% 0 5 1
Young-ORF1ab 923202 13819 6979 99.26% 5 1 0
Young-S 612115 39958 291927 69.08% 4 1 0
C1_COV_ORF1a 898984 17613 27403 97.10% 0 0 0
C2_COV_ORF1b 905522 26105 12373 98.69% 0 0 0
C3_COV_ORF3a 797075 119259 27666 97.07% 0 0 0
C4_COV_ORF8 628777 289062 26161 97.23% 0 0 0
C5_COV_S_gene 910578 13726 19696 97.91% 0 0 0
50

Updated to 1,057,192 COVID-19 WGS from GISAID.

Table 1. Results from PSET analysis. The five Noblis assays were compared alongside the four assays from Corman and three assays from the CDC. Along with assays from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore. Each assay was tested using SARS-CoV-2 as the intended target. All off-target hits (TN, PF, FP) are to entries in NCBI BLAST databases (nt and env_nt).

Algorithm updated 4/13/2021 to penalize Ns on the subject sequence.

assay PT TP FN Percent True PF FP TN
Noblis.12 1028325 20542 8325 99.21% 0 0 680
Noblis.40 1035399 16020 5773 99.45% 0 1 1420
Noblis.42 1037142 10391 9659 99.09% 0 2 1136
Noblis.44 1034163 11064 11965 98.87% 0 1 1786
Noblis.57 1015499 14692 27001 97.45% 0 1 853
ncov_e_gene 1042085 4344 10763 98.98% 361 15 53
ncov_n_gene 1015226 28772 13194 98.75% 0 340 72
ncov_rdrp_1 1 1035980 21211 97.99% 433 105 5967
ncov_rdrp_2 27 1036687 20478 98.06% 2 73 5916
cdc_n1 1004834 39319 13039 98.77% 0 2 381
cdc_n2 1007030 37156 13006 98.77% 0 1 372
cdc_n3 1005515 38103 13574 98.72% 1 353 36
China_N 434260 106553 516379 51.16% 0 2 691
China_ORF1ab 1016268 11093 29831 97.18% 1 0 367
France_nCoV_IP2 1024226 16518 16448 98.44% 1 0 367
France_nCoV_IP4 1016609 33520 7063 99.33% 0 0 1190
HKU-N 1014389 28203 14600 98.62% 327 33 34
HKU-ORF1b-nsp14 1035829 9211 12152 98.85% 324 29 452
Japan_NIID_2019-nCOV_N 0 1042146 15046 98.58% 0 0 384
Japan_NIID_WH-1_F24381 1015334 15551 26307 97.51% 0 1 601
Japan_NIID_WH-1_F501 1035167 10984 11041 98.96% 0 1 370
Japan_NIID_WH-1_F509 1023681 22161 11350 98.93% 1 0 369
Japan_NIID_WH-1_Seq_F24383 1015160 15621 26411 97.50% 0 0 606
Japan_NIID_WH-1_Seq_F519 1018188 14533 24471 97.69% 0 1 369
Japan_WuhanCoV-spk1 1020629 10209 26354 97.51% 1 1 594
Thailand_WH-NIC_N 1023410 20626 13156 98.76% 0 2 375
Chan-N 1012118 30301 14773 98.60% 4 3 353
Chan-ORF1ab 38 1040042 17112 98.38% 0 5 2
Chan-S 1018292 21245 17655 98.33% 5 0 2
Niu-E 1034200 12161 10831 98.98% 26 359 0
Won-E 7 1047277 9908 99.06% 0 384 1
Won-N 1020160 23987 13045 98.77% 8 7 358
Won-ORF1ab 1028461 7279 21452 97.97% 5 2 313
Won-S 1015126 9670 32396 96.94% 5 1 0
Yip-ORF1ab 1033026 9446 14720 98.61% 5 1 1
Young-N 1 1043455 13736 98.70% 0 5 1
Young-ORF1ab 1034522 14998 7672 99.27% 5 1 0
Young-S 654700 45248 357244 66.21% 4 1 0
C1_COV_ORF1a 1007061 20749 29382 97.22% 0 0 0
C2_COV_ORF1b 1014649 29291 13252 98.75% 0 0 0
C3_COV_ORF3a 893126 134309 29757 97.19% 0 0 0
C4_COV_ORF8 674434 354219 28539 97.30% 0 0 0
C5_COV_S_gene 1018242 15342 23608 97.77% 0 0 0
51

Updated to 1,228,199 COVID-19 WGS from GISAID using BLASTDB version 5.

Run Date: 04/26/2021

Table 1. PSET analysis using SARS-CoV-2 as the intended target. Perfect True Positive (PT), True Positive (TP), and False Negative (FN) counts for various assays were produced in each run. The five Noblis assays were compared alongside with assays produced in the US including the Corman (four assays) and CDC (three assays) brands. Assays originating from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore were also used in this comparison. Algorithms were set to penalize Ns on the subject sequence. Internal improvement to the PSET algorithms and snakemake (Snakemake — Snakemake 6.2.1 documentation) pipeline implementations decreased the Type I and Type II error rates compared with previous versions. We no longer calculate off-target hits due to the continued stability of these values.

Assay PT TP FN Percent True
Noblis.12 1195139 22765 914 99.93%
Noblis.40 1199037 22097 641 99.95%
Noblis.42 1204605 12295 341 99.97%
Noblis.44 1201137 13011 732 99.94%
Noblis.57 1175565 18669 1305 99.89%
ncov_e_gene 1209760 5025 1661 99.86%
ncov_n_gene 1179744 32628 976 99.92%
ncov_rdrp_1 0 1204176 1183 99.90%
ncov_rdrp_2 33 1204985 377 99.97%
cdc_n1 1166643 46652 165 99.99%
cdc_n2 1168898 43869 561 99.95%
cdc_n3 1167043 45309 597 99.95%
China_N 455092 132114 623915 48.48%
China_ORF1ab 1179747 13446 1854 99.84%
France_nCoV_IP2 1188667 21067 688 99.94%
France_nCoV_IP4 1163929 55175 875 99.93%
HKU-N 1179262 31843 666 99.95%
HKU-ORF1b-nsp14 1202615 10956 406 99.97%
Japan_NIID_2019-nCOV_N 0 1210503 904 99.93%
Japan_NIID_WH-1_F24381 1176945 18071 6170 99.49%
Japan_NIID_WH-1_F501 1201666 13439 735 99.94%
Japan_NIID_WH-1_F509 1187262 27454 1178 99.90%
Japan_NIID_WH-1_Seq_F24383 1176691 18167 6393 99.47%
Japan_NIID_WH-1_Seq_F519 1177642 18775 19501 98.40%
Japan_WuhanCoV-spk1 1183035 11961 5827 99.51%
Thailand_WH-NIC_N 1190244 22940 206 99.98%
Chan-N 1175743 35062 1023 99.92%
Chan-ORF1ab 42 1205868 1425 99.94%
Chan-S 1177715 29737 3652 99.70%
Niu-E 1200461 14190 665 99.95%
Won-E 7 1216111 840 99.93%
Won-N 1184667 27857 495 99.96%
Won-ORF1ab 1194309 9288 423 99.96%
Won-S 1179638 11385 10531 99.12%
Yip-ORF1ab 1199321 11457 3608 99.70%
Young-N 1 1211927 345 99.97%
Young-ORF1ab 1201332 17282 639 99.95%
Young-S 703119 53672 447080 62.86%
C1_COV_ORF1a 1170522 24631 1597 99.87%
C2_COV_ORF1b 1179752 33446 510 99.96%
C3_COV_ORF3a 1042529 151375 4831 99.60%
C4_COV_ORF8 728696 466365 2645 99.78%
C5_COV_S_gene 1179784 18269 1241 99.90%
52

Updated to 1,011,071 high coverage COVID-19 WGS from GISAID using BLASTDB version 5.

Run Date: 05/04/2021

Table 1. PSET analysis using SARS-CoV-2 as the intended target. Perfect True Positive (PT), True Positive (TP), and False Negative (FN) counts for various assays were produced in each run. The five Noblis assays were compared alongside with assays produced in the US including the Corman (four assays) and CDC (three assays) brands. Assays originating from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore were also used in this comparison. Algorithms were set to penalize Ns on the subject sequence. Internal improvement to the PSET algorithms and snakemake (Snakemake — Snakemake 6.2.1 documentation) pipeline implementations decreased the Type I and Type II error rates compared with previous versions. We no longer calculate off-target hits due to the continued stability of these values.

Assay PT TP FN Percent True
Noblis.12 987355 18700 214 99.98%
Noblis.40 986919 19473 90 99.99%
Noblis.42 996623 9734 25 100.00%
Noblis.44 995983 10284 33 100.00%
Noblis.57 993041 14943 102 99.99%
ncov_e_gene 1002017 3982 215 99.98%
ncov_n_gene 979433 26322 150 99.99%
ncov_rdrp_1 0 1005540 895 99.91%
ncov_rdrp_2 28 1006038 379 99.96%
cdc_n1 970106 36024 12 100.00%
cdc_n2 969226 36311 76 99.99%
cdc_n3 968441 37759 16 100.00%
China_N 350658 101313 552332 45.00%
China_ORF1ab 998615 7539 147 99.99%
France_nCoV_IP2 988320 17919 87 99.99%
France_nCoV_IP4 955144 51211 131 99.99%
HKU-N 979152 26143 62 99.99%
HKU-ORF1b-nsp14 997478 8909 62 99.99%
Japan_NIID_2019-nCOV_N 2 1005174 192 99.98%
Japan_NIID_WH-1_F24381 993111 14549 230 99.98%
Japan_NIID_WH-1_F501 994233 11753 74 99.99%
Japan_NIID_WH-1_F509 984296 21542 237 99.98%
Japan_NIID_WH-1_Seq_F24383 992903 13469 1538 99.85%
Japan_NIID_WH-1_Seq_F519 982455 15391 8244 99.18%
Japan_WuhanCoV-spk1 998282 9367 203 99.98%
Thailand_WH-NIC_N 987637 18482 66 99.99%
Chan-N 976309 28854 140 99.99%
Chan-ORF1ab 38 1005830 64 100.00%
Chan-S 980269 24956 404 99.96%
Niu-E 995796 10194 227 99.98%
Won-E 5 1006048 156 99.98%
Won-N 983658 22161 21 100.00%
Won-ORF1ab 998483 7934 12 100.00%
Won-S 995881 8201 689 99.93%
Yip-ORF1ab 996891 8979 62 99.99%
Young-N 1 1006230 47 100.00%
Young-ORF1ab 992298 13789 357 99.96%
Young-S 551310 42652 406416 59.37%
C1_COV_ORF1a 984120 17876 176 99.98%
C2_COV_ORF1b 978638 27732 19 100.00%
C3_COV_ORF3a 886664 117684 337 99.97%
C4_COV_ORF8 574988 428122 1573 99.84%
C5_COV_S_gene 993736 14337 82 99.99%
53

Updated to 1,081,736 high coverage COVID-19 WGS from GISAID using BLASTDB version 5.

Run Date: 05/7/2021

Table 1. PSET analysis using SARS-CoV-2 as the intended target. Perfect True Positive (PT), True Positive (TP), and False Negative (FN) counts for various assays were produced in each run. The five Noblis assays were compared alongside with assays produced in the US including the Corman (four assays) and CDC (three assays) brands. Assays originating from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore were also used in this comparison. Algorithms were set to penalize Ns on the subject sequence. Internal improvement to the PSET algorithms and snakemake (Snakemake — Snakemake 6.2.1 documentation) pipeline implementations decreased the Type I and Type II error rates compared with previous versions. We no longer calculate off-target hits due to the continued stability of these values.

Assay PT TP FN Percent True
Noblis.12 1055483 21103 220 99.98%
Noblis.40 1054779 22150 92 99.99%
Noblis.42 1066452 10448 25 100.00%
Noblis.44 1065821 10982 36 100.00%
Noblis.57 1062626 15899 103 99.99%
ncov_e_gene 1072331 4201 216 99.98%
ncov_n_gene 1048853 27418 152 99.99%
ncov_rdrp_1 0 1075806 1172 99.89%
ncov_rdrp_2 31 1076322 635 99.94%
cdc_n1 1038877 37797 12 100.00%
cdc_n2 1038361 37699 76 99.99%
cdc_n3 1036012 40716 16 100.00%
China_N 358119 106799 609811 43.26%
China_ORF1ab 1067798 8839 160 99.99%
France_nCoV_IP2 1058123 18653 90 99.99%
France_nCoV_IP4 1018659 58223 148 99.99%
HKU-N 1048396 27363 69 99.99%
HKU-ORF1b-nsp14 1067524 9406 64 99.99%
Japan_NIID_2019-nCOV_N 2 1075641 204 99.98%
Japan_NIID_WH-1_F24381 1062677 15456 234 99.98%
Japan_NIID_WH-1_F501 1063930 12551 75 99.99%
Japan_NIID_WH-1_F509 1053255 23074 242 99.98%
Japan_NIID_WH-1_Seq_F24383 1062389 14339 1659 99.85%
Japan_NIID_WH-1_Seq_F519 1050519 16955 9112 99.15%
Japan_WuhanCoV-spk1 1067642 10481 205 99.98%
Thailand_WH-NIC_N 1057475 19183 71 99.99%
Chan-N 1045151 30480 151 99.99%
Chan-ORF1ab 41 1076321 73 100.00%
Chan-S 1048072 27596 417 99.96%
Niu-E 1065378 11145 229 99.98%
Won-E 5 1076580 157 99.99%
Won-N 1052374 23961 23 100.00%
Won-ORF1ab 1068015 8946 12 100.00%
Won-S 1065942 8601 721 99.93%
Yip-ORF1ab 1066864 9538 63 99.99%
Young-N 1 1076761 48 100.00%
Young-ORF1ab 1061627 15000 358 99.97%
Young-S 563864 44899 461606 56.87%
C1_COV_ORF1a 1053672 18772 176 99.98%
C2_COV_ORF1b 1047179 29732 19 100.00%
C3_COV_ORF3a 953272 121543 346 99.97%
C4_COV_ORF8 589965 483426 1713 99.84%
C5_COV_S_gene 1063209 15402 84 99.99%
54

Updated to 1,185,134 high coverage COVID-19 WGS from GISAID using BLASTDB version 5.

Run Date: 05/17/2021

Table 1. PSET analysis using SARS-CoV-2 as the intended target. Perfect True Positive (PT), True Positive (TP), and False Negative (FN) counts for various assays were produced in each run. The five Noblis assays were compared alongside with assays produced in the US including the Corman (four assays) and CDC (three assays) brands. Assays originating from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore were also used in this comparison. Algorithms were set to penalize Ns on the subject sequence. Internal improvement to the PSET algorithms and snakemake (Snakemake — Snakemake 6.2.1 documentation 2) pipeline implementations decreased the Type I and Type II error rates compared with previous versions. We no longer calculate off-target hits due to the continued stability of these values.

Assay PT TP FN Percent True
Noblis.12 1155724 22678 222 99.98%
Noblis.40 1150262 28522 94 99.99%
Noblis.42 1167331 11396 31 100.00%
Noblis.44 1166570 12052 40 100.00%
Noblis.57 1163197 18119 110 99.99%
ncov_e_gene 1173766 4558 224 99.98%
ncov_n_gene 1146701 31317 174 99.99%
ncov_rdrp_1 0 1177536 1296 99.89%
ncov_rdrp_2 35 1178118 689 99.94%
cdc_n1 1137338 41167 19 100.00%
cdc_n2 1136718 41150 87 99.99%
cdc_n3 1134039 44491 18 100.00%
China_N 363658 118928 693614 41.03%
China_ORF1ab 1169105 9325 181 99.98%
France_nCoV_IP2 1157622 20989 95 99.99%
France_nCoV_IP4 1105699 72951 227 99.98%
HKU-N 1147909 29662 72 99.99%
HKU-ORF1b-nsp14 1167920 10863 64 99.99%
Japan_NIID_2019-nCOV_N 2 1177439 225 99.98%
Japan_NIID_WH-1_F24381 1164169 16723 254 99.98%
Japan_NIID_WH-1_F501 1164183 14129 79 99.99%
Japan_NIID_WH-1_F509 1152098 26057 253 99.98%
Japan_NIID_WH-1_Seq_F24383 1163797 15460 1909 99.84%
Japan_NIID_WH-1_Seq_F519 1147928 19471 11027 99.06%
Japan_WuhanCoV-spk1 1169458 11424 225 99.98%
Thailand_WH-NIC_N 1157768 20722 77 99.99%
Chan-N 1143755 33683 159 99.99%
Chan-ORF1ab 41 1178087 94 100.00%
Chan-S 1144316 34184 431 99.96%
Niu-E 1166294 12021 238 99.98%
Won-E 5 1178378 161 99.99%
Won-N 1150088 28002 29 100.00%
Won-ORF1ab 1167085 11730 12 100.00%
Won-S 1167839 9321 799 99.93%
Yip-ORF1ab 1167330 10843 71 99.99%
Young-N 1 1178564 55 100.00%
Young-ORF1ab 1161867 16601 363 99.97%
Young-S 590426 48252 533237 54.50%
C1_COV_ORF1a 1153187 20968 185 99.98%
C2_COV_ORF1b 1146761 32002 21 100.00%
C3_COV_ORF3a 1048420 128076 367 99.97%
C4_COV_ORF8 618586 556050 2047 99.83%
C5_COV_S_gene 1164343 17055 91 99.99%
55

Updated to 1,323,492 high coverage COVID-19 WGS from GISAID using BLASTDB version 5.

Run Date: 05/28/2021

Table 1. PSET analysis using SARS-CoV-2 as the intended target. Perfect True Positive (PT), True Positive (TP), and False Negative (FN) counts for various assays were produced in each run. The five Noblis assays were compared alongside with assays produced in the US including the Corman (four assays) and CDC (three assays) brands. Assays originating from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore were also used in this comparison. Algorithms were set to penalize Ns on the subject sequence. Internal improvement to the PSET algorithms and snakemake (Snakemake — Snakemake 6.2.1 documentation 2) pipeline implementations decreased the Type I and Type II error rates compared with previous versions. We no longer calculate off-target hits due to the continued stability of these values.

Assay PT TP FN Percent True
Noblis.12 1288871 25594 252 99.98%
Noblis.40 1278510 36349 144 99.99%
Noblis.42 1301684 13098 47 100.00%
Noblis.44 1300869 13816 65 100.00%
Noblis.57 1296740 20704 152 99.99%
ncov_e_gene 1309334 5008 269 99.98%
ncov_n_gene 1280038 33929 245 99.98%
ncov_rdrp_1 0 1313413 1554 99.88%
ncov_rdrp_2 47 1314043 877 99.93%
cdc_n1 1268555 46056 52 100.00%
cdc_n2 1268144 45377 167 99.99%
cdc_n3 1263742 51017 36 100.00%
China_N 376532 139139 798024 39.25%
China_ORF1ab 1303882 10515 196 99.99%
France_nCoV_IP2 1290867 23801 120 99.99%
France_nCoV_IP4 1221942 92737 285 99.98%
HKU-N 1280481 32751 285 99.98%
HKU-ORF1b-nsp14 1302598 12251 80 99.99%
Japan_NIID_2019-nCOV_N 2 1313059 308 99.98%
Japan_NIID_WH-1_F24381 1298051 18975 319 99.98%
Japan_NIID_WH-1_F501 1298268 16048 154 99.99%
Japan_NIID_WH-1_F509 1284458 29690 331 99.97%
Japan_NIID_WH-1_Seq_F24383 1297522 17690 2136 99.84%
Japan_NIID_WH-1_Seq_F519 1279397 22826 12243 99.07%
Japan_WuhanCoV-spk1 1304034 13004 286 99.98%
Thailand_WH-NIC_N 1291789 22782 95 99.99%
Chan-N 1275309 37758 234 99.98%
Chan-ORF1ab 42 1314058 241 99.99%
Chan-S 1272603 44049 1399 99.89%
Niu-E 1300602 13739 235 99.98%
Won-E 5 1314408 298 99.98%
Won-N 1283023 31015 109 99.99%
Won-ORF1ab 1295236 19657 25 100.00%
Won-S 1303465 10906 2532 99.81%
Yip-ORF1ab 1301352 12816 388 99.97%
Young-N 1 1314514 99 99.99%
Young-ORF1ab 1296251 18274 398 99.97%
Young-S 626128 54307 628418 51.99%
C1_COV_ORF1a 1286258 23711 224 99.98%
C2_COV_ORF1b 1279702 35114 51 100.00%
C3_COV_ORF3a 1172160 140157 661 99.95%
C4_COV_ORF8 660926 648983 2512 99.81%
C5_COV_S_gene 1297856 19658 175 99.99%
56

Updated to 1,415,475 high coverage COVID-19 WGS from GISAID using BLASTDB version 5.

Run Date: 06/08/2021

Table 1. PSET analysis using SARS-CoV-2 as the intended target. Perfect True Positive (PT), True Positive (TP), and False Negative (FN) counts for various assays were produced in each run. The five Noblis assays were compared alongside with assays produced in the US including the Corman (four assays) and CDC (three assays) brands. Assays originating from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore were also used in this comparison. Algorithms were set to penalize Ns on the subject sequence. Internal improvement to the PSET algorithms and snakemake (Snakemake — Snakemake 6.2.1 documentation 2) pipeline implementations decreased the Type I and Type II error rates compared with previous versions. We no longer calculate off-target hits due to the continued stability of these values.

Assay PT TP FN Percent True
Noblis.12 1379006 27207 259 99.98%
Noblis.40 1364879 41763 145 99.99%
Noblis.42 1392454 14102 53 100.00%
Noblis.44 1391559 14897 71 99.99%
Noblis.57 1387069 22129 161 99.99%
ncov_e_gene 1400826 5278 274 99.98%
ncov_n_gene 1367007 38182 287 99.98%
ncov_rdrp_1 0 1405108 1636 99.88%
ncov_rdrp_2 48 1405778 918 99.93%
cdc_n1 1357685 48617 57 100.00%
cdc_n2 1357351 47842 176 99.99%
cdc_n3 1351961 54524 40 100.00%
China_N 381708 155568 867964 38.23%
China_ORF1ab 1394965 11190 207 99.99%
France_nCoV_IP2 1380683 25745 129 99.99%
France_nCoV_IP4 1305290 101141 317 99.98%
HKU-N 1370215 34660 300 99.98%
HKU-ORF1b-nsp14 1393606 13023 85 99.99%
Japan_NIID_2019-nCOV_N 3 1404692 320 99.98%
Japan_NIID_WH-1_F24381 1388298 20391 343 99.98%
Japan_NIID_WH-1_F501 1388322 17644 164 99.99%
Japan_NIID_WH-1_F509 1373840 31944 357 99.97%
Japan_NIID_WH-1_Seq_F24383 1387693 18953 2389 99.83%
Japan_NIID_WH-1_Seq_F519 1368618 24802 12710 99.10%
Japan_WuhanCoV-spk1 1394779 13924 309 99.98%
Thailand_WH-NIC_N 1382094 24172 102 99.99%
Chan-N 1364669 40026 245 99.98%
Chan-ORF1ab 47 1405751 259 99.99%
Chan-S 1357925 50498 1499 99.89%
Niu-E 1391685 14414 243 99.98%
Won-E 5 1406170 310 99.98%
Won-N 1369388 35891 120 99.99%
Won-ORF1ab 1375505 31165 25 100.00%
Won-S 1394416 11587 2635 99.81%
Yip-ORF1ab 1391831 14076 400 99.97%
Young-N 1 1406187 111 99.99%
Young-ORF1ab 1385853 20446 403 99.97%
Young-S 654144 56697 688394 50.80%
C1_COV_ORF1a 1376415 25134 248 99.98%
C2_COV_ORF1b 1368614 37976 58 100.00%
C3_COV_ORF3a 1258645 145336 691 99.95%
C4_COV_ORF8 691359 709972 2624 99.81%
C5_COV_S_gene 1387728 21552 181 99.99%
57

Updated to 1,467,158 high coverage COVID-19 WGS from GISAID using BLASTDB version 5.

Run Date: 06/15/2021

Table 1. PSET analysis using SARS-CoV-2 as the intended target. Perfect True Positive (PT), True Positive (TP), and False Negative (FN) counts for various assays were produced in each run. The five Noblis assays were compared alongside with assays produced in the US including the Corman (four assays) and CDC (three assays) brands. Assays originating from China, Hong Kong, Thailand, Japan, France, South Korea, and Singapore were also used in this comparison. Algorithms were set to penalize Ns on the subject sequence. Internal improvement to the PSET algorithms and snakemake (Snakemake — Snakemake 6.2.1 documentation 2) pipeline implementations decreased the Type I and Type II error rates compared with previous versions. We no longer calculate off-target hits due to the continued stability of these values.

Assay PT TP FN Percent True
Noblis.12 1428580 28171 261 99.98%
Noblis.40 1410187 46998 146 99.99%
Noblis.42 1442321 14775 54 100.00%
Noblis.44 1441381 15606 75 99.99%
Noblis.57 1436632 23166 172 99.99%
ncov_e_gene 1451168 5450 284 99.98%
ncov_n_gene 1416576 39005 300 99.98%
ncov_rdrp_1 0 1455502 1782 99.88%
ncov_rdrp_2 53 1456202 1029 99.93%
cdc_n1 1406276 50549 59 100.00%
cdc_n2 1406225 49444 180 99.99%
cdc_n3 1400673 56336 42 100.00%
China_N 385695 171479 898540 38.27%
China_ORF1ab 1445057 11579 224 99.98%
France_nCoV_IP2 1429891 27059 137 99.99%
France_nCoV_IP4 1350568 106392 327 99.98%
HKU-N 1419280 36071 303 99.98%
HKU-ORF1b-nsp14 1443698 13470 88 99.99%
Japan_NIID_2019-nCOV_N 3 1455163 327 99.98%
Japan_NIID_WH-1_F24381 1438097 21126 362 99.98%
Japan_NIID_WH-1_F501 1438212 18234 172 99.99%
Japan_NIID_WH-1_F509 1422997 33253 383 99.97%
Japan_NIID_WH-1_Seq_F24383 1437450 19715 2422 99.83%
Japan_NIID_WH-1_Seq_F519 1417852 25623 13148 99.10%
Japan_WuhanCoV-spk1 1444755 14483 329 99.98%
Thailand_WH-NIC_N 1431790 25005 104 99.99%
Chan-N 1413915 41254 247 99.98%
Chan-ORF1ab 52 1456204 272 99.99%
Chan-S 1402833 56135 1541 99.89%
Niu-E 1441802 14812 250 99.98%
Won-E 5 1456689 324 99.98%
Won-N 1418985 36687 125 99.99%
Won-ORF1ab 1414489 42714 27 100.00%
Won-S 1445481 11994 2752 99.81%
Yip-ORF1ab 1441625 14801 411 99.97%
Young-N 1 1456685 115 99.99%
Young-ORF1ab 1435631 21200 405 99.97%
Young-S 675594 58563 712969 50.73%
C1_COV_ORF1a 1425815 26184 254 99.98%
C2_COV_ORF1b 1418115 38997 73 99.99%
C3_COV_ORF3a 1305249 149194 703 99.95%
C4_COV_ORF8 716173 735316 2789 99.81%
C5_COV_S_gene 1437520 22364 193 99.99%